My RC2014 Web Server
Tue 10 December 2019Tagged: rc2014, software
I wrote a web server for my RC2014. It runs from CP/M, which has no built-in concept of networking, so I had to implement every layer of the networking stack, which in this case is SLIP, IP, TCP, and HTTP. It totals about 1200 lines of C, all of which was written on the RC2014 in the ZDE 1.6 text editor, and compiled with the Hi-Tech C compiler.
SLIP is dead easy, but claiming that the project implements IP, TCP, and HTTP might be a bit of a stretch. It does the absolute bare minimum required to trick people into thinking that it works. The most obvious shortcomings are that it is really slow and it has no way to retransmit dropped packets, which I'll explain further down. But it does (appear to) work. You can browse a website being served by my RC2014 at http://moonship.jes.xxx:8118/, assuming it is still up by the time you read this. It is extremely slow, for various reasons, and only supports 32 simultaneous open connections, the combination of which means it is very easy to DoS (even by accident, but please don't do so on purpose).
I have made a short video showing the RC2014 serving a web page in case you want to see what it looks like with the console output and blinkenlights.
You can get the code on github, but I wouldn't recommend using it. It even includes a CP/M executable in case you want to try it on your own machine. It doesn't depend on anything RC2014-specific, so as long as your CPU supports Z80 instructions and your CP/M connects the "reader" and "punch" devices to a serial port, you can then plug that serial port into another computer that speaks SLIP and it ought to work. The IP address is fixed at 192.168.1.51. If you want to change that you can probably find it in the binary and hexedit it. I suspect you should only need to change it in one place, but I haven't actually checked.
Challenges
I've written a web server in C before, so I was expecting the lower-level networking stuff to be the only hard part about this project. If I had written it on a modern machine and cross-compiled to CP/M, that would probably have been true. But I wanted the full retro experience, so I wanted to develop it all inside CP/M. That meant all of the development tools I know and love are missing. It's just a text editor and a compiler. The text editor is no vim and the compiler is no gcc, and there's nothing remotely resembling gdb or valgrind. Fortunately, I was able to make use of Wireshark by running it on the Linux end of the SLIP connection. If I had to debug all of the networking code with printfs I would have certainly been driven mad.
It takes about 7 minutes to compile the server, which makes for a frustratingly-slow edit-compile-test cycle.
Compile-time memory usage
The "transient program area" on my system is only 54 kilobytes. There is more memory available via bank-switching, but the C compiler doesn't know how to use it. I've found that it runs out of memory very easily when compiling even small programs. Sometimes it gives you a helpful "Out of memory" message, but usually it gives unintelligible nonsense (either incorrect error messages, or just random bytes printed to the console). When this happens you need to split your source file into 2 smaller ones that can be compiled independently, which is quite annoying but relatively easy to do.
I ended up calling my networking files "net.c", "net2.c", and "net3.c" and the web server files "httpd.c", "httpd2.c", "httpd3.c", and "httpd4.c". The code in each file is vaguely grouped by purpose, but not much. If it was left up to me, there would probably have only been 2 source files (net.c and httpd.c).
Text editor
I haven't figured out how to copy and paste in ZDE and I'm not sure whether it's possible. This is a problem when it comes to splitting a source file into 2, because it means I don't know how to copy text out of one file and paste it in another. The solution I've come up with is to use CP/M's COPY.COM program to create a copy of the source file, and then delete the top half of the functions in one, and delete the bottom half in the other. That way I've moved some of the functions to a new file without having to type them all out again.
ZDE has other weirdness, such as using key combinations that I've never seen in any other software and sometimes spuriously drawing an "I" character at the end of a line. But it is a substantial improvement over ED.COM. It's usable.
Memset
The Hi-Tech C compiler's memset() prototype in stdlib.h is extern void * memset(void *, int, size_t). The usage of "int" and "size_t" implies that the second argument is the value, and the third argument is the length. This is the correct order of arguments for memset. Unfortunately, the implementation of memset uses the arguments the other way around, so you have to swap them in your code, which means your code is no longer portable to other compilers, and it also means all C programs that use memset won't quite work properly. I spent quite a few hours stuck on this.
Open files
The C library only supports 8 simultaneous open files, and 3 of these are taken up for stdin/stdout/stderr. If we want to support 32 simultaneous clients, we need to keep track of our position in 32 simultaneous files. I solved this problem by creating my own "jesfile_t" which is kind of like a "FILE *" except you're allowed more of them. It works by remembering the filename and byte offset within the file. Whenever you try to read from the file, it opens it, seeks to the current offset, performs the fread() call, updates its byte offset, and closes the file. That means it only ever has 1 file open at a time, but it can keep track of its position in all 32. The implementation is relatively simple. I only implemented "jesfopen()", "jesfclose()", and "jesfread()" because that's all I needed, but you could also use the same idea to replace fwrite(), fgets(), etc. Indeed, that is essentially all a "FILE *" does internally: it just keeps track of which file you're working with and where you are within it.
Stale connections
With a fixed number of connection slots, and no way to timeout idle connections, we can easily end up with a full connection table and no active clients! For example, if 32 clients opened connections and then just walked away, that would fill up all 32 slots, and no new clients would be able to connect. To work around this, the server keeps track of the order in which the connections were last active. In the event that it needs to allocate a new connection slot, and there are none empty, it just reuses the one which was least-recently active. Probably in this case it should send a "reset" packet to the other side of the connection that is being forgotten, but it doesn't. It just silently forgets about the old connection.
Reading from the serial port
The serial port is represented inside CP/M as the "reader" and "punch" devices. I guess it was originally expected that the only stream I/O device apart from the console would be a paper tape reader and punch. There is a blocking BDOS call (system call) for each of reading a byte from the reader and writing a byte to the punch. There are comparable blocking functions for the console, but the console also comes with a "get console status" call that will tell you whether reading a byte would block. There is no such function for the reader, which means you just have to ask for a byte and then sit there until you get one. CP/M literally has no way to find out whether reading a byte from the reader will block.
My RC2014 has the reader backed by a Z80 SIO/2 module, which does have a way to tell you whether or not a read will block, so it would be relatively easy to switch to driving the SIO module directly, instead of using the CP/M BDOS call, but currently there is nothing useful the web server can do in the absence of any input bytes, so there's no harm in blocking.
One downside of blocking on the reader is that it prevents ^C from exiting the program. There is in fact no way to exit the program until the read call returns, which means to exit it you need to either reset the machine, or type ^C and then send it some bytes. It seems as though the "read reader" BDOS call checks whether you've typed ^C at the console, but only upon entry. If you type ^C after it's started blocking, it doesn't notice until the read call returns and another read call is entered and it checks again.
Send buffering
The MTU of the SLIP link is 256 bytes. With TCP and IP headers, that becomes 296 bytes. In order to retransmit dropped packets, we'd need to store a copy of every packet that has been sent, and can only discard it after it has been ACK'd. We support 32 simultaneous clients, so if we allocated enough buffer space to store only 6 packets per client, that would exhaust the entire transient program area before we've even started. So far I'm just storing the last sent packet for each client (even though there's no facility to ever retransmit any). Thanks to CP/M's shortage of development tools, I have no idea how much memory the program uses while running. The binary is 21K on disk, but it does some dynamic allocation as well.
Packet retransmission
There are two times that a TCP implementation might want to retransmit a packet.
Duplicate ACK
If you receive 2 acknowledgments of the same sequence number, it means the remote side has discovered that it has missed a packet. For example, assume we sent packets with sequence numbers 1, 2, and 3. If the remote side receives no. 1, and acknowledges it, and then receives no. 3, it can't acknowledge no. 3 because it hasn't seen no. 2. In this case it sends a duplicate acknowledgment for packet no. 1. When the sender receives the duplicate acknowledgment of packet no. 1, it knows that packets may have been dropped. If the sender receives a third acknowledgment for packet no. 1, it must retransmit no. 2.
In our case, we only ever send 1 packet at a time, then wait for it to be acknowledged, before we'll send another. This is both bad for bandwidth (because we need to wait 1 entire round-trip-time between every 256 bytes of content), and it means we can never receive a duplicate ACK. So in this case, we don't retransmit because, even if that logic were implemented, we have no way to trigger duplicate ACKs in the first place.
Timeout
If the sender sees that too much time has passed since sending a packet, and no acknowledgment has yet been received, it must retransmit the packet. This means the sender must have some way to measure the passage of time. Unfortunately, my RC2014 does not have any way to measure the passage of time. The best way to do this would be with a Z80 CTC module, but I haven't bothered for now.
In the absence of some sort of timer interrupt, we could conceivably count instructions, or loop iterations, to get an estimate of how much time has passed, and retransmit once the estimate has got too high. Unfortunately, the CP/M BDOS call to read from the reader blocks until bytes are available. Firstly, we can't pre-empt it to retransmit a packet if we decided we wanted to. Secondly, we don't know how long it blocked for, so we don't even know if it's time yet to retransmit a packet.
For these reasons, my web server does not retransmit packets in this case either. This means that if a packet is dropped, the connection just hangs. Not ideal, but it's rare enough not to matter too much for a toy project.
Improvements
The most important improvement is to add a CTC to facilitate timed retransmission, and also drive the SIO device directly instead of via the BDOS so that it can be used in a non-blocking way. After that, to improve bandwidth it would be good to come up with a way to transmit several packets without getting the acknowledgments in between. We could allocate a fixed buffer large enough to store say 8K of sent-but-not-yet-acknowledged packets, and allocate space in this buffer across different clients as and when they need it. In the event that there is not enough space in the buffer, we just can't send another packet yet.
There are also probably some bugs in the TCP state machine. I made no effort to actually faithfully implement TCP (I haven't even read the RFC). It just goes through the motions enough to work in the normal case.
If you like my blog, please consider subscribing to the RSS feed or the mailing list: