TCP/IP in less than 200 lines of code

Thu, 3 Apr 2008 21:39:52 -0700

Lately I’ve been wishing for a much more expressive programming language. This is probably because I’ve spent the past year working on a grotty low-level data-access component in C++ that has to be portable to several *nix platforms. The variability of C++ compiler quality on different platforms is stunning -- “I can’t believe people pay for this” kind of stunning. We were forbidden from using the STL because it was completely broken on one of our target platforms. (Even worse, that target platform was dropped as a requirement after we had spent nearly six person-months developing and testing our own portable library... but that’s another story). Template support for many of our target compilers is pretty poor, so we ended up using templates in only the most rudimentary fashion. If you’ve done much heavy lifting in C++, you know how verbose it can get without a bit of meta-programming to automate things.

When we were nearing functional completion, I decided to fire up Construx Surveyor (Steve McConnell FTW!) and see how big the code base had become. I was shocked to find that it was nearly twice the size that I thought it would be! Suddenly it was clear to me why we were all working a lot harder to make our deadlines than we had expected. (Say what you will about the fuzziness of counting lines of code, it is still an important determinant of project effort.) Still, it was surprising because we had all been vigilant about avoiding and eliminating copy-paste code. Sure, there were a few areas where more abstraction could have saved some repetition, but the schedule didn’t allow for it at the time... catch-22. Besides that, what irked me was how the language itself kept forcing my hand by its lack of expressiveness -- lambdas and type inference would have helped in many situations. The lack of decent cross-platform template support also really hurt here.

To give my poor brain some relief from this bleak reality, I spend a lot of time learning other languages like Objective-C, Scala, and Smalltalk to see how their approaches to problem-solving differ from the usual dreck of C++. In my exploration of Smalltalk, I ran across a really cool lecture on iTunes U by Ian Piumarta about a simple and extensible object model he’s working on. The lecture is worth a look, as are the related papers. It’s part of the STEPS project at Viewpoints Research Institute, which aims to re-invent computer systems, especially programming. What appeals to me about it is the focus (albeit not directly stated) on taking the DRY principle to its ultimate extreme -- every aspect of the entire computer system shall have one and only one concrete representation. The STEPS paper explains with examples how this is achieved for certain domains like graphics, text layout, IDEs, and networking. What blew me away was the tiny TCP/IP implementation, which was expressed in a Yacc-like DSL in under 200 lines of code!

Obviously, the lines of infrastructure code required for implementing the DSL cannot be ignored, but it is amortized by being used for just about everything. The entire system is based on the idea of expressing a sub-system, component, or program in the most succinct and meaningful way possible, then using a series of tree-rewriting steps to produce optimized executable code. The difference from the modern OS/library/compiler trifecta is that you the developer can trivially intervene at any stage because the entire thing is dynamic, late-bound, and completely flexible.

When I think about how I might use tools like these to create a DSL for low-level cross-platform database access, it almost makes me want to spend time re-writing my entire project as a Smalltalk-based C code generator.

On second thought, I think I’ll stick with my honeymoon in Maui.

Bits For Thought

TCP/IP in less than 200 lines of code