Pure Danger Tech


Open Source Revolution

22 Nov 2009

As we close out a decade, I was reflecting on what it was like to write code 10 years ago. I had just started at a startup then called Quadrian (later MetaMatrix) and working with five other developers in the frontage space of a steel mill (super cheap rent).

We were using Java and JDK 1.2 was the new hotness with the collections library, Swing, and reflection. From a tools point of view, I did all my coding back then in Programmer’s File Editor (PFE), which had the basic advantage that it wasn’t Notepad. :) There were some Java IDEs at that point but they were slow and lacked many of today’s niceties. It was probably 3 more years before IDEs got fast enough and useful enough that it was faster than just using a text editor (for me at least).

We were building the 1.0 version of an information integration application and we tried to figure out how to take advantage of the brand spanking new EJB and J2EE 1.2 spec that were just coming out. However, we were building a bunch of things that required long-running background tasks which just didn’t exist in the J2EE lexicon back then. So we actually built a hybrid of J2EE services and RMI-based external Java services. Crazy stuff. Kinda worked. We tried for a couple more years to leverage EJB but ultimately rewrote it as an independent server application which made a whole lot more sense.

We were using the still nascent Weblogic 4.5 as the deployment platform back then and trying to understand the nuances of the new-fangled XML stuff. We used XML as a storage medium for our global metadata catalog. We later experimented with putting that metadata in the Versant object database (the API was an early precursor of JDO) and even LDAP (behind JNDI)! We ultimately used the emerging metadata MOF standards from the OMG, stored in a normalized relational DB but that was years in the future still.

We were a small team, so project planning largely consisted of writing some stuff down on a whiteboard. I can’t remember doing anything more than that. Things like agile methods were still a few years from emerging into the limelight. We did code reviews. Unit testing hadn’t really been invented yet, or at least not in the xUnit form we see today. I pushed through our initial unit testing suite in probably 2001 or 2002. We did have tests but they were usually buried in main() methods in each class you had something to test and were run manually.

Builds were a nightmare. We were using source control, but all we could afford was MS Visual Sourcesafe, which we had free along with our MS licenses. It sucked hard but was certainly better than nothing. The “build system” (quotes well-deserved) was a cobbled together set of batch scripts and some Java programs. Much later these got rewritten in Perl and eventually in Ant. I’d written these scripts at a previous job and had re-adapted them. They were mostly concerned with obtaining the software from VSS, compiling it, and creating a jar as far as I recall.

The actual deployment of our software was incredibly arcane as it was a first version of an app server-based app, RMI, etc along with connectors to various data feeds for news, stock quotes, etc. We didn’t have a kit or anything – it was a hand-built process installation at first. We actually shipped the first few versions to customers by buying a computer, installing the software, and traveling to the site with the configured computer. Seems incredibly primitive now.

Learning and trouble-shooting in those days was soooo much harder. Debuggers just barely worked, if at all. Logging frameworks didn’t exist yet that I recall (println was state of the art). Books were the primary source of learning with topical knowledge coming from magazines like Java Report, Java Developer’s Journal, etc. I got them all just to stay up to date. We had web search but this was pre-blog so if you were lucky you might find something on a vendor site or a forum or something. Most often you were just SOL and had to contact the vendor. From what I recall, most stuff was buggy and just didn’t work well.

We had user groups but there were hardly any conferences or other places to share experience. I did my first user group talk on JavaSpaces and TupleSpaces (and my framework abstraction that did both) in 1999.

In general, we rolled almost everything ourselves. We probably did use a few open source libraries, but I would guess they were restricted to XML parsers and things like that. Apache only had a handful of web-oriented libraries back then.

But in some ways, life was simple. We wrote a doc and built it from nothing to 1.0 in just 3 or 4 months. If we did it today, I’m sure we would do it completely differently, but I’m not sure we would do it any faster. These days, the bulk of the software in any app you build is probably open source libraries which probably significantly outpaces your own code. We have so many more resources in libraries, blogs, forums, user groups, conferences, and more but that means we also have to spend a lot longer choosing, configuring, and integrating all the dozens (or hundreds) of components we rely on to buidl our apps.

In general, I think this is a good thing. The quality of those libraries everyone uses is generally pretty high and actively improved all the time. Testing is now a standard practice and most common libraries have large test suites covering the bulk of the code. We now build apps and spend more time focusing on business value, not utilities. But I think we’ve paid for it on the integration side some too.

Certainly the tools (mostly open source) we have today are phenomenal compared to what we had a decade ago. We have a plethora of great IDEs, profilers, style checkers, static code analyzers, memory analyzers, structural analysis tools, and more. I love having all those choices and I think that’s an unqualified win.

Looking forward to seeing where we’ll be in 2019…. :)