Pure Danger Tech


Developer testing with the Testivus Manifestivus

09 Feb 2007

I heartily recommend a read of Alberto Savoia’s Testivus post. This is exactly the kind of pragmatic and useful advice that I find inspiring but not preachy.

I’m a huge advocate and fan of developer testing and I’ve done a whole heck of a lot of it. Over the years I’ve tried all sorts of stuff and have done my best to infect others with the need to test. Honestly, it’s hard to do that without sounding either like a crazy nut job or a preachy holier-than-thou know-it-all (I suspect I’ve been seen as both over the years). In fact, the risk of being seen as either is often enough to just downplay the whole thing and try to lead by example.

I know that I never saw the golden light part the heavens till I felt (viscerally) what it was like to make changes on a code base with a solid bed of tests by my side. The joy of being able to evolve the code as you know it should be without fear of breaking stuff is something that needs to be felt to be understood. The problem is that you have to do a lot of up-front (and sometimes tedious) work to feel that freedom later, and that can be a tough sell to new developers unfamiliar with the practice.

I’m not sure how I feel about the whole test-first and TDD movement. When doing maintenance work, test-first feels very natural and I work that way almost instinctively. But when doing new development, I have not been able to convince myself to actually work that way. I’m usually more of a test-right-after kind of guy (or something like that). The problem is that I think best in code and sometimes I want to work through a few designs until I find one that starts to work and that means a huge amount of churn at the beginning. Having a bunch of tests to evolve with the code in tandem is just too much of a drag for that phase. I know saying stuff like this will make the TDD extremists shudder. I want to believe that test-first could work even for this part of the dev cycle but somehow I just don’t (or at least don’t believe it enough to practice it).

I have separate concerns about how test-first code works on the design level. I’ve seen a lot of test-first code that was really awful. It had tests and it “worked” in that the tests passed, but the structure of the code was a horrifying unstructured organic mess. I couldn’t help but think that some up-front design would have given rise to a better end state.

So, bringing this back to the original topic, that’s why I like the “Testivus Manifestivus”. I think it’s important to make developer testing something that everyone can do, even if you don’t happen to write a test that fails to compile before every change. I think I would add a few items:

Automation pays for itself….eventually

I’ve spent a larger chunk of time than I care to admit writing testing automation in some form or another. Even so, I know that every time (and I mean 100%) I’ve automated something, it paid for itself. Sometimes it took weeks or months, but it ALWAYS paid for itself. If you’re on the fence about whether to automate something (database reset, integration testing, test scripts, etc), then just do it and trust it’ll be worth it.

Embrace feedback

So many things about testing are about creating feedback loops or making them tighter. Daily testing is better than weekly. Continuous integration testing is better than daily testing. Test feedback in your IDE as you make changes is even better. Same with metrics, or automation, or post-mortems, or just about anything.