Always test in context

03 May

When testing your work, it’s essential to always test it in the context that it’s going to be used in.

Last weekend, I helped a friend move her stuff down to the Orlando area. UHaul trailers are incredibly cheap compared to trucks, and my car has a hitch, so we got one of those, loaded up her stuff and headed south. Somewhere along the way, the hookup for the lights became disconnected and then dragged along the ground, getting all nice and melted in the process. I noticed this when we got to her new apartment and called UHaul. Awesomely, they have free roadside assistance for any time something happens to their trailer.

They sent out a guy, he looked at the hookup on the trailer, rewired it, and tested it with his wire testing doohickey before taking off. We had already disconnected the trailer from the car for parking purposes, and when he pulled in to fix the trailer he blocked access to my car, so we didn’t actually hook it up to my car to test it.

A few hours later, it’s time to go home and return the trailer and oh, crap, the lights don’t work. I had to call UHaul and have them come fix it again. Because of the way the trailer was hooked up originally, the tongue of the trailer managed to get on top of one of the wires on my side of the connector and eventually wore through it, something I hadn’t noticed previously. It also managed to blow a fuse at some point so my brake lights didn’t work at all even when the wiring was fixed. Fortunately, the UHaul guy was very friendly and also had all the parts one could ever need for fixing minor car problems, so he easily re-rewired the connections and fixed my fuse.

Still, a few extra minutes of checking the trailer in the context it was going to be used in (i.e. attached to my car) would have revealed the problems and saved UHaul several hours worth of labor (driving to and from the apartment complex; they were far away).

That was a non-technical example of the principle, but there are certainly plenty of technical ones as well:
When I wrote the code to handle our PayPal processing, PayPal helpfully provided a development sandbox for testing with. All my code was thoroughly tested, seemed to be handling every sort of error that I threw at it, processing successful charges properly and everything. Then when it was time to release we pointed it at the ‘real’ PayPal servers and suddenly it didn’t work anymore. A few (real) payments later and some minor differences between how the real servers worked and how the dev servers worked were revealed. No big deal, but again, some testing up front in the correct context would have prevented the issue from ever appearing in the first place.


Comments are closed.