This is a story about an interface and an automated test suite just like all the best stories.
The interface was complex and mysterious. A retinue of analysts toiled night and day, endeavouring to tame its complexities and to shine light on its mysteries. A hapless developer struggled valiantly to codify the requirements which seemed to change with every passing breeze.
However, a brave and enlightened tester pitched into the battle. He created a suite of automated tests which invoked the interface code in all the ways that were understood and, as requirements changed, updated those tests so that when the developer inadvertently broke stuff that was previously working, it was quickly spotted and rectified.
All was well in the kingdom and the interface was blessed by the release king and allowed to roam free.
Days went by and peace reigned until one day a rogue bug entered the kingdom in disguise and brought turmoil to the previously happy scene.
The developer and tester were summoned to court and pleaded for another chance to make things right. The king was wise and understood their protestations that the bug was brought about by circumstances beyond their control and, anyway, the analysts were really to blame.
The developer worked tirelessly until a fix was discovered and the tester once again unveiled his magic harness.
But alas, the harness didn’t work! In the days that had passed, some of the tests had started to fail because the lands around the interface were infested by shape shifters which confused and otherwise interfered with the tests.
But our heroic tester didn’t despair for he had an idea.
He would create a new type of harness: one which would be immune to the shape shifters and would in fact grow and adapt to the changing lands around the interface.
The task turned out to be far easier than when he created the original suite. The harness was instructed to look first at the system it was testing and then to create tests dynamically based on existing entities, thus avoiding the evil machinations of the shape shifters. He was even able to add additional tests that would again act only upon these existing entities. The coverage was high and destined to grow with the system itself. Of course, the comparison was only against the previous version of the software, but that suited well his regression testing requirements.
Some in the testing brotherhood at first dismissed these ideas as witchcraft or black magic but when they saw the evidence with their own eyes, they too were enlightened. They realised that this new method was in fact an addition rather than a replacement for their ‘first principles’ tests, a tool to be used for an existing system or once a system has reached a point of maturity and stability.
Never again would they be so burdened with ongoing test maintenance once a system was live and, somewhat inevitably, they all lived happily ever after!