Automated Regression - still running 10 years later
The Problem
Our client had bought an off-the-shelf market data repository that required extensive customisation to meet their requirements. The project was over-running and a rash of technical issues was hampering progress. The test team were a disparate group of contract testers. Under pressure to deliver, animosity existed between the developers, testers and business analysts which had resulted in a breakdown in communication. There were frequent software releases to address the burgeoning bug list, but no test automation. This meant that >60% of each test cycle was occupied with labour-intensive manual regression testing. This was disheartening the testers and distracting them from doing proper testing.
The Solution
We encouraged an amnesty and instituted a set of practical measures to right the sinking ship:
- We introduced a 'no communication, no defect' rule.
- We filtered the existing bug list removing spurious or non-reproducible defects.
- We collaborated with the developers to extend their existing unit tests into a fully fledged automated system test suite using JUnit.
- We coached the business analysts to ditch their 'War and Peace' specifications in favour of Excel worked examples.
By the time the project went live, there were >250K automated tests running nightly to prove the data validation and cleansing routines and the system's complex curve building logic.
These tests are still running 10 years later!