Automated Regression - still running 10 years later

 

The Problem

Our client had bought an off-the-shelf market data repository that required extensive customisation to meet their requirements.  The project was over-running and a rash of technical issues was hampering progress. The test team were a disparate group of contract testers.  Under pressure to deliver, animosity existed between the developers, testers and business analysts which had resulted in a breakdown in communication. There were frequent software releases to address the burgeoning bug list, but no test automation.  This meant that >60% of each test cycle was occupied with labour-intensive manual regression testing.  This was disheartening the testers and distracting them from doing proper testing.

The Solution

We encouraged an amnesty and instituted a set of practical measures to right the sinking ship:

  1. We introduced a 'no communication, no defect' rule.
  2. We filtered the existing bug list removing spurious or non-reproducible defects.
  3. We collaborated with the developers to extend their existing unit tests into a fully fledged automated system test suite using JUnit.
  4. We coached the business analysts to ditch their 'War and Peace' specifications in favour of Excel worked examples.

By the time the project went live, there were >250K automated tests running nightly to prove the data validation and cleansing routines and the system's complex curve building logic.  

These tests are still running 10 years later!