Dynamic Testing with Triangle

Why Dynamic testing?

If you already have a set of automated tests then the single biggest threat to your investment is the seemingly unavoidable prospect of high test maintenance costs.

It is often assumed that trading systems largely grow through addition; new functionality that requires new tests. In practice more effort can be spent maintaining old tests than writing new tests.

Spiralling maintenance can usually be attributed to two distinct classes of failure:

• The Scope Battle

• The Brittle Test Problem

The Scope Battle

When a new trading system goes live it is usually configured (and tested) to support a range of products; e.g. particular financial products or commodities.

But the business does not stand still, and immediately wants to trade variations on existing products (new currencies, new pricing options, new locations, etc). The Scope Battle has begun.

As the scope of the system evolves, test costs are driven up in two ways:

• The cost of keeping up

  • Testers spend an increasing amount of time writing new tests, that are minor variations on tests that have been written before.

  • The time taken to write new tests starts to slow down the process of on-boarding a new product.

  • Product owners get frustrated at the delays.

  • The calls to “skip” the test automation grow louder.

• The cost of doing nothing

  • Consciously, or unconsciously, the existing suite of tests is not updated to cover the increased scope.

  • The cost now manifests in a drop in quality assurance coverage, and specifically a drop in the area most likely to have problems (i.e. the new and the changed).

The Brittle Test Problem

“nothing has changed ... the test was working yesterday.... ...but there is no real problem ...it’s the test that is wrong”.

These phrases are all too common in the world of test automation. A test has failed, but the source of the failure is the test itself.

The root cause is a brittle test; one that does not respond to normal changes in the operating environment. If the functional requirement and the system hasn’t changed and a test starts failing then that’s a brittle test.

Testers with considerable experience can mitigate the risk through clever test design, but even the best tester will routinely face cases of tests that just “stop working”.

In a battle between a static test and a dynamic environment… the test will always lose.

Reduce the burden of test maintenance by harnessing the power of dynamic testing. Get in touch to learn more about how Triangle overcomes these problems.