Test Automation for Trading & Risk Management
AUTOMATION TOOLS
Energy & Financial Trading & Risk Management
Endur / Findur
Openlink's Endur/Findur systems are deservedly market leaders. Developed over more than 20 years, they are powerful and endlessly customisable products. However, the sheer flexibility and variety of configuration options presents difficulties for developing meaningful automated tests. Tests developed at the GUI level may be fragile and will likely fail to adequately test a client's specific implementation.
We have developed a test automation framework called Triangle that allows for the creation of meaningful automated tests, specified in business language and targeting the key functionality of the application (below the GUI level). The Triangle Test Automation® framework enables the rapid creation of useful automated tests, without the usual large, upfront investment.
Triangle provides a mechanism for writing executable acceptance and regression tests for Openlink's Endur/Findur. Trinitatum has created an extensive set of more than 1000 building blocks that enable the rapid development of comprehensive tests; from deal entry to complex financial reporting.
Armed with this capability a skilled tester can assemble a body of tests that form a long-term testing legacy; a set of tests that run every day to provide on-going quality assurance and an invaluable corpus of living documentation.
Too many consultancies "cut and run": the project is forced over the line and then the client is left to deal with the consequences.
At Trinitatum, we believe in designing for the long-term. That means tests that won't stop working three months after go live. It means comprehensive documentation; auto-generated from source so it remains up to date.
Most of all it means the investment in test automation continues to pay back; for the lifetime of the system.
Not content with automated execution of tests, Triangle enables the automatic generation of tests: tests that adapt dynamically to the system they are testing.
Our “Test Accelerators” provide templates that embody testing best practice which Triangle can utilise to rapidly generate extensive test suites, tailored to your configuration. No more lengthy, up-front investment… with the hope of rewards. Dynamic Testing and Test Accelerators produce a quick result to the present needs as well as the ability to adapt to future challenges.
“Regression testing for Endur releases has become ‘silent’ it’s just not an issue any more. Remember where we were a year ago?”
“Retrospective: Triangle automation, most voted success factor in a Findur upgrade project.”
Triangle Test Automation® is the result of more than a decade of designing, building and using test automation tools for Trading & Risk Management systems.
Triangle is designed to directly address the two biggest reasons why test automation fails:
Large, speculative, up-front development in test tools … with uncertain returns
Trinitatum has made the investment in test tools and has been operating them in complex trading systems for more than 5 years.
Ever increasing costs of maintaining tests over time
A tester can easily spend more than 50% of their time manually updating existing tests to ensure they remain valid.
It is equally common for old tests to be simply “turned off” because the cost of maintenance is too high, thereby signalling that the original investment is now returning zero benefit.
With Dynamic Testing, Triangle offers the ability to generate and regenerate tests automatically that respond to your evolving business.
The foundation of Triangle is a set of core capabilities that can be used to test your Trading & Risk Management systems. But such systems do not exist in isolation and Triangle also provides the capabilities to test the interaction with external systems.
Run arbitrary comparisons of databases.
Compare and manipulate common file types (Excel, CSV, XML, JSON, etc).
Produce easy to read reports of differences and test failures (HTML and Excel).
Use industry standard APIs (Web service, Rabbit MQ, etc).
Test execution timing.
Test automation of Endur/Findur presents a significant challenge due to the sheer range of its capabilities. Meaningful test automation is only possible after significant up-front investment in test capability.
With Triangle (Endur/Findur), Trinitatum has made the investment and after more than seven years of development, we have produced over 1000 testing building blocks designed specifically for testing complex Endur/Findur systems.
Automatically enter trades from more than 60 toolsets and perform the full range of trade life-cycle operations; validate, cancel, roll-over, buyout, closeout, etc.
Automatically generate, move and compare documents in the Settlement Desktop.
Run, Check and Compare Simulations.
Loading, Changing and Comparing Curves and Historical Prices.
Perform side-by-side comparison between two Endur/Findur environments.
And much more.
Take your testing to the next level with Dynamic Test Generation.
Say goodbye to writing tests by hand.
Expand and contract testing scope as needed.
Dynamically adapt your tests to evolving requirements.
With Triangle (Dynamic) you can take all of the operations of Triangle (Core) and Triangle (Endur/Findur) and dynamically generate new tests.
Leveraging our 10+ years experience of testing ETRM systems we have combined Triangle’s Core and Dynamic Test Generation capabilities with Allegro-specific functionality to provide a comprehensive test automation solution to:
Create and update static data such as contracts and counterparties.
Insert and update prices.
Create and update trades.
Run and check Valuations and Settlement/Accrual Validation.
Create Payable/Receivable Invoices and Cash Payments/Disbursements.
We have easily applied Triangle’s automation capabilities to a multitude of different applications. Any application with a database is a potential candidate.
Rapidly backfill automated regression testing into your application
Add a minimal set of custom steps to drive your specific application
Harness all of the test automation capabilities of Triangle at minimal cost
Use the capabilities of Triangle (Core) and Triangle (Dynamic) to dynamically generate new tests delivering unparalleled automated regression testing capability.
Watch this space for new Triangle platform coverage.
LNG Trading System
Automated Regression Testing using SpecFlow
The client has suffered before. Armies of consultants arrive on site, consume large amounts of paper & money and then leave when the system (just about) goes live. The client is stuck with long term maintenance costs and a system that is expensive to change and hard to understand.
This time they vowed it would be different. They sought a trusted partner to develop a long-term automated regression capability; one that would deliver immediate benefits but would still be driving down the cost of change 5-10 years from now.
Oh, and the project was behind schedule so all this was to be done without distracting core project resources.
The project has already been running for a year when Trinitatum arrive on site.
Within weeks, the Trinitatum lead has performed a full system review (including documentation, source code and ad hoc analysis) and identified an approach and set of test principles that were reviewed and accepted by the programme management.
At the end of month one, the first automated test scenario is running and reviewed by business analysts and domain experts.
In month two, tests are running overnight and a project dashboard is available every morning.
In month three 85% of defects are found by Trinitatum’s regression test team.
In month four the functional tests are leveraged to provide automated performance and concurrency tests.
In months five and six, a full set of P&L, Exposure and Attribution tests are developed from first principles.
Tests continue to run every night. Developers receive targeted defect reports, showing precisely what was broken “yesterday”. Developers become more productive and average time to fix comes down.
In months seven and eight test coverage grows (and is measured objectively) and is extended to include automated security testing. The daily project dashboard proves invaluable during final go-live preparation.
Nine months after arriving on site, and over two years after the project started, a complex LNG trading system is live. In the first four post go-live releases, the regression tests catch “critical” regression issues before they are released.
“The level of automated regression & performance testing from Trinitatum was excellent”
With confidence in the regression test capability, releases are planned more frequently and the functional payload is increased.
The long term support of a complex ETRM is entrusted to a junior Trinitatum tester... all made possible by the support of a comprehensive regression test pack.
Market Risk
Ex post facto testing
Our client had a Market Risk system that had evolved organically, was scarcely documented and made use of old technology. They needed to port that system to an enterprise level architecture due to the rapid expansion of their business, but feared the consequences.
We created a suite of automated tests that also served as the specification of the legacy system. The same test assets were then used to validate the new system resulting in a successful implementation with a regression test capability from day one.
Market Data Repository
A single source for prices & curves
Our client had bought an off-the-shelf market data repository that required extensive customisation to meet their requirements. The project was over-running and a rash of technical issues was hampering progress. The test team were a disparate group of contract testers. Under pressure to deliver, animosity existed between the developers, testers and business analysts which had resulted in a breakdown in communication. There were frequent software releases to address the burgeoning bug list, but no test automation. This meant that >60% of each test cycle was occupied with labour-intensive manual regression testing. This was disheartening the testers and distracting them from doing proper testing.
We encouraged an amnesty and instituted a set of practical measures to right the sinking ship:
By the time the project went live, there were >250K automated tests running nightly to prove the data validation and cleansing routines and the system's complex curve building logic.
These tests are still running 10 years later!
New Electricty Trading Arrangements
Regulatory change testing
The NETA programme was amongst the most complex to be undertaken since energy industry de-regulation. National Grid, as System Operator had over 50 internal and external interfaces, new or re-modelled as a result of these industry changes. The client required a project-wide integration test team to validate these critical systems.
Developed a suite of bespoke test tools and harnesses (including SQL, PL/SQL, VB and Open VMS DCL scripts) that could simulate a Production day including EDT (electronic data transfer) submissions and BOA (bid/offer acceptance) dispatch instructions
Conducted performance testing / benchmarking for the key energy balancing system that was integrated with external data providers
Provided out of hours second-line support for business-critical real-time systems
Openlink®, ION® Endur® and Findur® are trademarks, service marks or registered trademarks of Openlink Financial LLC. Trinitatum is unaffiliated with Openlink.
