MMPC

Case Study

Automated Testing for InterSystems Ensemble Productions

---

Overview

MMPC embarked upon an Epic EMR conversion and implementation project requiring 3 years, $40 million dollars, and 40+ people. In 2008, the head of this ream engaged Atomic Object to provide project management services and improve the quality and speed of Ensemble development projects.

The size and critical nature of the project demanded transparency and measurability not often present in traditional software project management techniques. The team lead desired to improve the process used within his team and sought AO’s experience with agile practices.

The critical nature of the conversion and permanent interface productions demanded rigorous, ongoing testing. MMPC saw an opportunity to pioneer automated testing with AO’s recognized software testing expertise, decreasing the cost of testing and improving project predictability.

Test-Driven Development

The Atomic Ensemble testing framework supports test-driven development (TDD) for Ensemble productions. TDD is a four-step process:

1. A test that specifies the correct behavior of the component is written.
2. The test is run and fails, showing that it does indeed test the component.
3. The production component is then programmed until the test passes.
4. When all tests necessary have been written, and are passing, the production can be refactored to be simpler, smaller, or more efficient, as required.

Because the tests are fully automated, the cost of running them is essentially zero — just push a button and see the results. Being able to run all tests for a production automatically means regression testing is very easy and inexpensive. By lowering the cost of regression testing, maintenance and modifications of existing interfaces become much more certain and safe. The automated test suite built around each production acts like a watchdog introduce a defect and the test suite barks to tell you about it. Automated test suites make Ensemble upgrades much less stressful. If a production behaves differently when run with a new version of Ensemble, the test suite for that production warns you in advance.

The Atomic Ensemble testing framework lets you run a single test at a time, or a suite of tests. Test results show up on the Studio console. For an entire suite, a test report can be viewed as a web page, with the results for each individual test. Running each production’s test suite when a change is made to the configuration, logic, or behavior of the production, or on a regular basis (say nightly), is the role of a continuous integration server. Continuous integration (CI) is achieved in the Atomic Ensemble test framework with a persistent production that runs all tests periodically. In the event the CI server finds an error in one of the production’s test suites, email is sent to notify the development team.

Results

Unit testing for production DTL logic provides confidence that transformations perform as expected for all possible cases of input. Configuration testing actively documents proper configuration of productions. Integration testing the behavior of routing rules proves that messages are routed through the proper DTL and outbound adapter.

Weʼve seen that developing Ensemble productions in a TDD fashion causes DTL code and routing rules to be simpler and smaller. Smaller modules are easier to test, and hence easier to understand and maintain. Complex logic moves to more easily testable and reusable functions.

Our experience at Michigan Medical shows traditionally trained staff can quickly and easily learn the Atomic Ensemble testing framework, can write tests for existing productions, and can develop new productions on their own in a test-driven fashion.

Customer Feedback

“Atomic Object’s team delivered on a challenging project by applying superior project management skills and applying highly developed testing skills to new problem domains. The AO team brings the goods and they consistently deliver very satisfying results.”

Matt Beaverson, MMPC Epic Implementation Lead