Test Incoming Changes

A Release Train is a set of multiple changes slated for release in the next version.

Testing and working with multiple changes in a single release train means that it is hard to debug and fix the problems that occur because system tests are hard at providing a trace to the exact code line for the failure. Unit tests on the other hand are typically adept at exacting the location of the failure, but are less suitable for Release Engineering systems where system integrity and functionality is the top priority.

As a Release Engineer, when you don't have confidence in each independent change-set getting integrated into the release train, the likelihood of failure increases. And it becomes extremely difficult to debug and determine which changeset is the culprit as a result.

You cannot completely eliminate the risk profile of a set of changes but to dramatically lower it, test each change-set independently against a subset or all of the system tests.

There is still a chance the combination of the two change-sets in the train may fail the pipelines, but the decrease in probability is worth the cost of testing these changes independently beforehand.

For TAS, we've built a GitHub Actions based Pull Request testing system that uses the same testing infrastructure as our release engineering system to create a receipt of record that the change-set getting integrated is tested and ready to go. This way, the Release Engineer has confidence that the changes work independently and are functionally working.