Simplicity is the Key

Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it's worth it in the end because once you get there, you can move mountains. – Steve Jobs

Oftentimes build and packaging systems tend to be a hodge-podge of scripts mangled together to spit out a version of a software that ends up getting distributed to customers. This increases friction in introducing changes while also making it hard to understand what is going on.

So it's important to consider releasing software as its own discipline and make conscious effort to strive for simplicity in its design. A clever approach may feel like an accomplishment, but keep asking why until you understand the outcomes and design simple pointed solutions (see first principles).

The same arguments that apply in software engineering apply here (see YAGNI). Release Engineering is an application of software engineering principles in a domain that aims to serve software out to customers. A poorly designed Release Engineering system contributes to delivering a poor quality software too.

Release Engineering is a craft and must be treated so. While it's orthogonal to the software it releases, it is influenced by it. Systems thinking allows one to take all such variables and forcing functions into account while solving problems in this domain. This is hard work. Designing for simplicity often requires complex hard thinking.

We had been using Concourse for many years before we switched our release system to GitHub Actions for Tanzu Application Service. Our goal of moving over required us to take the path that was hard and the whole project took us over a year.

Before we simplified, here was our state of the world:

• We had multiple repositories: one set had the product metadata while the other housed CI scripts.
• Having multiple repositories led to duplication of information among the these repositories, making it hard for people to understand the purpose of each. At times, a metadata change had to be made to all of these repositories.
• Our overall system already relied on GitHub for issues and pull requests, so using another system to run pipelines meant we had to keep an eye on Concourse as well as GitHub. On top of that, we would also keep track of stories in Pivotal Tracker.
• We kept building tools on top of these various abstractions to solve problems.
• Debugging was a pain. It needed understanding of all of the various repositories, their interactions, and their implementation to understand where the problem could be.

We needed to rethink the system at a fundamental level even if it meant significant work. We ended in a place where we now have a single repository that houses the CI changes and all product metadata for different product variants. This enables our contributors to also submit changes to the product while updating our CI scripts and configuration as needed. While there are still quirks we continue to work on as with any major change, the difference is night and day from where we were.

Simplicity requires hard thinking. It's easier to continue digging the rabbit hole you're in than to ask if it's the right hole to dig in the first place. To make a robust system, find paths to make your Release Engineering system simple to understand, contribute and debug.