Rizwan Reza

Tanzu Application Service

Tanzu Application Service (TAS) is VMware's Cloud Foundry implementation that enables developers to deploy their applications without worrying about installing dependencies or maintaining them. The developers code their applications and let the platform handle the rest. TAS provides companies an escape from large IT infrastructure teams and software and enables

Bump Dependencies Automatically

As a Release Engineer, one of the most time consuming tasks is to make sure that updated dependencies are shipped to customers. It's a crucial part of the release process as it helps customers get access to recent and up-to-date security patches. Important bug fixes and improvements are also shipped

Test Incoming Changes

A Release Train is a set of multiple changes slated for release in the next version. Testing and working with multiple changes in a single release train means that it is hard to debug and fix the problems that occur because system tests are hard at providing a trace to

Incorporate Supply Chain Management

Supply Chain Management encompasses the flow of physical goods, activities, processes and systems with the goal of delivering finished products to consumers. It is generally employed as a practice in large scale organizations to support and provide assembled products to customers in an efficient and reliable manner. As with most

Maintain Decision Records

A complex release engineering system is built from decisions that have consequential effects in and out of the system. Most of these decisions feel logical while others feel counterintuitive. It's foolish to rely on human memory as a way to make sure these decisions are transmitted across the team, notwithstanding

Do Post-Mortems

Agile teams typically conduct a retrospective at the end of a sprint or projects to determine what went well and what didn't to learn and work on improvements where needed. Similarly, each release train should be proceeded by a post-mortem. A release train retrospective meeting could include: * Metadata on the

Catalog Changes

Making changes to a product in a single repository is relatively simple. You've a series of commits that precede a deployment. While not trivial, it is easy to figure out what changes and upon error, easy to revert and make changes to fix the problem. If the team follows a

Write Playbooks

A playbook captures all of the details that are needed to ship a product out to customers. Instead of relying on word of mouth for context sharing, playbooks allow team members to ship products by following documentation step-by-step. When you begin implementing a process to regularly release your product, you'll

Single Source of Truth

A typical release engineering system uses different data as inputs to invoke necessary workflows. End-of-support dates can, as an example, determine whether a pipeline for a particular product version is triggered and whether it is published publicly to all customers. Such type of information typically includes: * Release dates for Beta,

Leave a Paper Trail

Each release train should have a designated conductor, one who keeps track of the progress in the system. The conductor's responsibility is to keep track and make notes as the progress is made. She should be noting down any discrepancies, creating incidents, noting problems in the workflow, and leading the

Simplicity is the Key

Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it's worth it in the end because once you get there, you can move mountains. – Steve Jobs Oftentimes build and packaging systems tend to be a hodge-podge of scripts

Implement Auto-Retries

We found that the majority of our incidents occurred due to test flakes. Test flakiness happens in most large software systems and complex integration test suites. For the longest time, we believed that it was important to feel the pain and resolve each flakiness by fixing the underlying issue. The

No Process Changes During a Release Train

Engineers are creative folk. It's tempting to see a problem in front and not find and implement the solution right away. But when it comes to changing an aspect of the release trains, don't change the process while a Release Train is going. Release Engineers should be substitutable—an engineer

Improve with Automation by Repetition

Any Release Engineering system is a complicated workflow system made up of lots of scripts, typically in a CI/CD tool. It can be confusing to determine what to improve at any point. To begin automating the system, the easiest low hanging fruit of a task can be a good

Every Stoppage is an Incident

"What's not measured is not improved" – Peter Drucker Any stoppage in a release train is a hurdle in delivering the product to the customer. It's a learning opportunity for the team to understand the problem and improve the system to be more resilient in the future. By cataloging the incident

Debug to Revert; Not to Fix

At any large company, a Release Engineering team receives a large amount of changes to incorporate into the product, test, and ship to customers. A release train therefore typically contains various changes for release. As it often happens, one of those changes fails and stops the release train. A typical

The Release Train

A Release Train is a release engineering pattern that helps define how a single release process can be instantiated and followed through. It takes the changes in the product through different stages including building, testing, compiling, and finally publishing.

Continuous Delivery and Release Engineering

Continuous Delivery: Reducing the cost, time, and risk of delivering incremental changes to users. Continuous Delivery develops a team's capability to have the code in an always deployable state. It mandates engineers to merge their code daily, never breaking the mainline, and delivering features incrementally. This is typically done by

Release Management and Release Engineering

While Release Engineering concerns with designing and automating the process of releasing software, Release Management deals with what makes up the release itself. Release Management assesses the importance of features and bug fixes going in from a business outcomes perspective. While Project Managers scope out the features to be designed

Release Engineering is not QA

A QA role in a typical tech company today comprises of finding and fixing bugs, typically by writing a set of test suites aimed at finding defects and mistakes in the system. With each release, they may aim at a specific subsystem or feature set and focus on finding defects.

Site Reliability Engineering and Release Engineering

Site Reliability Engineering focuses on running services reliably. It emerged from Google and has garnered attention rightfully as a fundamental discipline in operating and designing systems to run complex software machinery reliably. It provides tactics and embraces software engineering philosophy in a world traditionally looked upon as system administration. It’

Industrializing Release Process

Release Engineering is needed in circumstances where you cannot point the product as a whole to a single owning team anymore. This is a scaling problem, as noted before. Release Engineering focuses on the process so the product isn’t impacted by the lack of ownership on building, compiling, testing,

What problem does it solve?

Before we begin discussing the mechanics and components of Release Engineering, let’s set the context of its relevance and need. Release Engineering concerns itself with software that’s digitally or physically shipped to the customers. Given the dominance of cloud technologies, we’ll solely focus on the former—a

What is Release Engineering?

Release Engineering, sometimes shortened as RelEng, focuses on how software gets packaged, shipped, and distributed to the end-users. This discipline is more prevalent where complex software is shipped as a standalone product. In contrast to this, disciplines and concepts like Site Reliability Engineering [https://sre.google/sre-book/introduction/] and Continuous