Change Failure Rate
The percentage of deployments that result in failure, used to measure the reliability of an organization's change and release management practices.
ANALYSIS
The formula's numerator—the number of "failed" changes—lends itself to vagueness. A failure may range from a system outage to any negative impact on business users. With this kind of vagueness comes analysis, and inevitably debate, over what was or was not a failure, and why.
Definition & formula
(failed changes / total changes) * 100
The percentage of deployments that result in failure, used to measure the reliability of an organization's change and release management practices.
The Socratic view
The formula's numerator—the number of "failed" changes—lends itself to vagueness. A failure may range from system outage to any negative impact on business users. With this kind of vagueness comes analysis, and inevitably debate, over what was or was not a failure, and why.
Background
The Change Failure Rate is a software engineering metric that originated from the DORA Metrics developed at Google. It is used to measure the reliability of an organization's change and release management practices. The metric gained prominence as part of the DevOps movement, which emphasizes collaboration, automation, and continuous delivery. It started being used in the early 2010s as organizations recognized the need to quantify the effectiveness of their software engineering processes.
Use cases
The Change Failure Rate is used by software engineering teams and organizations to assess the quality of their change and release management practices. It helps identify areas of improvement and evaluate the impact of process changes over time. By tracking the percentage of deployments that result in failure, teams can prioritize enhancements to their deployment processes, testing strategies, and overall DevOps capabilities. This metric is particularly valuable for organizations that strive for rapid and effective feature releases.
Evaluating the metric
A good value for the Change Failure Rate depends on the historic performance of the organization. Generally, a value below 10% is seen as a good rate. A low change failure rate indicates effective testing processes, proper identification of errors before deployment, and a well-functioning DevOps team. On the other hand, a high change failure rate suggests inefficient or manual deployment processes, lack of testing before deployment, or an inefficient DevOps team.
The metric provides insights into the effectiveness of the software engineering team and the organization's ability to release features quickly and reliably. It highlights areas that require improvement, such as automation, testing, and collaboration. By consistently monitoring and evaluating the Change Failure Rate, organizations can strive for continuous improvement in their change and release management practices.