Why do we collect so many agile metrics?
Your agile metrics tell a story. If you are on a scrum team working independently from other teams, then the team’s metrics should be very straightforward quality(velocity, burndown, and burnup). However, if you are working in a Scale Agile Framework®, then you are practicing both Agile and Lean. Lean metrics provide a window to understand your current state and where to improve, for example, quality, lead time, and cycle time.
Metrics are not about assigning blame – they should lead to understanding. Aligned with goals, metrics guide future decisions. In a scaled agile environment, adding context to data-driven facts (like velocity) reveal the story of a team over time.
Getting Started
First, ensure that you have a baseline so you can see changes over time. To collect actionable metrics, review your metrics to determine if they are:
Simple: If the metrics are too complicated, simplify them, and then determine whether you need more. Thoughtfully add one or two at a time until you have arrived at the right level of information.
Accurate: If the metrics seem inconsistent, go back to the source to determine why. There could be a variety of issues from the tracking system not loading data properly to teams needing coaching on transparency. The key is to understand what went wrong and then determine the appropriate solution.
Complete: Are you collecting enough data of the right type? Consult your assessment tools to remind you of the dimensions. For example, what data points are you capturing to improve built-in quality?
Actionable: Data is only valuable if it is actionable towards improvement. If your data is not leading you to clear improvement actions it may be time to try different metrics.
Consistent: Collecting consistent metrics is easier as long as all teams are using the same data values in the same way. Apply data tracking standards that help the teams understand how your organization defines a data point. One common example is the story point; some Agile organizations treat one story point as equivalent to one day while others treat one story point as a measure of relative size. What standard does your organization apply?
Second, ask an experienced Release Train Engineer (RTE) or SAFe Program Consultant (SPC) to review your metrics. They are trained to see patterns (areas to improve) and missing information.
After two to three Program Increments you should have enough data to conduct an analysis and identify the trends that begin to appear over time. Use this information to discuss with team members and stakeholders what improvement actions might be needed. Once you have a list of actions it is time to plan them into the iteration, assigning each to an owner who will be expected to complete it.
The value of this exercise is in our reaction to the above process, e.g., continuous improvement through collaborative analysis and discussion. Metrics are meant to lead to action in your organization. Action leads to improvement. Improvement leads to growth.