You’ve been getting complaints that the floors in the procurement office are too sticky. As the administrator of the MRO contract that includes services for your building, you bring up the issue at your next meeting with the supplier. The supplier representative assures you that the floors aren’t sticky. Not only that, but she provides you with a list of data points showing how many times the floors have been mopped, how many total hours the staff have spent mopping, and how many bottles of cleaner they’ve used. You run through the building to show the data to your coworkers, proving to them that the floor isn’t sticky. When you’re done, you muscle your way back to your office, leaving a trail of your shoes and then your socks as they cling to the floor behind you. What’s wrong with this picture?
Aside from the questionable floor cleaner, you might have noticed that—despite the productivity of the cleaning staff—the underlying conditions just aren’t budging. In procurement, we need to keep track of the amount of work we’re doing and how efficiently we’re doing it. But we should also be assessing whether all that good work is moving the needle in the right direction.
Outputs vs. Outcomes
Many key performance indicators (KPIs) focus on outputs, tracking the amount of work performed by an office and the efficiency of related processes. Going beyond pure outputs, KPIs that focus more on outcomes help demonstrate how the work of an organization has contributed to the desired change in conditions. Outcome-level KPIs support several significant areas in the world of public procurement:
- Accountability: By demonstrating the deeper results of public investments, organizations can become more accountable to the priorities of citizens.
- Transferability: Other offices will be better able to justify the adoption of similar initiatives when they can see evidence of their ultimate effects.
- Mission Focus: Keeping the emphasis on higher-level goals can help offices stay true to their purpose and avoid common metric traps, like the use of vanity metrics to flaunt artificial progress.
Though it’s common to see output-level KPIs, many offices excel at considering outcomes when reporting on progress. The 2022 Cronin Award submissions included several examples of the importance of outcomes in state procurement. Take a look at the two summaries below, and check out the Cronin Awards page to read more about recent innovations in the states.
Examples from the 2022 Cronin Awards
Georgia: Effecting Change Through Procurement Officer Boot Camp
What can outcomes look like for a procurement office? The State of Georgia demonstrated outcome-level thinking as part of the Procurement Officer Boot Camp, a project that earned them recognition as a finalist in the 2022 George Cronin Awards. This accelerated training program provided 40 participants with 40.5 hours of training each, calculated to be $27,945 worth of training in 2022. Great outputs! How about what they wanted to achieve through the training? As it happens, the State has already seen several indicators of impact:
- Turnover rates among Boot Camp participants are significantly lower (6% in FY21 and 0% in FY22) than they are for non-participants in similar roles (around 20% in both FY21 and FY22). The reduction in turnover has saved the state between $800,000 and $1.2 million in FY22. Employee retention is often an important marker for procurement offices and contributes to several other higher-level outcomes down the road.
- As a result of their improved understanding of procurement requirements, two Boot Camp participants proactively identified compliance issues to the central procurement office. Overall, the State estimates that the Boot Camp saved $1.4 million in FY22 due to increased compliance and efficiency. The increased compliance likewise contributes to higher levels of transparency and trust.
Tennessee: Collecting and Disseminating Data Through DIT-II
Once you’ve identified what you want to track, what’s a good way to give others access to the information? The State of Tennessee’s Governor’s Office of Diversity Business Enterprise recently faced the need to do just that. Their updated public dashboard, Diversity Intelligence in Tennessee – II (DIT-II), compiles information from multiple sources to capture the State’s overall diversity spend by category and county. Apart from reducing reporting time, the dashboard supports the Governor’s priority of addressing accelerated transformation of rural, distressed, and at-risk counties in the State. This earned Tennessee the Bronze award for the 2022 Cronin Awards.
The State of Tennessee’s DIT-II public dashboard provides detailed information on diversity spend.
Both Georgia and Tennessee recognized the value of identifying outcomes and sharing the collected data. If you find that your office’s KPIs tend to stick to the output level, see below for a few simple ways to strengthen your assessment plans.
Getting Started with Outcomes
Tracking outcomes can be difficult. It’ll take time to establish baselines and find the right external resources to support your data collection efforts. However, even small steps taken now can make a big difference down the road in your organization’s ability to demonstrate impact. Keep these ideas in mind for your next cycle of KPI development:
- Focus on the long-term goals of the office or the state. Staff time is limited, and not all types of procurement activities need corresponding outcome metrics. The most far-reaching, high-investment initiatives are the ones most important to tackle.
- Look at your existing output-level KPIs. Ask yourself, “What is it we’re trying to achieve by reaching these output goals?” And then, “How can we tell whether we’ve achieved it?”
- Identify potential data partners. Even if an outcome seems difficult to measure, chances are that you’re not the first organization in the world to try to operationalize it. Find and bookmark some promising sources of data from inside and outside your organization that could help demonstrate the impact of your work.
- Be prepared to adjust your approaches if appropriate. It’s entirely possible to have impressive outputs and still be unsuccessful in achieving the desired change in conditions. There may be external factors at play, or maybe the floor cleaner just didn’t work the way we thought it would. For the next planning cycle, consider the feasibility of alternate tactics that might better support end users’ needs.
- Continue to celebrate strong outputs. Outputs are an important part of assessing an organization’s accomplishments. Procurement officials work their hearts out to meet productivity and efficiency goals, and these successes can’t be recognized enough.
Digging Deeper into KPIs
- NASPO’s Procurement Reform Guide provides examples of recent state efforts with outcome-level considerations, such as sustainable purchasing, risk management, recruitment and retention, and transparency.
- Procurement U’s skill-focused courses offer insights into data-driven decision-making at all levels of an organization. Introduction to Data Analytics and Data Analysis for Improving Organizational Performance can both assist procurement offices in collecting and using data.
- The USDA Forest Service’s information on program development walks you through the basics of logic models, which are a helpful framework for understanding outcomes in the context of your work.