Lead With Vision, Not Metrics: How OKR’s Can Be a Dangerous Tool

Data, if used properly, is key to success in any organization. It’s impossible to improve without metrics that help us understand performance and set a baseline. Most organizations start off with good intentions – using data to increase efficiency, drive accountability, and improve key metrics – but quickly devolve into a state of bureaucratic, “grade your own homework” deflection. This results in using data to either distract from the true state of the union or, worse yet, outright lie and cover it up. 

In the 1990s, the NYPD implemented CompStat, a citywide database system that used comparative statistics to more effectively track and reduce crime. It held officers responsible for crime in their areas and was credited for bringing crime rates down by 75%. Yet over the course of the last few decades, it has actually led to officers caring less about fulfilling their duties and more about showing improvement upon metrics that are tracked. Taken to the extreme, there have been accounts of cops reducing the severity of reported crimes just to improve metrics.

Managing by Metrics: Paved with Good Intentions

The tech industry has used OKR’s (Objectives and Key Results) for years to track performance but we’ve all seen how they become less meaningful over time. Over the course of quarters and years, people inevitably use OKR’s as a method to get promoted, rather than a way to track what’s going right and – probably more importantly – what’s going wrong. This means that the key results that are tracked end up becoming the same top-level metrics with slight tweaks to show small improvements to the rest of the organization: things like “our data will scale by 10%” or, “we’ll add five new customers.”

This type of problem doesn’t stand alone in one industry, but happens routinely when we set the wrong metrics for success. For example, schools are routinely judged on the rates of students that graduate. In response, teachers feel pressure to pass students that may not have achieved academic success. Rather than preparing students for the future, this practice endangers the quality of education that students receive.

Setting goals that solely track top-level metrics is a sign of bad leadership. I could come into any company with no outside knowledge and take charge by saying, “our organization needs to sell X% more” or, “become Y% more profitable.” A good captain won’t just prescribe where the ship needs to end up, but provide a vision that can empower the crew to make their own navigational decisions along the way.

Where Does it Go Wrong?

CompStat, along the lines of what we’ve seen in education and tech, started off well. As a centralized tracking database of all arrests, summonses, and other crime activity throughout a region, it had a tremendous ability to uncover major trends. Used properly, it helped leaders fight crime more efficiently by knowing, for example, how crimes are linked together or which areas need more police attention at which time.

However, two elements for any system like this to become abused:

1. Grading your own homework
2. Correlation to promotion and demotions

Unfortunately, these two concepts are at odds with each other. We would all love to think that we’d be objective when reporting our own success, but when performance correlates to personal career growth or achievements, even the morally zealous will, consciously or subconsciously, skew metrics towards those that look better for them.

How to Combat the Dark Side of Metrics

However you slice it, this is an extremely hard problem. It’s difficult to incentivize your organization to create and properly measure objective metrics that can’t be gamed. It’s also tough to ensure that employees have a clear and concise enough understanding of the vision that they’ll make the right decision for the organization at any juncture.

There are a few ingredients that might help organizations to achieve these goals.

1. Set the right vision for the company as a leader. 

An organization’s vision shouldn’t be composed of top level business outcomes. Instead, the vision should be a tool that employees use in their decision making process to understand and intuitively know the types of decisions that they should make. Amazon has been so incredibly successful partially because Jeff Bezos has made it abundantly clear that top level metrics aren’t what Amazon is driving towards. He’s building “the earth’s biggest store” and employees know that they can sacrifice short term revenue to achieve that objective.

2. Create objective metrics around organizational performance that aren’t related to individual performance.

Having a clearly defined vision makes everything easier and will help to ensure that employees track metrics that are actually important. For objectivity, metrics shouldn’t be related to individual promotion to ensure that there’s a clear view of company success which hasn’t been muddied by politics.

3. Drive accountability in individual performance measurement.

Grading your own homework is a good basis to determine performance, but any good system has checks and balances. Individuals need accountability which can be provided through management, peers, and end consumers.

How Does It Work in Practice?

Let’s try to take this process and apply it to the NYPD.

1. NYPD’s new vision using the Jeff Bezos principle.

Instead of setting a top level goal like “reduce crime rate for the 20th consecutive year”, we should think about why police exist and use that to define our vision. 

Finding the vision should be simple enough: “To ensure the safety, health and possessions of citizens while enforcing laws.” 

Ultimately, this means that police are civil servants which are tasked with helping make citizens more safe. Chasing metrics that don’t fit within this vision should not be prioritized.

2. Defining objective performance metrics.

While objectivity is usually hard to achieve, our job becomes a lot easier because we’ve both set a vision and decoupled measurement from promotion. As such, we shouldn’t have to worry about bad data when it comes to reporting crimes. Crime will be our first metric since it gets at the high level of understanding safety, health and possessions of our citizens.

Crime is clearly one measure of the vision since crime is the outward measure of how often laws are violated, but it doesn’t rigorously measure safety and health of citizens. We can add direct measures of this from external data and link it to crime to make sure they are accurately reported. We can also measure whether police have caused harm themselves through that method.

3. Driving accountability for individual performance.

A personal account of performance is a good start, but holding officers accountable also means gathering information from peers, and managers, and external information. 

In order to ensure that police are additive towards safety and the health of citizens, we have to measure the behaviors of officers themselves. Records of bad behavior in the force would need to be kept for police the same way that they are for citizens. 

Finally, we should add methods of extracting data on how police are doing from our citizens such as polls or anonymous tip lines. Gathering feedback from multiple sources and perspectives is crucial for driving accountability in performance measurement.

The Bottom Line

It’s been almost three decades since the introduction of CompStat. While metrics may be improving, relationships between citizens and police are strained more than ever. Of course, there is a greater societal context to take into consideration when discussing the state of policing in America. Nevertheless, by focusing on the wrong metrics, they are no longer working towards goals aligned with ensuring the safety and health of citizens. 

What can we learn from this? While the majority of people are inherently good and want to do best by their organization, it’s rare that leadership can distill their company’s vision to a set of guiding principles. Employees are very good at local optimization towards the success metrics that they’re given and have the tendency to drive towards short term metrics versus long term vision unless it’s made an explicit priority by the organization.

In many organizations, a lack of leadership vision results in misdirecting its people. OKR’s are a dangerous goal setting framework without a clearly defined vision and construct for objectively tracking and grading performance. Needless to say, we should all work to ensure that vision and goal-tracking are treated as first class citizens in our organizations, or find ourselves facing the same challenges as police institutions.