Analysts Who Ask Why? Business Targets and Goodharts' Law

August 20th 2020

Analysts Who Ask Why? Business Targets and Goodharts' Law

Roughly a 4 minute read by Nikki

Being an "analyst" is a broad term.

For many, the term probably conjures up images of someone looking at spreadsheets and databases to decipher data and extrapolate information. Whilst this is true to an extent, knowing the reasons behind why analysis results are what they are is another level of analytical thinking which is often overlooked. An effective analyst asks the question"why?" and understands what business processes have potentially led to a certain result. Within many of the articles I plan to write for The IG Group I will not only be aiming at providing information regarding the use of data analytics but I will also be asking what other factors we should consider as analysts.

During my career as an analyst, I have often been asked to monitor and track targets. Targets are undoubtedly important to any business, without targets there is no direction, without direction there is no business. Often the full effect of target implementation is not always possible via just the analysis of data. We have to sometimes analyse more than data, we have to analyse the impact of business practices and operational process to understand the cause and effect consequences of target implementation.

It is key for a business to monitor measures which are the basis of its success. Should every measure be targeted? Some suggest not and this where I introduce the concept of "Goodhart's Law". The always informative sketchplanations summarise it as below.

miro.medium.com/max/2560/1*8wje69e28yQt1EeGVb5K...

(SOURCE - Sketchplanations)

In a nutshell, Goodhart's law says that when you use measurements as targets, people will tend to optimise their processes to meet said targets regardless of the consequences of their optimisation of process. This can often lead to detrimental issues to the business such as:

  • Reduced quality of output.
  • Reduced levels of customer satisfaction.
  • Unethical treatment of fellow employees.

We can relate this to many areas, not just business. Think back to being at school and memorising a wealth of information to achieve a grade. How quickly did a lot of us forget all that information when the exams and coursework were finished?

As posited at the beginning of this blog, targets are a necessity for a business otherwise there would be no direction for the business. How can companies use measured targets whilst mitigating the unintended consequences of Goodhart's law?

1 - We should analyse the target measures themselves before implementing them.

Using a combination of data and understanding of the business you work in is crucial. Analysts can do more than just look at data. Looking at the data you can notice trends and patterns relating to employee and consumer behaviour as well as whether a targeted measure is actually conducive to overall business success. With this knowledge, an analyst can understand and advise on which measures are the most effective to track and target.

2 - Don't underestimate the overall human impact of targeted measures.

Employees are more inclined to take shortcuts and bend the rules if they have been given targets which rely very heavily on financial gain or even just keeping their job, this isn't to even mention the stress an employee can encounter when given unrealistic targets. Set targets but make them achievable and ensure rewards are given that are well proportioned. This is very much a trial and error approach, however, an effective analyst can create systems of reports which track and monitor targeted measures in a way which can spot potential issues before they turn into much bigger issues down the line.

3 - Don't rely on one measure.

Combine measures to make an overarching view of target performance. Too often companies rely on profit as a measure. Whilst obviously very important in the short term, is concentrating on just profit going to help establish a loyal customer base? What if we combine profit with positive reviews or positive satisfaction scores? Link these metrics to both new and returning customers and you only begin to scratch the surface of a combined set of metrics which benefit the business overall in a more meaningful way.

These are just a few key notions - I am sure many business leaders and analysts will read this and have many more ideas about how a business can mitigate the unintended consequences of targeting measures. Needless to say, we should always keep Goodhart's Law in mind. There is always the possibility that a poorly thought through target can have serious consequences on business success. Analysts are here to help businesses not just through data literacy and reports but also through understanding and knowledge.

No Image Available
NIKKI QUINN
Do you know someone in your network who might be interested in this information?
Back to the blog home