Probably the single biggest challenge in setting performance objectives is making them measurable.
This is important because …
Human beings adjust behavior based on the metrics they’re held against. Anything you measure will impel a person to optimize his score on that metric. What you measure is what you’ll get
Dan Ariely article “You are what you measure” in Harvard Business Review
And if we get it wrong, it can be dangerous and lead to the measure having an ineffective, or damaging, impact …
It[‘]s really easy to decide to measure something … and screw up a team beyond belief. For example, if I measure how productive individual programmers are, then it[‘]s to the advantage of individuals to focus on their own work and spend less (or no!) time helping others. Completely kills teamwork
Brian Button (Agile programmer and blogger) in “‘You get what you measure’ versus ‘what you measure you can manage'” – article no longer accessible)
So it’s worth getting it right … but it’s not so simple …
The most important things cannot be measured
This is even more true as most things sit within complex systems and have an impact over the long-term and so can’t be easily isolated or measured within a single twelve-month performance appraisal period.
What makes a good measure?
We tend to think of measures as being numeric, handy little objective numbers we can neatly slot into spreadsheets to form colourful graphics. I have worked in many places where data in colourful graphics (especially if simplified into traffic lights) is officially More Valid than any other form of data*.
I read quite a few websites when preparing this post, and most would agree.
All mentioned the importance of “quantifying”, asking “how much” and “how many” as the key aspects of measureability†.
I don’t agree.
I think they’re arguing that the most important thing about a measure is the measure itself, not the performance it tracks.
The numberyness of a measure is not the important factor.
I’d go further. The pursuit of a numerical measure means we often measure the wrong thing, just because it’s countable.
If a measure can be an objective number then that’s great, but much more important than that are these five factors:
- The individual can directly influence the measure through their actions
- Those actions have a positive impact on the organization
- Those actions are the key actions that really make the difference
- The act of measurement is efficient
- The measure has an appropriate balance of quality and quantity
How to measure the right thing
Objectives need to state two things:
- A specific deliverable (it states “what” needs to be done)
- The reason to bother doing it, i.e. what is the desired outcome (it states “why” it needs to be done)
The best thing to measure is not the first bullet above, but the second bullet!
For example, if we want to produce a quarterly report for the board that summarizes the previous quarter’s performance and makes predictions for the next quarter, how could we measure this?
We’d need to understand why it is being produced.
We could just measure that the data is collected and the work done (the input), or that the report is produced and distributed on time, but all we’d be doing is checking that the output we requested was completed.
It’s a numerical measure: the number of reports produced per quarter, one or zero.
But we have to move away from that.
We need to build in quality and try to understand if the actions taken (the input) led to the right end product (the output) which in turn had the desired impact (the outcome).
The outcome is the most important factor – we are producing this report for a reason – what is that reason?
It might be something like this:
So that the board feel that they have a good overview of the previous quarter, and have a solid prediction for the next, allowing them to make better decisions within their own areas.
That’s what we need to measure!
Do the board feel that they informed about previous performance, and have solid predictions for the near future?
This is the measure that counts!
It’s subjective, non-numbery, and beautifully ambiguous, but it’s really the only thing that matters!
People might not like this. It’s not a good measure because it’s subjective and woolly, but that’s not a fault of the measure. The measure reflects the objective. The objective is about producing a specific deliverable to meet a subjective woolly need.
But that’s fine, because in reality, the board will make their own subjective woolly judgements about the quality and usefulness of the report whether or not you capture that, and in reality, that subjective woolly judgement is what is going to count.
And if it’s what counts, you should count it – even if it’s subjective!
But sometimes you can’t.
Sometimes your desired outcome is just impossible to capture – or at least impossible to capture in an efficient and effective way. If you have to conduct in-depth market research to understand the impact of your deliverable, then this may not be viable.
So, unable to effectively measure the outcome, we go back a step and measure the output.
For example, if your objective is to run an event informing your suppliers of new arrangements for the order/payment/delivery process. The desired outcome may be to maintain relationships and remain on good terms so they continue to work with good will, and continue to provide quality products and services at good rates.
This is the real reason you’re doing a lovely event rather than just firing round an email.
The measurement of this outcome would be too great an overhead, far costlier than the event itself (and there’s probably no benchmark anyway), so let’s measure the output: do the suppliers attend the event and leave with the information about the new arrangements?
In this measure we are relying on a professional judgement that this output will lead to the desired outcome, and so it’s less good, but it’s still a fairly decent measure.
If we couldn’t even measure this, then we would measure the input.
This is least good, we’re now making even more assumptions.
We’re measuring that the event took place, not that anyone attended or that they left feeling informed, never mind the wider impact on good will and long-term relationships‡. We might measure that the venue was booked, that catering contracts were signed, that invitations were sent … we’re measuring essential actions (inputs), so it’s a third choice measure, but better than seeking irrelevant numerical measures just because they add up nicely.
So, in summary:
- Understand why an objective has been written: what is the desired outcome?
- If you can, measure the outcome, even if it’s subjective
- If you can’t, try to measure the output.
- If you can’t, as last resort, measure the input.
Measures are not everything
A final note.
A performance measure is not the same thing as a performance. It is a map of the terrain, not the terrain itself.
Understanding how people are performing in the workplace is hard work and yields ambiguous subjective results. This is why so many of us are tempted to make everything numerical, because that’s easier and harder to argue about – but measures are only the starting place to understanding performance, not the conclusion.
Brian Button again:
Metrics should never be used to judge a team or used to form conclusions about what is happening. … They should always serve as the starting point for a conversation, the beginnings of understanding an issue.
Useful reference
This article is useful for action words relating to Bloom’s Taxonomy – I think these are a great tool for writing objectives and thinking about measures.
FootNotes
* Yes, I said “data is” and not “data are“. This is because this post is written in English not Latin, and in English data is widely used as an uncountable noun (like information), not as the plural to datum. English is a language driven by usage, not a top-down Academy making up the rules (dictionaries are descriptive, not prescriptive), and so its use as an uncountable noun rather than a countable plural wins out
† Tim McMahon’s post (10 Characteristics of a Good Measure and 7 Pitfalls to Avoid) is good, I don’t agree with everything, but it’s a well-rounded list of things to bear in mind when considering how to measure an objective.
‡ This might sound a bit ridiculous, but how do you see objectives like “Run 4 sessions …” or “Deliver an event on …” – it happens people, it happens
John Tomlinson 