Bad decisions in life arise from having optimized for the wrong metric

Regretful choices arise from using a widely-accepted metric for success instead of a personally defined one.

Published: 2017-07-17 by Luca Dellanna

#wisdom

Something I have noticed:

Unfulfillment arises from using metrics chosen by others as proxies for personal success (e.g. a new car as a proxy for happiness).
Sustainability problems arise from using short-term success as a proxy for long-term one (e.g. boosting sales this quarter by delivering false promises to customers).
Failed relationships arise from choosing a person using widely-desirable traits as a proxy for personally-desirable ones.

In general,

Principle

Regretful choices arise from using a widely-accepted metric for success instead of a personally defined one.

Proxy optimization

I once wrote:

“Humans are extremely good at succeeding at their priorities, and extremely dishonest about them”.

With that, I meant that we are great at acting in such a way to succeed at the metric we chose, but often choose the wrong one or do not realize we subconsciously chose another one. I call this phenomenon commitment to failure.

Every time we fail at something we had the resources to succeed at, it is due to the fact that we considered an internal success to fail at it (because we were uncomfortable with the consequences of succeeding at it). In other words, we acted optimizing an internal metric (Expected Emotional Outcome) instead of an external one (success at whatever activity we were doing).¹

Even when we choose a proxy that is initially correlated with the ultimate outcome we desire, we often fail at misunderstanding a key concept:²

Principle

Optimizing for a proxy reduces the correlation with the ultimate metric.

As an example, imagine an author in the business of selling books online. He discovers that about 2% of his Twitter followers bought his book. He decides to run an advertisement to get more followers (the number of followers becomes a proxy for book sales). Such ad selects followers based on their sensitivity to advertisements, not on their propensity to buy. In other words, it tends to select people who are interested in following the author on Twitter, regardless of whether they are interested in buying his books. As a result, if before the advertisement 2% of followers would end up as customers, after the ad it it is likely on 1% will. The proxy optimization (running an ad optimized to get new followers) diluted the correlation between the proxy and the ultimate metric (the % of followers buying the book).

Alexander Wolfe noted that “What people who don’t risk don’t understand is that it is better to risk and fail at something that resonates deep within than to do nothing”. To which I replied: “That’s the difference between external failure (proxy) and internal success (actual metric to consider).” ↩
This is not (only) a repetition of Goodhart’s law. Goodhart focused on people gaming the system once a regulator introduced a policy. My heuristic is more general and works in single-player scenarios too. It is because the easiest way to improve an unweighted metric is often to include items whose weight towards the ultimate goal is lower than the past (e.g.: in the example in the text, the easiest way to improve the number of followers is to attract those who won’t buy anything). ↩
As far as I know, Marshall Goldsmith made the expression famous, in his homonymous book. ↩
Living a diverse life means acknowledging that because of the problems of addiction and tolerance, pursuing a single proxy will yield diminishing returns over time and lead to dissatisfaction. ↩
Did you notice any similarity between this list and the contents of Nassim Nicholas Taleb’s Incerto? That’s because the Incerto is, too, about how to spot and get rid of proxies. Fooled By Randomness is about spurious correlations between proxies and real metrics; The Black Swan is about frequencies used as proxies for impact; Antifragile is about averages used as a proxy for distributions; Skin In The Game is about incentives (and proxies are incentives). ↩

Bad decisions in life arise from having optimized for the wrong metric

Regretful choices arise from using a widely-accepted metric for success instead of a personally defined one.

Principle

Proxy optimization

Principle

Define success without using proxies

Eliminating proxies

Notes

Principle

Proxy optimization

Principle

Define success without using proxies

Eliminating proxies

Notes

Footnotes