Dead rats and dopamine - a new publication

My new paper, coauthored with Yohan John, Dakota McCoy and Oliver Braganza, is out in Behavioral and Brain Sciences.

"Dead rats, dopamine, performance metrics and peacock tails" is about the universal emergence of proxy failure. When you measure and incentivise performance by a single metric (a proxy), the proxy will always become a worse measure of performance than it was before you added the incentive.

This effect is also known as Goodhart's Law in economics, and by other names in other fields - but the same underlying process drives the effect across multiple domains. Our paper studies it in management, economics, biology, neuroscience and other areas.

Recent concerns about AI alignment are closely related to this phenomenon. The paperclip problem is a good example - if a sufficiently clever AI is given a single goal, to produce as many paperclips as possible, it may eventually destroy all of humanity and take over the whole universe in its efforts to maximise output.

Fortunately, our paper identifies a few constraints that can stop systems from running completely out of control - we might still be safe from our Clippy overlords.

Currently the journal is inviting comments on the article - a number of authors will be invited to write a commentary which will be published in the journal alongside the article. Please do feel free to sign up on the journal website and submit your proposal for a comment if you have something to add to the conversation.

It has been a long and very satisfying process for the four of us to put this paper together, so I hope you enjoy it - I would love to hear your informal thoughts if you don't want to go so far as submitting a formal response to the journal.

If you can't download the paper via the link above please let me know and I can send you a preprint copy.

Comments

Post a Comment

Popular posts from this blog

Is bad news for the Treasury good for the private sector?

An intriguing post from Robert Peston raising the spectre of the Treasury failing to sell all the bonds it wanted to sell yesterday. Is this, as he implies, plain bad news for the government because their funding is getting more expensive? Or is it good news for private lenders - implying that investors are fed up with their "flight to quality" and are now considering private sector investments instead? The answer is dependent on two further questions: is investors' money switching to foreign investments, UK private lending or UK equities? are investors actually running out of money to invest - the infamous savings trap, where people in aggregate try to increase their savings but fail to do so, because the attempt to save reduces total income in the economy and available savings are reduced too? Hopefully we can find out. How could we figure it out? by looking at total UK savings this month - this is not an exact science but we should hopefully be able to see if the tren
Read more

What is the difference between cognitive economics and behavioural finance?

This question was posed in an interesting LinkedIn discussion group (Behavioural Finance: Theory and Practice) by Kerry Pechter. I dashed off a quick answer which ended up being 500 words long. So I thought it might be useful to post on here. Can anyone tell me the difference between cognitive economics and behavioral finance? Cognitive economics is not yet a widely used phrase, though Marco Novarese and I have been using it as a name for a more microfounded version of what's typically called behavioural economics. So I'll answer the question based on how I use the term. The first difference is between "economics" and "finance". Economics is a broader field, including the trading of any kind of goods or services, whereas finance specifically focuses on investment and the value of financial instruments. Indeed I consider economics very broadly to be the study of how resources are allocated (by individuals, and across society). The more fundamental
Read more