Am I the person Dominic Cummings is looking for? A followup to The Times

For those interested in a bit more background to my Times article today, here are some details on how Cummings's topics have showed up in my cognitive economics research. You can judge for yourself if I have spotted what he is working towards.

Starting with Judea Pearl's modelling of causality. Pearl developed a way of using graphs (a kind of diagram showing a network of relationships between objects – like the chocolate example below) to express and work out cause-and-effect relationships. For example, you might use them to determine whether smoking causes cancer, or carbon dioxide causes global warming – or more locally, whether cutting Universal Credit reduces unemployment.

Quite often, we find that when scientists discover something about the structure of the world, the human brain has got there before us. The brain has evolved to seek out cause-and-effect relations in the world around us, and assemble them into a graph just like this. It learns the relationships by observation: if you see the sun come up and feel warm, you will naturally assume that one causes the other.

Our brains contain millions of these cause-and-effect links and much of our mental activity is based on navigating around these graphs. (One reason we have to keep traversing the graphs is that it's hard to distinguish causality from correlation. We first learn correlations, and then use mental simulation – daydreaming, planning, storytelling – to try out different scenarios and learn which relationships are genuinely causal. Sometimes the brain gets this wrong, and we end up with superstitions instead of accurate causal knowledge.)

A side-effect of this process is that the brain teaches itself shortcuts. For example, early in life I learned that having money causes me to be able to buy chocolate, which causes happiness. Readers familiar with Pavlov and his dogs will spot the logical conclusion: I learned that money itself was rewarding, and eventually was able to gain pleasure just from having money, even without going out to buy the chocolate. In computer science terms, the graph "caches" reward at key points to save the effort of recalculating it every time it is needed.

By identifying the cached reward points in the causal graph we can understand a lot of modern economic behaviour. People place value on mental states that are not directly related to their material interests: honesty, pride, compassion, dignity, status, identity, uniqueness and so on. These objects, because they generate mental reward, can become just as valuable as money, food, warmth and the more basic objects of economic life. These objects can be seen as our true values: if we get more reward from compassion than pride, or vice versa, that tells us something about who we are.

The reward that is generated from these mental objects is what drives our mind's endless habit of wandering over the causal graph, checking and recalculating the links. This wandering is what we think of as our imagination: daydreaming, speculating about the future, immersing in fictional worlds or replaying the past. I call it the mind's "System 3" – the counterpart to the System 1 and 2 made famous by Kahneman.

This process can influence how we build AI. To make artificially intelligent algorithms think like humans, we need to give them motivations like humans. One way to do this is to give them a similar structure to human minds. The causal graph with its powerful ability to support reasoning and decision making is a good basis for AI. "Big data" approaches to AI start with a blank sheet – an empty graph – and try to fill in the blanks just by throwing more and more data at the computer.

A better approach, in my view, is to give the computers an initial graph based on the human mind. There are ways of measuring what this graph looks like for a person or group of people – an online test that takes a few minutes – and the computer can be given a copy of this graph as its starting point. It will then learn much more like a human being – with the beliefs and values that those humans gave it.

When many people in a society share a similar graph, interpreting the world in a similar way, that society organises itself around the structure of their graph – and the values, beliefs and desires that it expresses. This is how the causal graph structure scales up to guide the economy, and the politics, of the larger world. The graph starts as our way of interpreting the world, but it ends up as our way of shaping it.

Agent-based modelling, also mentioned by Cummings, is a method for understanding how that happens. The 'agents' are computer simulations of human beings. The computer creates thousands or millions of agents, and imagines what would happen if they all interacted with each other. If each agent has its own causal graph inside its simulated mind, we can look at how those graphs evolve, how they spread from one person to another – and which techniques are most persuasive at changing their minds.

These agent-based testbeds – as I discuss in this talk – are powerful ways to find solutions to tough social problems. But it all depends on the values they start with. In Euristica, the simulated world I developed, the objective is to identify the causes of inequality and discrimination, and find solutions to them. Someone else might build an agent-based model with a different set of values.

Altogether, this collection of ideas provides a powerful set of insights into human cognition, the structure of society, and technologies that can simulate and run experiments using powerful AI. I will continue to use them to seek new solutions to the social challenges that are holding humanity back: from discrimination and division, to productivity and poverty. In the meantime, if Dom wants to chat, I'm right here.


Paul said…
Thanks Leigh for a thought provoking and interesting article. I was drawn to your blog after reading this morning's article in The Times. I come from a counselling/psychotherapy background myself, but am interested in what positive possibilities that AI can bring us without the diminishment of 'the human touch'
Unknown said…
I read your thoughtful article in the Times which led me to your blog - even more enlightening. One point concerns me - who decides the set of values implanted in the agent based model?
Leigh Caldwell said…
Good question, Unknown!

In the Times piece I suggested that these values should come from democratic debate. It shouldn't be down to either me or Dominic Cummings to decide on the country's values.

It would be very easy for a set of assumptions and beliefs about the world to be slipped into the programming surreptitiously, just as there is a value set embedded in conventional economic models. But at least with those models, the assumptions are relatively well known and can be explicitly discussed and challenged. Hidden values in an AI model would be tough to spot, let alone contest.
Matisa said…
In regards to ABM, I think it’s really important to separate toy models from empirical models. An empirical ABM needs to be calibrated with justifiable theory and empirical evidence. After running various simulations, the results need to be validated by comparing to actual empirical evidence. Without calibration/validation, the benefits of ABM are largely lost.

Popular posts from this blog

What is cognitive economics?

Book review: Alchemy, by Rory Sutherland

What makes a useful theory?