Thursday, 13 January 2011

Ben behaving oddly

Ben Goldacre (@bengoldacre) does some amazing work combatting folk mythologies in favour of science and real data. He has tens of thousands of readers and twitter followers who - mostly - agree with his philosophy. They spread the message at skeptics meetings, atheist book-signings and the pages of the Guardian.

He is a leader in showing how to expose and falsify unsubstantiated claims - not just those of homeopaths, fake-medicine enthusiasts and psychics, but also those of journalists, pharmaceutical companies and anyone at all who makes public claims.

So what happened today? Goldacre made the following assertion on twitter this evening:

What's the difference between this and a homeopath who doesn't need evidence because he "just believes" that his medicines work? Not much.

What's especially notable is that dozens or hundreds of his followers - science aficionados all, I would have thought - replied with their own stories about which particular prices they think might have gone up this year. Fuel, houses, food, DIY products - it's amazing that we can still afford to buy anything at all. Only a few lonely souls asked whether this personal anecdote can really be used to 

To be fair to him, Goldacre replied to me on this:

True. But any one person's "just reckoning" is inferior to the carefully calibrated (and openly published, and peer reviewed) inflation measure engineered by the ONS statisticians whose job it is to work out these figures. While technically there's nothing intellectually invalid about his claim, it's still a strange one to make.

The subject of inflation is certainly complex. Different inflation rates certainly apply to different people. Some economists argue that inflation is always lower than reported, because it can't take into account substitution effects (when some products get more expensive, people automatically stop buying as many of them). Scott Sumner doesn't believe in inflation at all, and thinks we should just measure total cash spent in the economy instead of trying to work out the average price of all products. So it's absolutely valid to raise questions about the use of a single aggregate measure across the whole economy.

But disagreeing with established data, which has the consensus of hundreds of professional economists, and a widely tested, peer-reviewed theory, behind it - on the basis of your feelings about personal experience - should automatically make any scientist wary. Economics may not be physics, but it's still more scientific than anyone's feelings. Even Ben Goldacre's.


bengoldacre said...

this is dumb, and it's clear - depressingly - that you misunderstand much of what i've written and said over the years.

it's absolutely fine to say "i reckon", to give a personal view, and describe personal anecdata, as long as it's clearly referenced as such. which is what i did. i also linked an excellent tool from xtophercook at the FT from ONS that you can use to see how closely, with your own personal spending patterns, your own inflation will match up to the quoted average.

in summary: disparities between personal experiences of inflation and summary national figures are well-documented and to be expected; posting personal experiences is fine as long as they're clearly labelled as such (mine was); and lastly, your post was dumb and timewastey.

Leigh Caldwell said...

Thanks for the comment, and apologies if the post came across like an attack - it wasn't meant that way.

Of course I agree that it's fine to cite personal experiences. I must have interpreted the comment "I just don't believe the inflation rate is 3%" differently than the way you intended it.

If I have a more substantive point, it's that economic data - just like most scientific data - is generated in a careful way by the accumulated expertise of a profession; and if our own personal experiences appear to disagree with the data, we ought to take them with a large pinch of salt.

I misread your comment as being a claim about the inaccuracy of the published figures, rather than just relating a personal experience.

Anonymous said...

I too respect and admire Dr Goldacre, read him every week etc, but Leigh's clearly right here. A simple substitution in the original tweet to "don't think my inflation rate..." would have made everything clear.
But if you say "I don't believe the inflation rate..." you're clearly questioning THE inflation rate, which, as Ben points out, all economists know is just a representative basket and not an individual one.
To come out and say Leigh was "dumb and timewastey" is, well, rude and timewastey.

TwistedByKnaves said...

I keep flip-flopping on this.

On the one hand, I do think your tweet was possibly slightly provocative.

After all, he was clearly referring to inflation in the context of his shopping, and saying that his experience differed from the published index is hard to argue with. "Inflation" here is not the same as "CPI", "RPI" or whatever metric we are using this month. This is perfectly correct usage.

On the other hand, the implication is clearly that we are using the wrong metric. This matters. Obviously.

On the other other hand, he could simply be raising awareness and empowering us all to see how we differ from the norm, and therefore if we are likely to be winners or losers in decisions taken on the basis of the published number. That feels right to me.

Treating our own experience with a pinch of salt compared to ANY scientific pronouncements sounds like driving by looking at the SatNav rather than the bus that has just pulled out in front of you. Actually, when the metric is so simplified, it's more like driving by comparing your current position to the grid reference of your destination. I don't think that you are seriously suggesting this.

Seriously, though, "experts" can advise or consult, but you always need to weigh their advice against your personal experience.

Kevin Denny said...

Ben may have been thinking "my inflation rate isn't 3%" but thats certainly not what he said so Leigh's interpretation seems reasonable to me. And I thought his (BG) response was churlish. One cannot expect doctors to be as economically literate as economists for the same reason that the latter are not as medically literate as the former.
Even an economist or a statistician would probably only have a vague idea of their "personal inflation rate".
[& being criticized as "time wastey" by someone who twitters: priceless!]

Mike said...

What's the difference between this and a homeopath who doesn't need evidence because he "just believes" that his medicines work? Not much.

No, quite a bit.

The important thing to remember is this is a TWITTER post. Twitter is a place for rapid off the cuff sharing of your thoughts. It is not the place for you to publish your PhD. Sometimes people will make mistakes and say silly things because when they post without thinking deeply. That should be all right.

Please can we forgive, or take take with a pinch of salt, what is said in this media, (and similar things such as Facebook status updates).

Ben G has probably spent much longer answering the criticism than he did composing the tweet, it will make him less likely to tweet.

Most liberals are also busy trying to stop "Twitter libel", when people criticize a rich and powerful person and then find themselves threatened from the courts.

Please let twitter be the place people can make stupid remarks without a big vendetta if they say something silly.

The alternative is that we will have to scrutinize everything for half an hour before we post, checking it is scientifically accurate and getting our Boss's approval.

If Ben had said this in a Wikipedia edit, then you could remove it. If he had said it in a 'learned article' then you could (and should) have peer reviewed it, but let's forgive twitter posts, folks.

Anonymous said...

Actually I disagree entirely with Mike. If you are 'no one' then fine, you can go ahead and tweet whatever you want. But the power of the popular followed tweet should not be underplayed as recently illustrated by the rapper 50 cent and his stock purchasing endorsement.

For better or for worse, public figures, speaking in public forums do need to be careful of exactly what they say and how they say it. Leigh's post was perhaps taking the comment in a way not intended, but then Ben should have had the grace to say, "I believe you misunderstood me" as opposed to "this is dumb".

Words have power, and those who make a business of using that power, have a responsibility to use it correctly. It doesn't cost anything to behave in a way that sets a good example for others, including when you slip up and may need to backtrack or explain yourself. Otherwise you deserve what flak you get.