How to debunk an electoral fraud claim

There are plenty of claims of US election fraud floating around this week. Most of them fall into three categories:

  1. Too vague to be meaningfully evaluated or investigated
  2. Too small to matter (a few individual ballots being challenged here and there, possibly valid but not enough to affect the results)
  3. Too wild to stand up to any kind of scrutiny
Together, these claims are certainly problematic: they create a fog of doubt about the legitimacy of the democratic process.

But a fourth category is more insidious. A twitter mutual retweeted the thread quoted below. You can click through to read the whole thread, but I have embedded the highlights. It's a well-told story, with several characteristics that make it effective - as well as dangerous.

To start with, the data comes from an authoritative source, the New York Times. Even better in this case: a source associated with "the other side". Surely the liberals can't deny the truths from their own newspaper?


But, the writer implies, the Times wouldn't want you to know this - it's only because we have cleverly figured out how to get access to their secret data (very enticing):

The source code used to generate the charts is provided, allowing any reader to 'check the workings' (of course, almost nobody will). A conspiratorial suggestion to download the programs because - presumably - the government will inevitably delete them if they reveal the truth.
(in fact, the provided source code is incomplete - it pulls the data down from the New York Times but it does not do any of the analysis used to produce the graphs later in the thread. Perhaps they don't want us to check the workings after all?)

The thread starts off in a neutral tone, holding back on its strongest claims until later. This allows the evidence to gradually persuade; your mind's skepticism is not activated by any bold accusations.

Charts and data lend authority to any claim, as they appear to provide evidence that validates the words used to describe them:


A series of near-reveals are followed by a retreat from the precipice - they tease us with a hint that we have found the fraud...only to pull back.

Good storytelling.

The explanations are well written, allowing readers to feel they are gaining a true understanding of what's going on. Your mind's desire to know, and flatter yourself with your knowledge, creates a confirmation bias where you increasingly go along with what is being claimed.



But finally, the dramatic buildup pays off.


Ta-da! A police car driving 169,000 absentee ballots to...somewhere. Fits nicely with narratives we might remember from old TV shows and historical rumour, about boxes of ballots being delivered by the Democratic "machine" just in time to swing an election.

Now we're on the downhill slope, and the storytelling can be ramped up for a couple of tweets before we get back to the data.


(a clarification: they are only supposed to 'stop accepting' ballots that arrive in the mail after polls close. They're still meant to transport and count the votes that have already arrived - which these had.)

You've been waiting, so here's the "explanation" of the big reveal. Sounds plausible, right? We'll get back to that.

To continue the story: if you aren't convinced by Wisconsin, let's look at Pennsylvania. And Georgia. And Michigan. The repetition lulls you into belief. Sure, someone could probably explain the 'anomaly' in one state. But all five of them?

Finally Virginia is thrown into the mix. Same pattern. You haven't heard anything about Virginia this election, have you? Well that just goes to show:


I'm not sure what 'the exception that proves the rule' means, but it serves its rhetorical purpose. If they were trying to trick you, they wouldn't have included Virginia. The fact that they have, shows their transparency and rigor.

A final summary to reinforce the key message:


Right! Quite a journey there. I can see why people are persuaded. But is any of it true?


The first thing that made me (and should make you) suspicious is that in each of these charts, the Democrat/Republican ratio of most batches of ballots is EXACTLY the same (apart from the so-called anomalies). Not just nearly, but exactly the same. The purported explanation is that they were 'shuffled in the mail like a deck of cards'. Well, anyone who has ever shuffled a deck of cards knows that this isn't what it looks like.

If this was true, a well-shuffled pack would always deal everyone at the poker table an equal hand. There would be no chance of winning - or losing - at blackjack. Tossing a coin ten times would always produce exactly five heads and five tails. But mathematical randomness does not work like that.

Anyone who works with real-world data will recognise that flat graphs like that are never what we see in reality.

In reality, a shuffled pack produces a distribution of outcomes clustered around the average. Sometimes it's higher, sometimes lower, and occasionally it's the exact average. It's not smooth, and there's no way every single batch would have an identical ratio. The probabilities are governed by the binomial distribution, and a real chart of these returns would look less like this (from the Pennsylvania graph above):




and more like this:






This chart was generated from a perfectly shuffled "deck" with exactly 50/50 proportions of each party's votes. It's fairly smooth - but not flat like the charts in the thread. The ratio of votes is never identical from one batch to the next, because real randomness is noisy.


The second reason to be suspicious of the original charts is that they do not reflect how votes are actually counted in those states. Ballots are not mailed into the state capital and shuffled together. Instead, they are mailed into a central point for each county. And every county has different politics, and a very different ratio of Democrat to Republican voters.

If you were watching the results on Tuesday night, Wednesday or Thursday you will remember the votes coming in from different counties - Waukesha, Kenosha, Milwaukee, Philadelphia, Allegheny, Maricopa - each with its own profile. Some are more Democrat and some are more Republican. Some got more of their reports in earlier in the night, others later. Mostly they got in-person votes counted first and mail ballots later. By 3am CST, more of the Republican-leaning counties were in and more Democrat-leaning ones still out, but there were a number of batches from Republican counties still to come.

Therefore the real pattern might have looked more like this, with GOP counties mostly coming early and Dem batches dominating later on:





The smoothness of the charts in the original thread is an indication - not of fraud - but that the data is probably wrong. Under no plausible scenario, not even a fraudulent one, would those lines look so flat.

So I asked myself: is it a straightforward fake? Has the author of the thread simply made up the data entirely? That didn't feel quite right either. Time to have a closer look.

I downloaded the program posted in the thread (fortunately the CIA hadn't deleted it yet, phew) and took a look at where the data was coming from. Indeed, it is the New York Times:

  state_results = requests.get('https://static01.nyt.com/elections-assets/2020/data/api/2020-11-03/race-page/{}/president.json'.format(formatted_state)).json()

Replace {} with the name of a state in lowercase and you can download it yourself. Here's the Wisconsin version.

So the data is real. Could it simply have been misinterpreted? One possibility was that they might have charted the cumulative D/R share instead of the D/R share of individual batches of ballots. That would explain why they converge on a stable value, because the cumulative share in a swing state quickly reaches a level close to 50% and changes very slowly after that.

However, this doesn't fit either. A cumulative graph would not jump around in the early hours like the graphs in the original thread.

In any case, the psychology doesn't point that way. This was quite a painstaking piece of work, from someone with at least a passing understanding of statistics and the ability to read data. It seems unlikely they would have confused cumulative with batch data. And - feel free to debate this with me - I think this kind of task is most likely to be carried out by someone who believes in what they are doing. No doubt there are people, perhaps even elected officials in the US government, who will say whatever suits them regardless of whether they believe it. But someone who puts together a detailed analysis like this is probably sincere, at least on some level. I'm not saying they don't have an agenda, or a bias. However I do think they are trying to put together a real argument out of real data.

Therefore, I didn't believe the data was either faked or crudely misinterpreted. And yet, it couldn't possibly be saying what the author was claiming.

So I took the next step: recreating what the author had done, to see how far I could get.

My computer didn't have all the right python modules to run their program, so I downloaded them, updated the source code to run under Python 3 (interestingly, the original was written in Python 2, which is more than ten years old), and was able to run the program and recreate the data behind the graphs. So the data is real, the graphs are really produced by the program (plus a second program that was not included) - so what was going on?

To answer that, we have to look in more detail at what the graphs are actually showing us.

Here is a chunk of the relevant data from the Times:


(apologies for poor image resolution).

Each data entry tells us:
  • The share of votes counted for Biden (so far)
  • The share of votes counted for Trump (so far)
  • The total number of ballots counted so far
  • The most recent time that new ballots were added to the total
As you can see, the number of votes continually goes up - this is cumulative data. That is, each entry includes all the same votes counted in the previous entry, plus the new ones that have just arrived.

The author (or their anonymous data scientist/programmer friend) has converted these cumulative totals into a differential time series. They take each entry and subtract the previous one, to work out how many new votes were added in this batch, and figure out how many were for Trump and how many for Biden.

For instance, take the first two rows of data here:



With 213,227 votes counted, Biden has 50.8% and Trump 47.4%. That's 108,319 for Biden and 101,070 for Trump (and 4,000 or so for third parties or write-ins).

After the next batch, the total votes counted are 279,037: Trump has now pulled ahead with 49.2%, and Biden has 48.9%. That means - in total - Biden has 136,449 and Trump 137,286.

We can simply subtract each candidate's totals before and after, to find out how many votes they got in this batch. Biden got (136449 - 108319) = 28,130. Trump got (137286 - 101070) = 36,216. Total votes in the batch, including third parties, is 65,810. This gives Trump 55.0% and Biden 42.7% - most likely this came from one of the more rural pro-Trump counties.

Straightforward enough arithmetic, right? But can you spot the hidden fallacy in the reasoning? There is a landmine in these figures which makes it impossible to draw the conclusion that the anonymous tweeter wants to draw.

Let's look at another set of figures and see if we can spot the landmine there.


This is from a little later in the night, when the big initial vote batches have all been counted, and the updates are much smaller. Most of the new batches being reported are only one or two thousand ballots. Try the same exercise with the first two rows of this data.

After batch 1: Trump = 50.6% of 1,087,928 = 550,492. Biden = 47.7% = 518,942.
After batch 2: Trump = 50.6% of 1,089,047 = 551,057. Biden = 47.7% = 519,475

New votes in batch: 1,119. Trump = 566, Biden = 534.

So what share of the votes in the new batch did each candidate get? Trump = 50.6% and Biden = 47.7%. Exactly the same as their vote share for the whole night up to this point.

This is surely an extreme coincidence! Whatever county these votes happened to come from, it has exactly the same breakdown of votes as the whole state? And it's not just that batch - the same pattern is seen with most of the vote batches in the above image.

If this were true, it would indeed indicate something weird was going on with the ballots. But it is not true. Have you spotted why yet?

It is all down to one simple reason: rounding errors.

The data provided by the New York Times is rounded to the nearest 0.1%. For its intended purpose, this is sufficient - they provide the data to support the graphics shown on their news pages, and news pages are fine with 50.6% vs 47.7% - it's close enough.

But the rounding means it is impossible to calculate the exact number of votes for Trump and Biden from the above data. The numbers are approximate, and could be off by 0.05% in either direction: in this case, 560 ballots either way. That doesn't affect the overall totals very much, but look at the number of votes in the individual batch we analysed above: 566 for Trump and 534 for Biden. If the totals in this batch are 560 off, there could be as few as 6 Trump votes in the batch, or more than 1000. That is, the real Trump share could be anywhere from 1% to 100%, and the Biden share 0 to 99%!

The rounding error makes it impossible to work out the vote share in each batch. And vote-share-per-batch is the one piece of information that the whole thread relies on, "anomalies" and all.

The original calculation, instead of treating the 50.6% figure on the first cumulative total as an approximation, treats it as an exact number. The next total also has a 50.6% score for Trump (which in itself is not surprising, since a small batch of new votes will not affect the cumulative share much). If you treat these as exact numbers instead of approximations, the calculation will tell you that the batch has exactly 50.6% Trump votes too, to keep the cumulative score identical.

In reality, that batch probably had somewhere between 400-700 Trump votes and 400-700 Biden votes - there is no way to know the real numbers. The proportions do not follow that artificially smooth line after all.

This calculating error also has some other strange consequences. Let's say the (real) Trump share is gradually creeping up, batch by batch, from 50.6% to 50.63% to 50.649% (all of which are rounded to 50.6%). A final tiny increase to 50.65% will cause the total to be rounded up to 50.7% instead of rounded down. It will appear that 1100 Trump votes have arrived all at once, when in reality they were spread over several batches. This makes it look as if there is a batch of votes almost completely dominated by Trump (or Biden - this effect goes either way). In fact, it could appear as if there are more Trump (or Biden) votes than the total number of ballots in the batch!

And this is exactly what you see in the original author's Wisconsin graph (look from midnight onwards): a long series of identical vote shares, and some wildly high and low outliers in both Trump's and Biden's direction. These outliers represent times when the vote share went up or down just enough for the total to be rounded up instead of rounded down. There are even bigger outliers than the ones shown, but the graph has been cut off at 2.00 presumably to make this less obvious.




So the mystery is solved. There is no "anomaly": just a genuine shift in vote share over the course of the night, because the votes that got counted later include a higher share of mail-in ballots (which take longer to count) and are from different counties. This shift is disguised by the rounding errors that these charts rely on.

Did the original author - or data scientist - realise this? Maybe not. Earlier I mentioned a Python module (it's called pandas - aw, cute) which they used for the data analysis. pandas has a time-series function which automatically works out the differences between batches - the calculation I did by hand above. Perhaps they called it without looking closely at the outputs, or thinking through the implications of the rounding. Maybe they didn't even realise the data was rounded (although it's fairly obvious). I hesitate to accuse someone of lying when it could be a mistake. But the upshot is the same: the whole set of claims made in this thread are wrong, and should be retracted.


Unfortunately there doesn't seem to be (yet) an authoritative source of the real cumulative vote counts which would allow us to create an accurate version of the above graph. Perhaps the Times or Edison will release that at some point. But in the absence of that, just think back to Tuesday night or Wednesday (or Thursday or Friday!) when you were watching the results come in. You may recall that every new batch of votes had a different share for Biden and Trump. In some reports (Arizona, for example) there were batches with a 51%, 57% or 59% Trump share. In Pennsylvania, most (not all) of the late-counted batches were pro-Biden, but by varying amounts - 60% in Allegheny, 73%, 75% or 79% in Philadelphia. The unpredictability of these vote shares was exactly what kept us waiting 4 days for the result to be called. Nobody knew what the Allegheny or Maricopa shares would be until they arrived, so we didn't know if the remaining votes would be enough to push either candidate over the edge and win the state.


If you'd like to see another view on this, here's a twitter thread from @cb_miller_, who independently did the same calculation as me. He has presented the same data in some alternative ways to show how the error works.


Some final thoughts, then - if you want to understand the truth of an assertion you see online, ask yourself:
  1. How good is the storytelling? If it follows a dramatic arc that confirms what you already suspect to be true, try to separate the persuasiveness of that rhetoric from the factual claims being made. Good storytelling doesn't mean that a claim is false, but it also doesn't mean it's true.
  2. Does the data look very clean and simple? Real data is usually a bit more messy, with the patterns not as easy to see.
  3. Does it accord with what you already know? In this case, a bit of thought about how counties report their vote totals would have shown that the claim could not be true.
  4. Does the claim rely on a long series of graphs and claims about data that are hard to check? If you don't have the expertise or time to check it yourself (and few of us do), see if someone you trust has checked and endorsed it for you. That might be a scientist, a trusted media organisation (although I'm aware that not everyone trusts traditional media organisations to the same degree) or a government agency (same). But even if you don't believe everything those authorities tell you, the fact that they are willing to put their name to something might still indicate they are willing to stake some of their credibility on it.
  5. A corollary of this: do you know the name of the person making the claim? Anonymous tweeters are not putting anything at stake by posting stuff like this, so you might choose to put less weight on what they say. Maybe different in a whistleblower situation where someone's safety is at risk, but that's why journalists protect their sources: so you can benefit from the fact-checking the journalist has done, without needing to know exactly where the original facts came from.
  6. Have you heard about it from anyone else? If something really is a genuine scandal, you'll probably hear about it in more than one place. It's much easier for someone to check the details of a report like this and amplify it, than for a whole conspiracy of journalists to get together and suppress it. And in a highly competitive media world, there are enough news sites with plenty of incentive to report any genuine issues - even if you think MSNBC wants to hide the truth, lots of other people don't.

Stay skeptical for sure, but eventually you have to trust someone. A good heuristic is: the more people involved in an activity, the harder it is to hide anything nefarious. There are a lot of people involved in counting votes in a US election, and it seems unlikely to me that widespread malarkey could go on without lots of people spotting it.



Comments

Max Magee said…
Seems like you're somewhat unfamiliar with Python versioning (which is one of the major weak-points of Python, so this isn't really your fault). Python 2.7 has been in maintenance mode for 10 years, but it was—-up until this year—-seeing maintenance releases. The core of the Python language is complete, but new modules are only being incorporated into Python 3.x. For example, the 2.7.18 release is date-stamped 2020-04-20. Meaning it was released in April (presumably bugfix, security, or performance improvements were added).
https://mail.python.org/archives/list/python-announce-list@python.org/thread/OFCIETIXLX34X7FVK5B5WPZH22HXV342/

As far as Pandas, yes, it is a cute name, but it's also a great data-analysis tool. Python has a history of silly names (starting with Python, which was named after Monty Python)—pandas is presumably Python for ANalysis of DAta and Statistics, however, they just named it pandas because it's cute and fun (or something about PANel Data AnalysiS, but that fits just as poorly).

Petty details aside, I think your analysis of the data is spot on: basically, that you can't draw those conclusions from this dataset because there isn't enough data to slice and dice it the way the original argument attempted. A.k.a.: Somebody skipped sig-fig day in data science school.

You charitably read the intent of the original author to be innocent and well-intentioned, but I would say it's part of the larger effort to discredit our election system and to sow doubt and confusion about our larger democratic institutions. Therefore, I wouldn't be so quick to impute altruistic or didactic motives.

There's an element of Dunning-Kruger at play here too. Folks think they've found a secret conspiracy because they don't understand the nuances and details of how numbers, statistics, or (and particularly the electoral count) data collection work. This author just climbed the peak of Mount Stupid and hit PUBLISH, instead of saying "Huh, maybe it's my analysis is wrong, rather than a vast nation-wide conspiracy."
https://www.psychologytoday.com/us/blog/mind-in-the-machine/201808/the-dunning-kruger-effect-may-help-explain-trumps-support

That all being said, all of this election fraud is completely plausible, and particularly, before strict rules and oversight were imposed, (even presidential) elections were stolen regularly when it seemed possible to do so—both major parties are complicit. See the election of 1876 for some particularly terrible two-party behavior.

https://www.chicagotribune.com/history/ct-opinion-flashback-hayes-tilden-presidential-election-20200925-s55522nok5burjnhopwtt2ijoi-story.html

All of the past history (up to and including 2000 and 2016's election results) puts our democratic systems on shaky ground, and is yet another definitive sign that the electoral college system is past due for an overhaul, if not outright abolition. Conspiracy theories are rife in every election, but when a presidential candidate espouses them, you expect there to be evidence and something real there. Time and time again, after experts (theoretically) made their best arguments in courts of law, they continue to show they had nothing but suspicion and lies to back them up.
https://theconversation.com/why-trumps-election-fraud-claims-arent-showing-up-in-his-lawsuits-challenging-the-results-150505
30 November 2020 at 16:22
Leigh Caldwell said…
Hi Max

Thanks for the Python details - I don't use it too often so it looks like I misunderstood the versioning situation slightly. My trusty java compiler usually gets more of my attention.

I suspect you're right about the original author's motives, but I didn't want to make my argument reliant on an assumption of bad faith. If the goal is to persuade people who might be tempted by the original argument, starting off with an accusation of bias may put up their mental defenses and stop them listening (e.g. see this paper by Hagmann and Loewenstein)

Electoral college reform does seem like a good idea, although it would involve the US making a fundamental political choice between privileging the separate interests of states versus the combined interests of the whole nation's population. That choice obviously has long historical roots and has had immense consequences over time, and I imagine it would be pretty hard to change.

Reform of the voting process is a different matter, and I could imagine this whole process motivating a new voting rights act. Perhaps by fighting this lost cause, Trump will inadvertently get his base on board with voting reform! Imagine a bunch of GOP senators being primaried until they commit to implementing a robust, (relatively) fraud-proof voting system that ensures everyone can vote. We can dream.

I did wonder briefly if a more consistently designed, straightforward, transparent nationwide voting process would help reduce conspiracy theorising, but being realistic - it probably won't. There are lots of systems in society without the idiosyncrasies and weirdness of the American voting process, and there are conspiracy theories about most of them. Still, it's a good thing to work towards on its own merits.
30 November 2020 at 19:53
Post a Comment

Popular posts from this blog

What is the difference between cognitive economics and behavioural finance?

This question was posed in an interesting LinkedIn discussion group (Behavioural Finance: Theory and Practice) by Kerry Pechter. I dashed off a quick answer which ended up being 500 words long. So I thought it might be useful to post on here. Can anyone tell me the difference between cognitive economics and behavioral finance? Cognitive economics is not yet a widely used phrase, though Marco Novarese and I have been using it as a name for a more microfounded version of what's typically called behavioural economics. So I'll answer the question based on how I use the term. The first difference is between "economics" and "finance". Economics is a broader field, including the trading of any kind of goods or services, whereas finance specifically focuses on investment and the value of financial instruments. Indeed I consider economics very broadly to be the study of how resources are allocated (by individuals, and across society). The more fundamental...
Read more

Is bad news for the Treasury good for the private sector?

An intriguing post from Robert Peston raising the spectre of the Treasury failing to sell all the bonds it wanted to sell yesterday. Is this, as he implies, plain bad news for the government because their funding is getting more expensive? Or is it good news for private lenders - implying that investors are fed up with their "flight to quality" and are now considering private sector investments instead? The answer is dependent on two further questions: is investors' money switching to foreign investments, UK private lending or UK equities? are investors actually running out of money to invest - the infamous savings trap, where people in aggregate try to increase their savings but fail to do so, because the attempt to save reduces total income in the economy and available savings are reduced too? Hopefully we can find out. How could we figure it out? by looking at total UK savings this month - this is not an exact science but we should hopefully be able to see if the tren...
Read more

Book review: Alchemy, by Rory Sutherland

Rory Sutherland's new book Alchemy: The Surprising Power of Ideas that Don't Make Sense  continues his 10-year campaign against the traditional, logical pursuit of business advantage, through a scientific lens that includes several cognitive economics themes. As ever, a curated series of amusing anecdotes about people or companies who took an unusual angle on marketing or product invention, fuel a philosophical wander. That philosophy could be summarised as: if it makes sense, someone's already tried it. So try something that doesn't . The ideas that underpin the book are broadly based on behavioural economics and cognitive science, with bits of evolutionary theory, statistics and old-fashioned advertising intuition thrown in. At first it doesn't look like a behavioural science book as such: the theoretical backbone takes a while to show. Rory's style is discursive: an after-dinner-talk of anecdotes, dismantling of conventional wisdom, ever-so-slightly outr...
Read more