Are both sides of the badger cull argument ‘mining the data’?

It has been claimed that the disagreement about the Randomised Badger Culling Trials (RBCT), and what that central study tells us about whether badger culling can reduce bovine TB breakdowns in cattle, is a result of ‘data mining’ from both sides. That is, both sides are selecting the methods of analysis that give them the result they want. Is this a true representation of the current situation which sees different analyses giving opposite results? Or is the claim just a bit of a smokescreen?

Basically, there are two models that are used to claim that badger culling produces a disease control benefit to cattle;

1. Donnelly et al 2006 produced the only Poisson model (there are many others available) that says badger culling ‘worked’ with around a 20% benefit (P 0.005%).

2. In the recent Godfray review of evidence, Bernard Silverman downgraded the Donnelly model and produced the only binomial model (of 4) that indicates badger culling had an effect, with around a 17% benefit (P 0.05%). A similar effect but at a much weaker level.

However, the Donnelly model did not correctly adjust the data for the number of herds in each trial area, nor the ‘time at risk’. And the recent Silverman analysis did not correctly adjust for ‘time at risk’. So neither analysis correctly adjusts for exposure to disease during the experiment.

Professor Torgerson’s reanalysis of the RBCT data in 2024, published In Nature Scientific Reports, reviewed a wider range of analyses in accordance with veterinary principles, the most appropriate/best fitting of which (based on model rating criteria and parsimony) suggested the superior models showed no benefit from badger culling.

If there was a convincing effect from what is a relatively small amount of data it would be visible in most or all of the analyses undertaken. Two selected models that have not been properly adjusted for important variables compare poorly to stronger ones that don’t. Claiming a benefit from badger culling from the RBCT is not verifiable because the claimed effect is not consistent. The correctly specified models consistently show badger culling produces no visible effect on disease in cattle.

For those wishing to read more about the recent reanalyses of the RBCT, our post on the 11th June Royal Society Open Science paper shows how the original 2006 analysis and subsequent attempts to support it were flawed and even ‘naive’, and our post from September 17th outlines statistical issues with new analyses in the recent Godfray review.

So the answer is that there is no data mining going on, at least not by those seeking to independently check the data. Just a thorough review of the best statistical way to understand the simple data involved, using the kind of approaches that the original scientists implied that they would use.