christl donnelly – The Badger Crowd – standing up for badgers

Government to review the last six years of bTB science for its ‘refreshed’ bovine TB strategy

On 30^th January 2025, Defra issued Terms of Reference (here) for the ‘comprehensive new bovine TB review’, that was announced last August. This included details of a scientific panel which will be reviewing ‘new’ evidence that has become available since the last review was published in 2018.

How objective will the new review be?

The panel, that last month began reviewing new evidence for the ‘refreshed’ bovine TB strategy, is largely a reprise of those who undertook the last review back in 2018, with one exception. The panel will be chaired, as previously, by Professor Sir Charles Godfray, University of Oxford. He will be familiar with the current scientific views of those whose work has been used to maintain badger culling for the last 12 years. He was personally involved in the statistical audit of the Randomised Badger Culling Trials (RBCT) (1998 – 2005), and so is extremely close to the discussion of issues relating to questionable statistical approaches raised since the last review.

As before, Godfray will be supported by Professor Glyn Hewinson CBE of Aberystwyth University, Professor Michael Winter OBE University of Exeter and Professor James Wood OBE of University of Cambridge. Wood has been vocal on TV and radio in his long-term support for Government publications that have suggested that badger culling might be working.

Professor Sir Bernard Silverman FRS, Emeritus Professor of Statistics at the University of Oxford University has stepped in to replace Christl Donnelly, Professor of Applied Statistics at Oxford University, who requested to be ‘recused’, for reasons that have not been stated, but may relate to recent scientific discussion over statistical elements of the RBCT. As one of the statistical auditors of the RBCT, Charles Godfray made recommendations in 2004 for tighter control of the data and analyses. Donnelly (et al.) statistics from the Randomised Badger Culling Trials (here) have been challenged in a new scientific paper by Torgerson et al. (here). And the debate has continued with Mills et al. (here) and (here) and Torgerson et al. (here). Whilst it is welcome that the ongoing dilemma will be reviewed, is the proper approach to have Oxford academics looking into an Oxford issue? Silverman describes himself on his CV as “Recognised as a world leader through ratings and awards. Wide experience within government, as chair or member of boards and committees and as a departmental chief scientific adviser, with specialist expertise in national security, modern slavery, official statistics, etc.”. Notably, he was on the panel of the Anderson Inquiry into the handling of the Foot and Mouth Epidemic in 2001, so has some experience of epidemiology.

The panel is expected to report their findings by the end June of 2025. Which is unfortunate for all the badgers that will be killed in the culls for which licences will be issued from June 1st (and September 1st) 2025. And for those that are victims of the escalating illegal culling that has been reported since ‘legal’ culling began.

One cannot help but think that if Labour had really wanted an objective review of the science around bovine TB and badger culling, they would have asked an independent set of scientists with less ‘skin in the game’, and perhaps more distanced from Oxford to undertake such a vital review. But once again it seems that it is largely the same set of academics who will be looking at the science in which they personally have a historical interest and potentially, future stake.

Defra have announced a £1.4 million badger vaccination project in Cornwall (here) suggesting that they may have already made their mind up on the science evidence; they are still treating badgers as a central issue in the control of bovine TB, despite the growing doubt. Yet they are still unable to provide any certainty that this is the case. Some are making robust claims about whole genome sequencing and what it can show, whilst others are modelling what they think might be happening using outdated assumptions and unproven associations. Meanwhile, the strongest evidence of inadequate control points to ineffective cattle testing being the crucial driver of bovine TB, and the solution must therefore lie with cattle controls.

On December 13^th 2022, a preprint was put up on the ResearchSquare platform. Entitled ‘Absence of effects of widespread badger culling on tuberculosis in cattle’ [1], it was a reanalysis of the Randomized Badger Culling Trials (RBCT). Following a protracted period of review, with a number of very long silences, and relatively few revisions, it was published in Nature Scientific Reports on July 15^th 2024 (Torgerson et al 2024) [2]. Finding a major anomaly, it uses a range of statistical models to re-examine the RBCT data and concludes that most standard analytical options did not show any evidence to support an effect of badger culling on bovine TB in cattle.

Torgerson et al 2024 [2] noted that the statistical model selected for use in the original study in 2006 [5] was one of the few models that did show an effect from badger culling. However, various model assessment criteria suggest that the original model was not an optimal model compared to other options available. You can read a short blog on the new Torgerson et al paper here, and the full paper is available here. Essentially, the more appropriate models in the latest study strongly suggest that badger culling does not bring about the disease reduction reported.

Following publication, the new analysis [2] was mentioned in an article in Vet Times on July 24^th, and in Vet Record in their 3/10 August edition. Neither publication noted its major significance. No other mainstream media reported on it at all. This is perhaps surprising since the government badger cull policy rests all but entirely on the conclusions from the RBCT. It is the science that DEFRA has used in court to defend their decisions to experiment with culling. It is the science that has resulted in 11 (12 including 2024) years of intensive and supplementary badger culling across huge areas of England, and around 230,000 dead badgers. In other words it is the pivotal piece of work for the decision-making around badger culling policy.

The reluctance of the media to report further on Torgerson et al. is a prelude to work by two of the authors of the original 2006 analysis (Christl Donnelly and Rosie Woodroffe) together with a DPhil Statistics student at Oxford University (Cathal Mills), who had, at the time of publication, two rebuttal papers in press with Royal Society Open Science [3,4]. Unusually, the abstracts and supplementary information for these new papers were posted online and available to view before publication without the main text. Enquiries regarding the posted information and the main papers resulted in in press versions being helpfully forwarded. The new analyses from Mills et al. are entitled An extensive re-evaluation of evidence and analyses of the Randomised Badger Culling Trial (RBCT) I: Within proactive culling areas [3], and An extensive re-evaluation of evidence and analyses of the Randomised Badger Culling Trial II: In neighbouring areas [4].

Published on August 21st, the two new papers largely duplicate the analyses in Torgerson et al 2024, but use different model assessment criterion to come to a different conclusion. In fact they double-down on the conclusions of the original 2006 analysis: “…we estimate substantial beneficial effects of proactive culling within culling areas, consistent with separate, existing, peer-reviewed analyses of the RBCT data.”

So in the year and a half since the posting of the Torgerson pre-print [1], Mills, Woodroffe and Donnelly have been working on their rebuttal to it. Torgerson and his team have looked through the in press versions of the Mills et al (2024) publications [3,4] for a few days and multiple problems stand out. In particular:

Incorrect statements regarding disease control outcomes since 2010
Use of a non-peer reviewed publication
Confusion between the offset and overdispersion leading to incorrect calculation of disease exposure
Incorrect and confused statements regarding model comparisons
Exaggerated claims during use of new modelling.
Failure to address the modern interpretation of SICCT test reactors
Failure to recognise the onward effect of the analytical failure on multiple subsequent publications and policy outcomes.

Somewhat unusually, the new Mills et al papers do not refer to or cite the published Torgerson et al paper [2], only the first pre-printed version of Torgerson et al. from 2022 [1]. So in essence, the new Mills et al papers [3,4] are out of date at the time of publication, failing to refer to the updated 2023 preprint or the final version in Scientific Reports published 15th July 2024 [2]. Following contact with the Royal Society, a response raising concerns with the newly published papers is being written and will be submitted shortly. An interim report has been put together by Torgerson and Langton with brief observations on the new papers (see here).

The Royal Society says it is committed to reproducibility. Reproducibility is the ability of independent investigators to draw the same conclusions from an experiment by following the documentation shared by the original investigators [6]. The issues identified in [2] and “rebutted” in [3,4] illustrate one of the major issues of the reproducibility crisis: poor statistical inference. It is hoped that the conclusions of this exchange will inform future bovine TB intervention policy this autumn.

References

[1] Torgerson P, Hartnack S, Rasmusen P, Lewis F, Langton T. 2022 Absence of effects of widespread badger culling on tuberculosis in cattle. In Review. (doi:10.21203/rs.3.rs-2362912/v1)

[2] Torgerson, P.R., Hartnack, S., Rasmussen, P. et al. Absence of effects of widespread badger culling on tuberculosis in cattle. Sci Rep 14, 16326 (2024). https://doi.org/10.1038/s41598-024-67160 (available: https://www.nature.com/articles/s41598-024-67160-0)

[3] Mills CL, Woodroffe R, Donnelly CA. 2024, An extensive re-evaluation of evidence and analyses of the Randomised Badger Culling Trial (RBCT) I: Within proactive culling areas. R. Soc. Open Sci. (doi:10.1098/rsos.240385)

[4] Mills CL, Woodroffe R, Donnelly CA. 2024, An extensive re-evaluation of evidence and analyses of the Randomised Badger Culling Trial II: In neighbouring areas. R. Soc. Open Sci. 11:240386.

[5] Donnelly CA, Woodroffe R, Cox DR, Bourne FJ, Cheeseman CL, Clifton-Hadley RS, Wei G, Gettinby G, Gilks P, Jenkins H, Johnston WT. Positive and negative effects of widespread badger culling on tuberculosis in cattle. Nature. 2006 Feb 16;439(7078):843-6.

[6] Gundersen Odd Erik 2021The fundamental principles of reproducibility. Phil. Trans. R. Soc. A. 37920200210