The peer-reviewer of the new scientific paper, Torgerson et al 2025, published on June 11th is the Director of Biomathematics and Statistics Scotland (BIOSS). The paper is concerned with analysis of the Ranomised Badger Culling Trial (RBCT). Below, you can see the text of the review together with the authors responses, as addressed to the editor. Also highlighted (bold italics) are a few points that seem particularly important & difficult to disagree with, even if you don’t understand statistics. It should be noted that the two reviewers of the Mills et al. (2024) papers that Torgerson et al. is rebutting, declined to submit a review of the new Torgerson paper. In other words, they praised the two Mills et al papers, recommended them for acceptance, but declined the opportunity to defend their decision by explaining their thinking on Torgerson et al (2024 & 2025). They remain anonymous, which is interesting from an integrity perspective.
These communications are available from a link (Review History) on the online paper.
Torgerson: The reviewer has made a number of comments. We have addressed these where necessary and made some amendments to the text. We would also like to thank the reviewer for these helpful comments which we hope have improved our manuscript.
Reviewer: Comment. Overall it seems there is strong disagreement over competing data analyses on what is already a highly contentious issue, which has significant implications in terms of ecology and agriculture. I have been asked to comment on one small part of the ongoing discussion, which I will do – however, my overall, strongest recommendation is that, given the importance of the underlying issue (both scientifically and politically) that a proper investigation be conducted to establish an agreed position involving all parties. A continuing to-and-fro among different sets of authors – each, I am sure, well-meaning in their own ways – serves little purpose, and there are better, more efficient and effective ways of resolving disagreements than in the pages of an academic journal.
Torgerson: The underlying issue (mass culling of largely healthy badgers across much of England and in Ireland) is the result of the original analysis by Donnelly et al, first published in Nature in 2006 and papers derived from that, and subsequent analyses. Although the merit of the analysis has been questioned, only by having alternative data analyses published in peer reviewed journals is it possible for the UK Governments to consider amending the policy. Nevertheless, we have contacted Donnelly et al previously in an attempt to meet and to find common ground, but our attempts were rebuffed. In order to give the reader a clear idea of the applied implications of both our re-evaluation and the peer review comments, we have added some text to the conclusions: “Accordingly a very substantial number of publications that rest extensively or completely on RBCT statistical analyses may require major qualification or retraction. The justification for lethal control of badgers to-date appears to have been based upon basic statistical oversight.” Also there is a reproducibility crisis in science. Therefore it is important for these issues to be published as we believe they provide an exemplar of a major driver to the reproducibility crisis and misdirection of disease epidemic management.
Badger Crowd Comment: A scientific seminar and evening presentation on the work of Paul Torgerson and his team (then a pre-print) was organised in Oxford in November 2024 in order to allow debate with the RBCT scientists. All declined the invitation. DEFRA and APHA sent no representatives. Natural England sent one staff member but gave no feedback. Badger Crowd very much welcomes the reviewers suggestion to hold a “proper investigation…to establish an agreed position involving all parties.”
Reviewer: Comment 1) Section 2.1. I agree with the authors of the current contribution here that use of an offset here is likely a requirement, and that the comments from the Mills et al. (4) are naïve at best – it isn’t helpful to think of an offset as equivalent to setting a regression coefficient to 1.0, as of course the issue is that this is a log-linear scale and the point is that (under the Poisson) we assume proportional rates. I agree that it does not seem to make sense that the herd breakdowns vary only very slightly (the parameter value of 0.04) with the number of herds, although I would caution here only that it is possible some other variable/term in the model might be related to the number of herds, hence suppressing the parameter value (when including number of herds as a regression term rather than as an offset) owing to collinearity.
Torgerson: This is quite possible, and the collinearity is likely to be between the co variates of “triplet” and “baseline herds at risk”. Hence the motivation to explore the results of models where triplet was removed (which always resulted in a considerable reduction in AICc). Although it is not clear if the reviewer required a response to this statement, we have inserted the following text into the manuscript “The removal of “triplet” may remove any hidden effects possibly due to collinearity with number of herds, as well as substantially reducing the number of covariates and hence largely eliminate the issue of over fitting.”
Reviewer: Comment 2) I have personally written (Brewer et al, 2016 – https://doi.org/10.1111/2041-210X.12541 ) about the dangers of relying on the theoretical distinctions between AIC/AICc and BIC, so it should be no surprise that I share the current authors’ suspicions on the preference expressed in Mills et al. (4) for BIC. Also, why are the AICc values for Models 1 and 3 so much higher than the null model in Table 1? Is this just a feature of the small-sample correction? Otherwise I would not expect this at all (if I’m understand what the null model is, correctly); in the absence of other explanations here, given the high number of parameters in models 1 and 3 I would suspect poor model fitting with inflated variances due to collinearity. So, I agree with the current authors here, on the basis of the evidence in front of me (Sections 2.2+2.3, Table 1, supplementary material).
Torgerson: This issue is well taken. This was part of our arguments. Although perhaps not completely clear. We have edited the relevant text which now reads “Brewer et al (16) have written about the dangers or relying on theoretical distinctions between AIC/AICc and BIC. Nevertheless, Mills et al. (4,5) state “…a wide array of statistical techniques and study periods allows us to make robust conclusions regarding the effects of proactive badger culling which are informed by consistent scientific evidence from trial data, irrespective of which approach to statistical inference is taken.” This is demonstrably untrue. The analysis of “confirmed breakdowns” (OTFW) show that results are highly dependent on the approach to statistical inference and information criteria used. However, absence of any cull effect on the incidence of bTB, when total breakdowns are considered, is robust, irrespective of statistical method.”
Reviewer: Comment 3) The discussion on Bayesian models claims that the authors of Mills et al. (4) made coding errors. I do feel that then Mills et al. should be able to examine and (if relevant) correct these errors, and formally issue a correction in the pages of the journal. Otherwise, I don’t feel I have sufficient information to comment further here.
Torgerson: Here the reader needs to refer to both the coding on GitHub, which is where Mills et al have made their statistical code available and the code we have given in the supplementary material. We give an example here.
In Mills et al, they claimed an offset was used with the following R code:
rs1aB←stan_glm.nb(Incidence~Treatment+log(hdyrsrisk),+log(Hist3yr), offset = log(hdyrsrisk), prior_intercept=normal(0,10), prior=normal(0,10), data = rbctconf, refresh=0)
Here you will note (in red) that log(hdyrsrisk) appears twice in the code, both as an explanatory variable and an offset. The effect of this is to shift the parameters of log(hrdyrsrisk) by a value of 1, whilst other parameter values remain unchanged, thus effectively having no offset.
The correct code if a parameter value is to be fixed as an offset is:
rs1aB<-stan_glm.nb(Incidence~Treatment+log(Hist3yr), offset = log(hdyrsrisk), prior_intercept=normal(0,10), prior=normal(0,10), data = rbctconf, refresh=0) or alternatively:
rs1aB<-stan_glm.nb(Incidence~Treatment+offset(log(hdyrsrisk))+log(Hist3yr), prior_intercept=normal(0,10), prior=normal(0,10), data = rbctconf, refresh=0)
We refer the reader to our supplementary material where it is fully explained together with the other errors in the code of Mills et al.
Reviewer: I agree with the current authors’ concerns on the statistical audit. I would go so far as to say that, given the important of the topic of this discussion, any audit should be carried out openly and transparently.
Torgerson: We have added short text at the end of section 4 “ It is important that trials include an audit that is open and transparent.”
Reviewer: Comment 5) Section 5 on the neighboring area study – again, from what I can see here, I would broadly agree with the concerns of the current authors.
Torgerson: No response required
Reviewer: Comment. 6) To clarify; I have no issue with the modelling of counts, as the use of a Poisson-form log-linear model is, in effect, modelling rates. To be more precise, I would suggest that the problem is not that Mills et al. (4) modelled counts, but that they did not properly scale those counts by use of an appropriate offset – and again, I am saying this on the basis of the evidence of the current work (only).
Torgerson: Here the reviewer appears to be agreeing with our approach by using the offset. But for clarity so readers can see the derivation of the offset (in the Poisson log-linear model) in our previous manuscript where there derivation is explained. Thus we have inserted the text “The mathematical derivation of the offset is explained in our previous manuscript on this issue (3).”
Reviewer: Comment. 7) Finally, I would like to address the quotation from Donnelly (16): “the suggestion of requiring independent replication of specific statistical analyses as a general check before publication seems not merely unnecessary but a misuse of relatively scarce expertise”. The point to me here is not that work should be “replicated” as such, but that work should be verifiable. The authors of Mills et al. (4) have apparently made their work – at least that related to the 2024 journal papers – available openly, and this is the key; openness and transparency are vital. I would even go as far to say that, in such a contentious area as this, it is naïve to imagine that a single analysis by a particular group of scientists should be seen as sufficient.
Torgerson: Yes we agree, which is one of the issues with the original RBCT proactive cull statistical findings published in 2006, which was led by Donnelly: it was a single analysis by a particular group of scientists, and the Mills et al papers are also led by Donnelly et al. We think this is obvious and it should be verifiable. Nevertheless we have modified the text surrounding the quotation of Donnelly (16). It now reads: The position of Donnelly (17) that “the suggestion of requiring independent replication of specific statistical analyses as a general check before publication seems not merely unnecessary but a misuse of relatively scarce expertise”, needs revisiting. This case underlines the need not only for rigorous checks of statistical analysis but also validation of the statistical models and assumptions used within submitted manuscripts to verify them.
Badger Crowd Comment: For the last 6 months, Sir Charles Godfray and his ‘expert’ panel have been reviewing badger cull and bovine TB science published since 2018. Godfray was involved in the RBCT audit, the 2013 restatement of badger cull science and the 2018 science review. In other words, the single analysis (Donnelly 2006) that has supported the badger cull policy is being reviewed by (largely) the same ‘particular’ group of scientists who have been associated with the work for nearly 20 years. Donnelly herself has been ‘recused’, but has been replaced by another Oxford statistician from the same department. There is no outside scrutiny, and there is a case that there has been no “proper investigation ….to establish an agreed position involving all parties” as recommended by the peer reviewer. A proper investigation would be free from conflict of interest. Defra have refused to address this issue over the last six months.
In case you missed the point of all this, the new Torgerson paper shows how for multiple reasons that there is no evidence that culling badger delivers a disease benefit of bovine TB control in cattle herds. The current ‘closed shop’ of science at DEFRA has fallen flat in the past and should not be allowed to continue. They are selecting the scientists and science that they want to suit a civil service agenda and they don’t want to admit that they have been wrong for very many years. It is a flagrant example of policy driven science. And everybody is losing out because of it: the public (because of the enormous costs of policy), the farmers (because it is a policy that can never achieve its aims) and the badgers – because they are being inhumanely killed in huge numbers.
This intolerable situation cannot be allowed to continue. Badger culling must stop now, and an independent investigation or inquiry must be set up.
Discover more from The Badger Crowd - standing up for badgers
Subscribe to get the latest posts sent to your email.