Tag Archives: godfray

Are both sides of the badger cull argument ‘mining the data’?

Posted on by

It has been claimed that the disagreement about the Randomised Badger Culling Trials (RBCT), and what that central study tells us about whether badger culling can reduce bovine TB breakdowns in cattle, is a result of ‘data mining’ from both sides. That is, both sides are selecting the methods of analysis that give them the result they want. Is this a true representation of the current situation which sees different analyses giving opposite results? Or is the claim just a bit of a smokescreen?

Basically, there are two models that are used to claim that badger culling produces a disease control benefit to cattle;

1. Donnelly et al 2006 produced the only Poisson model (there are many others available) that says badger culling ‘worked’ with around a 20% benefit (P 0.005%).

2. In the recent Godfray review of evidence, Bernard Silverman downgraded the Donnelly model and produced the only binomial model (of 4) that indicates badger culling had an effect, with around a 17% benefit (P 0.05%). A similar effect but at a much weaker level.

However, the Donnelly model did not correctly adjust the data for the number of herds in each trial area, nor the ‘time at risk’. And the recent Silverman analysis did not correctly adjust for ‘time at risk’. So neither analysis correctly adjusts for exposure to disease during the experiment.

Professor Torgerson’s reanalysis of the RBCT data in 2024, published In Nature Scientific Reports, reviewed a wider range of analyses in accordance with veterinary principles, the most appropriate/best fitting of which (based on model rating criteria and parsimony) suggested the superior models showed no benefit from badger culling.

If there was a convincing effect from what is a relatively small amount of data it would be visible in most or all of the analyses undertaken. Two selected models that have not been properly adjusted for important variables compare poorly to stronger ones that don’t. Claiming a benefit from badger culling from the RBCT is not verifiable because the claimed effect is not consistent. The correctly specified models consistently show badger culling produces no visible effect on disease in cattle.

For those wishing to read more about the recent reanalyses of the RBCT, our post on the 11th June Royal Society Open Science paper shows how the original 2006 analysis and subsequent attempts to support it were flawed and even ‘naive’, and our post from September 17th outlines statistical issues with new analyses in the recent Godfray review.

So the answer is that there is no data mining going on, at least not by those seeking to independently check the data. Just a thorough review of the best statistical way to understand the simple data involved, using the kind of approaches that the original scientists implied that they would use.

Godfray Review 2025: Defra Revision? Correction? Or just further scientific howlers?

Posted on by

On Friday 26th September, Defra changed the “Bovine TB strategy review update” but simply made things worse by adding another layer of confusion.

The correction of reported Information Criteria as now shown in the Godfray Review

An error was reported as “a revision to a line of code”. In the document itself, the change is referred to as a ‘cut and paste error.’ Red text in a box next to the correction (see above) suggests that ‘The rest of the analysis and interpretation is unaffected’.

The replacement red numbers, now show the  outputs from their 4 models described, but this time the difference between models 3 and 4 is insignificant. To put it in technical terms, a delta (difference) AICc of 0–2 suggests both models have similar support from the data.

However, far from the analysis and interpretation being unaffected as they suggest, the revision does have serious implications for the inferences drawn. The ‘update’ is actually a correction due to the wrong results being inserted into their models in Annex 4. The results reported as AICc (Akaike Information Criterion) were in fact the results from Baysian analysis (BIC) (which they did not report on).

However, BIC is not the correct IC for model selection, due to the small number of data points and large number of coefficients to estimate. The small sample size equivalent (was this really not tried?) would be appropriate (BICc), especially for logistic regression. Applying BICc favours a model which suggests that the incidence of herd breakdowns is independent of culling (i.e. culling produces no effect).

If AICc is used for model selection, a model suggesting a weak effect of culling is only slightly favoured, model 3: 173.58 vs model 4: 173.87. This difference is too small to draw any inference from; it is effectively a toss-up between the two models. The incorrectly reported IC results showed a bigger difference: model 3:151.58 vs model 4:159.38). It is this greater significant difference, which is what the peer-reviewers seem to have picked up on. Hence the big question, what were the peer reviewers sent? And given that one or both reviewer assumed the code was right, were they commenting on the wrong results?

Apparently Defra have been asked this all-important  question and an answer is pending. Maybe they don’t know and need to ask?  The answer could alter the Chapter 6 discussion hugely. A very large number of people are interested to find out who knew what and when.

The Information Criteria as originally presented in the Godfray review

One might pose the question “did the Godfray group try the small sample size BICc?” Given that they obviously tried BIC (as that is the output they incorrectly reported), the correct form of BIC would be the small sample size correction as there are only 20 data points. Interestingly, when BICc is applied, model 4 (without badger culling at 196.00) is favoured over model 3 (with badger culling at 200.96). I.e. badger culling has no effect.

However, notwithstanding all this incredible muddle,  it would have been more appropriate to select the most favourable model of a series of over-dispersed models, using the quasi version of AICc, which is QAICc, especially as the quasibinomial model was subsequently used for statistical inference.

When the QAICc is applied to their models, the results strongly favour a model which again suggests bTB incidence in cattle is independent of badger culling (i.e. culling produces no effect).

A pre-print outlining some of the problems with the new analysis is available here.

Reasonable people can disagree….

When dealing with the recent academic publications on the RBCT, (which have been reported on here), and which arrive at different conclusions, the Godfray review has stated at para 6.LXX:

“…reasonable people can disagree about the best way to analyse complex data such as these.”

That is an interesting response to the current differing views. A short article entitled “When Can Reasonable People Disagree?” is quite insightful on this. We borrow a couple of apposite quotes below:

“To say that “reasonable people can disagree” can encourage suspension of judgment in response to important matters of personal and social concern.”

“There is a related tendency to use the expression “reasonable people can disagree” to create an equivalence when it comes to conflicting beliefs that large groups of people hold. A common perception exists that if a large enough group of people believes something, it has met the litmus test for being a “reasonable’ belief and the people involved “reasonable people.” This is a familiar critical thinking error. The fact that a large group of people believes something does not provide us with a compelling reason for thinking it is true.”

“Respect for persons might require engaged listening. It might require careful consideration. It does not require treating weak evidence as if it is strong nor does it require treating unsound arguments as if they are sound. Far from being kind, we are actually doing one another an injustice when we engage one another as if all arguments are equally compelling.”

The point is, and it is a  point made by Prof mark Brewer earlier this year in his review of Torgerson et al 2025, it is more a question of can reasonable people agree?  And the answer to that appears to be not yet. But there  needs to be conversation. So far Defra has refused to have that conversation. Will that continue or does it have to wait until there is an inquiry into the crisis?

In any event, this muddle is so bad that the Godfray review will need to be rewritten. That is unavoidable now. The mistakes are too obvious, too profound and too important to remain as a part of the narrative.

While there is a ‘it doesn’t matter anyway’ type sentence to cover up any exposure, it is quite clear that it really does matter. The mistakes alter everything, including the economic forecast on badger culling benefit.  

Its time for Defra to change the way they do business.

 

 

2038 Bovine TB targets on the scrap heap, with details of failed policy increasingly exposed

Posted on by

Why was the use of Gamma interfreon testing, alongside use of the tuberculin test (SICCT), under-declared in APHA’s central bovine TB control evaluation paper (Birch et al 2024) last year, and also in the newly published ‘Godfray’ report?

Gamma reactors by cull year 2009 – 2023

A letter published today 19th September, in Veterinary Record, details the amount of supplementary Gamma testing that was introduced in the High Risk and Edge Areas before and during the post 2013 mass badger culls. It shows how Gamma testing was under-emphasised in the Birch et al. analysis on the effect of the poorly labelled ‘Badger Control Policy’, (see here). Considerable disease benefit is being claimed for badger cull in its first two years and beyond, but is far more likely to simply be linked to the increase in cattle testing with Gamma, as recorded publicly by number of gamma test reactors found.

What this shows is that Defra knew well that Gamma could find undisclosed infection, no surprise as it has been used in other countries for decades. They used it to help lower the spread of disease, then purposefully eased off, as badger culling was phasing down in more recent years. Who, you have to ask, was controlling this behaviour? And why?

As Badger Crowd has pointed out before (see here), all disease measures implemented, including extensive testing, were analysed together, with no control areas. There was no comparison of culled and unculled areas as the recent Godfray Review very strangely mis-reported.

Data presented with the Vet Record letter shows how easing off of the use of Gamma testing to supplement SICCT testing in 2022 is likely to have hindered disease control. This reduction in Gamma was said to be due to lack of EU funding, but this was surely predictable. So why were farmers and the public not told that the strategy was being derailed? A freedom of information disclosure in April 2025 concerning the likelihood of
bTB freedom being achievable under present conditions of testing and cattle controls, produced the following response: ‘APHA has not yet produced models suitable for predicting whether TB eradication will be achieved in England by 2038, or when TB eradication will be achieved.’ The 25-year bTB eradication strategy published in 2014, with a 2038 projected target of bTB freedom (elimination or near elimination) now, on current trajectory, has an end point beyond 2060.

As has been pointed out many times before, there is an urgent need for the use of other immune-assay tests and phage testing to be authorised alongside SICCT and Gamma IFN-γ tests, and for farm vets to be provided with extensive new advice and guidance. With freedom to test bTB out from herds – this is mentioned in the Godfray report but it should have been front and centre.

The money raid on central government funds requested in the Godfray review would have been credible with a report triaging essential priorities. But instead, like the 2018 report, the 2025 report is muddled and incoherent. Progress, especially in England, Northern Ireland and the Republic of Ireland simply will not materialise if Defra continue along his line.

References

Birch et al (2024)
Langton, Griffiths & Griffiths (2025)

 

Summer news roundup

Posted on by

The  parliamentary summer recess has begun. There can be no more Parliamentary Questions until the recall in September. Which is more than a shame, because there are questions that still need to be answered about the badger cull and bovine TB policy, by a government that does not engage properly with many stakeholders and the public. Supplementary badger cull (SBC) and Low Risk Area licenses were issued in May, and badger shooting is underway, with more authorisations expected for intensive culling shortly. These last intensive cull licenses will almost certainly be issued later this month to allow even more culling in the autumn. But the science to support this policy has been successfully challenged in the literature, with independent verification and a call for proper investigation – yet we still have silence from a government that just wants to finish its ugly killing spree.

Zeichner visit to Gatcombe Farm

The Minister of State for Food Security and Rural Affairs Daniel Zeichner visited Gatcombe Farm in Devon a few weeks ago. This is the farm at the centre of the ground breaking Save Me Trust BBC documentary last year that was attacked by some of the nastier elements of the bTB world, including Defra-funded bodies. Gatcombe is where an innovative protocol for cattle testing has been investigated over the last ten years or so, using carefully managed, newer and more sensitive tests. Each test can be used to target bTB to better increase chance of detection. Used in combination, in a manner prohibited for general use by current rules, the new protocol has been successful in identifying infection that would previously be left hidden in the herd. Let’s hope Zeichner sees the potential to finally start on changes to policy that were needed many years ago, using the cattle measures that DEFRA staff have fought so hard to resist.

Godfray Review report postponed

The current review of bovine TB science, the first one published back in 2018, was commissioned by the new Labour government last year and was due to report by the end of June. But in June, this was officially changed to ‘from the end of June’. Badger Crowd understands that it will now appear towards the end of the year, but an exact time has not been announced. This could, perhaps, be partly due to the publication on June 11th of a paper in Royal Society Open Science that confirmed that previous core Government reference science, the RBCT, was in fact based on ‘a basic statistical oversight’, and that more  plausible analyses of the results showed no effect of badger culling from the £50 Million experiment.

APHA produces a pre-print to oppose the 2022 appraisal finding no cull benefits

A pre-print has appeared on BioRxiv: ‘Evaluating the effect of badger culling on TB incidence in cattle: a critique of Langton et al. 2022’ authored by DEFRA’s Andy Robertson. Robertson has worked for TBHub, APHA, Natural England and is based at DEFRA. His publications have twice wrongly claimed badgers are a known maintenance host for cattle TB.

The new pre-print, three years in the preparation, claims that if badger culling had ‘worked’, (created disease decline benefit), the Langton et al analysis might not have detected it. As ever with DEFRA bTB publications, computer code for the model and simulations used is not provided, so it is impossible to check that what has been done is correct or plausible. Code was requested from DEFRA on July 21, but there has been no response at all.

Much of the text leans heavily on published studies that have now been shown to be uncertain at best. The Animal and Plant Health Agency (APHA) paper (Birch et al., published March 2024) in particular is misrepresented as evidence of a positive effect of badger culling. Accurate interpretation of that paper shows that there was no attempt in it to see if badger culling contributed to the general decline in bTB in herds under progressively tighter cattle testing methods. The critique glosses-over an important finding in Langton et al. 2022 (that Defra acknowledged at the time), that at the county level, bovine TB incidence stabilised, and started to decline, well before badger culling was rolled out.

Badger Vaccination

The governments new agreement to fund the NFU  £1.4 Mn badger vaccination trials in Cornwall has been widely reported since January. It has been in the news again recently, yet there are still scant details available on the scientific and analytical protocol of the work. Aspects follows a similar project in Wales many years ago, that led to it being dropped as a strategic option.

Requests for further information from DEFRA have met the usual wall of silence. DEFRA’s Minister Sue Hayman half-answered a PQ on the project last week saying “Unlike previous badger culling studies, the Cornwall Badger Project is focused on testing different methods of delivering badger vaccination, rather than evaluating the impact on bovine TB in cattle.” So the use of badger vaccination as a tool in cattle TB control is not being measured? This despite NFU saying that is the essential question that needs answering. It all looks so half-baked and ‘un-joined up’ at DEFRA.

Jeremy Clarkson’s herd is OTF-S

As reported here, it was bad news for Jeremy Clarkson recently. Positive and inconclusive tuberculin tests on his cattle mean that Diddly Squat Farm now has the status Officially TB Free-Suspended. With viewing figures of 4-5 million, Clarkson is in a good position to put the disastrous government bovine TB policy into the public consciousness. Costing over £100Mn a year, the result of the policy has been an immense waste of time and resources. With a hidden epidemic that is still not being effectively detected, and 250,000 mostly healthy badgers culled, many cruelly, due to ‘statistical oversights’ and a government mired in its inability to get a proper grip. If Ministers want to do farming a huge favour, they will get the right experts to look at the evidence, and having procrastinated for over a year, instigate immediate radical change. Forget badgers, it is correct cattle testing and movement control  procedures that will rapidly bring herds into manageable condition, as it did in the 1960’s.

Will anything new be offered before the intensive badger culling starts again in September? Probably not. The lack of urgency on this issue is incredibly disappointing. Whatever Labour’s manifesto intentions were, it seems that the civil servants have the whip hand here, holding on to their dogma and their wrong advice and roles, resisting rather than following the new science. It is the public purse, the farmers, cows and badgers who are paying the price of ineffective government.

Cull review details announced

Posted on by

Government to review the last six years of bTB science for its ‘refreshed’ bovine TB strategy

Dan Zeichner

On 30th January 2025, Defra issued Terms of Reference (here) for the ‘comprehensive new bovine TB review’, that was announced last August. This included details of a scientific panel which will be reviewing ‘new’ evidence that has become available since the last review was published in 2018.

How objective will the new review be?

Sir Charles Godfrey

The panel, that last month began reviewing new evidence for the ‘refreshed’ bovine TB strategy, is largely a reprise of those who undertook the last review back in 2018, with one exception. The panel will be chaired, as previously, by Professor Sir Charles Godfray, University of Oxford. He will be familiar with the current scientific views of those whose work has been used to maintain badger culling for the last 12 years. He was personally involved in the statistical audit of the Randomised Badger Culling Trials (RBCT) (1998 – 2005), and so is extremely close to the discussion of issues relating to questionable statistical approaches raised since the last review.

As before, Godfray will be supported by Professor Glyn Hewinson CBE of Aberystwyth University,  Professor Michael Winter OBE University of Exeter and Professor James Wood OBE of University of Cambridge. Wood has been vocal on TV and radio in his long-term support for Government publications that have suggested that badger culling might be working.

Professor Sir Bernard Silverman FRS, Emeritus Professor of Statistics at the University of Oxford University has stepped in to replace Christl Donnelly, Professor of Applied Statistics at Oxford University, who requested to be ‘recused’, for reasons that have not been stated, but may relate to recent scientific discussion over statistical elements of the RBCT. As one of the statistical auditors of the RBCT, Charles Godfray made recommendations in 2004 for tighter control of the data and analyses. Donnelly (et al.) statistics from the Randomised Badger Culling Trials (here) have been challenged in a new scientific paper by Torgerson et al. (here). And the debate has continued with Mills et al. (here) and (here) and Torgerson et al. (here). Whilst it is welcome that the ongoing dilemma will be reviewed, is the proper approach to have Oxford academics looking into an Oxford issue? Silverman describes himself on his CV as “Recognised as a world leader through ratings and awards. Wide experience within government, as chair or member of boards and committees and as a departmental chief scientific adviser, with specialist expertise in national security, modern slavery, official statistics, etc.”. Notably, he was on the panel of the Anderson Inquiry into the handling of the Foot and Mouth Epidemic in 2001, so has some experience of epidemiology.

The panel is expected to report their findings by the end June of 2025. Which is unfortunate for all the badgers that will be killed in the culls for which licences will be issued from June 1st (and September 1st) 2025. And for those that are victims of the escalating illegal culling that has been reported since ‘legal’ culling began.

One cannot help but think that if Labour had really wanted an objective review of the science around bovine TB and badger culling, they would have asked an independent set of scientists with less ‘skin in the game’, and perhaps more distanced from Oxford to undertake such a vital review. But once again it seems that it is largely the same set of academics who will be looking at the science in which they personally have a historical interest and potentially, future stake.

Defra have announced a £1.4 million badger vaccination project in Cornwall (here) suggesting that they may have already made their mind up on the science evidence; they are still treating badgers as a central issue in the control of bovine TB, despite the growing doubt. Yet they are still unable to provide any certainty that this is the case. Some are making robust claims about whole genome sequencing and what it can show, whilst others are modelling what they think might be happening using outdated assumptions and unproven associations. Meanwhile, the strongest evidence of inadequate control points to ineffective cattle testing being the crucial driver of bovine TB, and the solution must therefore lie with cattle controls.