Best practice guidelines for evidence synthesis

The methodological foundation for the SEBI-Livestock evidence synthesis programme.

To ensure that our evidence synthesis work is robust, transparent, and reproducible, we have developed a comprehensive set of internal guidelines. These guidelines standardise our approach to identifying, collating, and synthesising scientific evidence, ensuring that every review we undertake meets the highest academic and practical standards. By creating a shared reference point to guide the team, we minimise bias and maximise the reliability of the data used to inform decision-makers.

The challenge

Evidence synthesis is a complex process used widely to inform policy and practice. However, without standardised approaches and a priori protocols, reviews can suffer from inconsistency, lack of transparency, or hidden biases. For a service-oriented team like ours, which must respond rapidly to diverse research questions, there is a risk that speed could compromise rigour. We require a framework that allows us to be agile while strictly adhering to the principles of good evidence synthesis: inclusivity, rigour, accessibility, and transparency. Furthermore, as we integrate new technologies and address gaps in the literature, we require clear approaches to ensure our methods remain defensible and repeatable.

Our approach: standardisation and rigour

Our approach centres on the creation and maintenance of a living guidance document of best practices that guides every stage of our review process. 

Key components of our guidelines include:

  • Search Strategy Templates: We use common search terms and pre-defined search string templates to ensure comprehensive and consistent literature retrieval across different topics.
  • Methodological Guidance: The guidelines provide clear instructions on selecting the appropriate type of synthesis—whether a Rapid Review, Systematic Map, or full Systematic Review—based on the specific needs of the decision-maker.
  • Transparency Protocols: We mandate detailed documentation of every step, from inclusion criteria to data extraction, ensuring that our work can be audited and replicated.
  • Integration with Innovation: As we collaborate with informatics partners like CAMARADES and the Data Science Unit (DSU), these guidelines are updated to incorporate validated AI tools, ensuring our automation efforts align with our rigorous standards.

These guidelines serve as the bedrock for our "Evidence Base," supporting both our internal modelling efforts and our public-facing products. They that our syntheses capture the best available evidence that is suitable for high-stakes decision-making.

Strategic Value and Outcomes

By institutionalising best practices, we deliver significant value to our stakeholders:

  • Consistency and Trust: Standardised methods ensure that our findings are reliable and comparable across different projects, building trust with funders and partners.
  • Efficiency: Pre-defined templates and clear workflows reduce the time spent on methodological setup, allowing researchers to focus on analysis and synthesis.
  • Global Public Good: Our commitment to transparency means our syntheses contribute to the global knowledge base, highlighting both "gluts" and "gaps" in current research for the wider community. Where possible, we will make all our syntheses publicly available, via dashboards, journal articles, and shared datasets. 
  • Foundation for Modelling: High-quality, standardised evidence collation provides the accurate input parameters required for our animal health economic models, ensuring their outputs are meaningful.

Next Steps

We continue to refine these guidelines as new methodologies emerge and as we learn from our ongoing projects. Future updates will focus on integrating AI innovations in the evidence synthesis process, and strengthening our approaches for gender-sensitive evidence synthesis. By laying these strong foundations, we ensure that SEBI-Livestock remains a leader in producing timely, fit-for-purpose evidence for the livestock sector.

Updates