Our team’s reflections from the Evidence Synthesis Edinburgh Community Initiative meeting
by Katie Dirsmith and Isabell Orlishausen
Last week, SEBI-Livestock took part in the Evidence Synthesis Edinburgh Community Initiative event at the Edinburgh Futures Institute. The initiative’s mission is to connect Edinburgh’s evidence synthesis community and create space for cross-disciplinary learning—across university research groups, public sector and local public health organisations, and animal and human health teams. Many members of the Evidence Synthesis Edinburgh Community have shared lessons from their work in human health, offering valuable insights that help strengthen our evidence synthesis work in the livestock health sector.
Katie Dirsmith and Isabell Orlishausen presented on behalf of the SEBI-L team, sharing their work designing a data extraction tool for use in a rapid review to collate data from disparate sources on foot and mouth disease (FMD) impacts for cattle in East Africa. This collated data will help create model input parameters for animal health models used by SEBI-L and the wider livestock health community. The team found that commonly available data extraction software, which is often designed for use with controlled laboratory or clinical-based studies, did not suit the data needs of this rapid review. This is because data in field-based livestock studies is often heterogenous. To address this, SEBI-L has created a multi-level Excel data extraction template that works reasonably well, though this solution comes with its own shortcomings and is not easy to scale. Ultimately, we want a solution that allows us to fulfil our vision to make data available on dashboards and preserve the context of each data point.
Many teams will recognise the challenge of data extraction for heterogenous, non-clinical studies. One goal of the Community Initiative event was to identify and find solutions for common challenges encountered by multiple groups in evidence synthesis, so SEBI-L posed the following questions to the group:
- How do you keep extracted data tidy and usable, while preserving context and traceability (hierarchical/nested subgroups, overlaps, longitudinal observations)?
- What tools or interfaces handle complex, variable reporting, beyond “flat” forms?
- What is a realistic level of automation that saves time without losing meaning?
During the event, we received thoughtful reflections from event participants, including members of EDINA and Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES). The discussion helped situate our questions in the wider ecosystem of partial automation and workflow design in evidence synthesis.
The ensuing conversation revealed that many people working in evidence synthesis face these same questions. If anything, the conversation and the lack of quick answers helped confirm there is no easy fix. A takeaway from the session is that it may be worth re-thinking what success looks like; this could mean aiming for something more realistically achievable and custom-created than a perfect one-size-fits-all data extraction system. If custom-created data extraction systems are used, it is essential they use consistent ontologies throughout to facilitate the collation of data across data extraction systems.
If you have found workable ways to handle complex data extraction, SEBI‑L would love to learn from you.
- Have you faced (and ideally solved) similar technical challenges in capturing these data nuances in a tidy way, and developed or used any standards here?
- Have you used or evaluated user-friendly tools beyond Excel/Covidence that handle hierarchical/relational extraction well?
- We are also keen to save time through automation. We would love to discuss how to use pragmatic partial automation without losing meaning!
Please get in touch if you want to connect: sebi@ed.ac.uk
Katie Dirsmith is a Researcher (Livestock Insights & Modelling) and Isabell Orlishausen is a Data Analyst Programmer with SEBI-Livestock.