Abstract
Abstract
Systematic reviews and meta‐analyses are key evidence synthesis methods for informing future research, interventions and policy. As the validity of their conclusions depends on the primary studies they synthesise, assessing the internal validity of the included studies is essential. In some fields, such as medicine, this is the norm and is commonly done using Risk of Bias assessment tools.
Risk of Bias (RoB) assessment is however rare in ecology and evolutionary biology (EEB) even though several RoB tools have been developed in some ecological subfields and related fields. To identify potential reasons for a limited uptake, we conducted a survey of ecologists and evolutionary biologists with evidence synthesis experience and reviewed 275 journals that publish EEB research for guidelines on performing RoB or related assessments.
Only 28 of 232 (12%) survey respondents had correct interpretation of the RoB concept, while 46 (40%) of 116 that had heard of RoB have confused RoB with publication bias. Just 10 (4%) had conducted a RoB assessment, most of whom found it challenging. Out of the 209 EEB journals that explicitly solicit evidence synthesis (
N
= 58) or reviews (
N
= 151), only five (2%) directly mentioned standards for conducting evidence synthesis, which include RoB or related (e.g. critical appraisal) assessments. An additional 45 (22%) journals indirectly linked to RoB or a related assessment via referring to the guidelines for reporting evidence synthesis (e.g. PRISMA), despite such reporting guidelines not providing information on how to conduct RoB assessments.
To increase its uptake in EEB we recommend making RoB assessment: (1) known and recognised as an essential component of a reliable evidence synthesis by including it in training materials, and journals' and funders' guidelines and policies; (2) easy to perform by bringing the synthesis community together to determine the need for developing new or adjusting existing RoB tools; and (3) possible by further improving reporting standards for primary studies so that RoB assessment can be done on these studies. For those unfamiliar with the RoB assessment, we provide five key RoB questions that existing tools often cover. These questions can be considered to understand the basic composition of the evidence included in evidence synthesis.