Research Culture’s Role in Contributing to Research Waste

Lessons from Systematic Reviewlution

Abstract

Systematic reviews are the gold standard of evidence syntheses and underpin decision making which affects outcomes for patients globally. A research integrity project funded by the UK Research and Innovation Medical Research Council, entitled ‘Systematic Reviewlution’ aimed to understand and document problems with these highly cited and influential articles, which are often being published at a rate that outpaces primary clinical research. This living systematic review found 485 articles in the first iteration, documenting 67 discrete problems relating to the conduct and reporting of published systematic reviews. These problems potentially jeopardise the reliability or validity of systematic reviews. A variety of institutional factors are likely fuelling the publication of substandard systematic reviews and these factors are representative of issues affecting the entire evidence ecosystem. These factors are discussed in reference to themes identified through this meta-meta-meta-research initiative. The publish or perish perverse academic reward system is fuelling a lack of reproducible research. Paradoxically, the reputation of systematic reviews as a high-quality form of evidence is leading to an overproduction as they are likely seen as a certainty for publication. Wider issues of the influences of research culture generally, the fallibility of peer review and the importance of diversity and representation in research teams are emphasised.