This site is intended for healthcare professionals
Wooden dominoes falling
  • Home
  • /
  • News
  • /
  • News trends
  • /
  • Clinical Trials
  • /
  • How can flaws in clinical trial design, conduct an...
Original Medthority Content

How can flaws in clinical trial design, conduct and reporting impact clinical decision-making?

Last updated:19th Mar 2024
Published:30th Mar 2023
Author: Debra Kiss, PhD; Senior Medical Writer at EPG Health.

Clinical trials are considered an important source of information for healthcare professionals to keep up to date with the latest research in their field1. As part of evidence-based practice, the results of clinical trials can also influence prescribing decisions and patient management2,3.

When appropriately designed, conducted and reported, randomised controlled trials (RCTs) are considered the ‘gold standard’ in evaluating the safety and effectiveness of healthcare interventions4,5.

However, many factors can complicate interpretation of a study’s results6,7. Although no clinical trial is perfect in design and execution, limitations and flaws can impact a study’s validity, trustworthiness and credibility6,7. Poorly designed, conducted and reported clinical trial research can overestimate benefits and underestimate the harms of a treatment approach8, and may lead to inaccurate conclusions9,10.

In this first article of a 3-part series, explore key issues reported with quality and accuracy in clinical trial publications, and their potential impacts on clinical decision-making.

Fraudulent data, zombie trials and a growing list of retractions

Although some may assume that biased, inaccurate and false reporting are minimised through the peer review processes, this is not necessarily the case11. Among research and clinical trial findings published over the past decade, there has been a continuous stream of retractions, reports of poor quality, inaccurate or fabricated data, and fraudulent studies7,9,11-15.

A new term has even been minted – ‘zombie trials’ – to describe trials with flawed or untrustworthy data that would be considered sufficient for retraction11.

In a sample of 153 RCTs submitted to Anaesthesia during 2017–2020, 44% had false data and 26% were categorised as zombie trials – that is, the extent of fabricated data was sufficient to invalidate their scientific value15

Other key issues: Bias, misreporting, spin and limited applicability to real-world patients

A range of other issues can impact the accuracy, quality and applicability of clinical trial data. These include:

  • Bias in the design, conduct and reporting of published clinical trials7,16 – the validity of a trial’s findings may be questioned if it is considered to have a high risk of bias17
  • Use of patient populations, treatment regimens and/or outcome measures that have limited applicability to real-world patients and settings7,18
  • Small sample size7 – a critical consideration in trial design, sample size can affect interpretation of a study’s findings19
  • Misreporting of outcomes, or changing outcomes from those that were pre-specified in the study protocol (‘outcome switching’)16,20-22
  • Selective or incomplete reporting of an intervention’s impact, which can lead to misinterpretation or overstatement of impact10, or an under-reporting of adverse events18
  • Use of ‘spin’ when reporting results, which can overemphasise the benefits and safety profile of an intervention8,23
  • Extrapolating findings beyond the clinical trial population23

Issues with clinical trial publication quality are not just restricted to specific or niche journals20,24. In 2019, Goldacre and colleagues found misreported outcomes in 58 of 67 RCTs published across 5 high-impact journals during a 6-week period in 201520. This included trials with missing primary outcomes, incorrectly reported primary and secondary outcomes, and reporting of additional outcomes without declaring they were novel20. Since then, studies have continued to report issues with the quality and accuracy of randomised trials across various high-impact journals5,25.

Can flawed clinical trial data adversely affect clinical decision-making?

Findings from clinical trials are used to inform guideline recommendations7, clinical decision-making in practice26, and are also a basis for regulatory approval of medicines27-29 and decisions for government reimbursement30,31.  

There is potential that flawed or fabricated clinical trial results could lead to use of ineffective interventions in practice, or even patient harm14,32

The impact of poor-quality research on clinical decision-making was highlighted in the context of fast-moving research during the COVID-19 pandemic9,14. In a tidal wave of research into potential COVID-19 treatments, the availability of pre-print publications, low quality evidence, and studies with serious methodological limitations led to confusion about the effectiveness of medicines for treatment or prophylaxis of COVID-199,14. This caused confusion amongst not only the general population, but also healthcare professionals9. Some of these medicines were used to treat patients with COVID-19 in clinical practice, despite a lack of sufficient evidence to assess their efficacy or safety for this indication9,33.

Numerous publications on COVID-19 treatments were later retracted due to issues such as34-36:

  • Study methodology, including lack of appropriate randomisation
  • Inclusion of claims within the conclusion that were not supported by study methods
  • Flaws in source data, and concerns about data integrity and fraud

Concerningly, some of these publications continued to be cited in literature after being retracted34,37.

What about the impact of flawed clinical research on clinical decision-making outside the context of the COVID-19 pandemic?

While this is more challenging to define, it is clear that issues with the quality and accuracy of clinical research publications can have a flow-on effect, especially related to aggregation of data from multiple trials.

Flaws in the design, conduct and reporting of individual clinical trials may be carried over into systematic reviews – which can magnify the impact of inadequate data7

Systematic reviews may also be susceptible to the effects of fraudulent data7. One study reported that exclusion of fabricated trials would likely have changed the conclusions of at least one out of 25 systematic reviews in the field of analgesia and anaesthesiology38.

Flawed clinical research can also impact guideline recommendations. It has been suggested that probable fabricated data may have contaminated previous WHO guideline recommendations relating to use of high-concentration oxygen and surgical site infection7,39. These guidelines were originally based upon a meta-analysis of 15 RCTs, of which two were authored by a research group implicated in data integrity issues and plagiarism39. Although the guideline was later updated to exclude the studies in question39, this demonstrates the potential impact poor quality data can have on the conclusions of meta-analyses, as well as guideline recommendations that can directly inform clinical practice.

Why do these quality issues continue to occur in research?

In the context of the COVID-19 pandemic, the rapid pursuit of effective options to prevent and treat COVID-19 led to a focus on research quantity over quality, and may have contributed to numerous retractions14,40.

Other ongoing issues that likely contribute to poor quality and flawed published research include pressure on researchers to publish at high rates32,41, obtain research funds32 and deliver results for funding sources32. The quality and accuracy of published research may also be influenced by insufficient training on good research practice and ethics14, lack of adequate quality control checks7,21, and ineffective management of conflicts of interest32,42.

Improving transparency and detection: How far have we come?

Although steps have been taken towards improved transparency in publishing clinical trials over the past decade, there is still a long way to go.

Some journals require study protocols, their amendments and statistical analysis protocols to be submitted with manuscripts during the submission process43 (e.g., Annals of Internal Medicine44, The New England Journal of Medicine45). However, journal publishing requirements are variable11 and standards for reporting are not always followed in practice20. Overburdened peer reviewers may not have sufficient time – or be asked – to cross-check source data and other information21.

There is still a lot of trust placed into authors of clinical trial publications11 – we are trusting that they have followed the methods as stated, and have accurately reported complete data from the trial.

Where ‘problematic’ RCTs are published, the INSPECT-SR project aims to prevent them contaminating the results of meta-analyses. The aim of the project is to develop and prospectively test a tool to assess the trustworthiness of RCTs, helping researchers to identify and exclude those with a high probability of containing false data46.

Where to next?

RCTs remain an important source of evidence for evaluating the safety and effectiveness of healthcare interventions4,5. However, the quality and accuracy of data reported in clinical trial publications remains a cause for concern14.

Ongoing issues in clinical trial design, conduct and reporting support a cautious approach to interpreting results of clinical trial publications – particularly if results have not been replicated across multiple studies, or if the trial data have not been subjected to appropriate quality assessments or critical appraisal

So, how can healthcare professionals navigate these issues as they endeavour to implement evidence-based practice? Read the next article in this three-part series to explore this predicament and some potential solutions in more detail.

Read article 2 in this series - Interpreting clinical trial results through a frame of reference: Why take the time?

Would you prefer to skip to key practical considerations that may assist healthcare professionals in using a more critical approach when reading clinical trial publications? Click the link below.

Read article 3 in this series - Looking under the surface of clinical trials: Are the findings valid and relevant to your patients?

Explore medical education and publication summaries on Medthority

Explore a range of independent and sponsored medical education across therapeutic areas including cardiology, immunology, oncology, and more. Upskill on key developments via continuing medical education in various formats including Learning Zones, with expert insights from key opinion leaders in videos and podcasts, as well as webinars, quizzes and more. Medthority Learning Zones also feature publication digests, which provide concise summaries of key publications across different fields.

Explore medical education content by specialty on Medthority


  1. EPG Health. The gaps between HCP demand and pharma supply of medical information. 2021. Available at:
  2. Mooney LA, Fay L, DeCastro B, Zanki TA, Mansi B. Transparency and credibility of industry-sponsored clinical trial publications: a survey of journal editors. Curr Med Res Opin. 2019;35(7):1221-1230.
  3. Kristensen N, Nymann C, Konradsen H. Implementing research results in clinical practice- the experiences of healthcare professionals. BMC Health Serv Res. 2016;16:48.
  4. Clyne B, Boland F, Murphy N, Murphy E, Moriarty F, Barry A, et al. Quality, scope and reporting standards of randomised controlled trials in Irish Health Research: an observational study. Trials. 2020;21(1):494.
  5. McErlean M, Samways J, Godolphin PJ, Chen Y. The reporting standards of randomised controlled trials in leading medical journals between 2019 and 2020: a systematic review. Ir J Med Sci. 2023;192(1):73-80.
  6. Bauchner H, Golub RM, Fontanarosa PB. Reporting and Interpretation of Randomized Clinical Trials. JAMA. 2019;322(8):732-735.
  7. Moore A, Fisher E, Eccleston C. Flawed, futile, and fabricated—features that limit confidence in clinical research in pain and anaesthesia: a narrative review. Br J Anaesth. 2022.
  8. Lazarus C, Haneef R, Ravaud P, Boutron I. Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol. 2015;15(1):85.
  9. Garegnani LI, Madrid E, Meza N. Misleading clinical evidence and systematic reviews on ivermectin for COVID-19. BMJ Evid Based Med. 2022;27(3):156-158.
  10. Turner EL, Platt AC, Gallis JA, Tetreault K, Easter C, McKenzie JE, et al. Completeness of reporting and risks of overstating impact in cluster randomised trials: a systematic review. Lancet Glob Health. 2021;9(8):e1163-e1168.
  11. Smith R. Time to assume that health research is fraudulent until proven otherwise? Br Med J. 2021.
  12. Dabbous HM, Abd‑Elsalam S, El‑Sayed MH, Sherief AF, Ebeid FFS, El Ghafar MSA, et al. Retraction Note to: Efficacy of favipiravir in COVID‑19 treatment: a multi‑center randomized study. Arch Virol. 2022;167(1):277-277.
  13. Marcus A. RetractionWatch. More than 100 of an anesthesiologist’s papers retracted. Accessed 6 October 2022.
  14. Bramstedt KA. The carnage of substandard research during the COVID-19 pandemic: a call for quality. J Med Ethics. 2020;46(12):803.
  15. Carlisle JB. False individual patient data and zombie randomised controlled trials submitted to Anaesthesia. Anaesthesia. 2021;76(4):472-479.
  16. Mitra-Majumdar M, Kesselheim AS. Reporting bias in clinical trials: Progress toward transparency and next steps. PLoS Med. 2022;19(1):e1003894.
  17. Pirosca S, Shiely F, Clarke M, Treweek S. Tolerating bad health research: the continuing scandal. Trials. 2022;23(1):458.
  18. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1):122.
  19. Thabane A, Phillips MR, Wong TY, Thabane L, Bhandari M, Chaudhary V, et al. The clinician’s guide to randomized trials: interpretation. Eye. 2022;36(3):481-482.
  20. Goldacre B, Drysdale H, Dale A, Milosevic I, Slade E, Hartley P, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):118.
  21. Ioannidis JP, Caplan AL, Dal-Ré R. Outcome reporting bias in clinical trials: why monitoring matters. BMJ. 2017;356:j408.
  22. West J, Bergstrom C. Misinformation in and about science. Proc Natl Acad Sci USA. 2021;118(15):e1912444117.
  23. Steegmans PAJ, Di Girolamo N, Meursinge Reynders RA. Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews. Res Integr Peer Rev. 2019;4(1):27.
  24. Anderson C, Nugent K, Peterson C. Academic Journal Retractions and the COVID-19 Pandemic. J Prim Care Community Health. 2021;12:21501327211015592.
  25. Hemming K, Javid I, Taljaard M. A review of high impact journals found that misinterpretation of non-statistically significant results from randomized trials was common. J Clin Epidemiol. 2022;145:112-120.
  26. Turner M. University of Canberra. Evidence-Based Practice in Health.
  27. Therapeutic Goods Administration. Mandatory requirements for an effective application. Australian Government Department of Health; 2022. Available at: Accessed 24 February 2023.
  28. US Food and drug Administration. Development and approval process - drugs.
  29. European Medicines Agency. From laboratory to patient: the journey of a medicine assessed by EMA. 2019. Available at: Accessed 24 February 2023.
  30. Australian Government Department of Health. HTA for Australian Government subsidy.
  31. Dawoud D, Lamb A, Moore A, Bregman C, Rupniewska E, Paling T, et al. Capturing what matters: updating NICE methods guidance on measuring and valuing health. Qual Life Res. 2022;31(7):2167-2173.
  32. Poutoglidou F, Stavrakas M, Tsetsos N, Poutoglidis A, Tsentemeidou A, Fyrmpas G, et al. Fraud and deciet in medical research - insights and current perspectives. Voices in Bioethics. 2022;8.
  33. Mega E. Latin America’s embrace of an unproven COVID treatment is hindering drug trials. 2020. Available at: Accessed 27 February 2023.
  34. Valencise FE, Palamim CVC, Marson FAL. Retraction of Clinical Trials about the SARS-CoV-2 Infection: An Unaddressed Problem and Its Possible Impact on Coronavirus Disease (COVID)-19 Treatment. Int J Environ Res Public Health. 2023;20(3).
  35. Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19. N Engl J Med. 2020;382(25):e102.
  36. Mehra MR, Ruschitzka F, Patel AN. Retraction-Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 2020;395(10240):1820.
  37. Lee TC, Senecal J, Hsu JM, McDonald EG. Ongoing Citations of a Retracted Study Involving Cardiovascular Disease, Drug Therapy, and Mortality in COVID-19. JAMA Intern Med. 2021;181(11):1535-1537.
  38. Marret E, Elia N, Dahl JB, McQuay HJ, Møiniche S, Moore RA, et al. Susceptibility to fraud in systematic reviews: lessons from the Reuben case. Anesthesiology. 2009;111(6):1279-1289.
  39. Myles PS, Carlisle JB, Scarr B. Evidence for compromised data integrity in studies of liberal peri-operative inspired oxygen. Anaesthesia. 2019;74(5):573-584.
  40. Milanese A, Trerotoli P, Vestri A, On Behalf Of The Biostatisticians Collaborative G, Sismec Directive C. A Survey on Methodological Issues of Clinical Research Studies Reviewed by Independent Ethic Committees during the COVID-19 Pandemic in Italy. Int J Environ Res Public Health. 2022;19(18).
  41. Gandevia S. Publication pressure and scientific misconduct: why we need more open governance. Spinal Cord. 2018;56(9):821-822.
  42. Østengaard L, Lundh A, Tjørnhøj-Thomsen T, Abdi S, Gelle MHA, Stewart LA, et al. Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study. BMJ. 2020;371:m3764.
  43. Spence OM, Hong K, Onwuchekwa Uba R, Doshi P. Availability of study protocols for randomized trials published in high-impact medical journals: A cross-sectional analysis. Clin Trials. 2020;17(1):99-105.
  44. McCook A. In major shift, medical journal to publish protocols along with clinical trials. Accessed 6 October 2022.
  45. NEJM. New England Journal of Medicine. New Manuscripts. Accessed 6 October 2022.
  46. Wilkinson J, Heal C, Antoniou GA, Flemyng E, Alfirevic Z, Avenell A, et al. Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions. medRxiv. 2023.