This site is intended for healthcare professionals
Magnifying glass
  • Home
  • /
  • News
  • /
  • News trends
  • /
  • Clinical Trials
  • /
  • Interpreting clinical trial results through a fram...
Original Medthority Content

Interpreting clinical trial results through a frame of reference: Why take the time?

Read time: 10 mins
Last updated:31st Mar 2023
Published:31st Mar 2023
Author: Debra Kiss, PhD; Senior Medical Writer at EPG Health.

While all trials have their limitations, not all randomised controlled trials (RCTs) are designed, conducted, and reported with the same methodological rigour, and particular flaws can compromise the validity of findings1,2.

In the first article of this series, some key flaws in the design, conduct and reporting of clinical trials were explored, along with their potential impact on clinical decision-making. Although some steps towards improved standards of reporting and transparency have been implemented in recent years3,4, many issues still remain unresolved5-7.

While it is well-known that evidence-based practice involves using clinical trial findings to inform treatment decisions for individual patients1,8-10, knowing that some of the evidence base may be flawed adds further complexity for time-poor healthcare professionals when trying to decide if trial results could inform patient management11.

Clinicians may have the most challenging role in understanding the results of RCTs because they have to take the (necessarily) incomplete information provided in the scientific report, decide whether it is potentially actionable, and, if so, decide whether and how the findings can be applied to individual patients
Bauchner et al., 20191

In this second article of a 3-part series, we explore this predicament faced by healthcare professionals, and outline some potential solutions.

Is it feasible to expect healthcare professionals to be proficient in critical appraisal?

It is often noted that clinicians should practice critical appraisal skills, including risk of bias assessments, as part of evidence-based practice12-14. Critical appraisal skills are clearly beneficial as part of training in evidence-based medicine – particularly for healthcare professionals with a keen interest and sufficient time to practice these skills12. However, it may not be feasible to expect practising healthcare professionals to be proficient in these in-depth skills and practice them on a regular basis12.

Time constraints can be a serious barrier that can prevent healthcare professionals from keeping up to date with new research12 — not to mention undertaking detailed critical evaluations of original research studies

A frame of reference for a more informed interpretation of study findings

Healthcare professionals can benefit from an understanding – or ‘refresher’ — of key issues that influence the quality, certainty and trustworthiness of evidence, including study design, risk of bias, treatment effect, and precision12. The ability to interpret an RCT through this frame of reference, and consider its applicability to patients in the real world, may enable healthcare professionals to gain a more informed interpretation of the study’s findings2.

Reading the abstract is not enough

Although it may be tempting to rely solely on publication abstracts for clinical decisions — due to lack of time, limited access to full-text articles and limited expertise in critical appraisal15,16 — this practice can result in an incorrect interpretation of a study and misleading conclusions17,18

An analysis of over 49 systematic reviews and meta-analyses on RCTs in dentistry reported over 65% of abstracts failed to mention adverse events that occurred with the intervention, 2% omitted reporting on primary outcomes, and over 4% selectively reported positive results and omitted negative results for primary outcomes17. However, when the full-text paper was read, these proportions reduced to 14%, 0% and 0%, respectively17.

In addition, multiple groups have reported the presence of spin in abstracts of RCTs, non-randomised studies, systematic reviews and meta-analyses, whereby the benefits of an intervention are over-emphasised compared to the full text18-21. A two-armed parallel-group RCT tested the impact of spin in the abstracts of published RCTs on clinician interpretation, comparing to their interpretation of abstracts rewritten without spin18. The results indicated the presence of spin had a significant impact on the clinician’s interpretation of results, particularly in rating the treatment as more beneficial, compared to the version of the abstracts written without spin18.

It is crucial to remember that the study abstract is only a concise presentation of the key points in a study’s design and results, and is highly restricted by journal requirements and space limitations1

Secondary sources of trustworthy information

Secondary sources of trustworthy information can be a useful source of information and commentary on clinical trial publications12. This information may be available through:

  • Evidence-based clinical practice guidelines12
  • Systematic reviews12, particularly those from credible and trustworthy sources such as the Cochrane Collaboration

Such sources may contain useful information on quality and risk of bias assessments for key clinical trials in particular fields12. However, secondary sources of trusted information are limited.

It is also important to consider potential and declared sources of bias and conflicts of interest in secondary sources of information. The author(s) and/or experts should include appropriate declarations on conflicts of interest, and disclose any funding from the pharmaceutical industry. Based on this information, and the quality of evidence and commentary provided, the reader can then make an informed judgement about whether bias may have potentially influenced the materials.

What about editorials and expert commentaries?

Editorials and other commentaries can provide insights into the study’s results in the context of other research, and may explore clinical implications and highlight limitations to consider when interpreting study results1. However, it is up to healthcare professionals to read a publication in detail, considering its complexity, strengths and limitations1, particularly when using its results to inform clinical decision-making.

Key considerations when reading clinical trial publications: Suggestions for healthcare professionals

A more complete picture of a trial and its findings can be gained by reading the full publication. In particular, results of RCTs should be interpreted within the context of1,2,8,9,22,23:

  • The importance of the research question
  • The design and conduct of the trial
  • The study results, treatment effect, and their clinical importance
  • Applicability of the study to real-world patients
  • Other pre-existing evidence

These considerations require careful reading of the entire article, including study methods, results and discussion1. The discussion section will usually describe key study limitations and caveats that are important when interpreting the study’s findings1.

Up next: Looking under the surface of clinical trials - are the findings valid and relevant to your patients?

To assist healthcare professionals to determine if trial results are of sufficient quality to inform their decision-making in practice, the final article in this series will explore the above key aspects of clinical trials in more detail.

Read article 3 in this series - Looking under the surface of clinical trials: Are the findings valid and relevant to your patients?

Did you miss article 1 in this series? Click the link below to learn more about key issues reported with the quality and accuracy in clinical trial publications, and their potential impact on clinical decision-making.

Read article 1 in this series - How can flaws in clinical trial design, conduct and reporting impact clinical decision-making?

Explore medical education and publication summaries on Medthority

Explore a range of independent and sponsored medical education across therapeutic areas including cardiology, immunology, oncology and more. Upskill on key developments via continuing medical education in various formats including Learning Zones, with expert insights from key opinion leaders in videos and podcasts, as well as webinars, quizzes and more. Medthority Learning Zones also feature publication digests, which provide concise summaries of key publications across different fields.

Explore medical education content by specialty on Medthority

References

  1. Bauchner H, Golub RM, Fontanarosa PB. Reporting and Interpretation of Randomized Clinical Trials. JAMA. 2019;322(8):732-735.
  2. Thabane A, Phillips MR, Wong TY, Thabane L, Bhandari M, Chaudhary V, et al. The clinician’s guide to randomized trials: interpretation. Eye. 2022;36(3):481-482.
  3. Spence OM, Hong K, Onwuchekwa Uba R, Doshi P. Availability of study protocols for randomized trials published in high-impact medical journals: A cross-sectional analysis. Clin Trials. 2020;17(1):99-105.
  4. Mitra-Majumdar M, Kesselheim AS. Reporting bias in clinical trials: Progress toward transparency and next steps. PLoS Med. 2022;19(1):e1003894.
  5. Moore A, Fisher E, Eccleston C. Flawed, futile, and fabricated—features that limit confidence in clinical research in pain and anaesthesia: a narrative review. Br J Anaesth. 2022.
  6. Pirosca S, Shiely F, Clarke M, Treweek S. Tolerating bad health research: the continuing scandal. Trials. 2022;23(1):458.
  7. Poutoglidou F, Stavrakas M, Tsetsos N, Poutoglidis A, Tsentemeidou A, Fyrmpas G, et al. Fraud and deciet in medical research - insights and current perspectives. Voices in Bioethics. 2022;8.
  8. Sonbol MB, Firwana BM, Hilal T, Murad MH. How to read a published clinical trial: A practical guide for clinicians. Avicenna J Med. 2020;10(2):68-75.
  9. Turner M. University of Canberra. Evidence-Based Practice in Health. https://canberra.libguides.com/evidence.
  10. Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1):122.
  11. Wiffen P. Bad trials are a scandal that need to be stopped. Eur J Hosp Pharm. 2022;29(4):179-179.
  12. Tikkinen KAO, Guyatt GH. Understanding of research results, evidence summaries and their applicability—not critical appraisal—are core skills of medical curriculum. BMJ Evid Based Med. 2021;26(5):231-233.
  13. Mhaskar R, Emmanuel P, Mishra S, Patel S, Naik E, Kumar A. Critical appraisal skills are essential to informed decision-making. Indian J Sex Transm Dis AIDS. 2009;30(2):112-119.
  14. Al-Jundi A, Sakka S. Critical Appraisal of Clinical Research. J Clin Diagn Res. 2017;11(5):Je01-je05.
  15. Fontelo P. Rapid response to innovations in publishing BMJ research: Can journal abstracts alone be used for clinical decision making? BMJ. 2008;337:a3132.
  16. Barry HC, Ebell MH, Shaughnessy AF, Slawson DC, Nietzke F. Family physicians' use of medical abstracts to guide decision making: style or substance? J Am Board Fam Pract. 2001;14(6):437-442.
  17. Sensever FA, de Lucena Alves CP, Lima GDS, Loomans B, Opdam N, Pereira-Cenci T. Spin and reporting in systematic reviews with meta-analysis of randomized clinical trials in restorative dentistry. J Dent. 2022;125:104282.
  18. Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32(36):4120-4126.
  19. Tosatto D, Bonacina D, Signori A, Pellicciari L, Cecchi F, Cornaggia CM, et al. Spin of information and inconsistency between abstract and full text in RCTs investigating upper limb rehabilitation after stroke: An overview study. Restor Neurol Neurosci. 2022;40(3):195-207.
  20. Lazarus C, Haneef R, Ravaud P, Boutron I. Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol. 2015;15(1):85.
  21. Siex P, Nowlin W, Ottwell R, Arthur W, Checketts J, Thompson J, et al. Evaluation of spin in the abstracts of systematic reviews and meta-analyses covering surgical management, or quality of life after surgical management, of osteoarthritis of the knee. Osteoarthr Cartil Open. 2020;2(4):100121.
  22. Practice Committee of the American Society for Reproductive Medicine. Interpretation of clinical trial results: a committee opinion. Fertil Steril. 2020;113(2):295-304.
  23. Turner EL, Platt AC, Gallis JA, Tetreault K, Easter C, McKenzie JE, et al. Completeness of reporting and risks of overstating impact in cluster randomised trials: a systematic review. Lancet Glob Health. 2021;9(8):e1163-e1168.
Welcome: