Do the majority of clinical trials provide no meaningful evidence?

A recent review of data on ClinicalTrials.gov (2007 - 2010) reported that clinical trials are falling short of producing high-quality evidence needed to guide medical decision–making.

The analysis, published on May 2 in the Journal of the American Medical Association (JAMA), reported that ClinicalTrials.gov is dominated by small trials (<100 subjects) and that there is significant heterogeneity in methodical approaches, including randomising, blinding and the use of data monitoring committees. The authors concluded: "Our analysis raises questions about the best methods for generating evidence, as well as the capacity of the clinical trials enterprise to supply sufficient amounts of high-quality evidence to ensure confidence in guideline recommendations."

However many of the observations have been noted previously and, as the authors state, "most likely represent variations in appropriate approaches for particular diseases". For example, the analysis showed heterogeneity in median trial size, with cardiovascular trials tending to be twice as large as those in oncology and trials in mental health falling in the middle. There were also differences in the use of randomisation, blinding and data monitoring committees, critical issues often used to judge the quality of evidence for medical decisions in clinical practice guidelines and systematic overviews. But this too is perhaps an unsurprising observation considering the broad scope of the data set. It is not surprising that there are a significant number of ‘pilot’ or early phase studies that are more likely to be intended to establish safety in small numbers of healthy subjects following conservative study designs rather than guide future medical decision-making.

It is clear from this analysis that review of the data held on ClinicalTrials.gov is likely to encourage evolution of the database itself, the information it records and how we use it to reflect on our current practices. An analysis-ready copy of the ClinicalTrials.gov dataset has been made freely available (here). Research groups across various therapeutic disciplines are believed to be interrogating the data to better understand how the number and characteristics of clinical trials match the perceived needs of the research communities.