Publications

OBJECTIVES: To assess how often new treatments for childhood cancer assessed in phase III randomised trials are superior or inferior to standard treatments and whether the pattern of successes and failures in new treatments is consistent with uncertainty being the ethical basis for enrolling patients in such trials.

DESIGN: Observational study.

SETTING: Phase III randomised controlled trials carried out under the aegis of the Children's Oncology Group between 1955 and 1997, regardless of whether they were published.

MAIN OUTCOME MEASURES: Overall survival, event free survival, and treatment related mortality.

RESULTS: 126 trials were included, involving 152 comparisons and 36,567 patients. The odds ratio for overall survival with experimental treatments was 0.96 (99% confidence interval 0.89 to 1.03), indicating that new treatments are as likely to be inferior as they are to be superior to standard treatments. This result was not affected by publication bias, methodological quality, treatment type, disease, or comparator.

CONCLUSIONS: New treatments in childhood cancer tested in randomised controlled trials are, on average, as likely to be inferior as they are to be superior to standard treatments, confirming that the uncertainty principle has been operating.

CONTEXT: The superiority of innovative over standard treatments is not known. To describe accurately the outcomes of innovations that are tested in randomized controlled trials (RCTs) 3 factors have to be considered: publication rate, quality of trials, and the choice of the adequate comparator intervention.

OBJECTIVE: To determine the success rate of innovative treatments by assessing preferences between experimental and standard treatments according to original investigators' conclusions, determining the proportion of RCTs that achieved primary outcomes' statistical significance, and performing meta-analysis to examine if the summary point estimate favored innovative vs standard treatments.

DATA SOURCES: Randomized controlled trials conducted by the Radiation Therapy Oncology Group (RTOG).

STUDY SELECTION: All completed phase 3 trials conducted by the RTOG since its creation in 1968 until 2002. For multiple publications of the same study, we used the one with the most complete primary outcomes and with the longest follow-up information.

DATA EXTRACTION: We used the US National Cancer Institute definition of completed studies to determine the publication rate. We extracted data related to publication status, methodological quality, and treatment comparisons. One investigator extracted the data from all studies and 2 independent investigators extracted randomly about 50% of the data. Disagreements were resolved by consensus during a meeting.

DATA SYNTHESIS: Data on 12,734 patients from 57 trials were evaluated. The publication rate was 95%. The quality of trials was high. We found no evidence of inappropriateness of the choice of comparator. Although the investigators judged that standard treatments were preferred in 71% of the comparisons, when data were meta-analyzed innovations were as likely as standard treatments to be successful (odds ratio for survival, 1.01; 99% confidence interval, 0.96-1.07; P = .5). In contrast, treatment-related mortality was worse with innovations (odds ratio, 1.76; 99% confidence interval, 1.01-3.07; P = .008). We found no predictable pattern of treatment successes in oncology: sometimes innovative treatments are better than the standard ones and vice versa; in most cases there were no substantive differences between experimental and conventional treatments.

CONCLUSION: The finding that the results in individual trials cannot be predicted in advance indicates that the system and rationale for RCTs is well preserved and that successful interventions can only be identified after an RCT is completed.

OBJECTIVE: To determine whether poor reporting of methods in randomised controlled trials reflects on poor methods.

DESIGN: Observational study.

SETTING: Reports of randomised controlled trials conducted by the Radiation Therapy Oncology Group since its establishment in 1968.

PARTICIPANTS: The Radiation Therapy Oncology Group. Outcome measures Content of reports compared with the design features described in the protocols for all randomised controlled trials.

RESULTS: The methodological quality of 56 randomised controlled trials was better than reported. Adequate allocation concealment was achieved in all trials but reported in only 42% of papers. An intention to treat analysis was done in 83% of trials but reported in only 69% of papers. The sample size calculation was performed in 76% of the studies, but reported in only 16% of papers. End points were clearly defined and alpha and beta errors were prespecified in 76% and 74% of the trials, respectively, but only reported in 10% of the papers. The one exception was the description of drop outs, where the frequency of reporting was similar to that contained in the original statistical files of the Radiation Therapy Oncology Group.

CONCLUSIONS: The reporting of methodological aspects of randomised controlled trials does not necessarily reflect the conduct of the trial. Reviewing research protocols and contacting trialists for more information may improve quality assessment.

A 77-year-old man presented with sudden-onset epigastric pain and bilious vomiting following a light breakfast. Vagotomy and gastrojejunostomy for bleeding duodenal ulcer had been done 22 years ago. Barium meal study suggested jejunogastric intussusception. At laparotomy, a retrograde type II jejunogastric intussusception was confirmed and managed by reduction of the intussusception, disconnection of gastrojejunostomy and resection of the jejunum. Postoperative recovery was uneventful.

BACKGROUND: Increasingly, clinicians advocate the use of nonmyeloablative allogeneic stem-cell transplants (NM-allo-SCTs, "mini-transplants") to manage hematologic malignancies. They hypothesize that NM-allo-SCT is equally efficacious to standard allo-SCT but produces less regimen-related toxicity.

METHODS: To analyze available evidence on the benefits and harms of "mini-transplants," we identified 23 manuscripts, 1 abstract, and 1 letter that reported the outcome of mini-transplants in hematologic malignancies.

RESULTS: Data were compiled on 603 treated patients, with 118 transplants using stem cells from matched unrelated donors. All studies were small prospective case series, and most lacked concurrent or historical controls. Outcomes of interest were not uniformly reported. The studies were heterogeneous and used different patient selection criteria, conditioning regimens, and timing of transplant with respect to disease status. The transplant-related mortality rate was 32%, the relapse rate was 15%, and toxicities included acute and chronic graft-vs-host disease and veno-occlusive disease. The aggregate rate of complete remission was 45%. Survival at 1 year or longer ranged from 30% to 60% at 1 to 5 years of follow-up. All studies reported successful chimerism.

CONCLUSIONS: Disease-specific studies with longer follow-up are needed to evaluate this potentially promising therapy.

We have done a systematic review of all randomised studies in myeloma, identified through a comprehensive search. Our aim was to investigate and critically examine the effects of various treatment modalities on outcome in patients with multiple myeloma and address 22 specific clinical questions in the management of this disease. As a result of our analysis we identified two therapeutic advances in the management of myeloma that, according to the evidence, are most important for improving outcome. These advances were: introduction of high dose chemotherapy, which appears to be superior to conventional chemotherapy, and the use of bisphosphonates, which decrease the probability of pathological vertebral fractures. However, the overall quality of the body of evidence for myeloma management was poor. Many trials were done with small sample sizes, and did not include reporting power analysis. The majority of studies had inadequate allocation concealment, and few were analysed according to intention to treat principle. We conclude that the quality of total evidence supporting treatment recommendations in myeloma is modest at best and has an ample scope for improvement.