Statistics influence study results
Two studies assess the impact that pre-publication statistical analysis of data can have on published results. On the one hand, the results could be very different if different possible statistical methods are applied to the same raw data, as Canadian scientists led by Thérèse Stukel [1] show.
Researchers from Denmark also found that the involvement of statisticians with potential conflicts of interest is not publicized in many publications. According to Peter Gøtzsche and colleagues [2], commissioned evaluators, who are often financed by parties economically interested in the results of the study, have opaque influence.
The Danish researchers led by Gøtzsche from the Nordic Cochrane Center retrospectively analyzed 44 studies funded by various pharmaceutical companies and published in various medical journals after peer review. They compared the study protocols agreed upon with the ethics committee before data collection began with later publications.
In 31 cases, the statisticians responsible for the actual data analysis were not named later. Furthermore, none of the 44 papers made it clear whether the physicians commissioned with the data collection, the statisticians active after the data collection, or the sponsor was responsible for the planning of the study or the type of analysis.
"The involvement of ghostwriters is very common in industry-funded randomized trials," Gøtzsche and co conclude, suggesting that the practice "serves commercial interests."They demand measures that make all those involved in a publication public.
Stukel and her colleagues from the Institute for Clinical Evaluative Sciences in Toronto looked at the data from a large clinical study that followed 122,124 patients after a heart attack from 1994 onwards. The study was intended to clarify whether patients who had a cardiac catheter in place carried a lower risk of death. Factors that could influence the result had to be statistically taken into account in the evaluation, such as the fact that the 73,000 or so members of the patient group with a catheter were on average younger and had suffered less serious attacks than the members of the group that had not been operated on.
The various methods that can be used to calculate such distorting factors sometimes lead to different results, the authors write. The calculated relative decrease in the risk of death varied between 16 and 50 percent.
The result affects many medical studies, since in clinical practice, for medical and ethical reasons, test groups are often not randomized and randomly assembled. They are therefore rarely representative and cannot be evaluated statistically without taking into account possible falsifying factors. If studies are not randomized, the researchers state, then their results must be analyzed and interpreted with great care. According to Stukel, this applies in particular when clinical recommendations are derived from the results of the study. (yo)
