By Sheri Ann Strite and Michael E. Stuart, MD
A just-published article in the Journal of Evaluation in Clinical Practice titled "How Evidence-Based Medicine is Failing Due to Biased Trials and Selective Publication" has issued a rallying cry, asking all evidence-based medicine (EBM) science evaluators, such as ourselves, to "recognize the high risk of bias when the producers of evidence have a vested interest in the results." The authors have broadly recommended that evidence rating schemes be "formally modified" so that research with "conflict of interest bias" be "explicitly downgraded in value." They make it clear in their article that this means "funding source" and, further, it is imminently clear that the authors mean industry-sponsored research. They claim that, "…evidence for [certain] flaws is clearest in industry-funded studies."
The authors state, "…this is a necessary step if EBM critical appraisal is to remain credible." Is it? We do not agree. Consider, for example, a hypothetical three-tiered system that rates evidence as 1 = reliable evidence, 2 = evidence of possible to uncertain reliability and 3 = insufficient evidence or lacking direct evidence. If such a ranking scale has only three choices, this would mean that any reports of industry-sponsored evidence—even if of superlative quality with meaningful clinical outcomes — would, at best, achieve a ranking of 2 and potentially be disregarded by payers and prescribers. How does this serve credibility? How can this truly be an evidence-based approach?
We think that is it worth being attentive to industry involvement when conducting a critical appraisal of health care evidence; however, everyone involved in research should be assumed to have a bias in favor of what they are investigating. We believe that the evidence should speak for itself regardless of industry involvement. Quality science should be the goal of us all. This requires, at a minimum, optimal study design with outcomes that are meaningful to end users, reasonable populations for study, outstanding study execution and performance outcomes—such as successful blinding, no differences in the study groups except for the agent under study, high adherence, low risk for attrition bias and so on—and effective data collection and measurement, etc., along with appropriate reporting and results that are of clinical utility.
A big gap we often see lies in reporting and may result in studies being rejected by payers and prescribers. Transparency and clarity in reporting is paramount and must be done in ways that are meaningful to end users—meaning all "customers of medical science." All too often industry (especially scientific staff) limit their attention to the Food and Drug Administration (FDA), not fully comprehending that their scientific customers belong to a much, much broader target group which includes payers, health care systems, individual clinicians involved in health care decision-making, patients and people like ourselves who are guideline developers, evaluators of medical science, content providers and investigators such as meta-analysts. If the recommendations of Every-Palmer et al. are acted upon by various groups and if quality science will be downgraded simply because of funding or industry involvement, we believe that our health care professionals and patients will lose.
So what is the remedy? We believe that everyone working in industry who is involved in generating or communicating information about research, or who may face a research comment or question coming from a customer, or who may be instrumental in facilitating such a discussion should possess basic skills in the key principles of evidence-based critical appraisal of science. These essentials are not hard to acquire. Such staff should be knowledgeable about what constitutes reliable research, be aware of the strengths and weaknesses of their and their competitors' research and know how to effectively communicate about all of these issues. This is necessary to guard against draconian measures such as those proposed above, to repair industry's reputation and work to restore trust. This is also necessary to foster a good relationship with customers of medical science to help establish a common language and meaningful common ground, with the ultimate goal of best serving patients. And, finally, this is necessary to best conduct and to represent research of value to patients.
Our prescription, if you will, is a roadmap for industry to best engage with customers over evidence. Our roadmap consists of the following:
- Research: High Quality & Clinically Useful Evidence
Conduct high-quality research on appropriate populations using good comparators and selecting useful endpoints that are meaningful to customers. Strive for solid research design and good execution, working to maximize good study performance outcomes and utilizing effective measurement methods.
- Report: Clarity & Transparency
Provide transparent reporting that is meaningful to customers. Understand if there are any gaps between your research and its reporting and take steps to make needed information accessible.
- Appraise: Knowledge About The Evidence
Apply critical appraisal skills to relevant research. This includes your own research and that of your competitors.
- Prepare to Communicate About the Evidence: Know the Evidence Issues & Have Communication Aids
Prepare for effective communications about evidence using all of the above. Be familiar with critically appraised information and typical issues raised by customers. Have effective communication pieces, resources and options including education methods.
- Prepare to Connect Over the Evidence: Know the Customer to Interact with the Customer
Apply skills for enhancing connection over the evidence in your interactions with your specific customer in mind. This includes being able to assess your customer's climate for medical decision-making and their evidentiary knowledge and approach both organizationally and individually within the organization. This also includes being prepared to bridge gaps in evidentiary knowledge and skills. Have effective methods for addressing customers' questions.
Where quality industry-funded medical science exists, staff in industry roles—including, but not limited to, scientists, health outcomes research staff, medical science liaisons, medical communications staffs, brand teams, account managers and sales representatives—who are effectively trained in these areas and who put these attributes into practice, will have the potential to bring value to their companies, payers, clinicians and patients as well. Such staff can help nurture a better and more accepting environment for high quality industry-sponsored scientific work. Clearly, we all have benefited from good science from industry. Companies promulgating and acting upon these principles and practices can help mitigate efforts that run counter to a true evidence-based approach and threaten potentially quality care for patients.
Sheri Ann Strite and Michael E. Stuart M.D., of Delfini Group, LLC are the authors of EBM Guide For Scientists to Sales Reps—Medical Evidence in the World of Payers & Health Care Systems: A Handbook For Industry to Better Understand, Communicate and Navigate These Worlds. For more information, visit www.delfini.org.