Pragmatic clinical trials offer real-world insights into the effectiveness of a treatment in the context in which it’s expected to be used. This is a valuable difference from explanatory trials, which typically measure much more controlled and specific biological outcomes.
The difference is significant, yet making the adjustment from one to the other comes with the need to unlearn a lot of instincts in research. In this article, we’re going to look at what makes these trials so different and challenging, and follow up with ten best practices to minimize those challenges. Before that, let’s cover some definitions.
The Definition and Significance of Pragmatic Clinical Trials
The definition of pragmatic research came as a way of differentiating trials aiming to evaluate the efficacy of an intervention in a controlled setting (explanatory), from those designed to test the effectiveness of an intervention in a broader context of routine clinical practice (pragmatic).
A pragmatic trial is designed to test interventions in a real-world setting, across the full spectrum of clinical contexts. It’s a study of the applicability and generalizability of an intervention and aims to answer the question of whether the treatment works in reality.
This is a test in a routine clinical setting, usually as an evaluation against current competing treatments, either in the same or different class as the intervention being tested. The outcomes measured in pragmatic trials are mostly patient-centered and exist on a wide spectrum, which is one of the key differences between these studies and explanatory ones, as we’ll discuss in the next section.
Pragmatic trials are a hunt for maximal heterogeneity in aspects such as clinical settings and patients, and this requires a large enough sample size to detect small effects while keeping the design simple enough to plan, perform, and follow up effectively.
Essentially, pragmatic trials are designed to answer questions that decision-makers are asking; those relating to the comparative effectiveness of treatments in regular practice. Outside of cost-effectiveness, this may be the single most important factor in the widespread adoption of treatment, and studies that answer these questions help policymakers and healthcare providers make informed decisions about patient care.
Pragmatic Trials Vs Explanatory Trials: The Difference Between Effectiveness and Efficacy
Finding out whether a treatment is efficacious is the main goal of an explanatory study. The performance of an intervention under controlled circumstances is the major focus of this kind of trial, and treatment is usually matched against placebo as the control condition. In contrast, a pragmatic study looks at how effective the treatment will be under real-world conditions. The control for this will generally be the usual care, and the novel intervention will be compared with this in a clinical setting.+
In practice, most studies are neither one nor the other, and in fact, lie somewhere along the spectrum between the two. This makes study design a matter of finding a useful balance in which the outcomes are most reliable and relevant to the goal of the study.
The outcomes measured are a lot wider with pragmatic studies, and they’re going to be primarily patient-centered. This differs from studies leaning toward the explanatory end of the spectrum, which will be looking at more physiological or clinical markers to evaluate success.
These trials will be designed differently, with explanatory trials typically being more specialized, smaller, and more complex, and pragmatic trials having a much larger sample size, yet being simpler in design. This is due to the former focusing on homogeneity, and the latter needing to overcome the diluted effects of bias from a more heterogeneous approach.
Ways of Measuring the Continuum
Since the line between the two extremes of this continuum is not set, it can be useful to have some criteria to assess the explanatory/pragmatic nature of a trial. One way to do this would be to use a pragmatic-explanatory continuum indicator (PRECIS), as we’ve laid out below. Each of the following ten domains can be ranked from 1-5, on a scale between explanatory and pragmatic, respectively.
We’ve listed the domains along with a description of what would elicit a score of 5 for a fully pragmatic trial:
- Eligibility criteria – In a fully pragmatic trial, there would be no inclusion or exclusion criteria.
- Flexibility of the experimental intervention – Pragmatic trials do not require practitioners to be constricted by guidelines on how to apply the intervention being tested.
- Practitioner expertise (experimental) – In a pragmatic trial, the experimental intervention is applied by all practitioners, thus covering the full spectrum of clinical settings.
- Flexibility of the comparison intervention – The best alternative treatments are used for comparison with no restrictions on their application.
- Practitioner expertise (comparison) – Again, in a pragmatic setting, the comparative treatment is applied by all practitioners, covering the full spectrum of clinical settings.
- Follow-up intensity – No formal follow-up sections are required in pragmatic trials.
- Outcomes – The primary patient and clinical outcomes are not overly technical, and so have a lower training requirement to assess.
- Participant Compliance – Pragmatic trials don’t need plans to improve or alter compliance for the experimental or the comparative treatment.
- Practitioner adherence – They don’t need any special strategy to motivate the practitioner’s adherence to the trial’s protocol.
- Primary outcomes – The analysis includes all participants in an intention-to-treat fashion.
The goal of this indicator is to show and quantify the continuum to give a summary for systematic reviews. This should give a clearer picture of the differences between the two study types and suggest some of the key benefits of each, but navigating this balance brings with it numerous challenges when it comes to various elements of study design.
Pragmatic Study Challenges and Solutions
Pragmatic clinical trials need to be able to find participants who resemble those who would receive the intervention being tested in normal conditions.
Assuming the treatment became normal care, this means finding volunteers who would already be eligible for it, which tends to reduce the available pool of volunteers. Add to this competition from other trials, and the pool gets even smaller. Sample sizes need to be higher for pragmatic studies, so recruitment is one of the first challenges they commonly face.
Informed Consent is another area in which roadblocks present themselves. In some cases, waiving this requirement is possible, where it’s necessary to guarantee that everyone eligible is included. This is sometimes the case in emergency treatments, or in cases where the trial neither interferes with normal care nor adds any nonstandard activities or data collection. In other low-risk contexts, consent isn’t always required, but for more high-risk treatments the need for informed consent can complicate things.
Investigator recruitment can sometimes be a challenge in pragmatic studies. Since healthcare providers are often too busy, inexperienced, or disinterested in research to take on investigator responsibilities, access to the heterogeneity of practice required in this type of research is limited.
And there are other ways in which chasing heterogeneity can be a problem. Reaching into communities that will not represent the targeted users of the intervention (for example, trialing a treatment designed for low-income communities in a well-developed hospital) will not produce useful outcomes.
While this may seem obvious, it does represent an intellectual conundrum of whether to conduct studies in contexts that reflect current conditions, or in contexts that are anticipated in the future in the specialist area.
Many of the challenges faced in pragmatic research are similar to, or extensions of, those faced in clinical trials in general. Still, they face some unique obstacles along with this. By following some common practices, it’s possible to mitigate a lot of the risks and overcome many of the hurdles expected.
Pragmatic Trial Best Practices
New strategies are being proposed every year when it comes to the design, planning, recruitment, and conduct of pragmatic research. Many of these listed apply to trials as a whole but have particular benefits to those challenges mentioned above.
1. Investigator recruitment
Investigators need to be better incentivized to provide more efficient clinical services since the research elements of their responsibilities will necessarily be in addition to the day-to-day clinical care they are expected to take part in.
Clinical networks and research communities should be fostered and developed to increase exposure to, and engagement with, research as a whole among clinicians. Another incentive would be to credit healthcare workers as key contributors to the research.
These approaches have been demonstrated to improve the recruitment and retention of investigators and work better than to more common approach of developing a network for a specific trial which is then not reused in any subsequent research.
2. Site selection and patient recruitment
Centers to be used in a pragmatic trial should be selected from a range of patient care settings that resemble those in the target population, to whom trial designers expect to apply the findings of the study. Where study centers are limited, heterogeneity will take a hit.
One way to approach this is to lower the number of patients expected from a single site and increase the number of sites. For the recruitment of eligible patients, it may be necessary to relax the patient selection criteria. Eligibility criteria should not exclude people based on the likeliness of their response to treatment, or those who are deemed unlikely to complete the follow-up.
The target population should be represented as accurately as possible for the maximum amount of applicability of the results, but exclusion and inclusion criteria should typically be more flexible in order to accomplish this.
3. Avoid placebos
In comparative effectiveness research, control interventions are usually in the form of another widely-used active treatment. Controls should seldom be a placebo since placebos are not a form of usual care and this could alter the outcomes of the control group.
In addition to this, a placebo could contribute to an undesirable homogeneity among control group patients by affecting their behavior, particularly in the context of self-treatments that they may have otherwise pursued.
While this might seem straightforward, it throws up regulatory challenges, requiring a deeper level of communication between trial planners and regulatory bodies to come to an arrangement that will allow for the avoidance of placebos in the trial design while maintaining compliance.
4. Plan for non-compliance
While compliance is heavily regulated and monitored in clinical trials, the clinical setting is very different. When designing the sample size requirements for the study it’s important to take into account real-world levels of compliance to compensate for contamination that will likely occur when comparing usual-care interventions without blinding.
Sample size alone won’t be enough to address the inherent bias that will come from contamination, and neither will it address the increase in variability and decrease in sample size estimates. Address this issue with cluster randomization to limit contamination, but be aware that this itself may introduce a bias from the recruitment of participants if it occurs after randomization, which may be a larger issue.
5. Do not enforce compliance
As an extension of the above, it’s important to understand the differences in compliance within an explanatory and pragmatic context. Efforts to promote compliance to a higher degree than usual care would provide will create undesirable effects in a pragmatic trial since the focus is on the reality of the situation in which the treatment will be handled.
Address this by altering the behaviors of patients and carers no more than is necessary to administer the intervention as it would be expected in the future state of care. This practice can be counter-intuitive to experienced researchers and sponsors who are familiar with explanatory trials and may take some adjusting to.
6. Don’t be afraid of co-interventions
This is another way in which the study may trigger some discomfort in researchers more familiar with explanatory trials. The nature of confounding variables is seemingly flipped on its head in a pragmatic setting, allowing for additional treatments to be well within the acceptable range of events, as long as they are representative of usual care.
7. Compensate for missing data
When using the intent-to-treat analysis commonly useful in pragmatic research, missing data is an important issue to consider. This is something that needs to be accounted for in the study design, particularly in the statistical analysis plan.
Missing data is typically more prevalent in pragmatic studies, but preparing for it can be as simple as scheduling the right statistical methods to handle it, such as multiple imputation or covariate adjustment.
8. Focus on pragmatism over ancillary studies
It can be tempting to make the most of the data coming in from a pragmatic study to learn more about the assessed intervention. This usually comes from a series of process analyses to explore how the intervention was implemented. This can be valuable extra information, but it can incentivize trial design to alter either data collection or patient/investigator behavior with these post-study analyses in mind.
It’s important to consider any ancillary analyses as secondary to the main study objective, and ensure that nothing gets in the way of keeping the environment authentic and reflective of the real-care contexts expected for the treatment.
9. Outcome
Overall, remember that the pragmatic study is attempted as a way of answering questions that lead to decision-making by patients, caregivers, and policy-makers. This means designing outcomes that are routinely considered significant to these stakeholders.
The primary outcomes should correspond to those which are routinely assessed in usual care and are clinically important, with the mind that they should be used to affect policy on a larger scale.
10. Ethical and regulatory considerations
Ethical principles cannot be ignored in pragmatic trials any more than they must be in explanatory ones. While there may be some contexts where consent requirements can be waived, these are after legitimate and controlled observation and assessment of the contexts, and should not give the impression that ethical requirements are any less stringent.
Waivers have been granted in contexts in which written informed consent may jeopardize the science in a trial, but usually after finding another solution to informed consent such as integrated consent, in which a healthcare provider gets verbal consent from a patient during the usual course of care delivery. In this instance, the same goal of consent is accomplished with less intrusion.
Where heterogeneity is threatened by informed consent requirements, there may be the need for trial designers to collaborate on the proper balance between protection and trial power with ethicists and research ethics committees. In the end, ethical issues will always need to come before scientific endeavors in the hierarchy of significance.
Conclusion
Trials typically fall somewhere on a continuum between pragmatic and explanatory. Some need to be nudged closer to one end than the other, and others will naturally fall into one of the two categories. Because of the nature of the scientific method and the context of healthcare involved in clinical trials, very few are exclusively pragmatic, though it’s important to identify the relevant domains in which a trial can be placed for review.
The benefits of a pragmatic trial come from the real-world data in the context of usual care, and demonstrate the effectiveness of a treatment as it will likely be administered. This brings what may appear to be countless confounding variables into the mix, yet represent the true nature of the treatment’s application. Understanding and adjusting to this paradigm shift is the key to making powerful pragmatic research work.
