OR WAIT null SECS
Fig. 1. Quality, time and cost elements of major research approaches.
Observational research—evaluating outcomes in patients in a naturalistic rather than an experimental setting–has the potential to answer many research questions. But in order to do so effectively, researchers must select “the right research approach for the right question.” Doing so requires thorough evaluation of feasibility of delivery, specialized operational approaches with an understanding of pitfalls to avoid, and an upfront plan to communicate the results to stakeholders for whom the results are relevant.
The starting point for selecting “right research/right question” is to refine the question into something specific that can be transformed into a viable study. The research question represents the uncertainty that the sponsor wants to resolve by performing the study. Sometimes, the research question is confused with the research interest, which is a broader concept. To refine the research question requires understanding the strategic objectives of any research, and reviewing the current body of literature and any ongoing research to ensure that the question is relevant, specific and novel.
The right research approach is determined by practical considerations such as time and budget, ethical considerations and the strength of evidence required to meet healthcare stakeholder needs. Various study design options exist, with varying strengths and limitations that can be evaluated against these considerations. For example, a randomized controlled trial (RCT), while providing the highest strength of evidence, might cost too much and take too long. There may be ethical questions concerning randomization. A cross-sectional study conducted in a database might be quick and cheap, but might not provide sufficient certainty of evidence for decision-making. All of these factors should be weighed against each other to determine the right research approach.
The feasibility of a study is a key ingredient in its ability to generate the right quality evidence in the right time at the right cost. There are several research approaches to choose from (Fig. 1). The quality of data generated via each research approach varies, as does the time and cost to generate that evidence. In general, the time needed to gather data prospectively is longer than approaches using real-world data sources, and the cost of research can vary from less than $100,000 to tens of millions of dollars. RCTs and prospective observational cohort studies provide the most certainty of evidence for decision-makers, but take longest and cost most. In contrast, research using routine data sources—while relatively quick and cost-effective–may not be considered sufficient by some decision-makers.
The various types of research approaches—such as prospective observational cohort studies and patient registries, chart reviews, database studies, and hybrid designs–have different features, advantages and disadvantages.
For example, features of prospective observational cohort studies include the fact that they are non-randomized and non-interventional, and they monitor cohorts over time. Patient registries may focus on disease, product or exposure. Advantages include the fact that prospective observational cohort studies and patient registries can:
Disadvantages of prospective observational cohort studies and patient registries include the fact that they are prone to bias and confounding. Optimal design, conduct and analysis are critical for producing strong evidence.
Chart reviews and database studies use existing clinical or administrative data present in patient paper charts or electronic databases. Advantages of using existing data sources include the ability to assess benefits and harms across an extremely large population, and the fact that this is a cost- and time-efficient approach compared to prospective, longitudinal research. Disadvantages are:
Fig. 2. Advantages and disadvantages of hybrid designs.
Hybrid designs can capitalize on the speed, cost-efficiency and strengths of different methods, using a combination of primary and secondary data to shorten time and lower costs (Fig. 2). With this approach, sub-studies—focusing on health economic data or patient reported outcomes–are conducted alongside larger prospective studies, providing added value.
For all types of study design, strategies to ensure a feasible study include:
Specialized operational delivery
Importantly, observational research requires a different operational approach from that used in experimental clinical research. It is nonetheless common for companies to use clinical development teams to conduct observational research. Observational research requires specialized people, processes and technology in four major areas:
In practice, observational research requires the full integration of strategy, science and operations throughout the lifecycle of every study. Regulatory and commercial teams are needed to provide global and local insights into evidentiary needs and the tools for evidence dissemination. Epidemiologists and outcomes researchers are involved in writing the study protocol; reviewing data management and biostatistical deliverables; leading site and patient sampling; reviewing data for missing elements, bias and confounding; and interpreting, analyzing and reporting data throughout the study. Epidemiological input is needed to all aspects of operational delivery to minimize bias and confounding. Dedicated observational research operational teams work on specialized observational study delivery processes, and leveraging fit-for-purpose data capture technology ensure high-quality delivery.
Risks to avoid
Several pitfalls may be encountered during the design and implementation of observational studies. First, researchers may mistakenly assume that such studies are “simple” to design and conduct working off a light-touch clinical trial delivery template. This is far from true. Indeed, applying clinical trial operational approaches may lead to a lower quality study than appropriate observational study approaches.
Operational standards for the design, conduct and reporting of observational studies have not been harmonized. This means that each nation governs observational studies to its own standards and requirements. Moreover, specific standards may not exist in some countries, even today, and interpretation of the requirements may vary within countries. A need for representative sites brings the possibility of an extended site selection process, and the likelihood of working with many research-naïve sites. Such sites require training and monitoring approaches tailored to observational research.
The lack of randomization increases the risk of bias and confounding of results, requiring specialized approaches to design and analysis. The ‘real world’ provides a research environment that is less controlled than the gold standard randomized clinical trial, bringing with it specific design and operational challenges.
Second, there may be the temptation to omit robust pre-study feasibility to inform best practice design and conduct. However, working in the real world requires an understanding of patient pathways, product launch dates and other factors relevant to usual care. Understanding these factors is critical to ensuring robust study plans.
Third, researchers may fail to invest in extracting the value from studies through analyses, reports and publications. The value of the study resides in the results that are reported and published. For these to demonstrate value to decision-makers, the results must live far beyond the life of the study. A desire to meet study timelines can blind teams to strategic goals, leading them to invest less time in study outputs (analyses, reports and publications) and more time in study inputs (such as first patient in, first patient out or last site closed).
Communication of results to stakeholders
The final element of observational research design involves creating a comprehensive communication plan with specific strategies designed for each stakeholder involved. There are four critical success factors to maximize the impact of the study. First, identify appropriate target journals for publication, enabling the research to be tailored to meet their needs. Many journals provide guidance on what they look for. For example, the British Medical Journal lists factors that make study publication less likely, including: unethical research; an insufficiently important or clear research question; an insufficiently original, relevant or important overall message; an inappropriate study design for the research question; suboptimal study design; low internal validity/robustness of the study or external validity/generalizability of the study; or the study not being publicly registered. This highlights the importance of the second success factor, which is to register the observational study.
The third success factor is having a study-focused publication plan, including clarity around criteria for authorship during site selection, and a plan for early analyses to maximize the impact of the study from the start. Many long observational studies include interim analyses to disseminate results, and these should be coordinated with key journal and conference abstract submission dates. The process should be transparent with respect to the types of publications expected, and cross-company collaboration should be encouraged to optimize internal communication of results.
Engaging the study physicians is the fourth success factor. Payment for participation in observational studies can be low, and for most physicians, research does not play a major role in their day-to-day work. Additional approaches can boost physician motivation to participate in such studies, including:
Real-world evidence generation
Successfully selecting ‘the right research approach for the right question’ is critical for best practice real-world evidence generation. Robust feasibility, specialized operational delivery approaches, understanding and mitigating risks are critical. Engaging stakeholders through creation of a comprehensive communications and publication plan and establishing an advisory board will maximize the value of the study back to society. Delivering the right quality of evidence in the right time at the right cost will aid healthcare decision-making and ultimately improve patient care.
ABOUT THE AUTHOR
Louise Parmenter, PhD, is Global Head of Operations, Epidemiology & Outcomes Research, Real-World & Late Phase Research, Quintiles. She is a specialist in real-world and late-phase research with 23 years of related global operational and strategic experience. She is currently responsible for a team of epidemiologists and outcomes researchers primarily based in the United States, with growing teams in Europe and Asia. Dr. Parmenter holds a BSc in Physiology & Biochemistry and a PhD in Neurophysiology from the University of Southampton, UK. She is completing an MSc in Epidemiology from the London School of Hygiene and Tropical Medicine, UK.