Quality improvement (QI) aims to enhance the quality and delivery of health care through incremental changes. Examples of QI include introducing simple, low-risk initiatives like financial incentives and reminders to improve attendance at appointments or adherence to medication.

In practice, many health services make QI changes based on simple observations and anecdotal evidence. More robust evidence, such as clinical trials, is rarely used, mostly due to high cost and complexity.

In fields such as digital technology, more flexible, efficient trial designs have been in use for decades. ‘A/B’ testing, for example testing variations in a webpage, have often allowed for rapid, evidence-based outcomes to achieve companies’ aims. While there are obvious differences between this type of trial and those appropriate for health care, there is increasing evidence that these kinds of testing methods can be used for clinical research.

Researchers from the International Centre for Eye Health undertook a study to explore adaptive trial designs and their potential applications to drive QI in health service programmes. The team focused on outlining changes that lead to improvements, creating evidence with small population sizes and ensuring that potential trials could maintain patient safety.

Adaptive Trials

The team first aimed to define adaptive trials and describe examples of them.

Adaptive trials allow for the flexibility to modify and tailor the trial during its course. Modifications to the trial can include stopping early, adding or removing trial arms, adjusting randomisation or changing the sample size, among others.

For example, if the goal is to swiftly evaluate programmatic changes, stopping rules can be applied to terminate trials when early evidence demonstrates either efficacy or futility, allowing the change to be made within the programme and increasing efficiency immediately. Alternatively, trials can be monitored for benefit, quickly discontinuing ineffective or harmful adaptions or increasing beneficial ones.

The researchers recommend several considerations and ways to structure an adaptive trial design:

• Adaptive trials suit services where the duration between the intervention and the outcome are relatively short, for example attendance to appointments after receiving a reminder call
• a Bayesian trial design is recommended, which represents all available data up to the point and any prior beliefs, and allows diverse sources of information to be included
• well-structured interim analysis is key, avoiding premature analysis which ensures sufficient data is collected and any unexpected trends negated
• decision criteria on when to make changes to the trial should be incorporated, for instance if a variant in a programme is seen to have a 95% probability of being better
• early stopping can be built into the trial, when sufficient evidence has been collected to draw conclusions or continuing wouldn’t provide any further value. The stopping rules can be aligned with the trial objectives
• to test multiple variants, multiple trial arms can be created but then some dropped and the participants moved to another
• the tradeoff between sample size, accuracy, and bias should be managed carefully, ensuring that the benefits of adaptive design features outweigh their potential risks
• effect sizes may not need to be calculated, as positive changes among variants may be sufficient, depending on the service

The researchers also note a few limitations to the use of such trials, including still requiring considerable effort and resources (for setup and interim analysis) such as infrastructure and staffing. Similarly, simulations may not be feasible in some health services. Therefore, the viability of the trials should be considered during planning. Ethical review for each programme change may also be impractical and a hindrance in some settings.

Publication

Kim M, Prieto-Merino D, Nicholas J et al. Evidence‐Based Approaches to Quality Improvement: A Narrative Review of Integrating Bayesian Adaptive Trials Into Health Services. J Eval Clin Pract. July 2025. https://doi.org/10.1111/jep.70197