B-FREE – BiasFREEvidence

Appropriate and efficient decisions about the introduction and reimbursement of health technologies are critical for the sustainability of healthcare systems. Health technology assessment (HTA) is a multidisciplinary process and is widely used to inform decisions about the adequate use of technologies. The relative effects of different health technologies on health outcomes are key inputs for HTA processes. Gold standard estimates of these relative effects frequently come from systematic reviews and meta-analyses of randomized controlled trials (RCTs).

To ensure that optimal decisions are made, it is therefore essential that trustworthy inputs are available for HTA evaluations. However, a substantial number of trials with significant shortcomings in their conduct are often found in the medical literature and included in systematic reviews. Several studies have reported empirical evidence that the inclusion of such trials in a meta-analysis may introduce bias into the results.

When performing a meta-analysis, investigators are therefore confronted with the decision of restricting the analysis only to trials at low risk of bias (RoB), which may ignore a significant amount of the evidence and generate a result that is unbiased but imprecise, or adopt an “all available evidence” approach, which will yield higher precision at the expense of an increased bias. The choice of the latter option may deliver a spuriously precise effect estimate for HTA evaluations and generate inappropriate decisions.

Recently, Bayesian meta-analytical models were developed that make it possible to adjust and down-weight the effect estimates from studies at high RoB, through the incorporation of prior distributions for bias-related parameters estimated from meta-epidemiological data. These models are a cutting-edge methodology that could offer a more favourable compromise between bias and precision than simply restricting the analysis to low risk of bias studies.

Given its promising performance in generating more precise, valid, and reliable results than standard meta-analysis, this approach was recently included in the Cochrane Handbook for Systematic Reviews of Interventions as a potential strategy to eliminate the bias from meta-analysis results. However, at the present state of knowledge, the Cochrane Collaboration cannot yet recommend bias-adjustment using empirically based priors for routine use, given its still experimental nature. Issues about the lack of statistical power of meta-epidemiological studies, the assumption that the meta-analysis to be adjusted is exchangeable with those in the meta-epidemiological dataset and, the sensitivity of results to model and prior choice need to be clarified before this method can be recommended for routine use in HTA.

Investigating if and how this new methodology could be confidently applied in routine evidence synthesis and be advantageous to the process of decisionmaking is therefore of extreme relevance and importance in view of an improved and more efficient resource allocation that strengthens the sustainability of healthcare systems.

The overall goal of this project is to provide empirical evidence on whether bias-adjusted meta-analysis using empirically based priors is likely to be feasible, reliable, and valid to be confidently and routinely used in the setting of evidence synthesis of RCTs so that more trustworthy inputs could be available to HTA evaluations. We will address questions related to sample size requirements of meta-epidemiological datasets, context-dependency of bias, sensitivity to model inputs, and effects of bias-adjustment compared to restricting the meta-analysis to low RoB trials. These insights are critical to inform HTA stakeholders on the suitability of this innovative methodology to the process of evidence synthesis in the setting of decision-making in health care and HTA.

With this end in view, three tasks have been specified. The first one aims to address the feasibility of the method in terms of sample size requirements where through simulation studies we will explore the number of meta-analyses required for the analysis of meta-epidemiological data. The second task aims to address the validity of the methodology by assessing if prior distributions of biases-related parameters are exchangeable between meta-analyses with different outcomes and comparators and, if bias-adjustment models shift the pooled effect size of the meta-analysis in the direction of trials at low RoB. The third task aims to address the reliability of the method by measuring the agreement between bias-adjustment results using different model and prior assumptions.

FCT – Portuguese Foundation for Science and Technology

46,944.36 EUR

46,944.36 EUR

2022.09205.PTDC

16/01/2023 to 15/07/2024

Luís Azevedo (PI), Joana Pardal (CO-PI) and Mariana Lobo