Have you ever been asked, “how does your product compare to [your competitor]”? How do you answer if there are no or few direct trials comparing the two? Enter network meta-analysis, to answer that question and provide a statistically sound estimate of comparative effectiveness.
In a previous newsletter, we presented meta-analysis as a means of generating evidence for an intervention by combining results from smaller, possibly underpowered studies to calculate overall effects. Such analyses are of particular importance for medical devices, where the evidence base may be less extensive. When you need to compare against multiple products or a product for which no direct head-to-head evidence exists, meta-analysis can be extended to network meta-analysis (or indirect treatment comparison).
Network meta-analysis can generate evidence not only regarding efficacy of an intervention in direct comparison to other products (as in a meta-analysis), but also a ranking of how effective those interventions are relative to each other. In the already underpopulated (in comparison to pharmaceuticals) field of medical device evidence in the literature, the likelihood is slim of finding many high-quality studies such as randomized, controlled trials that include all the comparisons you require. Network meta-analysis is a rigorous, computational, and statistical method that allows you to generate evidence of relative ranks of interventions, even in the absence of multiple studies that directly compare them.
This analysis requires, at a minimum, one comparative study for each intervention of interest. In the figure below, usual care (UC) is compared to products A, B, and C, but direct comparisons only exist between UC and each of the three products, and between products A and C. There is no comparison between B and A or C. Nevertheless, a ranking of all products can be generated. In the example, product A has clearly the best outcomes, C perhaps slightly better than B, though the order is uncertain, and all are better than UC. Hopefully product A will be yours!
Beyond ranking data, mean differences in outcomes such as operative time, length of stay, or adverse events can be estimated. Data such as these, when generated robustly, can be invaluable to decision makers presented with multiple treatment options. As a payer, which one do I fund? As a provider, which one has the lowest reports of operative failure? As a patient, which one may result in the lowest incidence of chronic pain? Examination of multiple outcomes enriches the data available to inform choices made at any stage of the healthcare process. It also allows you to position your product appropriately, by identifying its strengths (and weaknesses) in the myriad landscape of treatment alternatives.
The thickness of connecting arrows indicates the number of studies with direct comparisons between the treatment alternatives, including no arrow where no direct comparison has been included.