MCDA: Too Much Talk, Too Little Progress?

The General MCDA Process

MCDA incorporates different approaches and methods into the decision-making process and attempts to account for the trade-offs that may occur when multiple criteria must be taken into consideration. This process involves identifying and defining relevant criteria and assigning a weight (ie, relative importance) to each. Weighting methods can involve direct ratings, such as visual analogue scales or the analytic hierarchy process (AHP), or indirect methods, such as discrete choice experiments (DCEs). After the specific criteria are established, the performance of each selected item is evaluated using identical metrics and, following this evaluation, all scores are aggregated to provide an overall value assessment. Each option is summarized in a numerical score, and the options can be far-ranging, depending on what is of interest. Within the context of healthcare, options could be alternative drugs to treat a particular disease.

There are 3 broad classifications for MCDA approaches: value measurement models, outranking models, and reference-level models. Value measurement models, the approach most commonly used in healthcare, generally assign the MCDA process results with numerical scores for each option to identify the preferred decision alternative. Figure 1 outlines the steps of a value measurement MCDA process.

Current Utilization of MCDA and Future Potential

MCDA has been used across many industries to help with decision making, both in the public and private sectors; these areas range from transportation, immigration, education, investments, and the environment. For healthcare, MCDA has been used for many types of decisions (Table 1); adoption was initially slow but there has been an increasing number of publications over recent years, mostly citing its application around healthcare resource utilization. This may be because clear, transparent decision-making processes are still a challenge for many healthcare systems.

Healthcare decision-making processes are quite complex; they often involve multiple stakeholders (eg, patients, clinicians, and payers) with differing objectives. Across these stakeholders, criteria for decision making includes a myriad of items, including the need for intervention, comparative outcomes, type of benefit, and economic consequences. Additionally, the effect of treatment options on indirect costs and health-related quality of life may also be of interest depending on the stakeholder. Because decisions within healthcare are multifaceted, utilizing MCDA can improve the consistency, transparency, legitimacy, and quality of these decisions by using a structured, explicit approach. MDCA has promising applications for decisions pertaining to innovative therapies (eg, orphan drugs or targeted therapies), which require a more specialized approach to assessment compared to most therapies because of several differences to standard or traditional treatments designed to treat common diseases. First, the target patient population size may differ, especially for orphan diseases, which would affect the adaption of treatment. Second, a lack of alternative options or knowledge in the disease state could further support utilizing an MCDA approach. 

HTA Utilization of MCDA

MCDA may be particularly valuable for health technology assessments (HTAs), since these complex decisions require a comprehensive approach to address the multitude of factors that must be taken into consideration. Current examples of where MCDA has already been adopted or piloted by various HTA agencies include those in Colombia, Germany, Hungary, Italy, and Thailand. Here we explore the application of MCDA in Germany and Italy to capture patient preferences and create a framework for adopting health technologies.

Germany

In 2010, the German Institute for Quality and Efficiency in Health Care (IQWiG) initiated a pilot study to examine the use of MCDA as a method for capturing patient preferences in its HTA process. As part of this pilot study, 2 value measurement MCDA techniques, AHP and DCE preference elicitation methods, were explored. Two workshops were completed in the AHP study; one workshop with patients and the other healthcare professionals. Workshop participants rated their preferences on the importance of endpoints related to an antidepressant treatment by pairwise comparison of individual endpoints. Comparisons were analyzed to generate the relative importance for each endpoint. In the DCE study, patient and healthcare professionals provided preferences for hypothetical hepatitis C treatment alternatives on treatment outcomes. The choices were analyzed using logistic regression models to calculate the relative importance of the individual treatment characteristics. In Germany, the assessment of patient-relevant endpoints is core to the HTA, and both pilot studies concluded that these types of MCDA techniques can be used to support the HTA process. However, the studies also illustrated some methodological challenges that still need to be resolved before they could be fully implemented. 

Italy

The Lombardy region of Italy utilized MCDA in the development of an HTA framework called the Valutazione delle Tecnologie Sanitarie (VTS), which includes elements of the EUnetHTA Core Model and the Evidence and Value: Impact on Decision Making (EVIDEM) framework. The VTS framework has been utilized in the Lombardy region since 2011 for decision making related to the listing and delisting of health technologies. This framework is applied in a 3-step process that starts with a quick prioritization of requests, a full assessment of the prioritized technologies based on the EUnetHTA criteria, and lastly, an appraisal of the assessed technologies grounded on the analysis of multiple criteria. The VTS framework demonstrates the ability to combine EUnetHTA assessment tools with MCDA methods to support HTA. The EVIDEM framework adapted by the Lombardy region has 20 qualitative and quantitative criteria centered on 3 ethical imperatives (alleviation of patient suffering, prioritizing patients while balancing benefits across the greater population, and sustainability), with the goal of producing valuable outputs to support decision making in healthcare. Since its inception, it has been adopted for use globally and is now on its tenth iteration.

MCDA Limitations

The use of additive models has been the most common application of MCDA. These models are appealing because, at face value, they seem to be simple, transparent, and easy to implement; however, upon application, there are additional complexities to consider. In an additive MCDA model, explicit criteria are identified and assigned weights for the relative importance, and a simple weighted sum encapsulates the benefit into a single number. To avoid inappropriate double-counting of value, the criteria in an additive model should not overlap and the criteria should be preferentially independent, meaning that the weight or relative value of one criteria should not be dependent or influenced by the performance of other criteria. 

Unfortunately, many applications of additive-model MCDAs break these key rules, with overlapping criteria and preferentially dependent weights. For example, including both improvements in efficacy and patient-reported outcomes (PROs) as separate criteria creates an overlap and potential for double counting unless it is clear that the PRO value will only reflect benefits over and above that of the general improvement in efficacy measure. The presence of these methodologic flaws undermines the usefulness, and greater research to overcome these challenges may be needed to advance the application of MCDA. 

For the aforementioned limitations, it has been suggested that clinical and cost assessments should not be combined into a single MCDA. Instead, MCDA could be used for the clinical assessment, and a separate cost-effectiveness analysis (CEA) could be conducted for cost assessments, since the factors involved in MCDA may not be suitable for economic analyses. For example, it may be difficult to assign a weight to stakeholders’ willingness to pay for health gains. Conversely, a counter-argument is that many organizations may not have the resources to conduct full CEAs on technologies, so MCDA can provide a structured approach to allow for stakeholders to determine how to best allocate their resources. 

Additionally, research involving senior management in health authorities shows that healthcare funding decision making is often driven by ad hoc favoritism, trend-based weak arguments, and political influence. Therefore, an MCDA approach to HTA would be an incremental enhancement to existing practice, and the increased transparency from adopting this approach could lead stakeholders to identify and continuously improve.

Manufacturer Implications

Regardless of the challenges to applying MCDA in an environment with constrained resources and decision makers with a desire to control healthcare spending, efforts to conduct valuation and resource allocation optimization across global markets open the door for MCDA. Given the potential incremental benefits with MCDA over existing decision-making practices, manufacturers should seek to better understand how MCDA applies to their own products and engage all healthcare stakeholders to fully understand each group of criteria and relative weight of benefits and tradeoffs. Early engagement with these stakeholders, including HTAs, will facilitate capturing and communicating patient perspectives. 

The article should be referenced as follows: 

Campbell D, Kang I. MCDA: Too much talk, too little progress? HTA Quarterly. Winter 2019. Feb. 25, 2019.

References

• Castro H, Tringali M, Cleemput I, et al. Advancing MCDA and HTA into coverage decision-making. In: Marsh K, Goetghebeur M, Thokala P, et al., eds. Multi-Criteria Decision Analysis to Support Healthcare Decisions. Cham, Switzerland: Springer International Publishing, 2017.
• Danner M, Hummel JM, Volz F, et al. Integrating patients’ views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. Int J Technol Assess Health Care. 2011;27:369-375.
• Diaby V, Campbell K, Goeree R. Multi-criteria decision analysis (MCDA) in health care: a bibliometric analysis. Oper Res Health Care. 2013;2:20-24.
• Endrei D, Molics B, Agoston I. Multicriteria decision analysis in the reimbursement of new medical technologies: real-world experiences from Hungary. Value Health. 2014;17:487-489.
• Goetghebeur MM, Cellier MS. Can reflective multicriteria be the new paradigm for healthcare decision-making? The EVIDEM journey. Cost Eff Resour Alloc. 2018 Nov 9;16(Suppl 1):54.
• Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and Value: Impact on DEcisionMaking--the EVIDEM framework and potential applications. BMC Health Serv Res. 2008 Dec;8(270):1-16.
• Hall W, Mitton C, Levy A. Realizing the "great potential" of MCDA in HTA. Value Health. 2018 Dec;21(12):1355-1356.
• Institute for Quality and Efficiency in Health Care (IQWiG). General methods. https://www.iqwig.de/download/IQWiG_General_Methods_Version_ 4-2.pdf. Accessed February 21, 2019.
• Institute for Quality and Efficiency in Health Care (IQWiG). Analytic hierarchy process (AHP) – pilot project to elicit patient preferences in the indication “depression”. 2015. https://www.iqwig.de/download/Executive-summary-of-working-paper_Analytic-Hierarchy-Process-pilot-project.pdf. Accessed February 21, 2019.
• Institute for Quality and Efficiency in Health Care (IQWiG). Choice-based conjoint analysis – pilot project to identify, weight, and prioritize multiple attributes in the indication “hepatitis C”. 2015. https://www.iqwig.de/download/GA10-03_Executive-summary-of-working-paper-1.1_Conjoint-Analysis.pdf. Accessed February 21, 2019.
• Marsh K, Lanitis T, Neasham D, et al. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. 2014;32:345-365.
• Marsh KD, Sculpher M, Caro JJ, Tervonen T. The use of MCDA in HTA: great potential, but more effort needed. Value Health. 2018 Apr;21(4):394-397.
• Radaelli G, Lettieri E, Masella C, et al. Implementation of EUnetHTA core models in Lombardia: the VTS framework. Int J Technol Assess Health Care. 2014;30:105-112.
• Smith N, Hall W, Mitton C, Bryan S, Urquhart B. What constitutes high performance in priority setting and resource allocation? Decision maker narratives identified from a survey and qualitative study in Canadian healthcare organizations. Health Serv Manage Res. 2014 Dec 4. pii: 0951484814559714.
• Thokala P, Devlin N, Marsh K, et al. Multiple criteria decision analysis for health care decision making--an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016 Jan;19(1):1-13.
• Youngkong S, Baltussen R, Tantivess S, et al. Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand. Value Health. 2012;15:961-970.