Network meta-analysis (NMA) is a powerful statistical technique for comparing the effectiveness of multiple interventions for a specific health condition. By synthesizing direct and indirect evidence from a network of studies, NMA provides a comprehensive understanding of the relative efficacy and safety of different treatments. However, missing data are a common problem in clinical trials 1, and their presence can compromise the validity and reliability of NMA results. This article explores the impact of different methods for handling missing data on effect estimates and uncertainty in NMAs. To achieve this, we conducted a comprehensive review of the literature, examining various methods for handling missing data, their impact on NMA results, and recent advancements in the field. This involved identifying relevant research papers and articles, summarizing different missing data handling methods, and analyzing studies that compared the impact of these methods on effect estimates and uncertainty. We also explored recent developments and advancements in the field.
Methods for Handling Missing Data in Network Meta-Analyses
Missing data in clinical trials can arise due to various reasons, such as patient dropout, loss to follow-up, or incomplete data collection. The mechanism by which data are missing can be categorized into three types: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) 1. Understanding these mechanisms is crucial for selecting an appropriate method for handling missing data.
Various methods have been proposed to address missing data in NMAs, each with its own assumptions and implications. These methods can be broadly categorized as follows:
- Complete Case Analysis (CCA): This method analyzes only the complete cases, i.e., individuals with available data for all variables 1. While simple to implement, CCA can lead to bias if the missing data are substantial, even when they are MCAR 1. Additionally, CCA can result in a loss of information and reduced statistical power.
- Single Imputation Methods: These methods replace missing values with a single imputed value, such as the mean, median, or value predicted from a regression model 2. Single imputation methods are easy to understand and implement but can underestimate uncertainty and distort the distribution of the data 1. For binary data, single imputation methods may involve imputing failure, the worst-case or best-case scenario, or using the last observation carried forward 1.
- Multiple Imputation (MI): MI generates multiple plausible imputed datasets, accounting for the uncertainty associated with the missing data 3. By analyzing each imputed dataset and pooling the results, MI provides more accurate estimates of standard errors and confidence intervals compared to single imputation 1.
- Model-Based Methods: These methods use statistical models to estimate the missing data, often relying on assumptions about the missing data mechanism 4. Examples include maximum likelihood estimation, which estimates parameters based on the observed data, and pattern-mixture models, which consider the relationship between the observed and missing data 5.
- Pattern-Mixture Model: This model groups subjects with similar missing data patterns and estimates a complete data analysis model within each pattern 6. It allows for investigating the impact of missing data on the conclusions by estimating a missingness parameter that measures the departure from the MAR assumption 5.
The following table summarizes the different methods for handling missing data, along with their pros and cons:
Method
Description
Pros
Cons
Complete Case Analysis
Removing cases with missing data
Simple to implement
Can lead to bias and loss of information, especially when missing data are not MCAR or substantial
Mean/Median Imputation
Replacing missing values with the mean or median
Easy to understand and implement
Can distort the distribution and relationships in the data, underestimates uncertainty
Multiple Imputation
Creating multiple plausible imputed datasets
Accounts for uncertainty in imputations, provides more accurate estimates
More complex to implement and interpret
Maximum Likelihood Estimation
Estimating parameters based on available data
Can provide unbiased estimates under MAR
Computationally intensive for complex models
Pattern-Mixture Model
Groups subjects with similar missing data patterns and estimates a complete data analysis model within each pattern
Allows for investigating the impact of missing data on the conclusions by estimating a missingness parameter
May require strong assumptions about the missing data mechanism
Impact of Missing Data Handling Methods on Effect Estimates and Uncertainty
The choice of missing data handling method can significantly influence the results of an NMA. Studies have shown that different methods can lead to varying effect estimates and levels of uncertainty. For instance, a study by Mavridis and Chaimani 7 found that using a pattern-mixture model to handle missing outcome data in mental health trials resulted in different effect estimates compared to a complete case analysis. This highlights the importance of considering the missing data mechanism and selecting an appropriate handling method.
Furthermore, a study by Spineli et al8. compared one-stage and two-stage approaches for handling missing binary outcome data in NMAs. They found that the one-stage approach, which involves analyzing all data in a single step, resulted in systematically larger uncertainty around the effect estimates, especially for networks with small trials or low event risks. This finding suggests that the choice of approach can affect the precision of NMA results.
The impact of missing data handling methods can also depend on the extent and pattern of missingness. When the proportion of missing data is small and MCAR, the different methods may yield similar results. However, as the amount of missing data increases or when the missing data are not MCAR, the choice of method becomes more critical. In such cases, more sophisticated methods like MI or model-based approaches are generally recommended to minimize bias and provide more accurate estimates of uncertainty 9.
It is crucial to recognize the potential for missing data to alter the conclusions of clinical trials. A study by Mihaela et al10. found that one in three trials with statistically significant results lost significance when plausible assumptions about missing data were applied. This emphasizes the substantial impact missing data can have on study conclusions and the importance of handling them appropriately.
Recent Advancements in Handling Missing Data in Network Meta-Analyses
Recent years have seen advancements in methods for handling missing data in NMAs. These include:
- Individual Participant Data (IPD) Network Meta-Analysis: This approach involves collecting and analyzing individual-level data from multiple trials, which can provide more detailed information on patient characteristics and outcomes 11. IPD NMAs can facilitate more accurate handling of missing data by allowing for more flexible modeling of the missing data mechanism and incorporating patient-level covariates. However, challenges associated with IPD NMAs include data collection, standardization, and potential biases related to participant selection 11.
- Bayesian Network Meta-Analysis with Missing Data: Bayesian methods have gained popularity in NMAs due to their ability to incorporate prior information and provide a more comprehensive assessment of uncertainty 12. Recent developments in Bayesian NMAs with missing data include the use of pattern-mixture models and sensitivity analyses to assess the robustness of results to different missing data assumptions.
- Machine Learning Methods for Missing Data Imputation: Machine learning techniques, such as random forests and neural networks, have shown promise in imputing missing data in various settings 3. These methods can capture complex relationships in the data and provide more accurate imputations compared to traditional methods. Advancements in machine learning for missing data imputation include the use of tree-based imputation, support vector machines, clustering-based imputation, and neural network-based methods such as artificial neural networks, flow-based models, variational autoencoders, generative adversarial networks, and diffusion models 13.
Recommendations for Handling Missing Data in Network Meta-Analyses
Based on the available evidence, the following recommendations can be made for handling missing data in NMAs:
- Assess the Extent and Pattern of Missingness: Before conducting an NMA, carefully evaluate the amount and distribution of missing data in the included trials. This can help in understanding the potential impact of missing data and selecting an appropriate handling method.
- Consider the Missing Data Mechanism: Whenever possible, try to understand the reasons for missing data and whether they are related to the outcome or other variables in the study. This can guide the choice of missing data handling method.
- Avoid Complete Case Analysis: Unless the proportion of missing data is very small and MCAR, avoid CCA as it can lead to biased results.
- Prefer Multiple Imputation or Model-Based Methods: When missing data are present, consider using MI or model-based methods, such as pattern-mixture models, to provide more accurate estimates of effect and uncertainty.
- Conduct Sensitivity Analyses: Perform sensitivity analyses to assess the robustness of the NMA results to different missing data assumptions 1. This can help in evaluating the potential impact of missing data on the conclusions. It is important to note that the choice of sensitivity analysis method can significantly impact the assessment of robustness in network meta-analyses 14.
- Understand how missing data were handled in each published report: Critically evaluate the methods used to handle missing data in the primary studies included in the NMA. This can help in identifying potential sources of bias and interpreting the results more accurately 1.
Conclusion
Missing data are a common challenge in NMAs, and their presence can affect the validity and reliability of the results. The choice of missing data handling method can significantly influence the effect estimates and uncertainty. Researchers should carefully consider the extent and pattern of missingness, the missing data mechanism, and the potential impact of different methods on the NMA results. Recent advancements in IPD NMAs, Bayesian methods, and machine learning techniques offer promising solutions for handling missing data in NMAs. By following the recommendations outlined in this article, researchers can minimize the impact of missing data and conduct more robust and reliable NMAs.
Synthesis of Findings
Missing data are a pervasive issue in network meta-analyses, potentially biasing effect estimates and inflating uncertainty. While complete case analysis remains a common approach, it is often suboptimal, especially with substantial or unbalanced missingness 9. Single imputation methods, while simple, may underestimate uncertainty 1. Multiple imputation and model-based methods, including pattern-mixture models, offer more robust solutions by accounting for uncertainty and potential biases due to missing data mechanisms. Recent advancements like individual participant data network meta-analysis and Bayesian approaches with sensitivity analyses further enhance the rigor of handling missing data.
The choice of the most appropriate method hinges on the extent and nature of missingness, the assumed missing data mechanism, and the specific research question. A thorough assessment of missing data, careful consideration of assumptions, and sensitivity analyses are crucial for ensuring the reliability and validity of network meta-analysis findings. Importantly, the choice of sensitivity analysis method can influence the assessment of robustness 14.
It is important to note that the effectiveness of different methods can vary depending on the specific characteristics of the data and the network meta-analysis 8. For example, the one-stage approach for handling missing binary outcomes can lead to systematically larger uncertainty, especially in networks with specific characteristics like small trials or low event risks 8. Additionally, it is crucial to recognize the potential for changes in statistical significance due to missing data, as highlighted by the finding that one in three trials with statistically significant results lost significance when plausible assumptions about missing data were applied 10. Further research is needed to evaluate the performance of these methods in different contexts and to develop new and improved techniques for handling missing data in network meta-analyses.
Works cited
1. Dealing with missing outcome data in meta‐analysis - PMC, accessed on January 15, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC7003862/
2. Effective Strategies to Handle Missing Values in Data Analysis, accessed on January 15, 2025, https://www.analyticsvidhya.com/blog/2021/10/handling-missing-value/
3. A Comparative Study of Various Methods for Handling Missing Data ..., accessed on January 15, 2025, https://www.mdpi.com/2077-0472/11/8/727
4. www.cmu.edu, accessed on January 15, 2025, https://www.cmu.edu/joss/content/articles/volume10/huisman.pdf
5. Continuous(ly) missing outcome data in network meta-analysis: A one-stage pattern-mixture model approach - PMC, accessed on January 15, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC8209314/
6. Implementation of Pattern-Mixture Models Using Standard SAS/STAT Procedures - PharmaSUG, accessed on January 15, 2025, https://pharmasug.org/proceedings/2011/SP/PharmaSUG-2011-SP04.pdf
7. Allowing for uncertainty due to missing outcome data in meta ..., accessed on January 15, 2025, https://training.cochrane.org/resource/allowing-uncertainty-due-missing-outcome-data-meta-analysis
8. Pattern-mixture model in network meta-analysis of binary missing outcome data: one-stage or two-stage approach? - PubMed, accessed on January 15, 2025, https://pubmed.ncbi.nlm.nih.gov/33413138/
9. Comparison of exclusion, imputation and modelling of missing ..., accessed on January 15, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC7049189/
10. Potential impact of missing outcome data on treatment effects in ..., accessed on January 15, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC7448113/
11. Using Individual Participant Data for Network Meta-Analysis: Advantages and Challenges, accessed on January 15, 2025, https://marksmanhealthcare.com/2023/10/19/using-individual-participant-data-for-network-meta-analysis-advantages-and-challenges/
12. Handling missing continuous outcome data in a Bayesian network meta-analysis, accessed on January 15, 2025, https://www.researchgate.net/publication/370026542_Handling_missing_continuous_outcome_data_in_a_Bayesian_network_meta-analysis
13. A Comprehensive Review of Handling Missing Data: Exploring Special Missing Mechanisms - arXiv, accessed on January 15, 2025, https://arxiv.org/html/2404.04905v1
14. How robust are findings of pairwise and network meta-analysis in ..., accessed on January 15, 2025, https://pmc.ncbi.nlm.nih.gov/articles/PMC8691029/