Doubly Robust Methods

Chattopadhyay, A., Cohn, E. R., & Zubizarreta, J. R. (2022).One-step weighting to generalize and transport treatment effect estimates to a target population. ArXiv Preprint ArXiv:2203.08701.

• Weighting methods are often used to generalize and transport estimates of causal effects from a study sample to a target population. Traditional methods construct the weights by separately modeling the treatment assignment and the study selection probabilities and then multiplying functions (e.g., inverses) of the estimated probabilities.
• These estimated multiplicative weights may not produce adequate covariate balance and can be highly variable, resulting in biased and/or unstable estimators, particularly when there is limited covariate overlap across populations or treatment groups.
• To address these limitations, we propose a weighting approach for both randomized and observational studies that weights each treatment group directly in ‘one go’ towards the target population. We present a general framework for generalization and transportation by characterizing the study and target populations in terms of generic probability distributions. Under this framework, we justify this one-step weighting approach. By construction, this approach directly balances covariates relative to the target population and produces weights that are stable. Moreover, in some settings, this approach does not require individual-level data from the target population. We connect this approach to inverse probability and inverse odds weighting. We show that the one-step weighting estimator for the target average treatment effect is consistent, asymptotically Normal, doubly-robust, and semiparametrically efficient.
• We demonstrate the performance of this approach using a simulation study and a randomized case study on the effects of physician racial diversity on preventive healthcare utilization among Black men in California.

Dahabreh, I. J., Haneuse, S. J. A., Robins, J. M., Robertson, S. E., Buchanan, A. L., Stuart, E. A., & Hernán, M. A. (2021). Study designs for extending causal inferences from a randomized trial to a target population. American Journal of Epidemiology, 190(8), 1632–1642

• In this article, we examine study designs for extending (generalizing or transporting) causal inferences from a randomized trial to a target population. Specifically, we consider nested trial designs, where randomized individuals are nested within a sample from the target population, and nonnested trial designs, including composite data-set designs, where observations from a randomized trial are combined with those from a separately obtained sample of nonrandomized individuals from the target population.
• We show that the counterfactual quantities that can be identified in each study design depend on what is known about the probability of sampling nonrandomized individuals. For each study design, we examine identification of counterfactual outcome means via the g-formula and inverse probability weighting. Last, we explore the implications of the sampling properties underlying the designs for the identification and estimation of the probability of trial participation.

Dahabreh, I. J., Robertson, S. E., Steingrimsson, J. A., Stuart, E. A., & Hernan, M. A. (2020). Extending inferences from a randomized trial to a new target population. Statistics in Medicine, 39(14), 1999–2014.

• When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations.
• In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust).
• We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease.
• We conclude by discussing issues that arise when using the methods in applied analyses.

Lee, D., Yang, S., Dong, L., Wang, X., Zeng, D., & Cai, J. (2021). mproving trial generalizability using observational studies. Biometrics.

• Complementary features of randomized controlled trials (RCTs) and observational studies (OSs) can be used jointly to estimate the average treatment effect of a target population. We propose a calibration weighting estimator that enforces the covariate balance between the RCT and OS, therefore improving the trial-based estimator’s generalizability.
• Exploiting semiparametric efficiency theory, we propose a doubly robust augmented calibration weighting estimator that achieves the efficiency bound derived under the identification assumptions. A nonparametric sieve method is provided as an alternative to the parametric approach, which enables the robust approximation of the nuisance functions and data-adaptive selection of outcome predictors for calibration.
• We establish asymptotic results and confirm the finite sample performances of the proposed estimators by simulation experiments and an application on the estimation of the treatment effect of adjuvant chemotherapy for early-stage non-small-cell lung patients after surgery.

Pressler, T. R., & Kaizar, E. E. (2013). The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias. Statistics in Medicine, 32(20), 3552–3568.

• While randomized controlled trials (RCT) are considered the “gold standard” for clinical studies, the use of exclusion criteria may impact the external validity of the results. It is unknown whether estimators of effect size are biased by excluding a portion of the target population from enrollment.
• We propose to use observational data to estimate the bias due to enrollment restrictions, which we term generalizability bias. In this paper we introduce a class of estimators for the generalizability bias and use simulation to study its properties in the presence of nonconstant treatment effects.
• We find the surprising result that our estimators can be unbiased for the true generalizability bias even when all potentially confounding variables are not measured. In addition, our proposed doubly robust estimator performs well even for mis-specified models.

Rudolph, K. E., Díaz, I., Rosenblum, M., & Stuart, E. A. (2014). Estimating population treatment effects from a survey subsample. American Journal of Epidemiology, 180(7), 737–748.

• We considered the problem of estimating an average treatment effect for a target population using a survey subsample. Our motivation was to generalize a treatment effect that was estimated in a subsample of the National Comorbidity Survey Replication Adolescent Supplement (2001-2004) to the population of US adolescents.
• To address this problem, we evaluated easy-to-implement methods that account for both nonrandom treatment assignment and a nonrandom 2-stage selection mechanism. We compared the performance of a Horvitz-Thompson estimator using inverse probability weighting and 2 doubly robust estimators in a variety of scenarios.
• We demonstrated that the 2 doubly robust estimators generally outperformed inverse probability weighting in terms of mean-squared error even under misspecification of one of the treatment, selection, or outcome models. Moreover, the doubly robust estimators are easy to implement and provide an attractive alternative to inverse probability weighting for applied epidemiologic researchers. We demonstrated how to apply these estimators to our motivating example.

Rudolph, K. E., Levy, J., Schmidt, N. M., Stuart, E., & Ahern, J. (2020).Using transportability to understand differences in mediation mechanisms across trial sites of a housing voucher experiment. Epidemiology (Cambridge, Mass.), 31(4), 523.

• Background: Randomized trials may have different effects in different settings. Moving to Opportunity (MTO), a housing experiment, is one such example. Previously, we examined the extent to which MTO’s overall effects on adolescent substance use and mental health outcomes were transportable across the sites to disentangle the contributions of differences in population composition versus differences in contextual factors to site differences. However, to further understand reasons for different site effects, it may be beneficial to examine mediation mechanisms and the degree to which they too are transportable across sites.
• Methods: We used longitudinal data from MTO youth. We examined mediators summarizing aspects of the school environment over the 10–15 year follow-up. Outcomes of past-year substance use, mental health, and risk behavior were assessed at the final timepoint when participants were 10–20 years old. We used doubly robust and efficient substitution estimators to estimate 1) indirect effects by MTO site and 2) transported indirect effects from one site to another.
• Results: Differences in indirect effect estimates were most pronounced between Chicago and LA. Using transport estimators to account for differences in baseline covariates, likelihood of using the voucher to move, and mediator distributions partially to fully accounted for site differences in indirect effect estimates in 10 of the 12 pathways examined.
• Conclusions: Using transport estimators can provide an evidence-based approach for understanding the extent to which differences in compositional factors contribute to differences in indirect effect estimates across sites, and ultimately, to understanding why interventions may have different effects when applied to new populations.