Targeting Internal and External Validity in Observational Studies: From Causal Inference to Adaptive Study Design

Probability sampling is the gold standard for achieving external validity, while randomized experiments are the gold standard for internal validity in causal inference. However, these designs are often infeasible in practice. Observational studies offer an important alternative for estimating population-level causal effects, such as the population average treatment effect (PATE), but they face two distinct threats: confounding due to nonrandom treatment assignment and selection bias when the study sample differs systematically from the target population. The HEALthy Brain and Child Development (HBCD) Study illustrates both challenges. HBCD aims to reflect the sociodemographic diversity of its target population, allowing researchers to characterize variation in child development and generalize the effects of prenatal substance exposure. However, voluntary participation among pregnant women may introduce sample selection bias, while self-reported and biomarker-based measures of prenatal substance use raise concerns about confounding effects on child developmental outcomes. In this talk, I examine both inference methods and design schemes for observational data collected through probability or nonprobability samples. Standard doubly robust estimators adjust for confounding in PATE estimation when either the treatment assignment model or the outcome model is correctly specified. However, these estimators may lose double robustness when applied to nonrepresentative samples. I introduce our proposed estimators that explicitly incorporate sample selection mechanisms. I also describe an adaptive recruitment strategy currently implemented in HBCD to improve the internal and external validity of subsequent analyses. I conclude with challenges and opportunities in improving study design and statistical inference to strengthen causal inference in population science.

Host: Joe Feldman

Yajuan Si is a Research Associate Professor in the Survey Research Center at the University of Michigan.  She earned her Ph.D. in Statistical Science from Duke University and completed postdoctoral training at Columbia University. Dr. Si’s research focuses on methodological development from study design to statistical inference. Her work spans Bayesian statistics, unifying design- and model-based approaches for survey inference, data integration, missing data imputation, confidentiality protection, and causal inference, with applications in the social and health sciences. Her research has been supported by grants from NIH and NSF. She is the recipient of the 2026 Gertrude Cox Award and an elected member of the International Statistical Institute.