Identifiable Discrete Latent Representation Learning: From Deep Generative Models to Latent Causal Structure

When is hidden structure uniquely determined by complex, high-dimensional observations? This question is central to interpretable representation learning. If different latent explanations produce the same distribution of the data, then the meaning of a learned representation, and any scientific conclusion built on it, is ambiguous. Modern deep generative models are typically overparametrized and non-identifiable. In this talk, I will show that discrete latent variables can make deep latent structure both expressive and identifiable, and hence interpretable. I will consider two questions: Can the layers of a deep generative model be uniquely recovered from data? And can we learn causal relationships among hidden variables we never observe directly?

I will first introduce Deep Discrete Encoders, a class of generative models with multiple binary latent layers and flexible observation models for binary, count, and continuous data. We derive identifiability conditions on the graphs connecting adjacent layers. These conditions have a concrete implication for interpretable architecture design: deeper layers should contain fewer latent variables. A layerwise spectral initialization followed by penalized stochastic approximation EM makes estimation feasible even when the number of latent configurations grows exponentially. I will then turn to Discrete Causal Representation Learning, which models an unknown directed acyclic graph among discrete latent variables and a sparse measurement graph linking them to mixed-type observations. Under suitable conditions, observational data alone, without interventions or multiple environments, identify the measurement graph and the latent causal graph (the latter up to the Markov equivalence class). We give a consistent procedure for recovering both graphs: fit the measurement model, resample latent configurations from the fitted model, and apply causal discovery to the resampled latents. Applications to text, image, and educational assessment data show that the learned latent variables and their causal relations are interpretable. Together, these results provide statistical foundations for interpretable representation learning, from deep generative modeling to latent causal discovery.

Host: Joe Feldman

Yuqi Gu is an Assistant Professor of Statistics at Columbia University and a member of the Data Science Institute. She received her Ph.D. in Statistics from the University of Michigan in 2020 and was a postdoctoral researcher at Duke University before joining Columbia in 2021. Her research focuses on latent structure and representation learning: (1) identifiable deep generative models and causal representation learning; (2) spectral, tensor, and likelihood-based methods for latent structure recovery from high-dimensional data, with sharp theoretical guarantees and uncertainty quantification; and (3) drawing on latent variable models from psychometrics, statistical frameworks for evaluating large language models.