Luhuan Wu

Hi, I am a 5th-year PhD student in Statistics at Columbia University, where I am very fortunate to work with Prof. John Cunningham. and Prof. David Blei. During my PhD, I’ve interned at Apple MLR hosted by Sinead Williamson. Previously, I received my master’s degree in Data Science at Columbia University, supervised by Prof. John Cunningham and Prof. Itsik Pe’er. I received my bachelor’s degree in Mathematics at Nanjing University.

Research Interests: In general I am interested in problems that invovle randomness and uncertainty, mostly through a probabilistic lens. Specifically, my recent focus has been:

Probabilistic modeling: deep generative models (e.g. diffusion models, VAE, deep state space models), and Bayesian inference (e.g. variational inference, SMC)
Uncertainty quantification: I’ve been mostly exploring the Bayesian approach, but I’m also open to other approaches
Invariant prediction and out-of-distribution generalization

Selected publications

Posterior Uncertainty Quantification in Neural Networks using Data Augmentation

Luhuan Wu, and Sinead A Williamson

In International Conference on Artificial Intelligence and Statistics 2024

Abs Bib PDF Code

In this paper, we approach the problem of uncertainty quantification in deep learning through a predictive framework, which captures uncertainty in model parameters by specifying our assumptions about the predictive distribution of unseen future data. Under this view, we show that deep ensembling (Lakshminarayanan et al., 2017) is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only – a situation rarely encountered in practice. To address this limitation, we propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques. MixupMP operates as a drop-in replacement for deep ensembles, where each ensemble member is trained on a random simulation from this predictive distribution. Grounded in the recently-proposed framework of Martingale posteriors (Fong et al., 2023), MixupMP returns samples from an implicitly defined Bayesian posterior. Our empirical analysis showcases that MixupMP achieves superior predictive performance and uncertainty quantification on various image classification datasets, when compared with existing Bayesian and non-Bayesian approaches.
title = {Posterior Uncertainty Quantification in Neural Networks using Data Augmentation}, author = {Wu, Luhuan and Williamson, Sinead A}, booktitle = {International Conference on Artificial Intelligence and Statistics}, pages = {3376--3384}, year = {2024}, organization = {PMLR}, }
Practical and Asymptotically Exact Conditional Sampling in Diffusion Models

Luhuan Wu Wu, Brian Trippe, Christian Naesseth, David Blei, and John Cunningham

In Conference on Neural Information Processing Systems 2023

Abs Bib PDF Code

Diffusion models have been successful on a range of conditional generation tasks including molecular design and text-to-image generation. However, these achievements have primarily depended on task-specific conditional training or error-prone heuristic approximations. Ideally, a conditional generation method should provide exact samples for a broad range of conditional distributions without requiring task-specific training. To this end, we introduce the Twisted Diffusion Sampler, or TDS. TDS is a sequential Monte Carlo (SMC) algorithm that targets the conditional distributions of diffusion models. The main idea is to use twisting, an SMC technique that enjoys good computational efficiency, to incorporate heuristic approximations without compromising asymptotic exactness. We first find in simulation and on MNIST image inpainting and class-conditional generation tasks that TDS provides a computational statistical trade-off, yielding more accurate approximations with many particles but with empirical improvements over heuristics with as few as two particles. We then turn to motif-scaffolding, a core task in protein design, using a TDS extension to Riemannian diffusion models. On benchmark test cases, TDS allows flexible conditioning criteria and often outperforms the state of the art.
@inproceedings{wu2023tds, title = {Practical and Asymptotically Exact Conditional Sampling in Diffusion Models}, author = {Wu, Luhuan Wu and Trippe, Brian and Naesseth, Christian and Blei, David and Cunningham, John}, booktitle = {Conference on Neural Information Processing Systems}, year = {2023}, }
Variational Nearest Neighbor Gaussian Process

Luhuan Wu, Geoff Pleiss, and John Cunningham

In International Conference on Machine Learning 2022

Abs Bib PDF Code Talk

Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix. In this work, we instead exploit a sparse approximation of the precision matrix. We propose variational nearest neighbor Gaussian process (VNNGP), which introduces a prior that only retains correlations within 𝐾 nearest-neighboring observations, thereby inducing sparse precision structure. Using the variational framework, VNNGP’s objective can be factorized over both observations and inducing points, enabling stochastic optimization with a time complexity of 𝑂(𝐾3). Hence, we can arbitrarily scale the inducing point size, even to the point of putting inducing points at every observed location. We compare VNNGP to other scalable GPs through various experiments, and demonstrate that VNNGP (1) can dramatically outperform low-rank methods, and (2) is less prone to overfitting than other nearest neighbor methods.
@inproceedings{wu2022variational, title = {Variational Nearest Neighbor Gaussian Process}, author = {Wu, Luhuan and Pleiss, Geoff and Cunningham, John}, booktitle = {International Conference on Machine Learning}, pages = {24114-24130}, year = {2022}, organization = {PMLR}, }
Bias-free Scalable Gaussian Processes via Randomized Truncations

Andres Potapczynski, Luhuan Wu, Dan Biderman, Geoff Pleiss, and John P Cunningham

In International Conference on Machine Learning 2021

Abs Bib PDF Code Talk Slides

Scalable Gaussian Process methods are computationally attractive, yet introduce modeling biases that require rigorous study. This paper analyzes two common techniques: early truncated conjugate gradients (CG) and random Fourier features (RFF). We find that both methods introduce a systematic bias on the learned hyperparameters: CG tends to underfit while RFF tends to overfit. We address these issues using randomized truncation estimators that eliminate bias in exchange for increased variance. In the case of RFF, we show that the bias-to-variance conversion is indeed a trade-off: the additional variance proves detrimental to optimization. However, in the case of CG, our unbiased learning procedure meaningfully outperforms its biased counterpart with minimal additional computation. Our code is available at https://github.com/ cunningham-lab/RTGPS.
@inproceedings{potapczynski2021bias, title = {Bias-free Scalable Gaussian Processes via Randomized Truncations}, author = {Potapczynski, Andres and Wu, Luhuan and Biderman, Dan and Pleiss, Geoff and Cunningham, John P}, booktitle = {International Conference on Machine Learning}, pages = {8609--8619}, year = {2021}, organization = {PMLR}, }
Hierarchical Inducing Point Gaussian Process for Inter-domian Observations

Luhuan Wu, Andrew Miller, Lauren Anderson, Geoff Pleiss, David Blei, and John Cunningham

In International Conference on Artificial Intelligence and Statistics 2021

Abs Bib PDF Code Talk

We examine the general problem of inter-domain Gaussian Processes (GPs): problems where the GP realization and the noisy observations of that realization lie on different domains. When the mapping between those domains is linear, such as integration or diﬀerentiation, inference is still closed form. However, many of the scaling and approximation techniques that our community has developed do not apply to this setting. In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions. HIP-GP, which relies on inducing points with grid structure and a stationary kernel assumption, is suitable for low-dimensional problems. In developing HIP-GP, we introduce (1) a fast whitening strategy, and (2) a novel preconditioner for conjugate gradients which can be helpful in general GP settings.
@inproceedings{wu2021hierarchical, title = {Hierarchical Inducing Point Gaussian Process for Inter-domian Observations}, author = {Wu, Luhuan and Miller, Andrew and Anderson, Lauren and Pleiss, Geoff and Blei, David and Cunningham, John}, booktitle = {International Conference on Artificial Intelligence and Statistics}, pages = {2926--2934}, year = {2021}, organization = {PMLR}, }