Ryan Tibshirani: Statistics, Nonparametric Regression, Conformal Prediction

Episode 121
I spoke with Professor Ryan Tibshirani (stat.berkeley.edu/~ryantibs) about:
* Differences between the ML and statistics communities in scholarship, terminology, and other areas.
* Trend filtering
* Why you can’t just use garbage prediction functions when doing conformal prediction
Ryan is a Professor in the Department of Statistics at UC Berkeley. He is also a Principal Investigator in the Delphi group. From 2011-2022, he was a faculty member in Statistics and Machine Learning at Carnegie Mellon University. From 2007-2011, he did his Ph.D. in Statistics at Stanford University.
Reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.
The Gradient Podcast on: Apple Podcasts (podcasts.apple.com/us/podcast/the-gradient-pod…) | Spotify (open.spotify.com/show/6onNcSqsP6hEEqmZ6TU2g8) | Pocket Casts (pca.st/itunes/1569777340) | RSS (api.substack.com/feed/podcast/265424/s/1354/pr…)Follow The Gradient on Twitter (twitter.com/gradientpub)
Outline:
* (00:00) Intro
* (01:10) Ryan’s background and path into statistics
* (07:00) Cultivating taste as a researcher
* (11:00) Conversations within the statistics community
* (18:30) Use of terms, disagreements over stability and definitions
* (23:05) Nonparametric Regression
* (23:55) Background on trend filtering
* (33:48) Analysis and synthesis frameworks in problem formulation
* (39:45) Neural networks as a specific take on synthesis
* (40:55) Divided differences, falling factorials, and discrete splines
* (41:55) Motivations and background
* (48:07) Divided differences vs. derivatives, approximation and efficiency
* (51:40) Conformal prediction
* (52:40) Motivations
* (1:10:20) Probabilistic guarantees in conformal prediction, choice of predictors
* (1:14:25) Assumptions: i.i.d. and exchangeability — conformal prediction beyond exchangeability
* (1:25:00) Next directions
* (1:28:12) Epidemic forecasting — COVID-19 impact and trends survey
* (1:29:10) Survey methodology
* (1:38:20) Data defect correlation and its limitations for characterizing datasets
* (1:46:14) Outro
Links:
* Ryan’s homepage (stat.berkeley.edu/~ryantibs)
* Works read/mentioned
* Nonparametric Regression
* Adaptive Piecewise Polynomial Estimation via Trend Filtering (stat.cmu.edu/~ryantibs/papers/trendfilter.pdf) (2014)
* Divided Differences, Falling Factorials, and Discrete Splines: Another Look at Trend Filtering and Related Problems (stat.berkeley.edu/~ryantibs/papers/dspline.pdf) (2020)
* Distribution-free Inference
* Distribution-Free Predictive Inference for Regression (arxiv.org/pdf/1604.04173.pdf) (2017)
* Conformal Prediction Under Covariate Shift (stat.berkeley.edu/~ryantibs/papers/weightedcp.…) (2019)
* Conformal Prediction Beyond Exchangeability (stat.berkeley.edu/~ryantibs/papers/nexcp.pdf) (2023)
* Delphi (delphi.cmu.edu) and COVID-19 research
* Flexible Modeling of Epidemics (journals.plos.org/ploscompbiol/article?id=10.1…)
* Real-Time Estimation of COVID-19 Infections (projecteuclid.org/journals/statistical-science…)
* The US COVID-19 Trends and Impact Survey (pnas.org/doi/full/10.1073/pnas.2111454118) and Big data, big problems: Responding to “Are we there yet?” (arxiv.org/abs/2109.00680)

Get full access to The Gradient at thegradientpub.substack.com/subscribe?utm_medi…