A list of topics we will cover.

Conformal Prediction

- Exchangeable Data: Full and Split Conformal Prediction
- Distribution Shift and Time Series Data
- Adaptive/Adversarial Conformal Prediction
- Threshold Calibrated Multivalid Conformal Prediction

(Multi)Calibration

- Proper Scoring Rules, Calibration, and Regret
- Algorithms for offline (batch) multicalibration
- Algorithms for online (adversarial) multicalibration
- Moment Multicalibration
- Applications of Multicalibration
- Downstream Unconstrained Optimization (Omnipredictors)
- Downstream Constrained Optimization
- Proxies for Downstream Measurement
- Distribution Shift

Tests and Predictors

- Ignorantly Passing Tests: Possibility and Hardness
- Outcome Indistinguishability

Other Topics

- Smooth Calibration
- Applications of Calibration in Mechanism Design
- The Reference Class Problem

A list of papers related to the topic that we will draw from and/or that you might use as a starting point for a project:

- Conformal Prediction
- A Gentle Introduction to Conformal Prediction
- A Tutorial on Conformal Prediction
- Predictive Inference with the Jackknife+
- Exact and Robust Conformal Inference Methods for Predictive Machine Learning with Dependent Data
- Mondrian Confidence Machines
- Adaptive Conformal Inference Under Distribution Shift
- Conformalized Online Learning: Online Calibration Without a Holdout Set
- Practical Adversarial Multivalid Conformal Prediction
- Batch Multivalid Conformal Prediction

- Calibration
- The Well Calibrated Bayesian
- Calibration Based Empirical Probability
- Asymptotic Calibration
- Calibration for the (Computationally Identifiable) Masses
- Moment Multicalibration for Uncertainty Estimation
- Online Multivalid Learning: Means Moments and Prediction Intervals
- Smooth calibration, leaky forecasts, finite recall, and Nash dynamics
- Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration
- Low Degree Multicalibration
- “Calibeating”: Beating Forecasters at Their Own Game
- Online Minimax Multiobjective Optimization: Multicalibeating and Other Applications
- Multicalibration as Boosting for Regression
- The Scope of Multicalibration: Characterizing Multicalibration via Property Elicitation

- Applications of Multicalibration
- Omnipredictors
- Multiaccurate Proxies for Downstream Fairness
- Universal adaptability: Target-independent inference that competes with propensity scoring
- Multicalibrated Regression for Downstream Fairness
- Omnipredictors for Constrained Optimization
- Loss Minimization Through the Lens of Outcome Indistinguishability
- Making Decisions Under Outcome Performativity

- Passing Distributional Tests Beyond Calibration
- Calibration and Game Theory/Mechanism Design
- The Reference Class Problem