Uncertain: Modern Topics in Uncertainty Estimation

CIS 7000: A Course at the University of Pennsylvania

Time: Fridays 1:45-4:45

Location: Meyerson Hall B13

Instructor: Aaron Roth

Discussion: Slack (click here to join the workspace)

Draft Notes: PDF Here (Incomplete/Work in Progress)

Overview: In this course we will study modern techniques in machine learning, statistics, and computer science for estimating the uncertainty of black box forecasts. This includes conformal prediction, calibration and multi-calibration, outcome indistinguishability, and recent techniques for producing worst-case empirical coverage guarantees without any distributional assumptions. Along the way we will explore applications of these techniques to problems of domain adaptation and their use in solving downstream optimization problems, economics and mechanism design, and algorithmic fairness. 

Format: This class will be lecture based, and will be evaluated on the basis of a semester long project. The goal of this course is to bring students up to the research frontier and equip them to do original research in this area — and the aim of the project should be to produce publishable research. 

Intended Audience: PhD students interested in mathematical topics in computer science and statistics

Prerequisites: There are no specific coursework requirements, but students are expected to be mathematically mature, able to navigate the research literature, and operate and manage their time independently to complete a substantial research project.