Learning with Conditional Guarantees

CIS 6200: A Course at the University of Pennsylvania

Time: 12:00-1:20pm

Location: Tuesday/Thursday WLNT 401B (3401 Walnut, 4th floor Active Learning classroom)

Discussion: Slack (Sign up at this link.)

Draft Notes: PDF Here (Incomplete/Work in Progress)

Overview: In this course we will study modern techniques in machine learning, statistics, and computer science for learning with guarantees that hold not just marginally, but conditionally on various events. We’ll show how to use these kinds of algorithms for various purposes. One major application will be estimating the uncertainty of black-box forecasts; techniques in this family include conformal prediction and (multi)calibration, both in the standard distributional setting and in online adversarial learning settings. We’ll also show how to solve more traditional learning problems with various conditional optimality guarantees, in both the batch and online adversarial settings.

We will apply these techniques to give promises to downstream decision makers that it is safe to “trust” our predictions and act accordingly, even though they may have a variety of different goals and abilities, thus giving a foundation for trustworthy machine learning. Along the way we’ll see other applications in economics, mechanism design, and machine learning.

Format: This class will be lecture based, and will be evaluated on the basis of a semester long project. The goal of this course is to bring students up to the research frontier and equip them to do original research in this area — and the aim of the project should be to produce publishable research.

Intended Audience: Graduate students and advanced undergraduates interested in mathematical topics in computer science and statistics

Prerequisites: There are no specific coursework requirements, but students are expected to be mathematically mature, able to navigate the research literature, and operate and manage their time independently to complete a substantial research project.