Research
Methods and evidence for deciding when machine-learning systems can be trusted under changing operating conditions.
My research addresses a basic deployment question: when should a machine-learning system no longer be trusted? I approach this question through three connected lines of work.
Distribution shift and monitoring §
Machine-learning models can fail when their operating conditions differ from their training data, often while remaining highly confident. I develop out-of-distribution (OOD) detection methods that help recognize inputs and situations for which a model’s predictions may be unreliable.
Representative work:
- Improving Out-of-Distribution Detection with Markov Logic Networks (ICML)
- Out-of-Distribution Detection with Adversarial Outlier Exposure (CVPR Workshop)
- Multi-Class Hypersphere Anomaly Detection (ICPR)
Reasoning with structured knowledge §
Unusual situations are not always visually or statistically unusual. My neuro-symbolic work combines learned representations with logical, probabilistic, or natural-language knowledge about a domain. This makes it possible to detect semantic inconsistencies and gives people a way to inspect or revise some of the assumptions used by a detector.
Representative work:
- Improving Out-of-Distribution Detection with Markov Logic Networks (ICML)
- Out-of-Distribution Detection with Logical Reasoning (WACV)
- Language Models as Reasoners for Out-of-Distribution Detection (WAISE)
Evaluation and evidence §
A detector is only useful if its evaluation supports clear, reproducible claims. I work on software and evaluation protocols that make OOD experiments easier to compare, reproduce, and interpret.
Representative work:
pytorch-ood, open-source evaluation infrastructure- Addressing Randomness in Evaluation Protocols for OOD Detection
- On Challenging Aspects of Reproducibility in Deep Anomaly Detection