Plain Language Summary. Machine learning (ML) models can help us solve complex problems—including those in medicine and science. However, ML models don't always work well when used with new groups of patients or in different lab settings—and sometimes, they can fail in serious ways. My research focuses on how we can build ML systems that are safer and more reliable. We achieve this by:

1. designing models that mimic how humans think and make decisions; and
2. learning when and where we can trust model predictions.

We show how these techniques can help us better solve real-world problems in both medicine and the lab.

Abstract. Machine learning systems are becoming increasingly adopted in high-stakes applications, including clinical medicine and scientific discovery. In these settings, the predictions made by learned algorithms can have profound consequences. While modern machine learning models have achieved impressive empirical performance, they often behave unpredictably outside their training distribution, raising concerns about their reliability, fairness, and safety. These limitations are especially pronounced in domains where failure can be costly or irreversible, such as healthcare and scientific discovery. As a result, there is a growing need for AI systems that are not only performant, but also safe and generalizable when faced with new, diverse, and unforeseen inputs in the wild.

This dissertation investigates how we can design such ML systems to make reliable predictions across the range of inputs they might encounter in the real-world. We explore this question through two complementary hypotheses. First, by incorporating structured priors generated from natural language and domain knowledge of biological systems directly into model architectures, we can build systems that are more generalizable. We show how such ML systems that are interpretable-by-design are better aligned with human reasoning to solve challenging domain-specific tasks. Second, we show how leveraging adversarial supervision from auxiliary neural networks can help us estimate when and where black-box models predictions can be trusted. We demonstrate how this framework can be readily adapted to solve a wide range of optimization problems in medicine and science. In summary, this dissertation provides a principled framework for making machine learning systems more aligned, robust, and actionable in safety-critical biomedical applications.