Intro

I completed my Ph.D. at the Machine Learning Lab at the University of Freiburg, where I was supervised by Prof. Frank Hutter. I am now a Senior Researcher in the Applied Sciences Group at Microsoft in Redmond, where I work on on-device LLMs. Prior to starting my Ph.D. studies, I received my master's degree from ETH Zurich in 2021, where I worked on efficient generative models at the Computer Vision Lab, supervised by Dr. Zhiwu Huang and Dr. Suryansh Kumar. I was awarded the prestigious ESOP scholarship for my masters studies. I received my Bachelor's degree from Vellore Institute of Technology (VIT). During my Ph.D. studies I also interned at the Applied Sciences Group (ASG) at Microsoft Cambridge, where I worked on 2-bit quantization and finetuning, supervised by Dr. Pashmina Cameron and Dr. James Hensman.

My research focuses on automating and optimizing foundation model inference-particularly for large language models (LLMs) and vision models-to facilitate inference efficiency in their real-world applications. While training these models incurs a significant but one-time expense, the costs of inference can escalate quickly over the model's lifecycle. To address this, my work develops automated techniques for pruning, quantization, and knowledge distillation, reducing the manual effort in tuning these methods. Ultimately, my goal is to make foundation models more accessible and sustainable across diverse domains. As an open-source effort towards this goal, I co-developed the library whittle, with several others.

My recent research interests include the following topics:

• Large Language Model Compression with Neural Architecture Search (NeurIPS 2024 Compression Workshop).
• HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models (NeurIPS DBT 2024)
• whittle: A Library for Sub-Network Extraction from Large Language Models (AutoML 2026, oral, Best Paper Award).

• Multi-Objective Differentiable Architecure Search (ICML 2024 WANT Workshop)
• Weight-Entanglement Meets Gradient-Based Neural Architecture Search (AutoML 2024)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition (NeurIPS 2023, oral).
• Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars (NeurIPS 2023).