Intro

I'm a final year Ph.D. student in the Machine Learning Lab at the University of Freiburg, supervised by Prof. Frank Hutter. Prior to starting my Ph.D studies, I received my masters degree from ETH Zurich in 2021, where I worked on efficient generative models at the Computer Vision Lab, supervised by, Dr. Zhiwu Huang and Dr. Suryansh Kumar. I was awarded the prestigious ESOP scholarship for my masters studies. I received my Bachelor's degree from Vellore Institute of Technology (VIT). During my Ph.D. studies I have also interned at the Applied Sciences Group (ASG) at Microsoft Cambridge, where I worked on 2-bit quantization and finetuning, supervised by Dr. Pashmina Cameron and Dr. James Hensman.

My research focuses on automating and optimizing foundation model inference-particularly for large language models (LLMs) and vision models-to facilitate inference efficiency in their real-world applications. While training these models incurs a significant but one-time expense, the costs of inference can escalate quickly over the model's lifecycle. To address this, my work develops automated techniques for pruning, quantization, and knowledge distillation, reducing the manual effort in tuning these methods. Ultimately, my goal is to make foundation models more accessible and sustainable across diverse domains. As an open-source effort towards this goal, I develop and maintain the library whittle, with several others.

My recent research interests include the following topics:

• Large Language Model Compression with Neural Architecture Search (NeurIPS 2024 Compression Workshop)
• HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models (NeurIPS DBT 2024)
• Where to Begin: Efficient Pretraining via Subnetwork Selection and Distillation

• Multi-Objective Differentiable Architecure Search (ICML 2024 WANT Workshop)
• Weight-Entanglement Meets Gradient-Based Neural Architecture Search (AutoML 2024)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition (NeurIPS 2023, oral)
• Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars (NeurIPS 2023)