Intro

I'm a 3rd year Ph.D. in the Machine Learning Lab at the University of Freiburg, supervised by Prof. Frank Hutter. Prior to starting my Ph.D studies, I received my masters degree from ETH Zurich in 2021, where I did my thesis at Computer Vision Lab, supervised by, Dr. Zhiwu Huang and Dr. Suryansh Kumar. I was awarded the prestigious ESOP scholarship for my masters studies. I received my B.Tech. degree from Vellore Institute of Technology (VIT).

My research focuses on automating and optimizing foundation model inference--particularly for large language models (LLMs) and vision models--to facilitate inference efficiency in their real-world applications. While training these models incurs a significant but one-time expense, the costs of inference can escalate quickly over the model's lifecycle. To address this, my work develops automated techniques for pruning, quantization, and knowledge distillation, reducing the manual effort in tuning these methods. Ultimately, my goal is to make foundation models more accessible and sustainable across diverse domains. As an open-source effort towards this goal, I develop and maintain the library whittle, with several others.

My recent research interest includes the following topics:

• Large Language Model Compression with Neural Architecture Search (NeurIPS 2024 Compression Workshop)
• HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models (NeurIPS DBT 2024)

• Multi-Objective Differentiable Architecure Search (ICML 2024 WANT Workshop)
• Weight-Entanglement Meets Gradient-Based Neural Architecture Search (AutoML 2024)
• Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition (NeurIPS 2023, oral)
• Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars (NeurIPS 2023)