TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training
Sebastian Loeschcke, David Pitt, Robert Joseph George, Jiawei Zhao, Cheng Luo, Yuandong Tian, Jean Kossaifi, Anima Anandkumar
Pioneering tensor methods and neural operators for scientific machine learning
I lead research at NVIDIA in the field of AI for Engineering and Scientific Simulation, where my work focuses on advancing new algorithmic paradigms to solve complex physics-based problems. My core research combines tensor methods with deep learning to develop efficient and powerful neural architectures.
A central part of my mission is to democratize advanced computational techniques. To that end, I created and lead the development to two widely-used open-source libraries: TensorLy, for tensor methods, and NeuralOperator, for scientific machine learning, helping to accelerate scientific discovery for the broader research community.
Prior to NVIDIA, I was a founding member of the Samsung AI Center in Cambridge. My academic foundation includes a French Engineering Diploma in Mathematics, Computer Science, and Finance and a BSc in advanced mathematics. I then completed my PhD in Artificial Intelligence at Imperial College London.
A selection of projects and research I'm particularly proud of.
Easy configuration of Python projects with minimal boilerplate. A clean, Pythonic approach to configuration management, particularly adapted for deep learning research.
Sebastian Loeschcke, David Pitt, Robert Joseph George, Jiawei Zhao, Cheng Luo, Yuandong Tian, Jean Kossaifi, Anima Anandkumar
Kamyar Azizzadenesheli, Nikola Kovachki, Zongyi Li, Miguel Liu-Schiaffini, Jean Kossaifi, Anima Anandkumar
Jean Kossaifi, Nikola Kovachki, Zongyi Li, David Pitt, Miguel Liu-Schiaffini, Robert Joseph George, Boris Bonev, Kamyar Azizzadenesheli, Julius Berner, Valentin Duruisseaux, others
Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos, Maja Pantic
Jean Kossaifi, Antoine Toisoul, Adrian Bulat, Yannis Panagakis, Maja Pantic
Jean Kossaifi, Yannis Panagakis, Anima Anandkumar, Maja Pantic
Majid Janzamin, Rong Ge, Jean Kossaifi, Anima Anandkumar
Guest speaker at the Oak Ridge National Laboratory's AI Expo, where I presented my work on Neural Operators for AI in Science and Engineering
Co-authored a book chapter, "Tensor methods in deep learning", in the book Signal Processing and Machine Learning Theory
Co-organized workshop with Topal team at INRIA Bordeaux on efficient scaling of neural architectures, covering re-materialization, offloading, scheduling and model pipelining
Co-organizer, ICML Workshop on Advancing Neural Network Training (WANT): Computational Efficiency, Scalability, and Resource Optimization