Design Space Exploration and Analysis for AI Compilers @Nvidia Redmond
Date:
Abstract
Abstract: Dr. Size Zheng received his Ph.D. in Computer Science from Peking University and is currently a Research Scientist at ByteDance. His primary research areas include compilers, high-performance operators, and hardware-software co-optimization. He has published 20 papers in top conferences such as ASPLOS, ISCA, MICRO, and HPCA.
In this talk, he will summarize his doctoral research achievements in AI compilers, introducing compilation analysis and optimization techniques required for both graph- and operator-level compilation. These techniques include compiler-based automatic differentiation, automatic fusion, automatic mapping, design space exploration, and auto-tensorization. The talk will also cover his recent progress in distributed compilers and future work, such as computation-communication overlapping via compilers in GPU systems, integration of compilers with template libraries, and the potential of large language models (LLMs) for compilers.