Random Matrix Methods for Machine Learning: “Lossless” Compression of Large Neural Networks

演讲人: Zhenyu Liao 华中科技大学
时间: 2022-08-18 10:30-2022-08-18 11:30
地点:FIT 1-202 + Tencent Meeting(ID:102-737-367)

The advent of the Big Data era has triggered a renewed interest in large-dimensional machine learning (ML) and (deep) neural networks. These methods, being developed from small-dimensional intuitions, often behave dramatically different from their original designs and tend to be inefficient on large-dimensional datasets. By assuming both dimension and size of the datasets to be large, recent advances in random matrix theory (RMT) provide novel insights, allowing for a renewed understanding and the possibility to design more efficient machine learning approaches, thereby opening the door to completely new paradigms.

In this talk, we will start with the “curse of dimensionality” phenomenon in high dimensions, and highlight many counterintuitive phenomena in ML that arise when large-dimensional data are considered. By focusing on the use case of neural network compression, and by considering the data dimension and/or the ML systems to be large, we discuss how RMT is able to provide a renewed understanding of modern ML.


Zhenyu Liao received his M.Sc. in Signal and Image Processing in 2016, and his Ph.D. in Computer Science in 2019, both from University of Paris-Saclay, France. In 2020 he was a postdoctoral researcher with the Department of Statistics, University of California, Berkeley. He is currently an associated professor with the School of Electronic Information and Communications, Huazhong University of Science and Technology (HUST), China. His research interests are broadly in machine learning, signal processing, random matrix theory, and high-dimensional statistics. He published more than 20 papers on top-tier machine learning conferences such as ICML, NeurIPS, ICLR, COLT, AISTATS, etc., and he co-authored the book “Random Matrix Methods for Machine Learning.” He is the recipient of the 2021 Wuhan Youth Talent Fellowship, the 2021 East Lake Youth Talent Program Fellowship of HUST, the 2019 ED STIC Ph.D. Student Award, and the 2016 Supélec Foundation Ph.D. Fellowship of University of Paris-Saclay, France.