Speaker: Wenzheng Chen University of Toronto
Time: 2023-05-09 16:00-2023-05-09 17:00
Venue: C19-2 or 腾讯会议：https://meeting.tencent.com/dm/pf1RylREXVgw

Abstract:

The ability to infer 3D properties, such as geometry, texture, material, and light from 2D photographs is one of the most fundamental problems in computer vision and plays a key in many domains such as AR/VR, robotics, autonomous driving, and Metaverse. To tackle this problem, various imaging systems and reconstruction algorithms are proposed. As a two-stage procedure, 3D perception consists of “imaging” and “reconstruction”, wherein the former involves the use of devices to measure the scene and acquire the measurement, such as photographs, while the latter applies reconstruction algorithms on the measurement to recover 3D properties. While previous methods focus more on reconstruction algorithms and ignore the knowledge in the imaging stage, in this talk, we show how to develop a holistic method to connect ‘‘imaging‘’ and ‘’reconstruction‘’ together, utilizing the information from both sides. Specifically, we propose to model various imaging systems in a differentiable way and connect them with the reconstruction algorithms, such that the information (e.g., gradients) can be freely propagated bidirectionally. Such a connection boosts 3D perception in 2 directions: “learning for imaging‘’ and “imaging for learning“. The former passes the information of imaging systems to the reconstruction algorithms, which helps design the algorithms that perfectly match the chosen imaging systems. The latter propagates the information from reconstruction algorithms to imaging systems, helping explore the best settings they should adopt under specific algorithms. We demonstrate that the proposed method brings huge improvement and successfully pushes the boundary of 3D perception.

Short Bio:

Wenzheng Chen is a final-year Ph.D. student at University of Toronto, supervised by Prof. Sanja Fidler and Prof. Kyros Kutulakos. He is also a research scientist at NVIDIA. His research mainly focuses on the intersection of computational photography and machine learning, with a special interest in estimating 3D properties from various imaging systems. His long-term research goal is to explore all kinds of optical and physical rules in the imaging systems and combine them with deep learning to boost 3D perception. He will join PKU as an assistant professor in 2024. For more information, please visit http://www.cs.toronto.edu/~wenzheng/.