Ze Yang    杨泽

I am a Ph.D. candidate at the University of Toronto and the Vector Institute, advised by Prof. Raquel Urtasun. I am also a Senior Research Scientist at Waabi, focusing on self-driving technology. Before joining Waabi, I was a Research Scientist at Uber ATG. Prior to moving to Toronto, I received a Master's Degree in Computer Science from Peking University, advised by Prof. Liwei Wang.

My research interests focus on the intersection of 3D computer vision, robotics, and machine learning. I'm particularly intersted in neural scene representation, 3D reconstruction and modeling, sensor simulation, and visual perception.

Email  /  CV  /  Google Scholar  /  Github  /  LinkedIn  /  Twitter

Ze Yang
News

  • NEW 2024/01: Our work on 4D World Model for Self-Driving was accepted by ICLR 2024. Congrats to Lunjun!
  • NEW 2023/12: I gave a talk at MILA Robot Learning Seminar on Learning in-the-wild Sensor Simulation for Autonomous Driving.
  • NEW 2023/09: LightSim (lighting simulation) was accepted by NeurIPS 2023. Congrats to Ava, Gary and Jingkang!
  • NEW 2023/07: I gave a talk at Shanghai AI Lab on in-the-wild 3D reconstruction, modeling and sensor simulation.
  • NEW 2023/07: I gave a talk at Toronto Computational Imaging Group on in-the-wild 3D reconstruction, modeling and sensor simulation.
  • 2023/07: NeuRas (large scene reconstruction & rendering) was accepted by ICCV 2023. Congrats to Jeffrey!
  • 2023/07: LiDAR-DG (simulation domain gap analysis) was accepted by ICCV 2023. Congrats to Siva and Andrei!
  • 2023/03: UniSim (3D scene reconstruction, modeling and closed-loop sensor simulation for self-driving) was accepted by CVPR 2023 as highlight.
  • 2023/01: NeuSim (3D object reconstruction for sensor simulation) was accepted by ICRA 2023.
  • 2022/10: CADSim (3D object reconstruction for sensor simulation) was accepted by CoRL 2022.
  • 2022/10: I gave a talk at ByteDance Research on in-the-wild 3D reconstruction for sensor simulation.
  • 2022/03: RBGNet (3D object detection) was accepted by CVPR 2022. Congrats to Haiyang!
  • 2021/06: I gave a talk at CVPR 2021 tutorial All About Self-Driving on dynamic asset reconstruction for simulation.
  • 2021/03: I joined Waabi to work on next-generation self-driving solution.
  • 2021/02: S3 (3D human reconstruction and modeling) was accepted by CVPR 2021.
  • 2020/10: LiME (3D human reconstruction, modeling and sensor simulation) was accepted by CoRL 2020.
  • 2020/09: I started my Ph.D. journey at the University of Toronro.
  • 2020/06: I joined Uber Advanced Technologies Group (Uber ATG) to work on self-driving technology.
  • 2020/06: I graduated from Peking University.
  • 2020/06: Follow-up work Dense RepPoints was accepted by ECCV 2020.
  • 2019/07: RepPoints (representing objects as point sets for visual perception) was accepted by ICCV 2019.
  • 2018/12: I started my internship at Microsoft Research Asia to work on visual perception.

Research

My research goal is to build controllable and realistic digital twins using real-world data, with the purpose of creating immersive virtual environments that facilitate the development and evaluation of robotic systems, such as self-driving vehicles, in a safe, controlled, reactive, and cost-effective manner. Towards this goal, I have delved into various areas over the past few years, such as 3D reconstruction spanning from individual objects to large-scale scene; 3D modeling encompassing both rigid and dynamic content; and closed-loop sensor simulation for camera and LiDAR data. During the earlier stages of my research, I'm interested in learning flexible and structural representation for visual perception. Representative papers are highlighted.

World Model Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion
Lunjun Zhang, Yuwen Xiong, Ze Yang, Sergio Casas, Rui Hu, Raquel Urtasun
International Conference on Learning Representations (ICLR), 2024
bibtex / paper / project / video
LightSim LightSim: Neural Lighting Simulation for Urban Scenes
Ava Pun*, Gary Sun*, Jingkang Wang*, Yun Chen, Ze Yang, Siva Manivasagam, Wei-Chiu Ma, Raquel Urtasun
Neural Information Processing Systems (NeurIPS), 2023
bibtex / paper / project / video / 4K demo
NeuRas Real-Time Neural Rasterization for Large Scenes
Jeffrey Yunfan Liu, Yun Chen*, Ze Yang*, Jingkang Wang, Siva Manivasagam, Raquel Urtasun
International Conference on Computer Vision (ICCV), 2023
bibtex / paper / project / video
Domain Gap Towards Zero Domain Gap: A Comprehensive Study of Realistic LiDAR Simulation for Autonomy Testing
Siva Manivasagam*, Ioan Andrei Bârsan*, Jingkang Wang, Ze Yang, Raquel Urtasun
International Conference on Computer Vision (ICCV), 2023
bibtex / paper / project / video
UniSim UniSim: A Neural Closed-Loop Sensor Simulator
Ze Yang*, Yun Chen*, Jingkang Wang*, Siva Manivasagam*, Wei-Chiu Ma, Anqi Joyce Yang, Raquel Urtasun
Conference on Computer Vision and Pattern Recognition (CVPR), 2023   (Highlight)
bibtex / blog / paper / poster / project / supp / video / 4K demo
NeuSim Reconstructing Objects in-the-wild for Realistic Sensor Simulation
Ze Yang, Siva Manivasagam, Yun Chen, Jingkang Wang, Rui Hu, Raquel Urtasun
International Conference on Robotics and Automation (ICRA), 2023
bibtex / paper / poster / project / reconstructions / video
CADSim CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Simulation
Jingkang Wang, Siva Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Bârsan, Anqi Joyce Yang, Wei-Chiu Ma, Raquel Urtasun
Conference on Robot Learning (CoRL), 2022
bibtex / paper / poster / project / video
RBGNet RBGNet: Ray-based Grouping for 3D Object Detection
Haiyang Wang, Shaoshuai Shi, Ze Yang, Rongyao Fang, Qi Qian, Hongsheng Li, Bernt Schiele, Liwei Wang
Conference on Computer Vision and Pattern Recognition (CVPR), 2022
bibtex / code / paper
S3 S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Ze Yang, Shenlong Wang, Siva Manivasagam, Zeng Huang, Wei-Chiu Ma, Xinchen Yan, Ersin Yumer, Raquel Urtasun
Conference on Computer Vision and Pattern Recognition (CVPR), 2021
bibtex / paper / poster / project / video
LiME Recovering and Simulating Pedestrians in the Wild
Ze Yang, Siva Manivasagam, Ming Liang, Bin Yang, Wei-Chiu Ma, Raquel Urtasun
Conference on Robotic Learning (CoRL), 2020   (Spotlight)
bibtex / paper / simulation / video

Dense RepPoints Dense RepPoints: Representing Visual Objects with Dense Point Sets
Ze Yang*, Yinghao Xu*, Han Xue*, Zheng Zhang, Raquel Urtasun, Liwei Wang, Steve Lin, Han Hu
European Conference on Computer Vision (ECCV), 2020
bibtex / code / paper / slides / video
RepPoints RepPoints: Point Set Representation for Object Detection
Ze Yang*, Shaohui Liu*, Han Hu, Liwei Wang, Steve Lin
International Conference on Computer Vision (ICCV), 2019
bibtex / blog / code / mmdet / paper / poster / slides
Relation Network Learning Relationships for Multi-view 3D Object Recognition
Ze Yang, Liwei Wang.
International Conference on Computer Vision (ICCV), 2019
bibtex / paper / poster
NTS-Net Learning to Navigate for Fine-grained Classification
Ze Yang, Tiange Luo, Dong Wang, Zhiqiang Hu, Jun Gao, Liwei Wang
European Conference on Computer Vision (ECCV), 2018
bibtex / code / paper / poster / slides
Super-Resolution Single Image Super-Resolution with a Parameter Economic Residual-Like Convolutional Neural Network
Ze Yang, Kai Zhang, Yudong Liang, Jinjun Wang
International Conference on Multimedia Modeling, 2017   (Oral)
bibtex / paper


Work Experience

Waabi Innovation

Senior Researcher, 2023/09 - Present
Researcher II, 2022/06 - 2023/09
Researcher, 2021/03 - 2022/06
Woking with Prof. Raquel Urtasun on next-gen sensor simulation for self-driving.

Uber Advanced Technologies Group

Research Scientist, 2020/06 - 2021/03
Research Intern, 2019/10 - 2020/06
Woking with Prof. Raquel Urtasun on human modeling and simulation for self-driving.

Microsoft Research Asia

Research Intern, 2018/12 - 2019/09
Working with Dr. Han Hu, Dr. Jifeng Dai, Dr. Steve Lin on visual perception.
Academic Service

  • Conference Reviewer: CVPR, ICCV, ECCV, ACCV, WACV, ICRA, IROS, NeurIPS

  • Journal Reviewer: TPAMI, TCSVT, TMM


Website template comes from Jon Barron.