StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation, 2022
Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, JJ Park, Ira Kemelmacher-Shlizerman
Our method is trained on single-view RGB data only while solving two main challenges in 3D-aware GANs: 1) high-resolution, view-consistent generation of the RGB images, and 2) detailed 3D shape.
HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video, 2022
Chung-Yi Weng, Brian Curless, Pratul Srinivasan, Jonathan T. Barron, Ira Kemelmacher-Shlizerman
We introduce a free-viewpoint rendering method -- HumanNeRF -- that works on a given monocular video of a human performing complex body motions, e.g. a video from YouTube. Our method enables pausing the video at any frame and rendering the subject from arbitrary new camera viewpoints or even a full 360-degree camera path for that particular frame and body pose.
A Light Stage on Every Desk
Soumyadip Sengupta, Brian Curless, Ira Kemelmacher-Shlizerman, Steve Seitz, ICCV 2021
While watching a YouTube video the monitor is projecting patterns on the face. Our algorithm shows how to leverage the patterns to relight and preserve privacy.
TryOnGAN: Body-Aware Try-On via Layered Interpolation
Kathleen M Lewis, Srivatsan Varadharajan, Ira Kemelmacher-Shlizerman, SIGGRAPH 2021
switch garments and change their size to adjust to new humans via stylegan latent space interpolation.
Real-Time High Resolution Background Matting
Peter Lin*, Andrey Ryabtsev*, Soumyadip Sengupta, Brian Curless, Steve Seitz, Ira Kemelmacher-Shlizerman, CVPR 2021 (oral, best student paper honorable mention; 0.08% of 7093 papers)
We achieve HD matting at 60fps, by estimating alpha per frame in a sequence of CNNs from coarse to refined estimation.
Reconstructing NBA players
Luyang Zhu, Konstantinos Rematas, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman, ECCV 2020 (spotlight)
Key idea: Use synthetic NBA2k data to estimate pose and mesh from photos of real NBA players. project page
Kemelmacher-Shlizerman, Shechtman, Garg and Seitz, SIGGRAPH 2011
Moving Portraits, I. Kemelmacher-Shlizerman, E. Shechtman, R. Garg, S.M. Seitz, Communications of the ACM, Research Highlights, 2014.