A Light Stage on Every Desk

Soumyadip Sengupta, Brian Curless, Ira Kemelmacher-Shlizerman, Steve Seitz, 2021


Key idea:
While watching a YouTube video the monitor is projecting patterns on the face. Our algorithm shows how to leverage the patterns to relight and preserve privacy.

project page

TryOnGAN: Body-Aware Try-On via Layered Interpolation

Kathleen M Lewis, Srivatsan Varadharajan, Ira Kemelmacher-Shlizerman, SIGGRAPH 2021


Key idea:
switch garments and change their size to adjust to new humans via stylegan latent space interpolation.
project page

Real-Time High Resolution Background Matting

Peter Lin*, Andrey Ryabtsev*, Soumyadip Sengupta, Brian Curless, Steve Seitz, Ira Kemelmacher-Shlizerman, CVPR 2021 (oral, candidate for best paper award)

Key idea:
We achieve
HD matting at 60fps, by estimating alpha per frame in a sequence of CNNs from coarse to refined estimation.
project page

Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild

Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman, 2021
project page

The Cone of Silence: Speech Separation by Localization

Teerapat Jenrungrot*, Vivek Jayaram*, Steve Seitz, Ira Kemelmacher-Shlizerman, NeurIPS, 2020 (oral)

Key idea:
Build a mic array that allows to use spatial info for speaker separation and denoising
project page | Science article

Reconstructing NBA players

Luyang Zhu, Konstantinos Rematas, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman, ECCV 2020 (spotlight)


Key idea: Use synthetic NBA2k data to estimate pose and mesh from photos of real NBA players. project page

Lifespan Age Transformation Synthesis

Roy Or-El, Soumyadip Sengupta, Ohad Fried, Eli Shechtman, and Ira Kemelmacher-Shlizerman, ECCV, 2020

Key idea: GAN for unpaired aging while preserving the identity of the person.
project page | paper | dataset | code | colab

Background Matting: The World is Your Green Screen

Soumyadip Sengupta, Vivek Jayaram, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman, CVPR 2020
project page | paper | code | Two Minute Papers video | Microsoft AI using our code | CEO of Microsoft Satya Nadella talks about our work.

Photo Wake-Up: 3D Character Animation from a Single Photo

Chung-Yi Weng, Brian Curless, Ira Kemelmacher-Shlizerman, CVPR 2019
project page | paper | MIT Tech Review | NVIDIA

Soccer on your Tabletop

Konstantinos Rematas, Ira Kemelmacher-Shlizerman, Brian Curless, Steve Seitz
CVPR 2018
project page | paper | code | Forbes | NVIDIA

Audio to Body Dynamics

Eli Shlizerman, Lucio M. Dery, Hayden Schoen, and Ira Kemelmacher-Shlizerman, CVPR 2018
project page | paper | code | CNBC | Facebook Research blog

Video to Fully Automatic 3D Hair Model

Shu Liang, Xiufeng Huang, Xianyu Meng, Kunyao Chen, Linda Shapiro, Ira Kemelmacher-Shlizerman, SIGGRAPH Asia 2018
project page | paper | data

Synthesizing Obama: Learning Lip Sync from Audio

Supasorn Suwajanakorn, Steve Seitz, and Ira Kemelmacher-Shlizerman, SIGGRAPH 2017,
project page | paper | training videos | code | Video @ Two Minute Papers
1Million + views on YouTube

Transfiguring Portraits

Ira Kemelmacher-Shlizerman, SIGGRAPH 2016
video| paper | The Daily Mail

Head Reconstruction from Internet Photos

Shu Liang, Linda Shapiro, Ira Kemelmacher-Shlizerman, ECCV 2016
project page | paper | dataset

The MegaFace Benchmark

  • The MegaFace Benchmark, Kemelmacher-Shlizerman, Seitz, Miller, Brossard. CVPR 2016

  • Level Playing Field for Million Scale Face Recognition, Nech and Kemelmacher-Shlizerman, CVPR 2017

project page | paper CVPR 2016 | paper CVPR 2017 | The Atlantic

What Makes Tom Hanks Look Like Tom Hanks

Suwajanakorn, Seitz, Kemelmacher-Shlizerman, ICCV 2015
project page | paper | video (450K views on YouTube)

The Meme Quiz: A Facial Expression Game Combining Human Agency and Machine Involvement

K. Tuite and I. Kemelmacher-Shlizerman, Foundations of Digital Games (FDG), 2015
project video

3D Face Hallucination from a Single Depth Frame

S. Liang, I. Kemelmacher-Shlizerman, L.G. Shapiro, International Conf. on 3D Vision (3DV), Tokyo, Dec 2014
project webpage

Total Moving Reconstruction

S. Suwajanakorn, I. Kemelmacher-Shlizerman, S.M. Seitz, European Conference on Computer Vision (ECCV), Zurich, Sep 2014
project page | paper | video

Illumination-aware Age Progression

Ira Kemelmacher-Shlizerman, S. Suwajanakorn, S.M. Seitz, CVPR 2014
project webpage

Exploring Photobios

  • Kemelmacher-Shlizerman, Shechtman, Garg and Seitz, SIGGRAPH 2011

  • Moving Portraits, I. Kemelmacher-Shlizerman, E. Shechtman, R. Garg, S.M. Seitz, Communications of the ACM, Research Highlights, 2014.

project page | paper SIGGRAPH | paper ACM Research Highlights | video | Face Movies of Google
Appeared on CACM cover, and SIGGRAPH cover.

Internet based morphable model

Ira Kemelmacher-Shlizerman

International Conf. on Comp. Vision (ICCV), Sydney, Dec 2013
project page | paper | video

3D Face Reconstruction from Single Two-Tone and Color Images
Ira Kemelmacher-Shlizerman, R. Basri and B. Nadler

Shape Perception in Human and Computer Vision, Springer London, 2013
book chapter

Global Motion Estimation from Point Matches

Mica Arie-Nachimson, Shahar Kovalsky, Ira Kemelmacher-Shlizerman, Amit Singer, Ronen Basri, 3DimPVT (3DV), 2012
paper

Collection Flow

Ira Kemelmacher-Shlizerman and Steven M. Seitz, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June, 2012
project page | paper | video

Face Reconstruction in the Wild

Ira Kemelmacher-Shlizerman and Steven M. Seitz, International Conference on Computer Vision (ICCV), Nov, 2011
project page | paper

Exploring Photobios

Ira Kemelmacher-Shlizerman, Eli Shechtman, Rahul Garg and Steven M. Seitz, ACM Transactions on Graphics (SIGGRAPH), Aug, 2011

SIGGRAPH cover and in the Technical papers trailer. Tech transfered to Google, as the Face Movies feature in Picasa.
project page | Face Movies

Being John Malkovich

Ira Kemelmacher-Shlizerman, Aditya Sankar, Eli Shechtman and Steven M. Seitz

In European Conference on Computer Vision (ECCV), 2010
paper | video

3D Face Reconstruction from a Single Image using a Single Reference Face Shape

Ira Kemelmacher-Shlizerman, Ronen Basri, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2010
project page

3D Shape Reconstruction of Mooney Faces

Ira Kemelmacher-Shlizerman, Ronen Basri and Boaz Nadler, IEEE Conf. on Computer Vision Pattern Recognition (CVPR), 2008
paper

A Theory of Locally Low Dimensional Light Transport

Dhruv Mahajan, Ira Kemelmacher-Shlizerman, Ravi Ramamoorthi and Peter Belhumeur, SIGGRAPH 2007
project page

Photometric Stereo with General, Unknown Lighting

Ronen Basri, David W. Jacobs and Ira Kemelmacher, International Journal of Computer Vision (IJCV), 72(3):239-257, 2007
project page

Molding Face Shapes by Example

Ira Kemelmacher and Ronen Basri, In European Conference on Computer Vision (ECCV), 2006
project page

Indexing with Unknown Illumination and Pose

Ira Kemelmacher and Ronen Basri, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2005
project page