LightPainter: Interactive Portrait Relighting with Freehand Scribble
Yiqun Mei1
He Zhang2
Xuaner Zhang2
Jianming Zhang2
Zhixin Shu2
Yilin Wang2
Zijun Wei2
Shi Yan2
HyunJoon Jung2
Vishal M. Patel1
1Johns Hopkins University
2Adobe Inc.
[Paper]
[Web Demo]
LightPainter is an interactive lighting editing system that takes in an input image with freehand scribbles drawn on top and renders the correspondingly relit portrait. It enables creative portrait lighting editing (left) and allows users to reproduce a target lighting effect with ease.

Abstract

Recent portrait relighting methods have achieved realistic results of portrait lighting effects given a desired lighting representation such as an environment map. However, these methods are not intuitive for user interaction and lack precise lighting control. We introduce LightPainter, a scribble-based relighting system that allows users to interactively manipulate portrait lighting effect with ease. This is achieved by two conditional neural networks, a delighting module that recovers geometry and albedo optionally conditioned on skin tone, and a scribble-based module for relighting. To train the relighting module, we propose a novel scribble simulation procedure to mimic real user scribbles, which allows our pipeline to be trained without any human annotations. We demonstrate high-quality and flexible portrait lighting editing capability with both quantitative and qualitative experiments. User study comparisons with commercial lighting editing tools also demonstrate consistent user preference for our method.


Demo Video



Method

An overview of LightPainter. As the user starts scribbling with our interactive application, the neural modules interactively render a realistic relit image that is faithful to the user input.


Creative Portrait Lighting



Paper and Supplementary Material

LightPainter: Interactive Portrait Relighting with Freehand Scribble
CVPR2023.





Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang.