SCALE@NTU Research Webinar Dec 2021

This research webinar on Teams is organized by Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU) to share the research work in the Corp Lab. For registration, please visit:

https://wis.ntu.edu.sg/pls/webexe88/REGISTER_NTU.REGISTER?EVENT_ID=OA21120715341081

Talk 1: Neural Portrait Relighting

Portrait relighting is a much sought-after advanced capability in digital photography. While professional photographers may take the time to set up perfect portrait shots meticulously with special equipment, normal users often only care about improving portraits retrospectively, e.g. selfie beautification, for which relighting can often help. Commercial applications such as film editing, telepresence and virtual background can also benefit from this. In this talk, we would like to introduce a referred-based neural portrait relighting system that can relight a casually taken portrait image by simply referring to another image with the desired lighting condition, and also introduce a shadow-aware portrait relighting to improve existing virtual background effect such that the user appears more realistically embedded in a given virtual background.

Speaker: Song Guoxian, Research Scientist, ByteDance | SCALE@NTU Alumnus

Guoxian Song received his B.S. in Mathematics from the University of Science and Technology of China in 2016, and finished his Ph.D. in Computer Science at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU) in 2021. Currently, he is a Research Scientist at ByteDance in California, USA. His research interests are image-based 3D portrait reconstruction and manipulation including relighting and stylization.

Talk 2: Domain Adaptive Video Segmentation via Temporal Consistency Regularization

Video semantic segmentation is an essential task for the analysis and understanding of videos. Recent efforts largely focus on supervised video segmentation by learning from fully annotated data, but the learnt models often experience clear performance drops while applied to videos of a different domain. We designed a domain adaptive video segmentation network that addresses domain gaps in videos by temporal consistency regularization (TCR) for consecutive frames of target-domain videos. This network consists of two novel and complementary designs. The first is cross-domain TCR that guides the prediction of target frames to have similar temporal consistency as that of source frames via adversarial learning. The second is intra-domain TCR that guides unconfident predictions of target frames to have similar temporal consistency as confident predictions of target frames. Extensive experiments demonstrate the superiority of our proposed domain adaptive video segmentation network which outperforms multiple baselines consistently by large margins.

Speaker: Dr Guan Dayan, Research Fellow, SCALE@NTU

Dayan Guan received his Bachelor's degree from Central South University in Jun 2014 and obtained his Ph.D. degree from Zhejiang University in Sep 2019. Since Nov 2019, he has been a Research Fellow at Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU). His research interests include artificial intelligence, scene understanding, and unsupervised learning.