Hi~ I am Jingyuan Liu, a Machine Learning Engineer at Adobe Lightroom working with Jimei Yang and Simon Chen. I received a M.S. in Data Science at Harvard University and a B.S. in Computer Science at Fudan University. Previously, I was a research assistant at Upenn GRASP lab advised by Prof. Jianbo Shi and Harvard EconCS lab advised by Prof. David C.Parkes.

My work focused on semantic segmentation and auto white balance. I am particularly interested in the intersection between language and vision. Recently, I am working on 3D depth estimation and AI photo critique.

Email: jingyliu@adobe.com / CV / Google Scholar / Github / Linkedin

Mobile-friendly transformer-based portrait segmentation
Jingyuan Liu, Jimei Yang, Qing Liu, Simon Chen, Yuhong Wu
US Patent, 2022

This work aims to develop a robustness mobile-friendly model with small model size, high inference speed and reasonable memory usage for the portrait segmentation. We introduced high-resolution features in the encoder part to supply more information and utilized two segmentation head, one of it served as refinement stage to improve accuracy across all categories. To leverage datasets with different annotation, we adopt an unified training strategy with adaptive loss design for different data. We lowered memory usage by reducing the embedding dimension and substitute concatenate operator with series sum inside segmentation head without sacrificing accuracy.

Semantic-Aware auto white balance
Jingyuan Liu, Simon Chen, Brian Price, Xin Lu, Calista Chandler, He Zhang
US Patent, P11299-US, 2022

We introduce a novel and unified WB algorithm that could handle both raw device RGB (scene referred raw image) and standard RGB images (output referred non-raw image). Our algorithm is semantic aware that we put emphasis on skin tone and are easily to extend to other objects colors. Our method is easy to personalize and more convenient to integrate into products with good performance compared to large deep learning models.

Weakly Supervised Image Retrieval via Coarse-scale Feature Fusion and Multi-level Attention Blocks
Xinyao Nie, Hong Lu, Zijian Wang, Jingyuan Liu, Zehua Guo
ACM International Conference on Multimedia Retrieval (ICMR), 2019   [Paper]  [Code]

We deisgn a network for image retrieval without human annotations like bounding box, which utilizes coarse-scale feature fusion and generates the attentive local features via combining the information from different intermediate layers. Two attentions blocks help extracted detailed information.

Deep Fashion Analysis with Feature Map Upsampling and Landmark-driven Attention
Jingyuan Liu, Hong Lu
Workshop of European Conference on Computer Vision (ECCV), 2018   [Paper]  [Code]

We propose an end-to-end fashion analysis network that addresses category classification and attribute prediction simultaneously, via improving the resolution of heatmaps through upsampling for more accurate landmark localization. A novel attention mechanism is introduced: Landmark heatmaps are used as references to generate a unified attention, so that the network has enough information to enhance or reduce features.