sample

Hongyu LIU

PhD Candidate


I am a second-year Ph.D. student at the Department of Computer Science and Engineering, HKUST advised by Prof. Qifeng Chen.
Before joining HUKST. I spend two wonderful years in Huya as a computer vision engineer led by Xintong Han and Jia Xu. I was an intern at Tencent AI Lab advised by Yibing Song and Wei Liu. Meanwhile, I have cooperated many times with my good friend Ziyu Wan. My research interests include 2D/3D Generation, digital human, neural rendering and contrastive learning.
I am now in Ant Research as an intern supervised by XuanWang. And I will focus on 3D Generation.
I love fishing very much, and my dream is to go fishing all over China!

πŸ”₯News

  • 07 / 2024  Follow-Your-Emoji paper is conditionally accepted to SIGGRAPH Asia 2024.
  • 05 / 2024  We released a survey paper: LLMs Meet Multimodal Generation and Editing: A Survey.
  • 03 / 2024  HeadArtist is accepted to SIGGRAPH 2024.
  • 02 / 2024  CAD is accepted to CVPR 2024.
  • 09 / 2023  GOAE is accepted to ICCV 2023.
  • 07 / 2023  I am joining AntGroup as an intern.
  • 03 / 2023  XFormer is accepted to IJCAI 2023.
  • 02 / 2023 CLCAE is accepted to CVPR 2023.
  • 01 / 2023 Human MotionFormer is accepted to ICLR 2023.
  • 09 / 2022 FFCLIP is accepted to NeurIPS 2022 as spotlight.
  • 09 / 2022 I am starting my PhD at HKUST.

πŸ“ Selected Publications



Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation

Yue Ma*, Hongyu LIU*, Hongfa Wang*, Heng Pan*, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung-Yeung Shum, Wei Liu, Qifeng Chen (* denotes equal contribution)

ACM Special Interest Group for Computer Graphics and Interactive Techniques Asia (SIGGRAPH Asia) 2024




LLMs Meet Multimodal Generation and Editing: A Survey

Yingqing He*, Zhaoyang Liu*, Jingye Chen*, Zeyue Tian*, Hongyu LIU*, Xiaowei Chi*, Runtao Liu*, Ruibin Yuan*, Yazhou Xing*, Wenhai Wang, Jifeng Dai, Yong Zhang, Wei Xue, Qifeng Liu, Yike Guo, Qifeng Chen (* denotes equal contribution)

ArXiv 2024




HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation

Hongyu LIU, Xuan Wang, Ziyu Wan, Yujun Shen, Yibing Song, Jing Liao, Qifeng Chen

ACM Special Interest Group for Computer Graphics and Interactive Techniques (SIGGRAPH) 2024




CAD: Photorealistic 3D Generation via Adversarial Distillation

Ziyu Wan, Despoina Paschalidou, Ian Huang, Hongyu Liu , Bokui Shen, Xiaoyu Xiang, Jing Liao, Leonidas Guibas

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024



Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding.

Ziyang Yuan*, Yiming Zhu*, Yu Li, Hongyu Liu, Chun Yuan (* denotes equal contribution)

International Conference on Computer Vision (ICCV) 2023



XFormer: Fast and Accurate Monocular 3D Body Capture.

Lihui Qian, Xintong Han, Faqiang Wang, Hongyu Liu, Huawei Wei, Zhe Lin, Haoye Dong, Zhiwen Li, Huawei Wei, Chengbin Jin

International Joint Conference on Artificial Intelligence (IJCAI) 2023



Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint

Hongyu Liu, Yibing Song, Qifeng Chen

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023



Human MotionFormer: Transferring Human Motions with Vision Transformers

Hongyu Liu, Xintong Han, Chenbin Jin, Lihui Qian, Huawei Wei, Zhe Lin, Faqiang Wang, Haoye Dong, Yibing Song, Jia Xu, Qifeng Chen

International Conference on Learning Representations (ICLR) 2023



One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations

Yiming Zhu*, Hongyu Liu*, Yibing Song, Xintong Han, Chun Yuan, Qifeng Chen, Jue Wang (* denotes equal contribution)

Neural Information Processing Systems (NeurIPS) 2022, Spotlight



Pd-gan: Probabilistic diverse gan for image inpainting

Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, Jing Liao

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021



DeFLOCNet: Deep Image Editing via Flexible Low-level Controls

Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, Jing Liao, Bin Jiang, Wei Liu

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021



Rethinking image inpainting via a mutual encoder-decoder with feature equalizations

Hongyu Liu, Bing Jiang, Yibing Song, Wei Huang, Chao Yang

European Conference on Computer Vision (ECCV) 2020, Oral



Coherent semantic attention for image inpainting

Hongyu Liu, Bin Jiang, Yi Xiao, Chao Yang

International Conference on Computer Vision (ICCV) 2019



πŸ™ˆ Projects




Anime Talking Heads

Anime talking heads. Control anime avatar with your expression in real time. We also build SDK, which only occupy 9% I7 CPU's performance (No GPU!).



3D Face reconstruction

3D Face reconstruction. Refining the UV Texture of FLAME.
We use DECA as our code base.





Consumer Digital Human

Consumer Digital Human. Given a single image(T pose), we can create an avatar and drive this avatar with target motion.





Paperman

We can drive a 2D image with skeleton.



πŸŽ– Honors and Awards


2022, Postgraduate Scholarship, HKUST.

2022, HuNan outstanding master graduation thesis.

2020, CCF outstanding master graduation thesis.

2020, School-level outstanding graduation thesis of HuNan University.

2020, School-level outstanding graduates of HuNan University.

2020, Provincial outstanding graduates of HuNan.

2019, National Scholarship.

2018, CCF BIG DATA 2018 best application paper award.

2018, Third place out of 390 teams, Few-Shot Classification in AI FUTURELAB 2018.

✏ Academic Reviewer


Conference: CVPR, ECCV, ICCV, ICLR, NeurIPS, ICML.

Journal: TNNCL, TVCG, TMM, TCSVT.

Outstanding reviewer: NeurIPS, 2022

πŸ’» Work Experience


2022.9 - 2023.5, Tencent AI Lab, Intern.

2020.07 - 2022.07, Huya Inc, Computer vision engineer.

2020.2 - 2020.7, Tencent AI Lab, Intern.

2019.10 - 2019.11, Sense Time, Intern.

2018.7 - 2018.9, Hua Wei, Intern.

πŸ’¬ Invited Talks


2021, Combining Deep Neural Networks for Efficient Image Restoration and Editing, Link.


Flag Counter