KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis with an Optimized StyleGAN Encoder | Computer Vision – ACCV 2022 (2024)

Article

Authors: Li-Chen Cheng, Shu-Chuan Hsu, Pin-Hua Lee, Hsiu-Chieh Lee, + 3, Che-Hsien Lin, Jun-Cheng Chen, Chih-Yu Wang (Less)

Computer Vision – ACCV 2022: 16th Asian Conference on Computer Vision, Macao, China, December 4–8, 2022, Proceedings, Part IV

Pages 105 - 120

Published: 02 March 2023 Publication History

Metrics

Total Citations0Total Downloads0

Last 12 Months0

Last 6 weeks0

New Citation Alert added!

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

Manage my Alerts

New Citation Alert!

Please log in to your account

    • View Options
    • References
    • Media
    • Tables
    • Share

Abstract

High-fidelity kinship face synthesis is a challenging task due to the limited amount of kinship data available for training and low-quality images. In addition, it is also hard to trace the genetic traits between parents and children from those low-quality training images. To address these issues, we leverage the pre-trained state-of-the-art face synthesis model, StyleGAN2, for kinship face synthesis. To handle large age, gender and other attribute variations between the parents and their children, we conduct a thorough study of its rich latent spaces and different encoder architectures for an optimized encoder design to repurpose StyleGAN2 for kinship face synthesis. The obtained latent representation from our developed encoder pipeline with stage-wise training strikes a better balance of editability and synthesis fidelity for identity preserving and attribute manipulations than other compared approaches. With extensive subjective, quantitative, and qualitative evaluations, the proposed approach consistently achieves better performance in terms of facial attribute heredity and image generation fidelity than other compared state-of-the-art methods. This demonstrates the effectiveness of the proposed approach which can yield promising and satisfactory kinship face synthesis using only a single and straightforward encoder architecture.

References

[1]

Ertugrul, I.Ö., Dibeklioglu, H.: What will your future child look like? modeling and synthesis of hereditary patterns of facial dynamics. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2017)

[2]

Ozkan, S., Ozkan, A.: KinshipGAN: synthesizing of kinship faces from family photos by regularizing a deep face network. In: IEEE International Conference on Image Processing (ICIP) (2018)

[3]

Gao, P., Robinson, J., Zhu, J., Xia, C., Shao, M., Xia, S.: DNA-Net: age and gender aware kin face synthesizer. In: IEEE International Conference on Multimedia and Expo (ICME) (2021)

[4]

Lin, C.H., Chen, H.C., Cheng, L.C., Hsu, S.C., Chen, J.C., Wang, C.Y.: StyleDNA: a high-fidelity age and gender aware kinship face synthesizer. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2021)

[5]

Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of styleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

[6]

Zhang, Y., Li, L., Liu, Z., Wu, B., Fan, Y., Li, Z.: Controllable descendant face synthesis. arXiv preprint arXiv:2002.11376 (2020)

[7]

Tov O, Alaluf Y, Nitzan Y, Patashnik O, and Cohen-Or D Designing an encoder for styleGAN image manipulation ACM Trans. Graph. (TOG) 2021 40 1-14

[8]

Robinson JP, Shao M, Wu Y, Liu H, Gillis T, and Fu Y Visual kinship recognition of families in the wild IEEE Trans. Pattern Anal. Mach. Intell. 2018 40 2624-2637

[9]

Qin, X., Tan, X., Chen, S.: Tri-subjects kinship verification: understanding the core of a family. In: 2015 14th IAPR International Conference on Machine Vision Applications (MVA), pp. 580–583 (2015)

[10]

Chong, M.J., Chu, W.S., Kumar, A., Forsyth, D.: Retrieve in style: unsupervised facial feature transfer and retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021)

[11]

Liu, S., et al.: Face aging with contextual generative adversarial nets. In: Proceedings of the 25th ACM International Conference on Multimedia (2017)

[12]

Tang, H., Bai, S., Sebe, N.: Dual attention GANs for semantic image synthesis. In: Proceedings of the 28th ACM International Conference on Multimedia (2020)

[13]

Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations (ICLR) (2019)

[14]

Zhao, J., et al.: Dual-agent GANs for photorealistic and identity preserving profile face synthesis. In: Advances in Neural Information Processing Systems 30 (2017)

[15]

Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

[16]

Abdal, R., Qin, Y., Wonka, P.: Image2styleGAN: how to embed images into the stylegan latent space? In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)

[17]

Zhang, L., Bai, X., Gao, Y.: SalS-GAN: spatially-adaptive latent space in styleGAN for real image embedding. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 5176–5184 (2021)

[18]

Sainburg, T., Thielk, M., Theilman, B., Migliori, B., Gentner, T.: Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions. arXiv preprint arXiv:1807.06650 (2018)

[19]

Mukherjee S, Asnani H, Lin E, and Kannan S ClusterGAN: latent space clustering in generative adversarial networks Proceed. AAAI Conf. Artif. Intell. 2019 33 4610-4617

[20]

Bojanowski, P., Joulin, A., Lopez-Paz, D., Szlam, A.: Optimizing the latent space of generative networks. arXiv preprint arXiv:1707.05776 (2017)

[21]

Robinson, J.P., Shao, M., Fu, Y.: Survey on the analysis and modeling of visual kinship: a decade in the making. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)

[22]

Ertuğrul IÖ, Jeni LA, and Dibeklioğlu H Modeling and synthesis of kinship patterns of facial expressions Image Vis. Comput. 2018 79 133-143

[23]

Sinha R, Vatsa M, and Singh R Bartoli A and Fusiello A FamilyGAN: generating kin face images using generative adversarial networks Computer Vision – ECCV 2020 Workshops 2020 Cham Springer 297-311

[24]

Ghatas FS and Hemayed EE GANKIN: generating kin faces using disentangled GAN SN Appl. Sci. 2020 2 166

[25]

Zaman, I., Crandall, D.: Genetic-GAN: synthesizing images between two domains by genetic crossover. In: European Conference on Computer Vision Workshops (ECCVW) (2020)

[26]

Cui, X., Zhou, W., Hu, Y., Wang, W., Li, H.: Heredity-aware child face image generation with latent space disentanglement. arXiv preprint arXiv:2108.11080 (2021)

[27]

Alaluf Y, Patashnik O, and Cohen-Or D Only a matter of style: age transformation using a style-based regression model ACM Trans. Graph. (TOG) 2021 40 1-12

[28]

Kafri, O., Patashnik, O., Alaluf, Y., Cohen-Or, D.: StyleFusion: a generative model for disentangling spatial segments. arXiv preprint arXiv:2107.07437 (2021)

[29]

Or-El R, Sengupta S, Fried O, Shechtman E, and Kemelmacher-Shlizerman I Vedaldi A, Bischof H, Brox T, and Frahm J-M Lifespan age transformation synthesis Computer Vision – ECCV 2020 2020 Cham Springer 739-755

[30]

Wang, X., Li, Y., Zhang, H., Shan, Y.: Towards real-world blind face restoration with generative facial prior. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

[31]

Chen, S., Liu, Y., Gao, X., Han, Z.: MobileFaceNets: Efficient CNNs for accurate real-time face verification on mobile devices. CoRR abs/1804.07573 (2018)

[32]

Richardson, E., et al.: Encoding in style: a styleGAN encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

Index Terms

  1. KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis withanOptimized StyleGAN Encoder

    1. Computing methodologies

      1. Artificial intelligence

        1. Computer vision

        2. Computer graphics

          1. Image manipulation

      Index terms have been assigned to the content through auto-classification.

      Recommendations

      • Frontal face synthesis based on multiple pose-variant images for face recognition

        ICB'07: Proceedings of the 2007 international conference on Advances in Biometrics

        Pose variance remains a challenging problem for face recognition. In this paper, a stereoscopic synthesis method for generating a frontal face image is proposed to improve the performance of automatic face recognition system. Through this method, a ...

        Read More

      • Robust Deep Auto-encoder for Occluded Face Recognition

        MM '15: Proceedings of the 23rd ACM international conference on Multimedia

        Occlusions by sunglasses, scarf, hats, beard, shadow etc, can significantly reduce the performance of face recognition systems. Although there exists a rich literature of researches focusing on face recognition with illuminations, poses and facial ...

        Read More

      • Achieving Better Kinship Recognition Through Better Baseline

        2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)

        Recognizing blood relations using face images can be seen as an application of face recognition systems with additional restrictions. These restrictions proved to be difficult to deal with, however, recent advancements in face verification show that there ...

        Read More

      Comments

      Information & Contributors

      Information

      Published In

      KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis withanOptimized StyleGAN Encoder | Computer Vision – ACCV 2022 (8)

      Computer Vision – ACCV 2022: 16th Asian Conference on Computer Vision, Macao, China, December 4–8, 2022, Proceedings, Part IV

      Dec 2022

      780 pages

      ISBN:978-3-031-26315-6

      DOI:10.1007/978-3-031-26316-3

      • Editors:
      • Lei Wang

        University of Wollongong, Wollongong, NSW, Australia

        ,
      • Juergen Gall

        University of Bonn, Bonn, Germany

        ,
      • Tat-Jun Chin

        University of Adelaide, Adelaide, SA, Australia

        ,
      • Imari Sato

        National Institute of Informatics, Tokyo, Japan

        ,
      • Rama Chellappa

        Johns Hopkins University, Baltimore, MD, USA

      © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.

      Publisher

      Springer-Verlag

      Berlin, Heidelberg

      Publication History

      Published: 02 March 2023

      Author Tags

      1. Kinship face synthesis
      2. StyleGAN Encoder

      Qualifiers

      • Article

      Contributors

      KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis withanOptimized StyleGAN Encoder | Computer Vision – ACCV 2022 (14)

      Other Metrics

      View Article Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Total Citations

      • Total Downloads

      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0

      Reflects downloads up to 30 Aug 2024

      Other Metrics

      View Author Metrics

      Citations

      View Options

      View options

      Get Access

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      Get this Publication

      Media

      Figures

      Other

      Tables

      KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis with an Optimized StyleGAN Encoder | Computer Vision – ACCV 2022 (2024)
      Top Articles
      Latest Posts
      Article information

      Author: Msgr. Refugio Daniel

      Last Updated:

      Views: 5812

      Rating: 4.3 / 5 (54 voted)

      Reviews: 93% of readers found this page helpful

      Author information

      Name: Msgr. Refugio Daniel

      Birthday: 1999-09-15

      Address: 8416 Beatty Center, Derekfort, VA 72092-0500

      Phone: +6838967160603

      Job: Mining Executive

      Hobby: Woodworking, Knitting, Fishing, Coffee roasting, Kayaking, Horseback riding, Kite flying

      Introduction: My name is Msgr. Refugio Daniel, I am a fine, precious, encouraging, calm, glamorous, vivacious, friendly person who loves writing and wants to share my knowledge and understanding with you.