NEURAL NETWORK-BASED IMAGE GENERATION METHODS IN DIGITAL ART AND GRAPHIC DESIGN
PDF

Keywords

neural networks, image generation, GAN, VAE, diffusion models, transformer, CLIP, digital art, graphic design, national ornament.

How to Cite

Yalg‘osheva Ozoda, & Ismailov Kamoliddin. (2026). NEURAL NETWORK-BASED IMAGE GENERATION METHODS IN DIGITAL ART AND GRAPHIC DESIGN. INTERNATIONAL MULTIDISCIPLINARY SCIENCE CONFERENCE, 1(9), 328-345. https://doi.org/10.5281/zenodo.23164334

Abstract

The article presents a systematic analysis of the main neural network-based image generation methods used in digital art and graphic design: generative adversarial networks (GANs), variational autoencoders (VAEs), diffusion models and multimodal transformer-based models. The study employs theoretical comparative analysis, mathematical modelling and a meta-analysis of experimental results reported in the international literature. The models are compared in terms of image quality, diversity, generation speed and controllability using the FID, IS and CLIP Score metrics. The results show that latent diffusion models with classifier-free guidance provide the highest quality and controllability in text-to-image generation, while GANs retain their advantage in tasks that require real-time performance. A hybrid approach based on a cultural consistency loss is proposed to address the problem of accurately reproducing national cultural elements (ornament, embroidery, miniature painting).

PDF

References

1. Goodfellow I., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative adversarial nets // Advances in Neural Information Processing Systems. – 2014. – Vol. 27. – P. 2672–2680.

2. Kingma D. P., Welling M. Auto-encoding variational Bayes // International Conference on Learning Representations (ICLR). – 2014. – arXiv:1312.6114.

3. Ho J., Jain A., Abbeel P. Denoising diffusion probabilistic models // Advances in Neural Information Processing Systems. – 2020. – Vol. 33. – P. 6840–6851.

4. Rombach R., Blattmann A., Lorenz D., Esser P., Ommer B. High-resolution image synthesis with latent diffusion models // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). – 2022. – P. 10684–10695.

5. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser Ł., Polosukhin I. Attention is all you need // Advances in Neural Information Processing Systems. – 2017. – Vol. 30. – P. 5998–6008.

6. Radford A., Kim J. W., Hallacy C. et al. Learning transferable visual models from natural language supervision // Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR 139. – 2021. – P. 8748–8763.

7. O‘zbekiston Respublikasi Prezidentining 2021-yil 17-fevraldagi “Sun’iy intellekt texnologiyalarini jadal joriy etish uchun shart-sharoitlar yaratish chora-tadbirlari to‘g‘risida”gi PQ-4996-son qarori // Qonunchilik ma’lumotlari milliy bazasi (lex.uz).

8. O‘zbekiston Respublikasi Prezidentining 2024-yil 14-oktabrdagi “2030-yilgacha sun’iy intellekt texnologiyalarini rivojlantirish strategiyasini tasdiqlash to‘g‘risida”gi PQ-358-son qarori // Qonunchilik ma’lumotlari milliy bazasi (lex.uz).

9. Karras T., Laine S., Aila T. A style-based generator architecture for generative adversarial networks // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). – 2019. – P. 4401–4410.

10. Karras T., Aittala M., Laine S., Härkönen E., Hellsten J., Lehtinen J., Aila T. Alias-free generative adversarial networks // Advances in Neural Information Processing Systems. – 2021. – Vol. 34. – P. 852–863.

11. Isola P., Zhu J.-Y., Zhou T., Efros A. A. Image-to-image translation with conditional adversarial networks // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). – 2017. – P. 1125–1134.

12. Zhu J.-Y., Park T., Isola P., Efros A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks // Proceedings of the IEEE International Conference on Computer Vision (ICCV). – 2017. – P. 2223–2232.

13. Podell D., English Z., Lacey K., Blattmann A., Dockhorn T., Müller J., Penna J., Rombach R. SDXL: Improving latent diffusion models for high-resolution image synthesis // International Conference on Learning Representations (ICLR). – 2024.

14. Ho J., Salimans T. Classifier-free diffusion guidance // arXiv preprint arXiv:2207.12598. – 2022.

15. Zhang L., Rao A., Agrawala M. Adding conditional control to text-to-image diffusion models // Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). – 2023. – P. 3836–3847.

16. Ruiz N., Li Y., Jampani V., Pritch Y., Rubinstein M., Aberman K. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). – 2023. – P. 22500–22510.

17. Hu E. J., Shen Y., Wallis P., Allen-Zhu Z., Li Y., Wang S., Wang L., Chen W. LoRA: Low-rank adaptation of large language models // International Conference on Learning Representations (ICLR). – 2022.

18. Esser P., Rombach R., Ommer B. Taming transformers for high-resolution image synthesis // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). – 2021. – P. 12873–12883.

19. Peebles W., Xie S. Scalable diffusion models with transformers // Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). – 2023. – P. 4195–4205.

20. Heusel M., Ramsauer H., Unterthiner T., Nessler B., Hochreiter S. GANs trained by a two time-scale update rule converge to a local Nash equilibrium // Advances in Neural Information Processing Systems. – 2017. – Vol. 30.

21. Salimans T., Goodfellow I., Zaremba W., Cheung V., Radford A., Chen X. Improved techniques for training GANs // Advances in Neural Information Processing Systems. – 2016. – Vol. 29.

22. Hessel J., Holtzman A., Forbes M., Le Bras R., Choi Y. CLIPScore: A reference-free evaluation metric for image captioning // Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). – 2021. – P. 7514–7528.

23. Brock A., Donahue J., Simonyan K. Large scale GAN training for high fidelity natural image synthesis // International Conference on Learning Representations (ICLR). – 2019.

24. Dhariwal P., Nichol A. Diffusion models beat GANs on image synthesis // Advances in Neural Information Processing Systems. – 2021. – Vol. 34. – P. 8780–8794.

25. Sauer A., Schwarz K., Geiger A. StyleGAN-XL: Scaling StyleGAN to large diverse datasets // ACM SIGGRAPH 2022 Conference Proceedings. – 2022.

26. Ramesh A., Dhariwal P., Nichol A., Chu C., Chen M. Hierarchical text-conditional image generation with CLIP latents // arXiv preprint arXiv:2204.06125. – 2022.

27. Saharia C., Chan W., Saxena S. et al. Photorealistic text-to-image diffusion models with deep language understanding // Advances in Neural Information Processing Systems. – 2022. – Vol. 35. – P. 36479–36494.

28. Gatys L. A., Ecker A. S., Bethge M. Image style transfer using convolutional neural networks // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). – 2016. – P. 2414–2423.