Adaptive Mask Optimization-Driven Hybrid GAN with Swarm-Tuned Multi-Component Loss for High-Fidelity Image Inpainting
DOI:
https://doi.org/10.22399/ijcesen.4862Keywords:
Generative Adversarial Networks (GANs), Image Inpainting, Adaptive Mask Optimization, Swarm-Based Hyperparameter Tuning, Deep Learning, High-Fidelity Image RestorationAbstract
Image inpainting restores damaged or missing image regions and is vital in fields like digital restoration, medical imaging, and remote sensing. While GAN-based methods synthesize textures well, they often fail to preserve structure and fine details. Recent transformer approaches improve global consistency, but they remain resource-heavy and sensitive to tuning. This work proposes an Adaptive Mask Optimization-Driven Hybrid GAN to improve structure preservation, texture realism, and computational efficiency using Swarm-Based Hyperparameter Tuning and a refined loss function. The model dynamically refines missing regions via adaptive mask learning, uses swarm optimization to tune parameters efficiently, and applies an enhanced loss to balance perceptual, structural, and pixel-level accuracy. Evaluations are conducted on Paris Street View and DPG datasets, with benchmarking against representative inpainting models. The proposed method achieves 34.72 dB PSNR, 0.942 SSIM, and reduces EPI to 0.087, outperforming baselines such as Mask Optimization GAN, PainterNet, and Attention Two-Stage GAN. Cross-dataset evaluation confirms generalization, and ablation studies highlight the gains from mask refinement and swarm-based tuning. Overall, the framework improves realism and structure while maintaining practical accuracy, efficiency and trade-off. Future work may explore lightweight attention mechanisms, multi-stage refinement, and real-time deployment.
References
[1] K. Shimosato and N. Ukita, “Inpainting-Driven Mask Optimization for Object Removal,” arXiv, arXiv:2403.15849, Mar. 2024, doi: 10.48550/arXiv.2403.15849 (accepted to IJCNN 2024)
[2] C. Dong, H. Liu, X. Wang, et al., “Image inpainting method based on AU-GAN,” Multimedia Systems, vol. 30, art. no. 101, 2024, doi: 10.1007/s00530-024-01290-3
[3] P. Saxena, R. Gupta, A. Maheshwari, and S. Maheshwari, “Semantic Image Completion and Enhancement Using GANs,” in High Performance Vision Intelligence: Recent Advances, A. Nanda and N. Chaurasia, Eds., Studies in Computational Intelligence, vol. 913. Singapore: Springer, 2020, pp. 151–170, doi: 10.1007/978-981-15-6844-2_11
[4] Q. Zhang, C. Wang, A. Siarohin, et al., “SceneWiz3D: Towards Text-guided 3D Scene Composition,” arXiv, arXiv:2312.08885, Dec. 2023, doi: 10.48550/arXiv.2312.08885
[5] R. Wang, J. Zhang, Q. Xie, C. Chen, and H. Lu, “PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control,” arXiv, arXiv:2412.01223, Dec. 2024, doi: 10.48550/arXiv.2412.01223
[6] S. He, Y. Zou, B. Li, F. Peng, X. Lu, H. Guo, X. Tan, and Y. Chen, “An image inpainting-based data augmentation method for improved sclerosed glomerular identification performance with the segmentation model EfficientNetB3-UNet,” Scientific Reports, vol. 14, no. 1, art. no. 1033, Jan. 2024, doi: 10.1038/s41598-024-51651-1
[7] Z. U. Rahman, M. S. M. Asaari, H. Ibrahim, I. S. Z. Abidin, and M. K. Ishak, “Generative Adversarial Networks (GANs) for Image Augmentation in Farming: A Review,” IEEE Access, vol. 12, pp. 179912–179943, 2024, doi: 10.1109/ACCESS.2024.3505989
[8] M. Isogawa, D. Mikami, K. Takahashi, et al., “Image quality assessment for inpainted images via learning to rank,” Multimedia Tools and Applications, vol. 78, no. 1, pp. 1399–1418, 2019, doi: 10.1007/s11042-018-6186-z
[9] L. Zhao, T. Zhu, C. Wang, F. Tian, and H. Yao, “Image inpainting algorithm based on structure-guided generative adversarial network,” Mathematics, vol. 13, no. 15, art. no. 2370, 2025, doi: 10.3390/math13152370.
[10] C. Cai, Y. Zeng, S. Yang, X. Jia, H. Lu, and Y. He, “Deformable dynamic sampling and dynamic predictable mask mining for image inpainting,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 12, pp. 18445–18454, 2024, doi: 10.1109/TNNLS.2023.3316123
[11] J. Wang, M. Luo, X. Chen, H. Xu, and J. Kim, “A novel image inpainting method based on a modified Lengyel–Epstein model,” Computer Vision and Image Understanding, vol. 249, art. no. 104195, Dec. 2024, doi: 10.1016/j.cviu.2024.104195.
[12] Y. Niu, L. Wu, D. Yi, J. Peng, N. Jiang, H. Wu, and J. Wang, “AnyDesign: Versatile Area Fashion Editing via Mask-Free Diffusion,” arXiv, arXiv:2408.11553v4, Oct. 2024, doi: 10.48550/arXiv.2408.11553
[13] W. Huang, Y. Deng, S. Hui, Y. Wu, S. Zhou, and J. Wang, “Sparse self-attention transformer for image inpainting,” Pattern Recognition, vol. 145, art. no. 109897, Jan. 2024, doi: 10.1016/j.patcog.2023.109897
[14] W. Miao, L. Wang, H. Lu, et al., “ITrans: Generative image inpainting with transformers,” Multimedia Systems, vol. 30, art. no. 21, 2024, doi: 10.1007/s00530-023-01211-w
[15] P. Wang, Z. Ma, B. Dong, X. Liu, J. Ding, K. Sun, and Y. Chen, “Generative data augmentation by conditional inpainting for multi-class object detection in infrared images,” Pattern Recognition, vol. 153, art. no. 110501, 2024, doi: 10.1016/j.patcog.2024.110501.
[16] E. Goceri, “GAN-based augmentation using a hybrid loss function for dermoscopy images,” Artificial Intelligence Review, vol. 57, art. no. 234, 2024, doi: 10.1007/s10462-024-10897-x
[17] Y. Chen, R. Xia, K. Yang, and K. Zou, “Image inpainting algorithm based on inference attention module and two-stage network,” Engineering Applications of Artificial Intelligence, vol. 137, pt. B, art. no. 109181, Nov. 2024, doi: 10.1016/j.engappai.2024.109181.
[18] Y. Zhang, X. Liu, and Y. Lv, “A hybrid swarming algorithm for adaptive enhancement of low-illumination images,” Symmetry, vol. 16, art. no. 533, 2024, doi: 10.3390/sym16050533.
[19] C. H. Yeh, H. F. Yang, M. J. Chen, and L. W. Kang, “Image inpainting based on GAN-driven structure- and texture-aware learning with application to object removal,” Applied Soft Computing, vol. 161, art. no. 111748, 2024, doi: 10.1016/j.asoc.2024.111748.
[20] M. Huang, W. Yu, and L. Zhang, “DF3Net: Dual frequency feature fusion network with hierarchical transformer for image inpainting,” Information Fusion, vol. 111, art. no. 102487, Nov. 2024, doi: 10.1016/j.inffus.2024.102487
[21] W. Wen, T. Li, A. Tolba, Z. Liu, and K. Shao, “Progressively multi-scale feature fusion for image inpainting,” Mathematics, vol. 11, no. 24, art. no. 4908, 2023, doi: 10.3390/math11244908
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
