Integration of multi-view images and deep learning for automated restoration and application of realistic textures in 3D building models

doi:10.13474/j.cnki.11-2246.2025.1203

Abstract

Abstract: 3D building models with both geometric accuracy and realistic textures have become an important component of the national new infrastructure construction.Due to constraints such as UAV flight conditions and building layout,a large number of real texture occlusion problems occur in the texture mapping of 3D building models,which affect their visualization effects and the functions of applications such as query and measurement.Existing methods are based on a single texture image for repair and treat the occluded area as an unknown random variable,leading to possible deviations of texture repair from the real facade features of buildings.Based on the characteristic that the occlusion range of the same facade of a building varies in images from different perspectives,this paper proposes an automatic facade texture occlusion repair algorithm combining multi-view images and deep learning networks.The algorithm extracts the occluded area by using the structural similarity of multi-view textures after texture alignment,automatically synthesizes the real facade texture through the graph-cut method,and uses the DeepFill model to repair and optimize the synthesized texture.Experiments show that this method can repair the real texture of more than 40% of the occluded area,and the SSIM and PSNR values of the repaired facade texture are improved compared with existing methods.

Key words: 3D real scene model, texture mapping, structural similarity, texture occlusion

CLC Number:

P237

LIU Yawen, TIAN Qin, GUO bingxuan, LI Demin. Integration of multi-view images and deep learning for automated restoration and application of realistic textures in 3D building models[J]. Bulletin of Surveying and Mapping, 2025, 0(12): 15-19.

References

[1] 陈军, 刘建军, 田海波.实景三维中国建设的基本定位与技术路径[J].武汉大学学报(信息科学版), 2022, 47(10): 1568-1575.
[2] 景慧莹, 刘欣怡, 张永军, 等.低秩矩阵辅助的无人机影像建筑物立面纹理自动修复[J].武汉大学学报(信息科学版), 2025, 50(1): 144-152.
[3] 范冲, 王学.三维城市建筑物的纹理映射综述[J].测绘与空间地理信息, 2014, 37(7): 1-4.
[4] 张春森, 张卫龙, 郭丙轩, 等.倾斜影像的三维纹理快速重建[J].测绘学报, 2015, 44(7): 782-790.
[5] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al.PatchMatch: a randomized correspondence algorithm for structural image editing[M]//Seminal Graphics Papers: Pushing the Boundaries.Volume 2.New York: ACM Press, 2023: 619-629.
[6] GHORAI M, MANDAL S, CHANDA B.A group-based image inpainting using patch refinement in MRF framework[J].IEEE Transactions on Image Processing, 2018, 27(2): 556-567.
[7] YU Jiahui, LIN Zhe, YANG Jimei, et al.Generative image inpainting with contextual attention[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City: IEEE, 2018: 5505-5514.
[8] ZHAO Shengyu, CUI J, SHENG Yilun, et al.Large scale image completion via co-modulated generative adversarial networks[EB/OL].[2025-03-20].https://arxiv.org/abs/2103.10428.
[9] SUVOROV R, LOGACHEVA E, MASHIKHIN A, et al.Resolution-robust large mask inpainting with Fourier convolutions[C]//Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).Waikoloa: IEEE, 2022: 3172-3182.
[10] LEMPITSKY V, VEDALDI A, ULYANOV D.Deep image prior[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE, 2018: 9446-9454.
[11] 李明, 张卫龙, 范丁元.城市三维重建中的自动纹理优化方法[J].测绘学报, 2017, 46(3): 338-345.
[12] WANG Zhou, BOVIK A C, SHEIKH H R, et al.Image quality assessment: from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[13] HAN Yuexing, YANG Shen, CHEN Qiaochuan.Recognition and segmentation of complex texture images based on superpixel algorithm and deep learning[J].Computational Materials Science, 2022, 209: 111398.
[14] 龚元夫, 熊忠招, 陈关州, 等.实景三维模型缺陷修复关键技术[J].科学技术与工程, 2024, 24(19):8339-8345.
[15] 徐海燕, 郭为人, 李德民, 等.实景三维模型纹理的敏感目标自动识别与脱密方法[J].测绘通报, 2023(12): 153-158.
[16] YU Jiahui, LIN Zhe, YANG Jimei, et al.Free-form image inpainting with gated convolution[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV).Seoul: IEEE, 2019: 4470-4479.