融合多视角影像与深度学习的三维建筑模型真实纹理自动修复与应用

doi:10.13474/j.cnki.11-2246.2025.1203

测绘通报 ›› 2025, Vol. 0 ›› Issue (12): 15-19.doi: 10.13474/j.cnki.11-2246.2025.1203

• 学术研究 • 上一篇

融合多视角影像与深度学习的三维建筑模型真实纹理自动修复与应用

刘亚文^1,2, 田沁¹, 郭丙轩³, 李德民⁴

1. 自然资源部城市国土资源监测与仿真重点实验室, 广东深圳 518034;
2. 湖北工业大学, 湖北武汉 430068;
3. 武汉大学测绘遥感信息工程全国重点实验室, 湖北武汉 430072;
4. 浙江安防职业技术学院, 浙江温州 325016

收稿日期:2025-05-07 发布日期:2025-12-31
通讯作者: 李德民。E-mail:23086567@zjcst.edu.cn
作者简介:刘亚文(1991—),女,博士,讲师,研究方向为计算机图形学、数字城市实景三维模型构建。E-mail:liuyawen@hbut.edu.cn
基金资助:
自然资源部城市国土资源监测与仿真重点实验室开放基金(KF-2022-07-003);温州市基础性公益科研项目(2024G0153)

Integration of multi-view images and deep learning for automated restoration and application of realistic textures in 3D building models

LIU Yawen^1,2, TIAN Qin¹, GUO bingxuan³, LI Demin⁴

1. Key Laboratory of Urban Land Resources Monitoring and Simulation, Ministry of Natural Resources, Shenzhen 518034, China;
2. School of Computer Science, Hubei University of Technology, Wuhan 430068, China;
3. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China;
4. Zhejiang College of Security Technology, Wenzhou 325016, China

Received:2025-05-07 Published:2025-12-31

摘要/Abstract

摘要： 兼具几何精度与真实纹理的三维建筑模型已成为国家新型基础设施建设的重要组成部分。由于无人机飞行条件、建筑物布局等因素限制,三维建筑模型纹理映射出现大量的真实纹理遮挡问题,影响其可视化效果及查询、测量等应用功能。现有方法以单张纹理影像为修复基础,将遮挡区域视为未知随机变量进行处理,导致纹理修复可能偏离建筑物真实的立面特征。本文基于建筑物同一立面在不同视角的影像中遮挡范围不同的特点,提出了一种结合多视角影像和深度学习网络的立面纹理遮挡自动修复算法,根据纹理对齐后的多视角纹理的结构相似度提取遮挡区域,通过图割方法自动合成立面真实纹理,并利用DeepFill模型对合成纹理进行修复优化。试验表明,该方法可以修复40%以上遮挡区域的真实纹理,修复后立面纹理的SSIM值和PSNR值相较于现有方法有所提升。

关键词: 实景三维模型, 纹理映射, 结构相似度, 纹理遮挡

Abstract: 3D building models with both geometric accuracy and realistic textures have become an important component of the national new infrastructure construction.Due to constraints such as UAV flight conditions and building layout,a large number of real texture occlusion problems occur in the texture mapping of 3D building models,which affect their visualization effects and the functions of applications such as query and measurement.Existing methods are based on a single texture image for repair and treat the occluded area as an unknown random variable,leading to possible deviations of texture repair from the real facade features of buildings.Based on the characteristic that the occlusion range of the same facade of a building varies in images from different perspectives,this paper proposes an automatic facade texture occlusion repair algorithm combining multi-view images and deep learning networks.The algorithm extracts the occluded area by using the structural similarity of multi-view textures after texture alignment,automatically synthesizes the real facade texture through the graph-cut method,and uses the DeepFill model to repair and optimize the synthesized texture.Experiments show that this method can repair the real texture of more than 40% of the occluded area,and the SSIM and PSNR values of the repaired facade texture are improved compared with existing methods.

Key words: 3D real scene model, texture mapping, structural similarity, texture occlusion

中图分类号:

P237

刘亚文, 田沁, 郭丙轩, 李德民. 融合多视角影像与深度学习的三维建筑模型真实纹理自动修复与应用[J]. 测绘通报, 2025, 0(12): 15-19.

LIU Yawen, TIAN Qin, GUO bingxuan, LI Demin. Integration of multi-view images and deep learning for automated restoration and application of realistic textures in 3D building models[J]. Bulletin of Surveying and Mapping, 2025, 0(12): 15-19.

参考文献

[1] 陈军, 刘建军, 田海波.实景三维中国建设的基本定位与技术路径[J].武汉大学学报(信息科学版), 2022, 47(10): 1568-1575.
[2] 景慧莹, 刘欣怡, 张永军, 等.低秩矩阵辅助的无人机影像建筑物立面纹理自动修复[J].武汉大学学报(信息科学版), 2025, 50(1): 144-152.
[3] 范冲, 王学.三维城市建筑物的纹理映射综述[J].测绘与空间地理信息, 2014, 37(7): 1-4.
[4] 张春森, 张卫龙, 郭丙轩, 等.倾斜影像的三维纹理快速重建[J].测绘学报, 2015, 44(7): 782-790.
[5] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al.PatchMatch: a randomized correspondence algorithm for structural image editing[M]//Seminal Graphics Papers: Pushing the Boundaries.Volume 2.New York: ACM Press, 2023: 619-629.
[6] GHORAI M, MANDAL S, CHANDA B.A group-based image inpainting using patch refinement in MRF framework[J].IEEE Transactions on Image Processing, 2018, 27(2): 556-567.
[7] YU Jiahui, LIN Zhe, YANG Jimei, et al.Generative image inpainting with contextual attention[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City: IEEE, 2018: 5505-5514.
[8] ZHAO Shengyu, CUI J, SHENG Yilun, et al.Large scale image completion via co-modulated generative adversarial networks[EB/OL].[2025-03-20].https://arxiv.org/abs/2103.10428.
[9] SUVOROV R, LOGACHEVA E, MASHIKHIN A, et al.Resolution-robust large mask inpainting with Fourier convolutions[C]//Proceedings of 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).Waikoloa: IEEE, 2022: 3172-3182.
[10] LEMPITSKY V, VEDALDI A, ULYANOV D.Deep image prior[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE, 2018: 9446-9454.
[11] 李明, 张卫龙, 范丁元.城市三维重建中的自动纹理优化方法[J].测绘学报, 2017, 46(3): 338-345.
[12] WANG Zhou, BOVIK A C, SHEIKH H R, et al.Image quality assessment: from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[13] HAN Yuexing, YANG Shen, CHEN Qiaochuan.Recognition and segmentation of complex texture images based on superpixel algorithm and deep learning[J].Computational Materials Science, 2022, 209: 111398.
[14] 龚元夫, 熊忠招, 陈关州, 等.实景三维模型缺陷修复关键技术[J].科学技术与工程, 2024, 24(19):8339-8345.
[15] 徐海燕, 郭为人, 李德民, 等.实景三维模型纹理的敏感目标自动识别与脱密方法[J].测绘通报, 2023(12): 153-158.
[16] YU Jiahui, LIN Zhe, YANG Jimei, et al.Free-form image inpainting with gated convolution[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV).Seoul: IEEE, 2019: 4470-4479.

融合多视角影像与深度学习的三维建筑模型真实纹理自动修复与应用

Integration of multi-view images and deep learning for automated restoration and application of realistic textures in 3D building models

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	徐敬海, 刘阳, 齐梦轩, 经皓然. 无人机实景三维模型驱动的建筑屋顶光伏面板优化布设[J]. 测绘通报, 2025, 0(12): 1-6.
[2]	杨光, 彭林, 陈国梁. 无人机多尺度航摄数据联合解算对复杂区域实景三维模型构建的影响分析[J]. 测绘通报, 2025, 0(1): 121-126.
[3]	符惠伟, 岑铭, 廖超明, 吕华权. 无人机倾斜摄影技术的不动产测量应用与探讨[J]. 测绘通报, 2024, 0(7): 123-128.
[4]	史与正, 陈梦华, 黄煜, 张彤蕴, 周宪. 实景三维模型的建筑物单体模型框架搭建[J]. 测绘通报, 2023, 0(6): 161-166.
[5]	罗振威, 李骁, 刘承承, 张勇. 利用倾斜摄影与激光扫描技术构建城市级实景三维模型[J]. 测绘通报, 2023, 0(12): 116-120.
[6]	刘丽, 魏海霞, 祖为国, 谭金石. 无人机倾斜摄影测量技术在可建设城市屋顶光伏资源调查中的应用[J]. 测绘通报, 2023, 0(12): 142-146.
[7]	兰峰, 张帆, 高云龙, 黄先锋. 顾及地物类别的倾斜摄影三维模型简化方法[J]. 测绘通报, 2022, 0(4): 44-50.
[8]	任旭斌, 康建锋, 于东海. 无人机倾斜摄影测量在房屋建筑面积测算中的应用[J]. 测绘通报, 2022, 0(3): 116-120,126.
[9]	王珍, 张涛, 丁乐乐, 潘宇明, 史芙蓉. 融合高度和光谱信息的倾斜摄影数据建筑物自动提取[J]. 测绘通报, 2022, 0(2): 10-15.
[10]	席思远, 张西童, 王宁, 张兆江, 李剑明, 张国栋. 倾斜摄影设备选型及像控点布设对高精度实景三维模型重建的影响[J]. 测绘通报, 2022, 0(10): 86-92.
[11]	张会霞, 李豪, 李晋宏, 赵菲. 倾斜摄影测量三维模型单体化及服务发布[J]. 测绘通报, 2021, 0(9): 79-82.
[12]	周阳阳, 徐尚昭, 陈斌, 张青峰. 无人机倾斜摄影测量技术应用于自然保护区勘界[J]. 测绘通报, 2021, 0(9): 43-48.
[13]	齐永良, 李海鹏, 王永国. 明代石碑高精度真实纹理三维建模应用[J]. 测绘通报, 2021, 0(6): 98-102.
[14]	张守权, 李大军, 郭丙轩, 刘丹, 彭哲. 一种快速高效的无缝纹理匀光方法[J]. 测绘通报, 2021, 0(4): 98-104.
[15]	郭如宝, 郝利娟, 刘旭东, 刘军, 刘尧晖, 张炜. 基于倾斜摄影的实景三维单体化模型构建方法[J]. 测绘通报, 2021, 0(1): 103-107.