作者：张涛1, 刘刚1, 朱冀涛1, 徐晓雨2, 徐岩3
作者单位：1. 国网辽宁省电力有限公司，辽宁 沈阳 110003;
2. 国网辽宁省电力有限公司建设分公司，辽宁 沈阳 110003;
3. 华北电力大学电气与电子工程学院，河北 保定 071003
关键词：图像修复;目标识别;最小二乘生成式对抗网络;迁移学习;长短时记忆神经网络
摘要：
目前建筑工地视频监控图像模糊，部分图像有遮挡，监控智能性差，耗费大量的人力物力仍无法实现高效管理，针对此问题，提出一种基于最小二乘生成式对抗网络(LSGAN)及迁移学习的智慧工地监控图像修复和识别方法。首先，利用生成式对抗网络的判别器与生成器之间的零和博弈，引入最小二乘损失函数，修复工地监控图像；其次，引入迁移学习思想提取图像特征，将修复后的图像在预训练的GoogleNet模型上进行训练，微调网络参数；最后，利用长短时记忆(LSTM)神经网络对目标图像进行检测与识别，判别现场是否存在安全隐患及人员违规行为。实验结果表明：该方法能够高效预测图像语义缺失，图像修复速度快，视觉效果逼真，且目标识别准确率较高，具有一定的实用价值。
Intelligent site monitoring image restoration and recognition method based on LSGAN and transfer learning
ZHANG Tao1, LIU Gang1, ZHU Jitao1, XU Xiaoyu2, XU Yan3
1. State Grid Liaoning Electric Power Supply Co., Ltd., Shenyang 110003, China;
2. State Grid Liaoning Electric Power Co., Ltd. Construction Branch, Shenyang 110003, China;
3. School of Electrical and Electronic Engineering, North China Electric Power University, Baoding 071003, China
Abstract: At present, the video surveillance image of the construction site is fuzzy, some images are blocked, the surveillance intelligence is poor, it consumes a lot of human and material resources, and it can not achieve efficient management. An intelligent surveillance image restoration and recognition method based on least squares generative countermeasure network (LSGAN) and transfer learning is proposed to solve this problem. Firstly, using the zero-sum game between the discriminator and the generator of the generative countermeasure network, the least square loss function is introduced to repair the site monitoring image. Secondly, the idea of transfer learning is introduced to extract the image features, and pre-train the restored images on a pre-trained GoogleNet model and fine-tune the network parameters. Finally, the long-term and short-term memory (LSTM) neural network is used to detect and recognize the target image to judge whether there are potential safety hazards and personnel violations. The experimental results show that this method can effectively predict the semantic loss of the image, the image repair speed is fast, the visual effect is realistic, and the accuracy of target recognition is high. It has certain practical value.
Keywords: image restoration;target recognition;least squares generative countermeasure network;transfer learning;long-term and short-term memory neural network
2022, 48(9):125-132 收稿日期: 2021-08-11;收到修改稿日期: 2021-10-08
基金项目: 国网辽宁省电力有限公司科技项目(KJ-2021-021)
作者简介: 张涛（1968-），男，河南周口市人，高级工程师，主要从事输变电工程建设管理方面的研究
参考文献
[1] 崔志诚, 马胜. 于物联网技术的智慧工地[J]. 电子技术应用, 2021, 47(2): 33-35, 40
[2] 李建奎, 陈阳, 黄小星, 等. Adaboost算法的面部识别技术在智慧工地中的应用[J]. 单片机与嵌入式系统应用, 2021, 21(4): 61-63, 67
[3] 王昕, 赵飞, 蒋佐富, 等. 迁移学习和卷积神经网络电力设备图像识别方法[J]. 中国测试, 2020, 46(5): 108-113
[4] YEH R A, CHEN C, YIAN T, et al. Semantic image inpainting with deep generative models[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[5] BAE S H. Object detection based on region decomposition and assembly[C]// Association for the Advance of Artificial Intelligence, 2019.
[6] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014.
[7] 刘云鹏, 许自强, 和家慧, 等. 基于条件式Wasserstein生成对抗网络的电力变压器故障样本增强技术[J]. 电网技术, 2020, 44(4): 1505-1513
[8] 卢倩雯, 陶青川, 赵娅琳, 等. 基于生成对抗网络的漫画草稿图简化[J]. 自动化学报, 2018, 44(5): 840-854
[9] 邵杭, 王永雄. 基于并行对抗与多条件融合的生成式高分辨率图像修复[J]. 模式识别与人工智能, 2020, 33(4): 363-374
[10] MAO X D, LI Q, XIE H R, et al. Least squares generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017.
[11] 方明宽, 宁静, 陈春俊. 高速列车横向蛇行失稳的EEMD-CNN-LSTM预测方法[J]. 中国测试, 2021, 47(7): 79-83
[12] 韩要昌, 王洁, 史通, 等. 基于改进GoogLeNet的遥感图像分类方法[J]. 弹箭与制导学报, 2019, 39(5): 139-142, 153
[13] 刘长德, 顾宇翔, 张进丰. 基于小波滤波和LSTM神经网络的船舶运动极短期预报研究[J]. 船舶力学, 2021, 25(3): 299-310
[14] 王玉静, 李少鹏, 康守强, 等. 结合CNN和LSTM的滚动轴承剩余使用寿命预测方法[J]. 振动. 测试与诊断, 2021, 41(3): 439-446, 617
[15] 安珊, 林树宽, 乔建忠, 等. 基于生成对抗网络学习被遮挡特征的目标检测方法[J]. 控制与决策, 2021, 36(5): 1199-1205
