基于深度特征表达的目标检测与故障识别方法研究

项目来源

国家自然科学基金(NSFC)

项目主持人

翟永杰

项目受资助机构

华北电力大学

项目编号

61773160

立项年度

2017

立项时间

未公开

研究期限

未知 / 未知

项目级别

国家级

受资助金额

51.00万元

学科

信息科学-自动化-自动化检测技术与装置

学科代码

F-F03-F0306

基金类别

面上项目

关键词

目标检测 ; 深度特征表达 ; 复杂背景 ; 飞行机器人 ; 绝缘子 ; target recognition ; deep feature representation ; flying robot ; complex background ; insulator

参与者

赵振兵;仝卫国;程海燕;张婷;张木柳;赵海龙;刘金龙;陈瑞;刘业鹏

参与机构

华北电力大学(保定)

项目标书摘要:复杂背景和干扰下的目标检测与故障识别是飞行机器人自主巡检的关键技术。应用深度学习方法时遇到目标样本不完备、特征表达困难、人工特征未融入等问题。本项目借鉴人类对三维物体的空间认知能力,形成基于模拟图像的目标软识别方法,建立“模拟—真实”平行结构的研究框架。结合虚拟仿真技术,构建符合实际分布规律的高真实感模拟图像,解决样本不完备问题,并建立绝缘子目标图像公开数据库;对深度网络神经元的响应进行可视化研究,分析目标与神经元响应的关系,定义响应度指标,提出融合人工特征的深度卷积神经网络模型建立方法,实现端对端的目标一体化检测方案;针对绝缘子故障特性,提取多区域块深度特征,并融合形态特征生成深度特征表达,实现故障识别。项目重点研究目标深度特征表达这一科学问题,提高算法的鲁棒性和适应性,为实现飞行机器人对输电线路自主巡检提供理论及技术基础,同时为机器人导航定位、环境感知、工业无损检测等领域提供参考。

Application Abstract: Target detection and fault recognition under complex background and interference are the key technologies in the flying robot autonomous inspection for overhead powerline.There are problems when using deep learning method in this area,such as insufficient samples,obscure feature representation,uninvolved artificial features.Based on the human's spatial cognitive ability of three-dimensional objects,the target soft-recognition method using simulated images is formed,and the research framework of"simulated-real"parallel structure is established.In order to solve the problem of incomplete collection,insufficient quantity and unbalanced distribution of image samples,we construct highly realistic simulated images that conforms to the actual distribution using the virtual simulation technology,and establish the public database of insulator images;To achieve the end-to-end target integration detection scheme,the method of establishing the depth neural network structure with the artificial feature is proposed by studying the visualization response of deep network neurons,analyzing the relationship between the target and the response of neurons,and defining the responsiveness index.Aiming at the fault characteristics of insulators,the multi-region block depth feature is extracted,and the depth feature expression is generated by fusion morphological features to achieve fault recognition.This project focuses on deep feature representation of target,and aims to improving the robustness and adaptability of the fault recognition methods,providing the theory and technology for the tasks such as flying robot autonomous detection on overhead powerline,robot navigation positioning,environment perception and industrial non-destructive testing,etc.

项目受资助省

河北省

项目结题报告(全文)

目标检测和故障识别问题是图像处理和机器学习方法的一个应用,复杂背景和干扰下的目标检测与故障识别是无人机自主巡检的关键技术。无人机在电力行业输电线路自主巡检中担负着越来越重要的任务,其采集的航拍图像中,背景非常复杂且有干扰,目标检测及故障识别难度很大,应用深度学习方法时遇到目标训练样本不完备、航拍图像背景复杂且干扰严重、故障特征难以表达、人工特征未融入等问题。本项目针对典型的“高风险应用”,研究强鲁棒性的机器学习方法,即开放环境下的机器学习。针对上述问题,项目研究工作在平行视觉框架下展开。研究内容包括四方面:训练样本增广方法、复杂背景和干扰下的绝缘子目标检测方法、深度特征表达方法、目标检测与故障识别方法验证。(1)在训练样本增广方法研究方面,研究工作从两个方面着手,一是采用建模软件建立绝缘子结构模型,以生成模拟图像样本,二是通过GAN网络生成模拟图像样本;收集航拍图像并分类整理,包括真实航拍样本和人工图像样本,共计20万张。建立了绝缘子图像样本库;(2)在复杂背景和干扰下的绝缘子目标检测方法研究方面,进行了基于混合样本迁移学习的研究;提出了人工特征与深度特征融合的方法。分析像素级特征、中层语义特征、深度特征之间的关系,构成鲁棒性高的特征表达方法;(3)针对绝缘子表面故障特性,提取多区域块深度特征,融合人工形态特征生成深度特征表达,实现故障识别方法。提出了融合三种不同层次特征的模型建立和训练方法:像素级特征融合、语义级特征融合、深层深度特征融合;(4)针对实际采集的巡检图像,进行了算法的验证。项目重点研究目标深度特征表达这一科学问题,提高算法的鲁棒性和适应性,为实现无人机对输电线路自主巡检提供理论及技术基础,同时为机器人导航定位、环境感知、工业无损检测等领域提供参考。项目按照计划书要求执行,完成了计划书中所列的研究内容,达到了预期目标,并结合项目开展了电力视觉技术的研究、宣传与推广工作。

  • 排序方式:
  • 1
  • /
  • 1.Target Recognition Framework and Learning Mode Based on Parallel Images

    • 关键词:
    • Deep learning;Algorithm performance;Artificial image;Insulator string;Large datasets;Large scale data sets;Parallel images;Recognition accuracy;Target recognition
    • Yin, Zihui;Meng, Rong;Zhao, Zhilong;Yin, He;Hu, Zhedong;Zhai, Yongjie
    • 《15th Conference on Image and Graphics Technology and Applications, IGTA 2020》
    • 2021年
    • September 19, 2020 - September 19, 2020
    • Virtual, Online
    • 会议

    In the application of deep learning algorithms based on large-scale data sets, some problems, such as insufficient samples, imperfect sample quality, and high cost of building large data sets, emerge and restrict algorithm performance. In this paper, a target recognition framework and learning mode based on parallel images are proposed, and the application verification is carried out by taking the insulator target recognition in the transmission line as an example. This paper uses the artificial image generation technology to establish the insulator data set NCEPU-J, and then proposes the target recognition framework PITR and three learning modes, namely OriPITR, TrsPITR, and MutiPITR. The insulator strings with the piece number of 7, 11 and 14 are verified, and the recognition accuracy is significantly improved. The results show that the target recognition framework and learning mode based on parallel images are feasible and effective.
    © 2020, Springer Nature Singapore Pte Ltd.

    ...
  • 2.Bolt defect classification algorithm based on knowledge graph and feature fusion

    • 关键词:
    • Bolt and nut pair; Knowledge graph; Feature fusion; Decision fusion;Defect classification
    • Kong, Yinghui;Liu, Xu;Zhao, Zhenbing;Zhang, Dongxia;Duan, Jikun
    • 《8th International Conference on Power and Energy Systems Engineering》
    • 2021年
    • SEP 10-12, 2021
    • Fukuoka, JAPAN
    • 会议

    At present, there is a problem of insufficient utilization of regional features in the application of the existing knowledge graph based on GGNN in the defect classification of the bolt and nut pair of transmission lines. Therefore, a decision-making method of combining the bolt and nut pair with the original regional feature and the bolt and nut pair knowledge graph feature is proposed. For this reason, a method is proposed to combine the decision-making of the bolt and nut pair on the original regional features and the bolt and nut pair on the characteristics of the knowledge graph. First, the characteristics of the bolt and nut pair knowledge graph are combined with the adaptive normalization of the bolt and nut pair to the features of the original area. Then, the classification score vector based on fusion features and the classification score vector based on the bolt and nut pair to the original area feature are derived from the classifier respectively; Finally, the classification score vector of the fusion feature and the bolt and nut pair are fused to the classification score vector of the original region feature to obtain the final classification result. On this basis, this article uses bolt and nut pair to conduct multiple sets of defect classification experiments on the data set of the knowledge graph experiment. The experimental results show that the method of decision fusion of the bolt and nut pair to the original regional feature fusion the bolt and nut pair to the knowledge graph feature is better than the bolt and nut pair to the knowledge graph average precision, precision, and recall rate. It is effective Prove the improvement of the algorithm. (c) 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

    ...
  • 3.Bolt defect classification algorithm based on knowledge graph and feature fusion

    • 关键词:
    • Bolt and nut pair; Knowledge graph; Feature fusion; Decision fusion;Defect classification
    • Kong, Yinghui;Liu, Xu;Zhao, Zhenbing;Zhang, Dongxia;Duan, Jikun
    • 《8th International Conference on Power and Energy Systems Engineering》
    • 2021年
    • SEP 10-12, 2021
    • Fukuoka, JAPAN
    • 会议

    At present, there is a problem of insufficient utilization of regional features in the application of the existing knowledge graph based on GGNN in the defect classification of the bolt and nut pair of transmission lines. Therefore, a decision-making method of combining the bolt and nut pair with the original regional feature and the bolt and nut pair knowledge graph feature is proposed. For this reason, a method is proposed to combine the decision-making of the bolt and nut pair on the original regional features and the bolt and nut pair on the characteristics of the knowledge graph. First, the characteristics of the bolt and nut pair knowledge graph are combined with the adaptive normalization of the bolt and nut pair to the features of the original area. Then, the classification score vector based on fusion features and the classification score vector based on the bolt and nut pair to the original area feature are derived from the classifier respectively; Finally, the classification score vector of the fusion feature and the bolt and nut pair are fused to the classification score vector of the original region feature to obtain the final classification result. On this basis, this article uses bolt and nut pair to conduct multiple sets of defect classification experiments on the data set of the knowledge graph experiment. The experimental results show that the method of decision fusion of the bolt and nut pair to the original regional feature fusion the bolt and nut pair to the knowledge graph feature is better than the bolt and nut pair to the knowledge graph average precision, precision, and recall rate. It is effective Prove the improvement of the algorithm. (c) 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

    ...
  • 4.A knowledge-guided model of fitting detection in aerial transmission line images

    • 关键词:
    • Fitting detection; Spatial attributes; Co-occurrence model; Graph; Deeplearning
    • Zhao, Zhenbing;Wu, Xueliang;Qi, Yincheng;Nie, Liqiang;Lv, Bin
    • 《7th International Conference on Power and Energy Systems Engineering》
    • 2020年
    • SEP 26-29, 2020
    • Fukuoka, JAPAN
    • 会议

    Transmission lines are one of the most important infrastructures of energy Internet in China, and the effective detection of fittings is a necessary prerequisite to ensure their safety and stability. The fitting detection method based on classic deep learning only focuses on a single region and extracts the visual features of the region proposal into the classifier to identify the target, but never considers the connection between fittings. There is a certain regularity between fittings when constructing a complete and intact transmission line, interdependent and interact with each other. Therefore, we propose a knowledge-guided aerial transmission line image fitting detection model (KGFD) which uses two modules to learn the regularity of the fittings. The first module is an implicit module, starting from the spatial layout of the fittings on the image and taking the relative geometric features of the region proposal as the input knowledge of the spatial position of the fittings on the image. The second module is an explicit one. By introducing the prior knowledge of the fittings expressed in co-occurrence mode and using the region proposal as the node while the prior knowledge as the edge, a knowledge graph is constructed. The information spreads on the knowledge graph via a gated graph neural network, which realizes a combination of the prior knowledge of the fitting with the features of the region proposal. These two modules are added to the deep learning framework to detect the fitting targets, and the typical fitting data set of aerial transmission lines are used for testing. The AP value increased by 4.4% when IoU={0.5:0.95}and the AP value by 3.9% when IoU=0.5 compared with those of Faster R-CNN classification. (C) 2020 The Authors. Published by Elsevier Ltd.

    ...
  • 5.A knowledge-guided model of fitting detection in aerial transmission line images

    • 关键词:
    • Fitting detection; Spatial attributes; Co-occurrence model; Graph; Deeplearning
    • Zhao, Zhenbing;Wu, Xueliang;Qi, Yincheng;Nie, Liqiang;Lv, Bin
    • 《7th International Conference on Power and Energy Systems Engineering》
    • 2020年
    • SEP 26-29, 2020
    • Fukuoka, JAPAN
    • 会议

    Transmission lines are one of the most important infrastructures of energy Internet in China, and the effective detection of fittings is a necessary prerequisite to ensure their safety and stability. The fitting detection method based on classic deep learning only focuses on a single region and extracts the visual features of the region proposal into the classifier to identify the target, but never considers the connection between fittings. There is a certain regularity between fittings when constructing a complete and intact transmission line, interdependent and interact with each other. Therefore, we propose a knowledge-guided aerial transmission line image fitting detection model (KGFD) which uses two modules to learn the regularity of the fittings. The first module is an implicit module, starting from the spatial layout of the fittings on the image and taking the relative geometric features of the region proposal as the input knowledge of the spatial position of the fittings on the image. The second module is an explicit one. By introducing the prior knowledge of the fittings expressed in co-occurrence mode and using the region proposal as the node while the prior knowledge as the edge, a knowledge graph is constructed. The information spreads on the knowledge graph via a gated graph neural network, which realizes a combination of the prior knowledge of the fitting with the features of the region proposal. These two modules are added to the deep learning framework to detect the fitting targets, and the typical fitting data set of aerial transmission lines are used for testing. The AP value increased by 4.4% when IoU={0.5:0.95}and the AP value by 3.9% when IoU=0.5 compared with those of Faster R-CNN classification. (C) 2020 The Authors. Published by Elsevier Ltd.

    ...
  • 6.Channel-Wise and Feature-Points Reweights Densenet for Image Classification

    • 关键词:
    • ;
    • Zhang, Ke;Guo, Yurong;Wang, Xinsheng;Yuan, Jinsha;Ma, Zhanyu;Zhao, Zhenbing
    • 《26th IEEE International Conference on Image Processing, ICIP 2019》
    • 2019年
    • September 22, 2019 - September 25, 2019
    • Taipei, Taiwan
    • 会议

    Recent network research has demonstrated that the performance of convolutional neural networks can be improved by introducing a learning block that capture spatial correlations and channel-wise correlations. In this work, we propose a novel Channel-wise and Feature-points Reweights DenseNet (CAPR-DenseNet) architecture. The CAPR-DenseNet improves the representation power of the DenseNet by adaptively recalibrating the channel-wise feature responses and explicitly modeling the interdependencies between feature-points. First, in order to perform dynamic channel-wise feature recalibration, we construct the Channel-wise Feature Reweight DenseNet (CFR-DenseNet) by introducing the Squeeze-and-Excitation Module (SEM) to DenseNet. Then, we present a novel CAPR-DenseNet by adding a Feature-points Reweight Module (FPRM) to the CFR-DenseNet. Through massive experiments, we demonstrate that by recalibrating the channel-wise feature and the feature-points responses. Our CAPR-DenseNet performs better than DenseNet across challenging datasets CIFAR-10 and CIFAR-100.
    © 2019 IEEE.

    ...
  • 7.Study on Insulator Recognition Method Based on Simulated Samples Expansion

    • 《第30届中国控制与决策会议》
    • 2018-06-09
    • 中国辽宁沈阳
    • 会议

    In the application of the deep learning to the unmanned aerial vehicle(UAV) autonomous inspection, a problem about the insufficiency of both quantity and quality for insulator images emerges. In the light of this situation, a sample expansion method based on a combination of statute and 3 D modelling technology is proposed. Its feasibility is verified in a deep convolutional neural network by using five kinds of insulator simulated samples. The classification accuracy acquired by the proposed method is higher verified by the comparison of the experimental results, which proves its superiority to the traditional method containing no simulated samples. It is concluded that the simulated intensive samples of pure background have a significant effect on the accuracy of network classification. And when the ratio of real samples to simulated intensive samples goes to an appropriate value, the classification has the best accuracy.

    ...
  • 排序方式:
  • 1
  • /