Real-time multiple damage mapping using autonomous UAV and deep faster region-based neural networks for GPS-denied structures

An autonomous unmanned aerial vehicle (UAV) system integrated with a modified faster region-based convolutional neural network (Faster R-CNN) is proposed to identify various types of structural damage and to map the detected damage a GPS-denied environment. The proposed method reduces the number of false positives significantly using a real-time streaming protocol and multi-processing, particularly in the case of very small cracks in blurry videos due to the UAV vibrations. In comparative studies, the modified Faster R-CNN using ResNet-101 as the base network showed superior performance in detecting small and blurry defects with a mean average precision of 93.31% and mean intersection-over-union of 92.16% in video frames captured by the lowcost autonomous UAV. The autonomous flights of the UAV were tested in a real large-scale parking structure to account for the high wind effects during flight. The UAV successfully followed the desired trajectories, and the Faster R-CNN detected defects accurately.