Automatic Detection of Unsafe Acts in Mine Roadway Based on Adaptive Background Subtraction and Deep Learning

China’s coal-based energy resources endowment and the current stage of its economic and social development determine that the economic and social development will remain inseparable from coal for a considerable period of time in the future.Even with the “dual carbon” target,coal still  plays its role as a basic energy source,and provides energy support for economic and social development.

The coal mining industry is recognized as a high-risk industry where human behavioral factors are the direct cause of the vast majority of accidents.The monitoring system enables timely detection of unsafe miner behavior and timely intervention or change of unsafe miner behavior,which can effectively prevent accidents from occurring.However,safety and security in coal mines can be compromised by the responsibility,work fatigue and efficiency of safety monitoring staff.Therefore,it is important to study automation methods based on machine learning to replace manual monitoring using artificial intelligence technology in order to achieve automatic identification of unsafe  behavior in underground mines in a safe and efficient manner over a long period of time to ensure coal mine safety.

Specifically,it is hoped that the high-frequency,high-resolution image data captured by surveillance cameras will be used as input to develop appropriate statistical and deep learning models for the timely detection and correction of unsafe behaviors in mine production operations.Taking the identification of illegal behaviors by miners on monkey vehicles as an example,the essence of the problem is target detection and target identification.The current mainstream algorithms can be divided into two-stage methods and one-stage methods.Two-stage methods,such as R-CNN and Faster R-CNN series algorithms based on candidate regions.One-stage methods,such as Yolo and SSD algorithms,which are currently receiving great attention.Although these methods have played an important role in target detection,they are difficult to apply to the mine safety management issues we are concerned about,mainly for two reasons.Firstly,annotation is costly.Specifically,there are many types of roadways in coal mines and complex working faces,so it is very expensive and time-consuming to do bounding box annotation for the massive amount of data collected by each surveillance camera.Secondly,the method requires a large sample size.Although a large number of images can be extracted from mine roadway monitoring video,there are fewer images with miners present.This kind of image can be extracted mainly during rush hours.It is also worth mentioning that these images have extremely high similarity and cannot provide too much variability for model training.At the same time,due to the strong implementation of safety management,the vast majority of miners have not engaged in unsafe behavior,so relevant images constitute a large number of negative examples and a very low proportion of positive examples (images with unsafe behaviors).There is therefore an urgent need for a new methodology,applicable to smaller sample sizes.

For these reasons,we will mainly face two challenges:Firstly,how to quickly identify key pixel regions containing miners without the support of annotation boxes.Secondly,based on the discovered key areas,how to construct a high-precision deep learning model with the support of small samples.With regard to the first challenge,we use for reference to the background subtraction algorithm to obtain the target region by performing a subtraction operation between the current frame in the image sequence and the background image.The characteristic of this algorithm is its fast calculation speed,but its disadvantage is that the calculation results are unstable.To solve the stability problem,we propose a three-stage solution.In the first stage in outlier discovery,the median method is used to estimate the standard deviation instead of the traditional moment estimation,so as to obtain a more robust outlier region.In the second stage,the outlier area is smoothed on the pixel plane to obtain more stable results.In the third stage,the smoothed outlier region,bounded by a certain threshold,is adaptively extracted for important pixels and then the classical connected domain algorithm is used to do the integration in order to obtain the complete local region and thus obtain a high-quality effective miner sub-image dataset.With regard to the second challenge,we adopt the transfer learning method to leverage the pre trained results on the classic big data set to minimize the requirements on our sample size.Specifically,for the obtained miner sub-images,category annotation is directly performed to avoid complex bounding box annotation,saving expensive manual annotation costs.Transfer learning is adopted,and Google’s open source MobileNet model is selected as the framework of transfer learning.

The experimental results show that,compared with the traditional estimation,the thresholds constructed based on standard deviation obtained from the median estimation make the miner sub-images extraction accurate and robust.Finally,the authors use the MobileNet model for transfer learning,and the results further show that the miner sub-image dataset obtained by the median estimation can provide higher precision than that of the traditional estimation.The resulting out-of-sample prediction accuracy and AUC are excellent in classifying the miners’ actions into safe or unsafe category.

Finally,some feasible research directions for future work are proposed.Firstly,the images in the video frames are highly similar over a period of time in close proximity.For convenience,this article extracts images at equal intervals,which does not fully utilize temporal similarity.In the future,further research will be conducted to improve the efficiency of sub-images extraction by utilizing temporal similarity.Secondly,further in-depth research should be conducted on a series of safety behavior identification problems in mine roadways such as environmental monitoring and equipment operation,in order to improve the intelligent analysis technology and level of mining safety monitoring.