To automate manual work of operators who search and select road defects from road laboratories video footage, like cracks, holes and patches.
As they say, there are two major problems in Russia, and one of them are bad roads. According to Wikipedia, the total length of Russia's road network is 1,396,000 km. Russia is ranked fifth in the world by length of the road network. Given that and the fact that climatic conditions are not always favorable, the task of maintaining the good condition of the entire road network in Russia is a non-trivial. It is important to understand that the problem requires a systemic solution. Investing in a large number of disparate repairs — it is a struggle with the symptoms rather than treating the disease.
Systematic approach assumes that in the beginning we have to understand the problem: what is the condition of the road, which roads need to be repaired first? Here's an example: suppose, there are two roads that need repair. A detailed diagnosis is made, and, as a result, it is concluded that the cost of repairing the two areas is approximately the same at the moment. Another conclusion on the results of the diagnosis may consist in the fact that after a year the costs of first road repair will require ten times more money than the second. Situations like this could be predicted as a result of a detailed analysis of the structure and nature of damage on the roadway. Thus, diagnosis becomes very important in terms of limited resources.
The following steps has had to be made to complete road surface diagnosis. A special car (road laboratory), equipped with a variety of measuring instruments (in the simplest case, video cam) drives the road. Sensor's output are recorded and later handled by operators manually. As a result, we get some road-«passport» that shows the scope and urgency of repairs.
We have one serious bottleneck in the described method of diagnosis: «manual» data processing. The task of an operator is manual digitization of defects of road surface. Armed with a keyboard, mouse, and specialized software, operators must specify (ie circle) all defects in the target road section. Imagine, how people encircle patches and holes on the roads. Well, it is still possible, but how to circle all the cracks and other minor defects?
There are two solutions for this problem: either to evaluate defects «roughly», seriously impairing the quality of diagnosis, or to search for defects automatically, using modern methods of machine learning.
The next step is to build a model that based on the portion of the image can predict the presence or absence of a defect. Machine learning is very extensive and offers a variety of possible architectures, models and algorithms of their training. In the case of image recognition, models based on artificial neural networks are the most effective. More specifically, we have used convolutional neural network.
An important neural networks feature is capability of using the power of parallel computing. We cannot use this advantage on modern CPUs, optimized for sequential calculations. On the other hand, the graphics processor (GPU) from NVIDIA are specifically designed for the efficient operation of parallel computing. Typically, the performance of learning and recognition on GPU is 50-100 times higher than running the same model on CPU.
Our software has been successfully build into existing business processes. A scalable cluster solution was created to parallelize the process of training models and recognition of defects on several levels
Levels of parallelization:
Special software was developed. It allows operators to create, modify and populate training sample, to review the results of recognition, to perform post-processing of the results using a specially crafted script language, that provides reproducibility of results and allows you to quickly get a predetermined sequence of commands and modify it.
Also, web dashboard was developed, which allows: