Monitoring states of road surfaces provides valuable information for the planning and controlling vehicles and active vehicle control systems. Classical road monitoring methods are expensive and unsystematic because they require time for measurements. This article proposes an real time system based on weather conditional data and road surface condition data. For this purpose, we collected data with a mobile phone camera on the roads around the campus of the Karlsruhe Institute of Technology. We tested a large number of different image-based deep learning algorithms for road classification. In addition, we used road acceleration data along with road image data for training by using them as images. We compared the performances of acceleration-based and camera image-based approaches. The performances of the simple Alexnet, LeNet, VGG, and Resnet algorithms were compared as deep learning algorithms. For road condition classification, 5 classes were considered: asphalt, damaged asphalt, gravel road, damaged gravel road, pavement road and over 95% accuracy performance was achieved. It is also proposed to use the acceleration or the camera image to classify the road surface according to the weather and the time of day using fuzzy logic.
For vehicle safety and comfort, knowledge of the road surface (anomalies, weather condition, type) is essential [1]. By integrating surface condition data into the vehicle's control and perception framework, autonomous systems can adapt their driving strategies such as speed, acceleration, or braking profiles to current road states, thereby enhancing both safety and passenger comfort [2]. In addition, proper road data is needed to ensure safe and comfortable driving with advanced driver assistance system(ADAS) [3]. Continuous road surface monitoring is vital, as changing surface conditions directly affect braking, traction, and stability. Real-time awareness enables autonomous systems to adapt to driving decisions, ensuring safer and more reliable operation.To avoid future problems such as accidents and road damage, road sections with a high density of problems should be inspected regularly [4].
Over the last few decades, problem detection and classification in pavement analysis has improved significantly. Traditionally, visual inspection along the route has been used by human experts to perform these tasks and calculating a certain index, the IRI [5], for the road surface condition [6]. The current practice is labour intensive, it is impossible to monitor all roads and the measurements take a while, so there is a need for automating monitoring of roads as a perfect case for automation.
Other instruments, such as dipstick profilers and profilographs, are much more accurate. However, they are either time-consuming or impractical to use due to slow scanning speeds.
Today, for many roadway applications, including automated inspection and monitoring, imaging technologies have been chosen [7]. A good road should have low smoothness, tire-road friction, and noise level [8]. Manually extracted features are usually not the most appropriate representation of road images [9]. Therefore, the use of convolutional neural networks (CNN) structures that do not require feature extraction provides advantages.
Another method of detecting road anomalies is to place a camera outside a vehicle and capture real-time 2D images. In this way, information about the size and location of the problem and the surface quality of the road can be collected. Nolte et al. [10] compared ResNet50 and InceptionV3 architectures for classifying six road surface types using vehicle-mounted camera images. Their focus was mainly on evaluating CNN performance for visual surface recognition. In contrast, Roychowdhury et al. [11] extended this approach by also estimating the road friction coefficient, linking visual information to vehicle dynamics and safety.. In [12], the extraction of road areas using CNNs was achieved with an accuracy of 98.33%, a precision of 97.74% and a recall of 95.21%. Some shaded areas were misclassified, but overall performance was unaffected [12]. In the detection of cracks in concrete surfaces, the fully convolutional encoder-decoder network achieved impressive performance. The model effectively identified cracks while minimizing false positives and negatives, demonstrating strong capabilities in terms of both accuracy and recall [13]. Gopalakrishnan et al. (2017) use deep CNNs and transfer learning for automatic pavement distress detection, aiming to classify issues like cracks and potholes [14]. Pereira et al. (2018) propose a deep learningbased approach for road pothole detection in Timor Leste. Their model utilizes convolutional neural networks (CNNs) for efficient detection of potholes from road images. Pereira et al. achieved impressive performance, with an accuracy of 99.80%, precision of 100%, recall of 99.60%, and an Fmeasure of 99.60% [15].
Since camera or sensor-based systems are expensive, many studies have been conducted on identifying road problems using mobile phones. Tedeschi et al [16] use an Android mobile device to perform real-time detection of anomalies, such as potholes and cracks. Optical, microphone, acceleration, laser, polarimetric radar, ultrasonic and microwave sensors are used for road surface classification [17]. However, all of these techniques are expensive approaches. Many studies also focus on the application of inertial sensors in smartphones to the classification of road conditions, as it does not require additional costs. Studies [18] and [19] used accelerometer and GPS data with SVM, achieving 69.4% accuracy in classifying road surfaces. Study [20] used smartphone sensors, achieving 87.68% accuracy in classifying asphalt, cobblestone, and dirt roads. Ngwangwa and Heyns [21] used acceleration sensors and neural networks to estimate road roughness. Vittorio et al [22] detected potholes with over 80% accuracy on the basis of accelerometer data from smartphones. Kyriakou et al [23] trained an artificial neural network on accelerometer and gyroscope data and achieved an accuracy of more than 90% in the detection of road anomalies. Bajic et al. [24] used machine learning and Z-axis accel
This content is AI-processed based on open access ArXiv data.
