Study on Accuracy Metrics for Evaluating the Predictions of Damage Locations in Deep Piles Using Artificial Neural Networks with Acoustic Emission Data

Accuracy metrics have been widely used for the evaluation of predictions in machine learning. However, the selection of an appropriate accuracy metric for the evaluation of a specific prediction has not yet been specified. In this study, seven of the most used accuracy metrics in machine learning were summarized, and both their advantages and disadvantages were studied. To achieve this, the acoustic emission data of damage locations were collected from a pile hit test. A backpropagation artificial neural network prediction model for damage locations was trained with acoustic emission data using six different training algorithms, and the prediction accuracies of six algorithms were evaluated using seven different accuracy metrics. Test results showed that the training algorithm of TRAINGLM exhibited the best performance for predicting damage locations in deep piles. Subsequently, the artificial neural networks were trained using three different datasets collected from three acoustic emission sensor groups, and the prediction accuracies of three models were evaluated with the seven different accuracy metrics. The test results showed that the dataset collected from the pile body-installed sensors group exhibited the highest accuracy for predicting damage locations in deep piles. Subsequently, the correlations between the seven accuracy metrics and the sensitivity of each accuracy metrics were discussed based on the analysis results. Eventually, a novel selection method for an appropriate accuracy metric to evaluate the accuracy of specific predictions was proposed. This novel method is useful to select an appropriate accuracy metric for wide predictions, especially in the engineering field.

Automated summary

This study summarized and analyzed seven commonly used accuracy metrics in machine learning, and found that the TRAINGLM training algorithm exhibited the best performance in predicting damage locations in deep piles. Subsequently, the study discussed the correlations between the accuracy metrics and proposed a novel selection method for choosing an appropriate accuracy metric for specific predictions.