Relationship Between Standard Deviation and Root Mean Square Error (RMSE)
Facts about RMSE:
- Includes random and systematic errors
- More useful to use as it reveals biases (systematic error)
- It tells us how accurate the data is
Facts about Standard Deviation:
- Includes only random error
- Reflects only how precise the data is
- It does not tell us how accurate the data is in the presence of biases. It only tells us how precise the data is.
Table 1 illustrates the difference between standard deviation and the RMSE in revealing the presence of biases in measurements. The table represents a vertical accuracy evaluation for points cloud derived from UAS imagery by comparing it to a higher accuracy elevation model derived from a mobile lidar mapping system. The UAS-derived elevation model needed to meet 5-cm (0.164-ft) accuracy. If we used standard deviation alone, the data would meet the specifications with a value of 0.076-ft. However, looking at the high value of 0.246-ft. (7.5-cm) of the mean, it is obvious this data set contains a bias and the only way to catch it is by either evaluating the value of the mean or using the RMSE as the accuracy measure. The high value of the RMSE = 0.257-ft. (7.83-cm) will flag the data as not meeting specifications. The far right column contains the error values after removing the bias of 0.246-ft. (7.5-cm) from the measurements. Once we remove the bias, the values for the RMSE and the standard deviation are equal and they both meet the project accuracy specifications. Removing a bias from elevation data could be as simple as shifting the entire dataset up or down by the magnitude of the bias itself, such practice is called z-pump.