Evaluation metrics for Regression problems
As Machine Learning has grown in today’s world, one of the most prominent problems is a regression problem. These problems are the ones ranging from predicting house prices in a certain influential area to tomorrow’s stock prices. There is no doubt that regression problems are not only basic but also one of the most crucial aspects of machine learning. Like any other problem, regression also needs a metric to decide how effective the solution is, which would determine if it is worth implementing or not.
The metrics for regression problems differ from those used for classification problems. A simple metric like accuracy would not be effective in a regression problem. However, there are certain metrics for regression that are more useful than others, and that is decided on the structure of data( is the data scaled, are there any outliers, etc.). These are some of the basic metrics used for regression.
1. R-Squared (R²)
R-Squared is a metric that is used to determine how well the model has fit itself to the data. Also known as the coefficient of determination, r-squared will be 0.0 for a baseline model but 1.0 for a perfect fit model.
To understand this metric better, consider the following:
A data-set has “n” values marked y₁,y₂…yₙ with the prediction of those values as f₁,f₂…fₙ
If “yₘ” is the mean of all the “y” variables, then the variability of data can be measured with 2 formulas:
- Total Sum of Squares = SSₜₒₜ = Σ(yᵢ-yₘ)²
- Residual Sum of Square = SSᵣₑₛ = Σ(yᵢ-fᵢ)²
With the help of these two formulas, R² can be calculated. The most general definition of R² is:
R² = 1-(SSᵣₑₛ/ SSₜₒₜ)
In the best-case scenario, where the predictions are 100% correct, fᵢ will be equal to yᵢ. In that case, SSᵣₑₛ = 0. As a result, R² will be 1 or a perfect fit.
In a baseline model where the model always predicts the mean value yₘ, SSₜₒₜ will be equal to 0, thus bringing R² to 0.
However, in the worst-case scenario where R² lies outside the constraints of (0,1), the model performs worse than the baseline.
2. Mean Squared Error (MSE)
One of the most commonly used metrics, MSE is a metric that makes outliers stand out more. It is simply the squared error between the actual and predicted values.
In mathematical terms, MSE = (1/n)* Σ(yᵢ-yₘ)²,
where yᵢ = observed predictions and yₘ = predicted predictions.
MSE is very similar to the next metric, however, there is a special use case where you should use MSE instead of MAE.
Since MSE makes outliers stand out more and is extremely sensitive to them, you should use it if being 10% off is way worse than being 5% off. That being said, the closer values are to 0, the better in this case.
3. Mean Absolute Error (MAE)
Similar to MSE (only in terms of mean), MAE calculates the mean ABSOLUTE error. It can be used where MSE is not suitable. For. e.g When all errors are on the same scale. (Predicting 99 and 101 when trying to predict 100 produces the same error).
MAE = Σ |yᵢ-xᵢ|/n
Simply said, MAE is the arithmetic average of absolute errors.
4. Root Mean Squared Error (RMSE)
One of the most widely used metrics and arguably a better metric than MSE, RMSE is simply the square root of MSE.
Since it takes the square root, it has the same units as the quantity plotted on the vertical or the Y-axis.
In this way, interpretation is easier and it is a better measure of fit than a correlation coefficient.
