Evaluating Regression Model Performance - Practical

Hello,

I am (name) from LearnVern. (7 seconds gap, music)

You are welcome to the Machine Learning course and this tutorial is in continuation of our previous tutorial.

So, let's see ahead.

In today's session we will see evaluating regression model performance.

We had earlier seen in the same way that we can evaluate classification Models,

However in classification it's metrics are different as it is categorical, but the metrics of this are different.

So, here you can see I have listed the metrics that are there in regression.

First is Mean Absolute Error.

Second is Mean Squared Error.

This is called the R2 score.

So, let's understand this one by one.

This is very simple. I will explain it to you, and you will understand.

Mean Absolute Error.

Now, we will first understand the meaning of this word Mean Absolute Error,

Mean Absolute Error, let’s understand the meaning first ok?

What is the meaning of Mean?

Mean meaning AVERAGE, average.. (read slowly, typing).

What does absolute mean?

Absolute meaning absolute value, neither we will take it as negative nor positive, so “not negative and not positive”, meaning we are just talking about its magnitude. (read slowly, typing). right?

So, we are talking about Mean Absolute Error, how are we going to calculate that?

"Average of the absolute difference between the prediction and actual values"

So, now here we will assume, for instance, let's take 1 as actual value like this, and 1.2 is the value that is predicted, next, 2 is the actual value and 2.1 is the predicted.

In this way, 3 is the actual value and 3.2 is predicted…

Now, here if you remove the absolute difference in this, it is of 0.2, and here up the absolute difference is of 0.1, and above this what is the the absolute difference? 0.2.

Now, we have to get the average of, as here it is saying: so let's take out its sum, what do we get adding these? so we added, sum is equal to 0.2 + 0.1 +0.2 this is 0.5 divided by 3 because there are three values.

What will be the answer?

3 multiplied by one is one, and the remaining is twenty, two zero.

So, here we will get 0.16.

3 multiplied by 6 is 18.

So, this 0.16 is the mean absolute error, so this is very less, this much error is fine to have, but when the data is large in real time application then this is also high, as it should come like 0.00 and something, then it's good.

So, this mean absolute error, that we were talking about.

So, this is the first concept that we discussed of mean absolute error.

Now, we will go onto the next one, but before that let's execute this and see.

And walk through the code, you must have started getting handy with the codes.

See, import pandas,

From SKlearn import model selection

Why are we importing model selection?

Because with model selection we can use Kfold and use multiple samples.

You, must have understood the multiple samples,

So, in multiple samples, here you will understand, if I have this as an entire data, so by k fold if K says take 3 samples so I will remove 3 samples like this,

So, this is one sample, this is second, and this is third…

So, where is it coming from?

It is coming from the same data, if I want 4 samples then I can remove 4 from here, and it will also tell the size as to what size each sample should be of, 10, 10 or 100, 100, accordingly it will remove samples like 4 samples of 10, or 4 samples of 100, from the actual data itself.

So, what is the benefit of this?

So,the benefit of this is that if you are picking the sample randomly there is no pattern followed in it. right?

Because sometimes what happens in the data, as you must have seen in the iris dataset also, is that there is first Setosa, then verci colour, and then virginica, so there is a sequence, a pattern which is followed.

So, this kind of thing is completely absent here.

Ok!

So, model selection,

Then after this, "from sklearn dot linear model import linear regression".

So, we imported linear regression, so there is regression over here and not classification.

And here, we are using housing data.

The names are,:

Crim

ZN,

indus.

CHAS, (pronounce - ces)

So these are some of the parameters.

Now, here you can see, MED-V, this value is the output and the rest are inputs.

Then we converted this entire thing into a dataframe, so the data frame is equal to pandas dot read csv.

So, through url use read csv.

In the delimiter, there is white spaces, and the names are passed from here.

Now, convert this into an array.

Here took all the rows, and from 0 up till 12 columns have been chosen, 13 one will not be chosen from here.

So, till 12 have put in X and

Have put 13 in y.

Then take the seed value, in the random state here have put the seed value in this.

Now, k fold is equal to model selection dot kfold, n split, I took 10 here, so in how many samples this data will be splitted?, In 10.

Now, here the model is equal to Linear regression.

Then scoring, negative means absolute error, so negative here also means negative.

Here the value of zero indicates no error.

also you can see the meaning of 0 value is perfect prediction,

So, "like log loss this metric is also inverted", so that is also inverted and that is the reason that we are getting negative values here.

Then in results, Model Selection dot cross val score, and here have passed the model, x and y, in CV have passed Kfold and in scoring, have passed scoring.

Then calculated mean and standard deviations.

So, here we have the output, and you can see that.

So, minus 4 point something was the output, let it get executed once again.

So, minus 4 point 0 this much is the output.

So, this is the first MAE, mean absolute Error.

Ok!

Now, in mean absolute error we found out the absolute difference, so what is this?

This is showing us the magnitude of the error.

Now, we will come here,

In Mean Squared Error.

Now, this is also the same, there is no difference in it.

What happens here is that when we are performing this difference, assume that, instead of 3.2, here we will have the value as 2.8, so because of this 2.8, this will become minus 2.

So, this will become negative, so our answer will become 0.1 and then we will again divide this by 3, then our problem will increase.

So, in such cases when we get negative, positive, then we take out its square, for each of them .

Raise to 2, so if you take this square, then wherever there are negative values, that value will also become positive.

So, in this way when you square the values..

So, "it is much like mean absolute error", but but "it provides a gross idea of the magnitude of the error".

So, here what are we going to do?

"Taking the square root of the mean squared error converts"..

So, ok !

So, in mean squared error first take their square and then take their average.

So, in this way the mean squared error is found out.

Then take the square root of the that also, then it is called as root mean squared error, , it is called RMSE, so these term becomes really confusing, but it is nothing, take their square, and then take out their sum and calculated their average,i should have written AVG, so this becomes mean squared error.

And ,after this average of you take out the square root of it, then it will become RMSE, root mean squared error.

So, here we will execute this.

And here you can see I have already executed this, and the answer is minus 34.

So, this is showing the magnitude of the error.

So, this will also be inverted,

So, you can see all RMSE are inverted.

So, "Remember to take the absolute value before taking the square root value, before taking the square root if you are interested in calculating RMSE."

Ok!

So, which means, if you are taking RMSE, then take the absolute value discarding the minus value.

When You are removing it's square root of it at that time.

Now, see this R squared,

This R squared is an important metric, as it tells us about goodness of fit, in the set of predictions, to the actual value.

So, in goodness of fit you must have seen, when we are trying to do improvement upon any model, then what are we trying to improve?

So, we are trying to improve this only for goodness of fit.

Meaning we don't want overfitting nor underfitting.

So, we should have one appropriate fitting.

So, this metric provides us with an indication of goodness of fit.

Its value comes between 0 to 1.

Ok!

So, “zero and one for no fit and perfect fit.”

So, zero meaning no fit, and one meaning perfect fit.

So it means, higher the value we are proceeding towards perfect fit.

So, let's execute this also and see.

So here you can see zero point two zero, only this much is the value, which is very poor, so this is a very low and not so great value.

So, here you can see the prediction has a poor fit to the absolute values, so it is a poor fit, with the value close to zero and less than 0.5, so the value should have been above 0.5.

So, what should we do here? here we should try to improve through hyper parameter tuning, or we try to use another model.

So, we should try with something else and then check,

So, this is done with our mean absolute error, mean squared error, and R2 score.

So, in this way we find out from the metrics as to how the model is performing.

So, try with your algorithm, so try with your dataset and algorithm and let us know about it.

So, we will stop today's session here. And it's further parts we will cover in the next session.

Till then keep learning and remain motivated.

Thank you.

If you have any questions or comments related to this course.

then you can click on the discussion button below this video and post it there.

So, in this way you can discuss this course with many other learners of your kind