Support Vector Regressor

Hello,

I am (name) from LearnVern.

Today we are studying in continuation of our Machine Learning session.

And today we will see about Support vector regressor

We have studied Support Vector machines already during the classification module.

So, this support vector machine works on support vectors, hyperplanes.

And our support vector regressor works on the same principle as support vector classification.

But what is the difference between them

Here the support vector regressor works for the prediction or outputs which are in the form of continuous variables, and this gives the output in discrete values.

For example,

If you want to predict, how much it will rain?

So, here regressor will give a specific value,

If you want to take the example of the house pricing, there also you can use a regressor, this uses a hyperplane and with the help of this hyperplane it gives us predictions about the output.

This can solve both linear and non linear problems.

So,let's understand this in depth.

Now, sometimes what happens is this, there is a confusion in understanding linear non linear problems.

So, let me explain to you that also.

For that, I will draw in a paint window and explain,

So, if I have some data points, there are some which are circular data points and there are few which are in some other shape.

So, we will denote these two shapes as class1 and class2.

Now, you tell me if we want to create a distinct line between these two shapes, can we do that?

We will call one as shape 1 and the other as shape 2, so can we make a distinct line as a boundary line between the two, where at one side we can have shape1 and on the other shape 2.

This over here is very simple. I can select a line and draw in between them like this.

So, such an example is linearly separable.

Where with the help of one line both the classes got separated,

So, here this was so simple,

But sometimes what happens is that,

I have a datapoint and the data points are like this, and after that I have some other shapes like triangular.

So, if we have data points scattered this way, and you want to separate them with the help of a linear line, then it will become very difficult,

If I draw from here then you can see there are so many miss classifications that will happen, or from here also, and here also if I draw, meaning if I draw from any angle, it will not give me a proper solution, because in each line's front and back wrong classifications have been done.

And this is why there are so many confusions, so we cannot do it this way.

So, the best solutions for such problems is that we can use SVR, and if it is a classification problem then we can use SVM,

So what SVR and SVM do is that through higher dimensions, it creates a hyperplane, so these two types of initial data that we have transform them, and convert them in higher dimensions.

So, if I talk about this first example here we had only x and y as 2D dimensions,

But if I try to represent this example in 3D Dimension, so I took here some x y and z axis,so here it is possible that this circular point will lie between this x and y on this plane,

Similarly, this triangular data would be present in between y and z,

So, when we transform the data into higher dimension, then we come across that the two shapes lies in some other planes, and if this is actually happening, then we can create a plane in between them,

So, earlier in the other example we were making a line, where it's easy to draw a line in 2D, but here we have created a 2D plane which will separate both the classes.

So, this is the difference that this support vector machine or regressor has the capability of solving these nonlinear problems also.

Ok! So let's come back on our presentation now,

So, here we were discussing that it solves both linear and nonlinear related problems, and it works on the principle of support vector machines only.

So, let's move ahead, nd this concept we had already studied

So here we have to create a hyperplane and we here in the centre you can see, this is a hyperplane.

And we call this as optimal hyperplane, why because we draw a decision boundaries parallel to the hyperplane,

So this dotted line over here is the decision boundaries, and this created through support vector, now you will see that here this star point over here is very close to the decision boundary so, this is the support vector here, similarly this blue datapoint is our support vector because this is nearest to the hyperplane.

And this parallel line if we draw from the hyperplane they are called as decision boundaries, and we try to maximise the distance in between these decisions boundaries as long as possible, so that we can achieve a correct classifications based on maximum margin, as we had studied the concept of soft margin and hard margin, so this we had studied in SVM.

Now, if we talk about SVR, it's concept also is very similar,

Here this centre line is hyperplane, and here the maximum number of points lies on the hyperplane, or close to the hyperplane, and here also we have two boundaries, they are called as decision boundaries and support vector lies inside these decisions boundaries, so you can clearly see that support vectors are inside.

So, this is what we do in SVR, because it's Regression so we try to fit in a best line, which itself is the meaning of regression where our goal is to have a line that can fit best between the data points, so as we will proceed towards x that line will also keep getting increased, and if we pick any value of x then that line would predict us the similar value for y also as an output, so this is our prediction.

So, here in SvR the concept is the same as SVM, the only difference is that we need to fit in the best line.

So, let's move ahead and go through important terminology, such as kernel, hyperplane, support vectors and boundary lines.

So, here we will begin with kernel, so this is a function and its work is to help in creating a hyperplane with its function,

So to create that first we will check the data is in Lower dimension then it will have to be converted into a higher dimension.

So this is the work of kernel function.

When the data is converted into higher dimensions then it becomes easier for the model to create hyperplanes in between them. Whereas if the 2D data is linearly separable as we just saw in the paint window then there is no problem as such, just draw a line in between so if you have a complex data which is overlapping as we saw in the paint window, then we use the kernel function which helps in creating the hyperplane between them.

So in this way, the function of the kernel becomes very important.

Also there are different types of kernel function which we will discuss later on.

Now the second thing is the hyperplane that we have already understood, so it is a boundary which bifurcates between the two classes, aur in the case of regressor it is a line which is the best fit line between the two classes, which will help us in predictions.

After this the next function is the boundary line, so the boundary line is a parallel line which runs parallel to the hyperplane line, which creates a margin and is called a boundary line.

After the boundary line the next part is about support vectors, this helps us in creating the boundary line, because without the support vectors we will not be able to create the margin, as there is one side which is positive and other side is the negative side, there we will not be able to create this boundary or additional margin that we want, so support vectors helps has in creating this margin or these boundary lines.

Now let's talk about applications,

So this is regression so we will try to to predict the revenue generation,

It can also be used for price prediction,

Also to understand the number of covid cases that are rising or decreasing.

So in these predictions we can also use SVR.

So friends let's conclude for today we will stop this session here, and its upcoming part we will cover in our next session. Till then keep learning and remain motivated thank you.

If you have any queries or comments, click the discussion button below the video and post there. This way, you will be able to connect to fellow learners and discuss the course. Also, Our Team will try to solve your query.