Non-Linear SVM - Scaler Topics

Overview

There are two types of machine learning models: supervised and unsupervised. Supervised learning, unlike unsupervised, deals with labelled data. Supervised learning has two types: classification and regression. Classification deals with discrete data, and regression deals with continuous data. This article revolves around a classification algorithm called Support Vector Machines.

Transform Your Career

Choose from our industry-leading programs designed for career success

NSDC Certified

Modern Software and AI Engineering Program

Master full-stack development with AI integration

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

Modern Data Science and ML with specialisation in AI

Advanced data science techniques with AI specialization

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

Advanced AIML with Specialisation in Agentic AI

Deep dive into AIML with focus on Agentic systems

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

DevOps, Cloud & AI Platform Engineering

Build and manage AI-powered cloud infrastructure

12 MonthsDuration

AI-LedCurriculum

Career SupportSupport

+1000 more

Go to Program

NSDC Certified

AI Engineering Advanced Certification by IIT-Roorkee

Premier AI engineering certification from IIT-Roorkee

3 MonthsDuration

AI-LedCurriculum

Career SupportSupport

Go to Program

Introduction to SVM

Consider the following data points:- Change the color scheme A supervised learning model such as the Support Vector Machine would try to determine a hyperplane that separates the data: Change the color scheme as the previous one. This new hyperplane can be used to classify the new data points.

Support Vector Machine, or SVM, is a popular supervised learning algorithm. It is used primarily for classification but can also be used for regression tasks. The SVM algorithms create the best decision boundary that segregates an n-dimensional space into classes so that we can categorize new data points in the future. This decision boundary is termed as hyperplane. The SVM algorithm uses certain extreme data points or vectors known as the support vectors to create the hyperplanes so that we get the decision boundary which is at a maximum equal distance from the concerning vectors. Change the design SVM algorithm is of two types:-

Linear SVM: When the data points are linearly separable into two classes, the data is called linearly-separable data. We use the linear SVM classifier to classify such data.
Non-linear SVM: When the data is not linearly separable, we use the non-linear SVM classifier to separate the data points.

Scaler Placement Report and Statistics

₹23L

AVG CTC

SCALER PLACEMENT PROOF

Scaler learners achieved 2.5x salary growth with average post-Scaler CTC reaching ₹23L.

11,000+placements

650+companies

Verified data

See full placement report

What are Non-linear SVMs?

Change the color We cannot separate the above data points with a single line. Also, if we give more than two classes, it is impossible to separate them with a single straight line. Consider the following example:- Change the colors There are two classes of data points that a straight line cannot separate. But, a circular hyperplane can separate them, hence we can introduce a coordinate Z, with the help of X and Y, where $Z = X^2+Y^2$ . Now after introducing the third dimension, the graph changes to:-

The above depiction of data points is linearly separable and can be separated by a straight-line hyperplane. final

This representation is in 3-d with z-axis. In 2-D, the graph looks like this:- straight-line hyperplane

This is what a non-linear SVM does! It takes the data points to a higher dimension to be linearly separable in that dimension, and then the algorithm classifies them.

Radial Basis Function Kernel

The radial basis kernel is a kernel function used in machine learning to find a non-linear classifier or regression line.

Turn Learning into Career Growth

1200+Hiring Partners

89%Placement Rate

11,000+Placements

147%Avg Salary Increment

2.5XCareer Growth

₹23 LPAAvg Post-Scaler Salary

1200+Hiring Partners

89%Placement Rate

11,000+Placements

147%Avg Salary Increment

2.5XCareer Growth

₹23 LPAAvg Post-Scaler Salary

What is a Kernel Function?

This function transforms the n-dimensional input space to an m-dimensional space where n>>m so that we can efficiently do the required calculations in a higher dimension. We have seen before that a non-linear curve in the lower dimension becomes a linear curve in a higher dimension.

The mathematical formula behind RBF is:-

K\left(x, x^{\prime}\right)=e^{-\left.\gamma\left\|x-x^{\prime}\right\|\right|^2}

Gamma is a scalar that defines how much influence a single training example (point) has.

Let's implement non-linear SVM using the RBF kernel function in python. We will take the help of the sklearn library.

Here's what the data looks like:- data look like

Output:- data looks like

The Kernel Trick

A method where Non-Linear data is projected onto a higher dimension space such that it is easier to classify the data where it could be linearly divided by a plane. The trick is that we are not transforming the data points. We are achieving this mathematically for our convenience.

Linear SVM vs Non-Linear SVM

Linear SVM	Non Linear SVM
The data points are separated using a single line	The data points are hard to separate, so other shapes are used as a decision boundary.
Data is classified with the help of a hyperplane.	Kernels are used to classify data points.
difficult to classify more than two labels.	Used for classifying more than two labels.

Conclusion

The key takeaways from this article are:-

There are two types of SVM: linear and non-linear, they are used depending on the type of data.
Non-linear SVM uses the Radial Basis Function Kernel that takes the data points to a higher dimension so that they are linearly separable in that dimension, and then the algorithm classifies them.
Radial Basis Function Kernel is implemented using sklearn's SVC model.