Support Vector Machines Explained

By

What is a Support Vector Machine?
In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a non-probabilistic binary linear classifier.

 SVM performs classification by finding the hyperplane that maximizes the margin between two classes. The vectors that define the hyperplane are called "support vectors".

 Why is it important to find the optimal hyperplane?
There can be many possibilities when it comes to finding the hyperplane that separates the different classes.
If we compare those 3 hyperplanes in the graph, hyperplanes 1 & 2 are very close the points and the margin/gap between the points are very less. In that case, points can be easily misclassified. Whereas the optimal hyperplane is positioned right at the middle with a larger margin from the support vectors both sides.
It is important to have the largest margin/gap possible when it comes to separating different groups of data.

 NOTE: Here, only the support vectors are important, as these points decides where the hyperplane sits. Other points are ignorable.
Non-Linear Support Vector Machines
Lets look at the example below:
Lets say we are presented with data that is in a straight line (1-dimensional), it is impossible to separate the 2 classes. We can apply a function that separates our data into higher dimensional space (2-dimensional) that makes it easier to draw our hyperplane.

 Lets look at another example:

 The data in 2-dimensional space is inseparable, whereas when it is projected in 3D feature space, we see that there is a clear separation between the groups.
And thus, these 2 groups are separated as shown below.

 Problem:
The only problem in transformation to a higher dimensional feature space is that it is computationally very expensive.

 Solution:
"KERNEL TRICK" : this reduces the computational cost.

 A function that takes input vectors in original space and returns the dot products of the vectors in feature space is called a "kernel function".
Using this kernel function we can apply the dot products between vectors so that every point is mapped to a higher dimensional space via some transformation.

 Advantages:
1. effective in higher dimensional space.
2. different kernels for different decision functions.
3. can add kernel functions together to achieve more complex hyperplanes.

 Disadvantages:
1. if #features > #samples, this can lead to poor performance.
2. do not provide probability estimates.

Comments

Post a Comment

Popular posts from this blog

A/B Testing: Bernoulli Model VS the Beta-Binomial Hierarchical Model

By
Image
What is A/B Testing? A/B testing  is a method of comparing two versions of a webpage or app against each other to determine which one performs better.  AB testing is essentially an experiment where two or more variants of a page are shown to users at random, and based on the data it receives, statistical analysis is used to determine which variation performs better for a given conversion goal. How A/B Testing works? Here, 2 variations of the same webpage or app are created and are randomly assigned to the users (say 50% to Variation A and 50% to Variation B).  As users are served either the A or B, their engagement with each variation is measured and collected and analyzed through a statistical engine. You can then determine whether changing the webpage or app experience had a positive, negative, or no effect on user behavior. Why A/B Testing should be used? A/B testing allows individuals, teams, and companies to make careful changes to thei...
Continue Reading »

Recurrent Neural Networks and LSTM explained

By
Image
What are Recurrent Neural Networks ? Recurrent Neural Networks are the state of the art algorithm for sequential data  This is because it is the first algorithm that remembers its input, due to an internal memory, which makes it perfectly suited for Machine Learning problems that involve sequential data.   In simple terms, they are networks with loops in them, allowing information to be saved. Here, A is the network, Xt is the input with output  ht.  Although, these loops makes the RNN kind of hard to interpret, but in reality it is very simple. This is how the RNN looks when we unroll them. A RNN can be thought of as multiple copies of the same network , each passing message to  the next. Because of their internal memory, RNN’s are able to remember important things about the input they received, which enables them to be very precise in predicting what’s coming next. This is the reason why they are the preferred algorithm for se...
Continue Reading »

Exploratory Data Analysis and Hypothesis Testing - Loan Prediction

By
Image
Problem Statement: About Company Dream Housing Finance company deals in all home loans. They have presence across all urban, semi urban and rural areas. Customer first apply for home loan after that company validates the customer eligibility for loan. Problem Company wants to automate the loan eligibility process (real time) based on customer detail provided while filling online application form. These details are Gender, Marital Status, Education, Number of Dependents, Income, Loan Amount, Credit History and others. To automate this process, they have given a problem to identify the customers segments, those are eligible for loan amount so that they can specifically target these customers.  Introduction The objective of this project is to do “Exploratory Analysis” and “Hypothesis Testing” on the features of this dataset to find various insights as to how each feature affects the chances of getting a loan. Dataset The Loan Prediction dataset consists of 613 ...
Continue Reading »