top of page
Search

# What is Support Vector Machines(SVMs)?

Support vectors are data points that maximize the margin around a hyperplane that separates positive and negative instances in dataset.

Basic idea:

Another method for building a classifier where we view the data “spatially”, and predict a new instance’s class based on where it is “located in space”

For simplicity, we’ll assume decision is binary (positive/negative, yes/no, etc.)

Assume numeric attributes (easier to work with as “points in space” and perform arithmetic calculations); however, not restricted to 2D

Hyperplane: plane (or line if 2D space) that separates positive and negative instances in dataset

Support vectors: data points (actual instances from training dataset) that maximize the margin around the hyperplane; they “anchor” the hyperplane

Linear separability

One equation for defining a line is ax + by = c

Find a, b, c such that ax + by ≥ c for red points and ax + by ≤ c for green points In general, many possible solutions for a, b, c

Defining the hyperplane (assuming you knew the support vectors)

Ex: In the dataset below, assume decision attribute is z

Easy to see that support vectors are s1 = (1 0), s2 = (3 1), and s3 = (3 -1)

Augment vectors with 1 more dimension; assign value of 1 for this dimension in each support vector s1 = (1 0 1), s2 = (3 1 1), s3 = (3 -1 1)

Set up equations we need to solve:

φ is a function that maps instance data to a “feature space”

α‘s are parameters which, when “combined” with support vectors and respective decisions, will help us define the hyperplane

α1φ(s1) • φ(s1) + α2φ(s2) • φ(s1) + α3φ(s3) • φ(s1) = -1 s1’s decision

α1φ(s1) • φ(s2) + α2φ(s2) • φ(s2) + α3φ(s3) • φ(s2) = 1 s2’s decision

α1φ(s1) • φ(s3) + α2φ(s2) • φ(s3) + α3φ(s3) • φ(s3) = 1 s3’s decision

Let φ( ) be I (identity vector):

Since data in this example are linearly separable, we’re going to be able to use a linear SVM; so we’re just going to use identify function for φ

α1s1 • s1 + α2s2 • s1 + α3s3 • s1 = -1

α1s1 • s2 + α2s2 • s2 + α3s3 • s2 = 1

α1s1 • s3 + α2s2 • s3 + α3s3 • s3 = 1

Compute dot product in each equation:

2α1 + 4α2 + 4α3 = -1 e.g., s2 • s1 = (3 1 1) • (1 0 1) = (3*1) + (1*0) + (1*1) = 4

4α1 + 11α2 + 9α3 = 1

4α1 + 9α2 + 11α3 = 1

Solve for α1, α2, and α3:

α1 = -3.5, α2 = 0.75, α3 = 0.75

Define the discriminating hyperplane:

w’ = α1s1 + α2s2 + α3s3

= -3.5 (1 0 1) + 0.75 (3 1 1) + 0.75 (3 -1 1)

= (1 0 -2)

For hyperplane in 2D, (a b c) translates to ax + by + c = 0

For hyperplane in 3D, (a b c d) translates to ax + by + cz + d = 0 etc.

So in this case (1 0 -2) translates to 1x + 0y + -2 = 0 or simply x = 2

Doing Linear SVM in Python

Import Libraries

```import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.svm import SVC ```

Read Data

`df = pd.read_csv('svm_exampleData.csv')`

Change non numeric value into numeric value

```# Just for graphing purposes, change non-decision attr's to have numeric values
df= df.replace({'z': r'positive'}, {'z':1}, regex=True)
df= df.replace({'z': r'negative'}, {'z':-1}, regex=True)```
```X = df.iloc[:, 0:2].values
y = df.iloc[:, 2].values
r,_ = df.shape ```
```# Display original data points
plt.scatter(X[:, 0], X[:, 1], c=y, s=r, cmap='winter');
plt.show() ```

Make Linear SVM

```# Make a linear SVM
model = SVC(kernel='linear')
model.fit(X, y)
print(model.support_vectors_)         # The support vectors
print(model.coef_)                    # The weights (x and y)
print(model.intercept_)               # The intercept ```

```# Function to plot SVM boundary lines - cool!
def plot_svc_decision_function(model, ax=None, plot_support=True):
if ax is None:
ax = plt.gca()
xlim = ax.get_xlim()
ylim = ax.get_ylim()

# Create grid to evaluate model
x = np.linspace(xlim[0], xlim[1], 30)
y = np.linspace(ylim[0], ylim[1], 30)
Y, X = np.meshgrid(y, x)
xy = np.vstack([X.ravel(), Y.ravel()]).T
P = model.decision_function(xy).reshape(X.shape)

# Plot decision boundary and margins
ax.contour(X, Y, P, colors='k',
levels=[-1, 0, 1], alpha=0.5,
linestyles=['--', '-', '--'])

# Plot support vectors
if plot_support:
ax.scatter(model.support_vectors_[:, 0],
model.support_vectors_[:, 1],
s=300, linewidth=1, facecolors='none');
ax.set_xlim(xlim)
ax.set_ylim(ylim)

# Call the function to show points and SV boundaries
plt.scatter(X[:, 0], X[:, 1], c=y, s=50, cmap='winter')
plot_svc_decision_function(model);
plt.show()

# Make predictions (will be array([-1]) or array([1]))
model.predict([[-3, 4]])
model.predict([[7, 5]])
```

Thanks for your support!

You can send your request directly at realcode4you@gmail.com if you need any SVMs related help, machine learning related help, data science assignment help, python assignment help or any project related to machine learning and get instant help with an affordable prices.
32 views0 comments

bottom of page