Why are the regions/decision boundaries overlapping with multi-class classification using SVM in sci-kit?
I am using the SVM in scikit-learn library for doing multiclass classification. I am wondering why these regions (decision boundaries) are overlapping (as seen in the picture below)?
Could someone please explain the difference between whether I do one-vs-one or one-vs-all in terms of the regions overlapping? I assumed one-vs-one would have clearly delineated regions with no overlap since it's maximizing the margin against each other class and that one-vs-all could have regions overlapping, but perhaps this is inaccurate because 3 of the 4 models I am training are one-vs-one, and they show overlapping regions.
I've considered maybe it's a plotting issue as well, but could not determine any issues. 392
If the alpha is 1, then the regions no longer overlap, but I assume this is expected since it's just covering up the other regions it overlays (which is to be expected and doesn't solve the problem).
Here is the function which creates, trains, and plots 4 different SVM models #(3 different kernels using SVC and 1 with LinearSVC).
def createSVMandPlot(X,y,x_name,y_name):
h = .02 # step size in the mesh
# we create an instance of SVM and fit out data. We do not scale our
# data since we want to plot the support vectors
C = 1.0 # SVM regularization parameter
svc = svm.SVC(kernel='linear', C=C).fit(X, y) #1 vs 1
rbf_svc = svm.SVC(kernel='rbf', gamma='scale', C=C).fit(X, y) #1v1
poly_svc = svm.SVC(kernel='poly', degree=3, gamma='scale',C=C).fit(X, y) #1v1
lin_svc = svm.LinearSVC(C=C).fit(X, y) #1 vs rest
print(str(x_name)+' vs. '+str(y_name))
for i, clf in enumerate((svc, lin_svc, rbf_svc, poly_svc)):
X_pred=clf.predict(X)
X_pred1=np.asarray(X_pred).reshape(len(X_pred),1)
A=confusion_matrix(X_pred1, y)
print(A)
c=0
for r in range(len(X_pred)):
if X_pred[r]==y[r]:
c+=1
print(str(c)+' out of 34 predicted correctly (true positives)')
=============================================================================
with warnings.catch_warnings():
warnings.filterwarnings("ignore")
=============================================================================
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
# title for the plots
titles = ['SVC w/ linear kernel',
'LinearSVC (w/ linear kernel)',
'SVM w/ RBF kernel',
'SVM w/ poly(degree 3) kernel']
plt.pause(7)
for i, clf in enumerate((svc, lin_svc, rbf_svc, poly_svc)):
# point in the mesh [x_min, x_max]x[y_min, y_max].
plt.subplot(2, 2, i + 1)
plt.subplots_adjust(wspace=0.4, hspace=0.4)
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
# Put the result into a color plot
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z, alpha=.5)
# Plot also the training points
plt.scatter(X[:, 0], X[:, 1], s=13,c=y)
plt.xlabel(x_name)
plt.ylabel(y_name)
plt. 392