<p>There is no way to avoid overlapping regions with Multiclass SVM. From <a href="https://arxiv.org/ftp/arxiv/papers/0711/0711.2914.pdf" rel="nofollow noreferrer">https://arxiv.org/ftp/arxiv/papers/0711/0711.2914.pdf</a> , you have a fairly clear explanation : </p>

<blockquote>
  <p>As mentioned before, SVM classification is essentially a binary (two-class) classification technique,
  which has to be modified to handle the multiclass tasks in real world situations e.g. derivation of
  land cover information from satellite images. Two of the common methods to enable this adaptation
  include the 1A1 and 1AA techniques. The 1AA approach represents the earliest and most common
  SVM multiclass approach (Melgani and Bruzzone, 2004) and involves the division of an N class
  dataset into N two-class cases. If say the classes of interest in a satellite image include water,
  vegetation and built up areas, classification would be effected by classifying water against non-water
  areas i.e. (vegetation and built up areas) or vegetation against non-vegetative areas i.e. (water and
  built up areas). The 1A1 approach on the other hand involves constructing a machine for each pair of
  classes resulting in N(N-1)/2 machines. When applied to a test point, each classification gives one
  vote to the winning class and the point is labeled with the class having most votes. This approach
  can be further modified to give weighting to the voting process. From machine learning theory, it
  is acknowledged that the disadvantage the 1AA approach has over 1A1 is that its performance can
  be compromised due to unbalanced training datasets (Gualtieri and Cromp, 1998), however, the 1A1
  approach is more computationally intensive since the results of more SVM pairs ought to be
  computed. In this paper, the performance of these two techniques are compared and evaluated to
  establish their performance on the extraction of land cover information from satellite images. </p>
</blockquote>

<p>So you have either N classifiers, or N(N-1)/2 classifiers, that use the whole available space. As these are (for the purpose of this explanation) independant, the only way to have the decisions boundaries not cross would be to have parallel decision boudaries, and even then the regions would be overlapping (I feel like this sentence may not be the clearest, don't hesitate to ask for more explanations if needed be). </p>

<p>If you want clear non overlapping regions I suggest you use another algorithm that handles the multiclass problem better, such as KNN.</p>
Why are the regions/decision boundaries overlapping with multi-class classification using SVM in sci-kit?
<p>I am using the SVM in scikit-learn library for doing multiclass classification. I am wondering why these regions (decision boundaries) are overlapping (as seen in the picture below)?</p>
<p><a href="https://i.stack.imgur.com/1uCGV.png" rel="nofollow noreferrer">Results</a></p>
<p>Could someone please explain the difference between whether I do one-vs-one or one-vs-all in terms of the regions overlapping? I assumed one-vs-one would have clearly delineated regions with no overlap since it's maximizing the margin against each other class and that one-vs-all could have regions overlapping, but perhaps this is inaccurate because 3 of the 4 models I am training are one-vs-one, and they show overlapping regions.</p>
<p>I've considered maybe it's a plotting issue as well, but could not determine any issues. If the alpha is 1, then the regions no longer overlap, but I assume this is expected since it's just covering up the other regions it overlays (which is to be expected and doesn't solve the problem).</p>
<h1>Here is the function which creates, trains, and plots 4 different SVM models #(3 different kernels using SVC and 1 with LinearSVC).</h1>
<pre><code>def createSVMandPlot(X,y,x_name,y_name):
    
    h = .02  # step size in the mesh
    
    # we create an instance of SVM and fit out data. We do not scale our
    # data since we want to plot the support vectors
    C = 1.0  # SVM regularization parameter
    svc = svm.SVC(kernel='linear', C=C).fit(X, y) #1 vs 1 
    rbf_svc = svm.SVC(kernel='rbf', gamma='scale', C=C).fit(X, y) #1v1
    poly_svc = svm.SVC(kernel='poly', degree=3, gamma='scale',C=C).fit(X, y) #1v1
    lin_svc = svm.LinearSVC(C=C).fit(X, y) #1 vs rest
    
    print(str(x_name)+' vs. '+str(y_name))
    for i, clf in enumerate((svc, lin_svc, rbf_svc, poly_svc)):
        
        X_pred=clf.predict(X)
        X_pred1=np.asarray(X_pred).reshape(len(X_pred),1)
        A=confusion_matrix(X_pred1, y)
        print(A)
        c=0
        for r in range(len(X_pred)):
            if X_pred[r]==y[r]:
                c+=1

        print(str(c)+' out of 34 predicted correctly (true positives)')

    
    =============================================================================
    with warnings.catch_warnings():
        
        warnings.filterwarnings(&quot;ignore&quot;)
        =============================================================================
    
    
        x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
        y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
        xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
                             np.arange(y_min, y_max, h))
        
        # title for the plots
        titles = ['SVC w/ linear kernel',
                  'LinearSVC (w/ linear kernel)',
                  'SVM w/ RBF kernel',
                  'SVM w/ poly(degree 3) kernel']
        
        plt.pause(7)
        for i, clf in enumerate((svc, lin_svc, rbf_svc, poly_svc)):
            # point in the mesh [x_min, x_max]x[y_min, y_max].
            plt.subplot(2, 2, i + 1)
            plt.subplots_adjust(wspace=0.4, hspace=0.4)
        
            Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
        
            # Put the result into a color plot
            Z = Z.reshape(xx.shape)
            plt.contourf(xx, yy, Z, alpha=.5)
            
            # Plot also the training points
            plt.scatter(X[:, 0], X[:, 1], s=13,c=y)
            plt.xlabel(x_name)
            plt.ylabel(y_name)
            plt.xlim(xx.min(), xx.max())
            plt.ylim(yy.min(), yy.max())
            plt.xticks(())
            plt.yticks(())
            plt.title(titles[i])
            
            plt.show() 
</code></pre>
<p>The result from this is an image with decision bounaries/regions overlapping. It implies that if a point is at a specific 2D coordinate (x1,y1), then it could be classified as two or more classes instead of just one which is not what is desired or expected. Could someone explain what might be going on? Thanks</p>
<p>EDIT: I included a picture of the results with the overlapping decision boundaries.</p>