SVM
vapnik
['harold urey', 'conway', 'robert bunsen', 'carlos torres', 'henri cartan', 'maurice martenot', 'lyudmila zhuravleva', 'karl reinmuth', 'brouwer', 'rudolf diesel', 'paolo maffei', 'birmingham university', 'benjamin franklin', 'gridshell', 'bernhard schmidt', 'albert einstein', 'thomas newcomen', 'evangelista torricelli', 'mikhail lazarev', 'eugene shoemaker', 'eric elst', 'georg cantor', 'bill mollison', 'heinrich ruhmkorff', 'alonzo church', 'icao', 'indiana asteroid program', 'alexander grothendieck', 'nikola tesla', 'clyde tombaugh', 'urata', 'william hyde wollaston', 'cornelis johannes van houten', 'george berkeley', 'george van biesbroeck', 'louis boyer', 'niels bohr', 'blackberry', 'lord rayleigh', 'carl wilhelm scheele', 'james bradley', 'ephraim shay', 'lewis swift', 'christopher latham sholes', 'charles darwin', 'werner heisenberg', 'kazuro watanabe', 'alessandro malaspina', 'sir arthur evans', 'george maciunas', 'august kopff', 'ernest rutherford', 'ward cunningham', 'humphry davy', 'ernest johnson', 'babbage', 'nikolai chernykh', 'bartolomeo cristofori', 'oak ridge observatory', 'sir nigel gresley', 'takashi murakami', 'urbain le verrier', 'edward said', 'harlow shapley', 'leibniz', 'marie curie', 'niels henrik abel', 'karl drais', 'seymour papert', 'schelte bus', 'turing', 'marguerite laugier', 'michael faraday', 'otto hahn', 'hyperboloid', 'henry bessemer', 'james dunlop', 'hyperloop', 'ingrid', 'edward bowell', 'sir winston churchill', 'pieter zeeman', 'george westinghouse', 'eleanor helin', 'frederick scott archer', 'sequoyah', 'noam chomsky', 'robert luther', 'purple mountain observatory', 'edwin hubble', 'odin', 'lionel penrose', 'sergey belyavsky', 'liisi oterma', 'nicolas bourriaud', 'august derleth', 'ernest hemingway', 'tsutomu seki', 'hermann goldschmidt']
Regularization perspectives on support vector machines
Regularization perspectives on support vector machines provide a way of interpreting support vector machines (SVMs) in the context of other machine learning algorithms. SVM algorithms categorize multidimensional data, with the goal of fitting the training set data well, but also avoiding overfitting, so that the solution generalizes to new data points. Regularization algorithms also aim to fit training set data and avoid overfitting. They do this by choosing a fitting function that has low error on the training set, but also is not too complicated, where complicated functions are functions with high norms in some function space. Specifically, Tikhonov regularization algorithms choose a function that minimize the sum of training set error plus the function's norm. The training set error can be calculated with different loss functions. For example, regularized least squares is a special case of Tikhonov regularization using the squared error loss as the loss function. Regularization perspectives on support vector machines interpret SVM as a special case Tikhonov regularization, specifically Tikhonov regularization with the hinge loss for a loss function. This provides a theoretical framework with which to analyze SVM algorithms and compare them to other algorithms with the same goals: to generalize without overfitting. SVM was first proposed in 1995 by Corinna Cortes and Vladimir Vapnik, and framed geometrically as a method for finding hyperplanes that can separate multidimensional data into two categories. This traditional geometric interpretation of SVMs provides useful intuition about how SVMs work, but is difficult to relate to other machine learning techniques for avoiding overfitting like regularization, early stopping, sparsity and Bayesian inference. However, once it was discovered that SVM is also a special case of Tikhonov regularization, regularization perspectives on SVM provided the theory necessary to fit SVM within a broader class of algorithms. This has enabled detailed comparisons between SVM and other forms of Tikhonov regularization, and theoretical grounding for why it is beneficial to use SVM's loss function, the hinge loss.