title: SVM - Difference between Energy vs Loss vs Regularization vs Cost function
body: <p>I am reading <a href="http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf" rel="noreferrer">A Tutorial on Energy Based Learning</a> and I am trying to understand the difference between all those terms stated above in the context of SVMs. This <a href="https://www.quora.com/What-is-the-difference-between-a-*cost-function*-and-a-*loss-function*-in-machine-learning" rel="noreferrer">link</a> summarizes the differences between a loss, cost and an objective function. Based on my understanding,</p>
  
  <p><strong>Objective function</strong>: Something we want to minimize. For example <code>||w||^2</code> for SVM.</p>
  
  <p><strong>Loss function</strong>: Penalty between prediction and label which is also equivalent to the regularization term. Example is the hinge loss function in SVM.</p>
  
  <p><strong>Cost function</strong>: A general formulation that combines the objective and loss function.</p>
  
  <p>Now, the 1st link states that the hinge function is <code>max(0, m + E(W,Yi,Xi) - E(W,Y,X))</code> i.e. it is a function of the energy term. Does that mean that the energy function of the SVM is <code>1 - y(wx + b)</code> ? Are energy functions are a part of a loss function. And a loss + objective function a part of the cost function ?</p>
  
  <p>A concise summary of the 4 terms would immensely help my understanding. Also, do correct me if my understanding is wrong. The terms sound so confusing. Thanks !</p>