<h1>The short answer</h1>

<p>Many of the answers here rely on the widely-used mathematical definition [1]:</p>

<blockquote>
  <ul>
  <li>Discriminative models directly learn the conditional predictive distribution <code>p(y|x)</code>.</li>
  <li>Generative models learn the joint distribution <code>p(x,y)</code> (or rather, <code>p(x|y)</code> and <code>p(y)</code>).
  
  <ul>
  <li>Predictive distribution <code>p(y|x)</code> can be obtained with Bayes' rule.</li>
  </ul></li>
  </ul>
</blockquote>

<p>Although very useful, this <strong>narrow definition</strong> assumes the supervised setting, and is less handy when examining unsupervised or semi-supervised methods. It also <strong>doesn't apply to many contemporary approaches for deep generative modeling</strong>. For example, now we have implicit generative models, e.g. Generative Adversarial Networks (GANs), which are sampling-based and don't even explicitly model the probability density <code>p(x)</code> (instead learning a divergence measure via the discriminator network). But we call them "generative models” since they are used to generate (high-dimensional [10]) samples.</p>

<p>A <strong>broader and more fundamental definition</strong> [2] seems equally fitting for this general question:</p>

<blockquote>
  <ul>
  <li>Discriminative models learn the boundary between classes.
  
  <ul>
  <li>So they can <em>discriminate</em> between different kinds of data instances.</li>
  </ul></li>
  <li>Generative models learn the distribution of data.
  
  <ul>
  <li>So they can <em>generate</em> new data instances.</li>
  </ul></li>
  </ul>
</blockquote>

<p><a href="https://i.stack.imgur.com/mipQS.png" rel="noreferrer"><img src="https://i.stack.imgur.com/mipQS.png" alt="From http://primo.ai/index.php?title=Discriminative_vs._Generative"></a>
<em><a href="http://primo.ai/index.php?title=Discriminative_vs._Generative" rel="noreferrer">Image source</a></em></p>

<hr>

<h1>A closer look</h1>

<p>Even so, this question implies somewhat of a false dichotomy [3]. The generative-discriminative "dichotomy" is in fact a <strong>spectrum</strong> which you can even smoothly interpolate between [4]. </p>

<p>As a consequence, this distinction gets arbitrary and confusing, especially when many popular models do not neatly fall into one or the other [5,6], or are in fact hybrid models (combinations of classically "discriminative" and "generative" models).</p>

<p>Nevertheless it's still a highly useful and common distinction to make. We can list some clear-cut examples of generative and discriminative models, both canonical and recent:</p>

<ul>
<li>Generative: Naive Bayes, latent Dirichlet allocation (LDA), Generative Adversarial Networks (GAN), Variational Autoencoders (VAE), normalizing flows.</li>
<li>Discriminative: Support vector machine (SVM), logistic regression, most deep neural networks.</li>
</ul>

<p>There is also a lot of interesting work deeply examining the generative-discriminative divide [7] and spectrum [4,8], and even transforming discriminative models into generative models [9].</p>

<p>In the end, definitions are constantly evolving, especially in this rapidly growing field :) It's best to take them with a pinch of salt, and maybe even redefine them for yourself and others.</p>

<hr>

<h1>Sources</h1>

<ol>
<li>Possibly originating from "Machine Learning - Discriminative and Generative" (Tony Jebara, 2004).</li>
<li><a href="https://developers.google.com/machine-learning/gan/generative" rel="noreferrer">Crash Course in Machine Learning by Google</a></li>
<li><a href="http://www.trivialorwrong.com/2016/05/22/the-generative-discriminative-fallacy.html" rel="noreferrer">The Generative-Discriminative Fallacy</a></li>
<li><a href="https://arxiv.org/abs/1705.09011" rel="noreferrer">"Principled Hybrids of Generative and Discriminative Models" (Lasserre et al., 2006)</a></li>
<li><a href="https://stats.stackexchange.com/questions/408421/is-the-only-difference-between-conditional-generative-models-and-discriminative">@shimao's question</a></li>
<li><a href="https://stats.stackexchange.com/a/191320/188037">Binu Jasim's answer</a></li>
<li>Comparing logistic regression and naive Bayes: 

<ul>
<li><a href="https://cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf" rel="noreferrer">cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf</a></li>
<li><a href="https://papers.nips.cc/paper/2020-on-discriminative-vs-generative-classifiers-a-comparison-of-logistic-regression-and-naive-bayes" rel="noreferrer">"On Discriminative vs. Generative classifiers"</a></li>
<li><a href="https://link.springer.com/article/10.1007/s11063-008-9088-7" rel="noreferrer">Comment on "On Discriminative vs. Generative classifiers"</a></li>
</ul></li>
<li><a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2016/04/DengJaitly2015-ch1-2.pdf" rel="noreferrer">https://www.microsoft.com/en-us/research/wp-content/uploads/2016/04/DengJaitly2015-ch1-2.pdf</a></li>
<li><a href="https://arxiv.org/abs/1912.03263" rel="noreferrer">"Your classifier is secretly an energy-based model" (Grathwohl et al., 2019)</a></li>
<li><a href="https://deepgenerativemodels.github.io/notes/introduction/" rel="noreferrer">Stanford CS236 notes</a>: Technically, a probabilistic discriminative model is also a generative model of the labels conditioned on the data. However, the term generative models is typically reserved for high dimensional data.</li>
</ol>
What is the difference between a generative and a discriminative algorithm?
<p>What is the difference between a <strong>generative</strong> and a
<strong>discriminative</strong> algorithm?</p>