<p>I never heard the term &quot;differentiable programming&quot; before reading your question, but having used the concepts noted in your references, both from the side of creating code to solve a derivative with <a href="https://www.cs.utexas.edu/users/novak/asg-symdif.html" rel="nofollow noreferrer">Symbolic differentiation</a> and with <a href="https://en.wikipedia.org/wiki/Automatic_differentiation" rel="nofollow noreferrer">Automatic differentiation</a> and having written interpreters and compilers, to me this just means that they have made the ability to calculate the numeric value of the derivative of a function easier. I don't know if they made it a <a href="https://en.wikipedia.org/wiki/First-class_citizen" rel="nofollow noreferrer">First-class citizen</a>, but the new way doesn't require the use of a function/method call; it is done with syntax and the compiler/interpreter hides the translation into calls.</p>
<p>If you look at the <a href="https://github.com/FluxML/Zygote.jl" rel="nofollow noreferrer">Zygote</a> example it clearly shows the use of <a href="http://web.mit.edu/wwmath/calculus/differentiation/notation.html" rel="nofollow noreferrer">prime notation</a></p>
<pre><code>julia&gt; f(10), f'(10)
</code></pre>
<p>Most seasoned programmers would guess what I just noted because there was not a research paper explaining it. In other words it is just that obvious.</p>
<p>Another way to think about it is that if you have ever tried to calculate a derivative in a programming language you know how hard it can be at times and then ask yourself why don't they (the language designers and programmers) just add it into the language. In these cases they did.</p>
<p>What surprises me is how long it to took before derivatives became available via  syntax instead of calls, but if you have ever worked with scientific code or coded neural networks at at that level then you will understand why this is a concept that is being touted as something of value.</p>
<p>Also I would not view this as another <a href="https://en.wikipedia.org/wiki/Programming_paradigm" rel="nofollow noreferrer">programming paradigm</a>, but I am sure it will be added to the list.</p>
<blockquote>
<p>How does it relate to automatic differentiation (the two seem conflated a lot of the time)?</p>
</blockquote>
<p>In both cases that you referenced, they use automatic differentiation to calculate the derivative instead of using symbolic differentiation. I do not view <em>differentiable programming</em> and <em>automatic differentiation</em> as being two distinct sets, but instead that <em>differentiable programming</em> has a means of being implemented and the way they chose was to use <em>automatic differentiation,</em> they could have chose <em>symbolic differentiation</em> or some other means.</p>
<p>It seems you are trying to read more into what differential programming is than it really is. It is not a new way of programming, but just a nice feature added for doing derivatives.</p>
<p>Perhaps if they named it <em>differentiable syntax</em> it might have been more clear. The use of the word <em>programming</em> gives it more panache than I think it deserves.</p>
<p>EDIT</p>
<p>After skimming Swift <a href="https://forums.swift.org/t/differentiable-programming-mega-proposal/28547" rel="nofollow noreferrer">Differentiable Programming Mega-Proposal</a> and trying to compare that with the Julia example using Zygote, I would have to modify the answer into parts that talk about Zygote and then switch gears to talk about Swift. They each took a different path, but the commonality and bottom line is that the languages know something about differentiation which makes the job of coding them easier and hopefully produces less errors.</p>
<p>About the Wikipedia quote that</p>
<blockquote>
<p>the programs can be differentiated throughout</p>
</blockquote>
<p>At first reading it seems nonsense or at least lacks enough detail to understand it in context which is why I am sure you asked.</p>
<p>In having many years of digging into what others are trying to communicate, one learns that unless the source has been peer reviewed to take it with a grain of salt, and unless it is absolutely necessary to understand, then just ignore it. In this case if you ignore the sentence most of what your reference makes sense. However I take it that you want an answer, so let's try and figure out what it means.</p>
<p>The key word that has me perplexed is <em>throughout</em>, but since you note the statement came from Wikipedia and in Wikipedia they give three references for the statement, a search of the word <em>throughout</em> appears only in one</p>
<p><a href="https://arxiv.org/pdf/1907.07587.pdf" rel="nofollow noreferrer">∂P: A Differentiable Programming System to Bridge Machine Learning and Scientific Computing</a></p>
<blockquote>
<p>Thus, since our ∂P system does not require primitives to handle new
types, this means that almost all functions and types defined
throughout the language are automatically supported by Zygote, and
users can easily accelerate specific functions as they deem necessary.</p>
</blockquote>
<p>So my take on this is that by going back to the source, e.g. the paper, you can better understand how that percolated up into Wikipedia, but it seems that the meaning was lost along the way.</p>
<p>In this case if you really want to know the meaning of that statement you should ask on the Wikipedia <a href="https://en.wikipedia.org/wiki/Talk:Differentiable_programming" rel="nofollow noreferrer">talk page</a> and ask the author of the statement directly.</p>
<p>Also note that the paper referenced is not peer reviewed, so the statements in there may not have any meaning amongst peers at present. As I said, I would just ignore it and get on with writing wonderful code.</p>
What is differentiable programming?
<p>Native support for differential programming has been added to Swift for the <a href="https://www.tensorflow.org/swift" rel="nofollow noreferrer">Swift for Tensorflow</a> project. Julia has similar with <a href="https://github.com/FluxML/Zygote.jl" rel="nofollow noreferrer">Zygote</a>.</p>
<p>What exactly is differentiable programming?</p>
<ul>
<li><p>what does it enable? <a href="https://en.wikipedia.org/wiki/Differentiable_programming" rel="nofollow noreferrer">Wikipedia</a> says</p>
<blockquote>
<p>the programs can be differentiated throughout</p>
</blockquote>
<p>but what does that mean?</p>
</li>
<li><p>how would one use it (e.g. a simple example)?</p>
</li>
<li><p>and how does it relate to automatic differentiation (the two seem conflated a lot of the time)?</p>
</li>
</ul>