title: DDPG (Deep Deterministic Policy Gradients), how is the actor updated?
body: <p>I'm currently trying to implement DDPG in Keras. I know how to update the critic network (normal DQN algorithm), but I'm currently stuck on updating the actor network, which uses the equation:</p>
  
  <p><img src="https://i.stack.imgur.com/A2UKs.png" alt="equation"></p>
  
  <p>so in order to reduce the loss of the actor network wrt to its weight dJ/dtheta, it's using chain rule to get dQ/da (from critic network) * da/dtheta (from actor network).  </p>
  
  <p>This looks fine, but I'm having trouble understanding how to derive the gradients from those 2 networks.  Could someone perhaps explain this part to me?</p>