Forward-mode pushforward analogous to the pullback defined by tf.gradients.
        With tf.gradients, grad_ys is the vector being pulled back, and here d_xs is
        the vector being pushed forward, i.e. this computes (∂ys / ∂xs)^T ∂xs.

        This is adapted from https://github.com/HIPS/autograd/pull/175#issuecomment-306984338

        :param ys: list of variables being differentiated (tensor)
        :param xs: list of variables to differentiate wrt (tensor)
        :param d_xs: list of gradients to push forward (same shapes as ys)
        :return: the specified moment of the variational distribution

def _forward_gradients(ys, xs, d_xs):
        """
        Forward-mode pushforward analogous to the pullback defined by tf.gradients.
        With tf.gradients, grad_ys is the vector being pulled back, and here d_xs is
        the vector being pushed forward, i.e. this computes (∂ys / ∂xs)^T ∂xs.

        This is adapted from https://github.com/HIPS/autograd/pull/175#issuecomment-306984338

        :param ys: list of variables being differentiated (tensor)
        :param xs: list of variables to differentiate wrt (tensor)
        :param d_xs: list of gradients to push forward (same shapes as ys)
        :return: the specified moment of the variational distribution
        """
        # this should be v = [tf.placeholder(y.dtype) for y in ys], but tensorflow
        # wants a value for the placeholder, even though it never gets used
        v = [tf.placeholder_with_default(tf.zeros(y.get_shape(), dtype=y.dtype),
                                         shape=y.get_shape()) for y in ys]

        g = tf.gradients(ys, xs, grad_ys=v)
        return tf.gradients(g, v, grad_ys=d_xs)