title: Implementing simple probabilistic model with negative log likelihood loss
body: <p>First a quick disclaimer would be that I posted this question on Reddit, in the Deep Learning and Learning Machine Learning first, but I thought I might also request your expertise here too. Without further ado:</p>
  
  <p>I am currently challenging myself on this year <a href="https://sites.google.com/view/berkeley-cs294-158-sp19/home" rel="nofollow noreferrer">Deep Unsupervised Learning Course of Berkeley University</a> and although I just started the warmup exercise of week 1, I am already having 'technical' difficulties.</p>
  
  <p>The exercise in question is the "1. Warmup" in the following document: <a href="https://drive.google.com/file/d/1DtYllaV4Yk8ljgYcLBmdXNplEDTG6HT6/view" rel="nofollow noreferrer">Week 1 Exercises</a>. (My apologies as I am not familiar enough with Reddit formating to seemlessly include images.</p>
  
  <p>In my understanding, we have a variable <code>x</code> which can take values from <code>1..100</code> which a specific probability of being sampled ( defined in <code>sample_data()</code> function).
  The task is therefore to fit a vector of parameters <code>theta</code> which is passed to a softmax function, and is supposed to give the likelihood of a specific element <code>x_i</code> to be sampled. Namely, <code>theta_1</code> should the parameter which "bumps up" the soft-max value corresponding to the variable <code>x = 1</code> and so on.</p>
  
  <p>Using Tensorflow, I think I was able to create such a model, but when it comes to training, I believe I am missing a crucial point as the program cannot compute gradients with respect to the <code>theta</code> parameters.</p>
  
  <p>I would like to know if am not misunderstanding the task, and if there is any better method to achieve the result of the exercise.</p>
  
  <p>Here is the code, where the failing par is located from the <code># Computing gradients</code>.</p>
  
  <pre class="lang-py prettyprint-override"><code>import numpy as np
  import tensorflow as tf
  import tensorflow_probability as tfp
  
  if __name__ == "__main__":
      # Sampling function of the x variable provided in the exercise
      def sample_data():
          count = 10000
          rand = np.random.RandomState(0)
          a = 0.3 + 0.1 * rand.randn(count)
          b = 0.8 + 0.05 * rand.randn(count)
          mask = rand.rand(count) &lt; 0.5
          samples = np.clip(a * mask + b * (1 - mask), 0.0, 1.0)
          return np.digitize(samples, np.linspace(0.0, 1.0, 100))
  
      full_data = sample_data()
      train_ds = full_data[:int(.8*len( full_data))]
      val_ds = full_data[int(.8*len( full_data)):]
  
      # Declaring parameters theta
      w_init = tf.zeros_initializer()
      params = tf.Variable(
          initial_value=w_init(shape=(1, 100),
          dtype='float32'), trainable=True, name='params')
  
  
      softmax = tf.squeeze( tf.nn.softmax( params, axis=1))
  
      #Should materialize the loss of the model
      def get_neg_log_likelihood( inputs):
          return - tf.math.log( softmax)
  
      neg_log_likelihoods = get_neg_log_likelihood( softmax)
  
      dist = tfp.distributions.Categorical( probs=softmax, dtype=tf.int32)
  
      optimizer = tf.keras.optimizers.Adam()
  
      for epoch in range( 100):
          minibatch_size = 200
          n_minibatches = len( train_ds) // minibatch_size
  
          # Running over minibatches of the data
          for minibatch in range( n_minibatches):
              # Minibatching
              start_index = (minibatch*minibatch_size)
              end_index = (minibatch_size*minibatch + minibatch_size)
  
              x = train_ds[start_index:end_index]
  
              with tf.GradientTape() as tape:
                  tape.watch( params)
                  loss = tf.reduce_mean( - dist.log_prob( x))
  
              # Computing gradients
              grads = tape.gradient( loss, params)
              print( grads) # Result: None
              # input()
              optimizer.apply_gradients( zip( grads, params))
  </code></pre>
  
  <p>Thank you in advance for your time. </p>
  
  <p>PS: I mainly have a background in Deep Reinforcement Learning, therefore I can understand the various models used there ( policy, value functions ...), but I am trying to refine my grasp over the internals of the models themselves, namely in generative probabilistic models (GAN, VAE) and other unsupervised learning models in general ( RealNVP, Norm Flows, ...)</p>