title: Multi-Label Classifier in Tensorflow
body: <p>I want to develop a multi-label classifier with TensorFlow, i try to mean there exists multiple label which contains multiple classes. To illustrate you can image the situation like: </p>
  
  <ul>
  <li>label-1 classes: lighting raining, raining, partial raining, no raining </li>
  <li>label-2 classes: sunny, partly cloudy, cloudy, very cloudy. </li>
  </ul>
  
  <p>I want to classify these two labels with neural network. For now, i used different class label for every (label-1, label-2) pair classes. Which means i have 4 x 4 = 16 different label. </p>
  
  <p>By training my model with </p>
  
  <h1>current loss</h1>
  
  <pre><code>cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction), reduction_indices=[1])) 
  
  # prediction is sofmaxed
  loss = cross_entropy + regul * schema['regul_ratio'] # regul things is for regularization 
  train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
  </code></pre>
  
  <p>However i think that a multi-label training will work nicer in such a condition. </p>
  
  <ul>
  <li>My features will be [n_samples, n_features]</li>
  <li>My labels will be [n_samples, n_classes, 2]</li>
  </ul>
  
  <p>n_samples of [x1, x2, x3, x4 ...]  # features</p>
  
  <p>n_samples of [[0, 0, 0, 1],  [0, 0, 1, 0]] # no raining and cloudy </p>
  
  <p>How can i make a softmax probability distribution predictor with tensorflow. Is there any working example for multilabel problem like this one. How will be my loss tensor like?</p>