title: multi-label classfication project: how to improve bad performance
body: <p><strong>Contextualization</strong><br>
  I am building a model that identifes three categories in a given image.<br>
  There are three categories:  </p>
  
  <ul>
  <li><p><em>Gender:</em> male/female</p></li>
  <li><p><em>Type clotes:</em> Blazers,Blazers&amp;Jacket,Blouses,Cardigan,Coats,Coats&amp;Jackets,Culottes,Dresses,Jeans,Jumper,Jumpsuits&amp;Dungarees,Knitwear&amp;Sweatshirts,Leggings&amp;CigaretteTrousers,Maternity,Polo,Pololong,Shirts,Shorts,Skirts,Suits,Swimwear,Swimwear &amp; Beachwear,T-shirts,T-shirts &amp; Tops,Trousers,Tunics.  </p></li>
  <li><em>Color clothes:</em> beige,black,blue,brown,green,grey,orange,pink,red,white,yellow.  </li>
  </ul>
  
  <p>In other words every image should have these 3 categories ( one value per category).  </p>
  
  <p><strong>Data</strong><br>
  My database for <strong>training my model</strong> is composed of <strong>1812</strong> images and an excel file containing the values of the three categories for each image ( and its url).<br>
  Here are some information about my database:</p>
  
  <ul>
  <li>number of males: 759</li>
  <li>number of females: 1055</li>
  <li>I also have the number of times each attribute ( from each category) is appearing in my database  ( I can't list all of them, but if it's something usefull for interpretation I can send you that information)</li>
  <li>most appearing triplet is (Male,Jeans,blue):  66 times</li>
  </ul>
  
  <p><strong>What I have done</strong><br>
  It seemed pretty clear that I am in a multi-label classification context.<br>
  <strong>1. Clean &amp; preprocess my data</strong><br>
  I created <strong>(X,Y)</strong> data where <strong>X</strong> is is of shape <strong>(1814,204,204,3)</strong> and <strong>Y</strong> is of shape <strong>(1814,39)</strong> '' 39 corresponds to the dummy variable : category 1 has 2 attributes, category 2 has 26 attribute , category 3 has 11 attributes: so total makes 39 ''.<br>
  <strong>2. Building my neural neural network</strong><br>
  the different parameters of my model are :  </p>
  
  <ul>
  <li>epochs = 100</li>
  <li>lrate = 0.001</li>
  <li>decay = lrate/epochs</li>
  <li>sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)</li>
  <li>loss=binary_crossentropy </li>
  <li>optimizer=sgd </li>
  <li>metrics=accuracy</li>
  </ul>
  
  <p>The structure of my trained network is as follows:</p>
  
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #   
  =================================================================
  conv2d_3 (Conv2D)            (None, 204, 204, 32)      896       
  _________________________________________________________________
  dropout_3 (Dropout)          (None, 204, 204, 32)      0         
  _________________________________________________________________
  conv2d_4 (Conv2D)            (None, 204, 204, 32)      9248      
  _________________________________________________________________
  max_pooling2d_2 (MaxPooling2 (None, 102, 102, 32)      0         
  _________________________________________________________________
  flatten_2 (Flatten)          (None, 332928)            0         
  _________________________________________________________________
  dense_3 (Dense)              (None, 512)               170459648 
  _________________________________________________________________
  dropout_4 (Dropout)          (None, 512)               0         
  _________________________________________________________________
  dense_4 (Dense)              (None, 39)                20007     
  _________________________________________________________________
  activation_1 (Activation)    (None, 39)                0         
  =================================================================
  Total params: 170,489,799
  Trainable params: 170,489,799
  Non-trainable params: 0
  </code></pre>
  
  <p><strong>Questions</strong> </p>
  
  <ol>
  <li>I don't think my metric ='accuracy' is  a good choice: it doesn't reflect the reality of what happens. For example, I executed an evaluation test on my training data and got 0.95 ( 95% of all bins are well classified), but that doesn't mean the model is doing well because in every output: there should be 3 ones out of the 39 components and all the remaining are zeros.. so there is a high probability to have a lot of zeros and thus even in the worse case the 39 values are predicted to be 0 ( nothing is detected) than we have 36/39 accuracy: which doesn't reflect the real issue? ( That's what I got as a result, for a given X, i got many Values near to zeros , so when I use a threshold (0.5) it makes everything equal to zero: I tried to implement a threshold based on statistical approach using mathiew correlation coefficient: but still doesn't add anything usefull) .. So , what do you think is a good metric for a multi-label classification problem with many (39) labels output?  </li>
  <li>Do you think that the bad performance of my model comes from the fact that I don't have many images? ( images per label ) ?</li>
  <li>Do you think my model has bad performance because the structure of Neural network is not good? I tried to used a pretrained model where I fix first layers since state of the art says that these layers are responsible for detecting edges, and I trained only next layers: In that case I have also bad results (calcaluated through the 'accuracy' metric? </li>
  <li>To sum up , i Know that maybe there is not a universal solution to my issues, but at least I want to know what may cause these kind of issues and how to remedy them? So any help especially from people who worked on multi-label classifications with many labels at the output, is welcome</li>
  </ol>