<p>Accuracy could be misleading as a metric for your problem, with high class imbalance, I would use the <strong>F1</strong> score.</p>

<p>As for the loss, you could use the <a href="https://arxiv.org/pdf/1708.02002.pdf" rel="nofollow noreferrer"><strong>focal loss</strong></a> it is an variant of the <strong>categorical cross-entropy</strong> that focuses on the <strong>least represented</strong> classes. You can find an example <a href="https://github.com/umbertogriffo/focal-loss-keras/blob/master/losses.py" rel="nofollow noreferrer">here</a>, in my experience, it helps a lot with little classes on NLP classification tasks.</p>
how to handle unbalanced data for multilabel classification using CNN in Keras?
<p>My dataset shape is <code>(91149, 12)</code></p>

<p>I used CNN to train my classifier in text classification tasks</p>

<p>I found Training Accuracy: <code>0.5923</code> and Testing Accuracy: <code>0.5780</code></p>

<p>My Class has 9 labels as below:</p>

<pre><code>df['thematique'].value_counts()
Corporate                   42399
Economie collaborative      13272
Innovation                  11360
Filiale                      5990
Richesses Humaines           4445
Relation sociétaire          4363
Communication                4141
Produits et services         2594
Sites Internet et applis     2585
</code></pre>

<p>The model structure:</p>

<pre><code>model = Sequential()
embedding_layer = Embedding(vocab_size, 300, weights=[embedding_matrix],   input_length=maxlen   ,   trainable=False)
model.add(embedding_layer)
model.add(Conv1D(128, 7, activation='relu'))
model.add(GlobalMaxPooling1D())
model.add(Dense(9, activation='sigmoid'))
model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics= ['categorical_accuracy'])
</code></pre>

<p>My data for multilabel classification is imbalanced. I need to handle imbalanced data for multipabel classification using CNN in Keras.</p>