title: keras: sample weight multiple imbalanced output
body: <p>I have the following model with 28 outputs. Each output has 3 very imbalanced classes. So I use a dict of sample weights - one sample weight array per output- in as<code>sample_weight</code> in the <code>fit</code>function. y_train and y_valid are lists of 28 one-hot encoded arrays.</p>
  
  <p>Whether I use the sample_weight or not, the model always predicts the most frequent class in all the 28 labels. Therefore, I think that I use sample_weight incorrectly. I also tried to decrease or increase the learning rate, but that has no effect... Hence there should be something wrong with my weights, but what is it?</p>
  
  <pre><code>#create sample weights
  from sklearn.utils import compute_sample_weight
  
  train_weight = {}
  for i in range(0,28):
      train_weight[i] = compute_sample_weight('balanced', y_train[i])
  
  valid_weight = {}
  for i in range(0,28):
      valid_weight[i] = compute_sample_weight('balanced', y_valid[i])  
  
  #Model:  
  
  Earlystop = EarlyStopping(monitor='val_loss', patience=20, mode='min', mverbose=1, min_delta=0.02, restore_best_weights=True)
  checkpoint = ModelCheckpoint('nn_statarb', monitor='val_loss', verbose=1, save_best_only=True, mode='min', save_weights_only=False)
  optimizer = optimizers.Adam(lr=0.00001, beta_1=0.9, beta_2=0.999, epsilon=1e-8, decay=5e-8, amsgrad=False)
  
  input_ = Input(batch_shape=(batch_size, 1, 1, X_train.shape[3], X_train.shape[4])) 
  
  cnn = Sequential()
  cnn.add(Conv2D(16, kernel_size=(5, X_train.shape[4]), padding='same', 
  data_format="channels_first", input_shape = (1, X_train.shape[3], 
  X_train.shape[4])))
  cnn.add(LeakyReLU(alpha=0.01))
  cnn.add(Conv2D(16, (1, 1)))
  cnn.add(LeakyReLU(alpha=0.01))
  cnn.add(Conv2D(16, (10, 2), padding='same'))
  cnn.add(Flatten())
  rnn = Sequential()
  rnn.add(LSTM(X_train.shape[4], return_sequences=False, stateful=True, batch_size=batch_size))
  rnn.add(Reshape((1, -1, X_train.shape[4])))
  
  pricemodel = TimeDistributed(cnn)(input_) 
  pricemodel = rnn(pricemodel) 
  pricemodel = Model(inputs=input_, outputs=pricemodel)
  out_ = pricemodel(input_)
  
  tower_1 = Conv2D(64, (1,1), padding='same', activation='relu')(out_)
  tower_1 = Conv2D(64, (3,3), padding='same', activation='relu')(tower_1)
  tower_2 = Conv2D(64, (1,1), padding='same', activation='relu')(out_)
  tower_2 = Conv2D(64, (5,5), padding='same', activation='relu')(tower_2)
  tower_3 = MaxPooling2D((3,3), strides=(1,1), padding='same')(out_)
  tower_3 = Conv2D(64, (1,1), padding='same', activation='relu')(tower_3)
  
  output = concatenate([tower_1, tower_2, tower_3], axis=1)
  output = Flatten()(output)
  
  x1 = Dense(3, activation='softmax')(output)
  x2 = Dense(3, activation='softmax')(output)
  x3 = Dense(3, activation='softmax')(output)
  x4 = Dense(3, activation='softmax')(output)
  x5 = Dense(3, activation='softmax')(output)
  x6 = Dense(3, activation='softmax')(output)
  x7 = Dense(3, activation='softmax')(output)
  x8 = Dense(3, activation='softmax')(output)
  x9 = Dense(3, activation='softmax')(output)
  x10 = Dense(3, activation='softmax')(output)
  x11 = Dense(3, activation='softmax')(output)
  x12 = Dense(3, activation='softmax')(output)
  x13 = Dense(3, activation='softmax')(output)
  x14 = Dense(3, activation='softmax')(output)
  x15 = Dense(3, activation='softmax')(output)
  x16 = Dense(3, activation='softmax')(output)
  x17 = Dense(3, activation='softmax')(output)
  x18 = Dense(3, activation='softmax')(output)
  x19 = Dense(3, activation='softmax')(output)
  x20 = Dense(3, activation='softmax')(output)
  x21 = Dense(3, activation='softmax')(output)
  x22 = Dense(3, activation='softmax')(output)
  x23 = Dense(3, activation='softmax')(output)
  x24 = Dense(3, activation='softmax')(output)
  x25 = Dense(3, activation='softmax')(output)
  x26 = Dense(3, activation='softmax')(output)
  x27 = Dense(3, activation='softmax')(output)
  x28 = Dense(3, activation='softmax')(output)
  
  model = Model(inputs=input_, outputs=[x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14, x15, x16, x17, x18, x19, x20, x21, x22, x23, x24, x25, x26, x27, x28])
  model.compile(loss='categorical_crossentropy', optimizer=optimizer,  metrics=['accuracy'])
  history = model.fit(X_train, y_train, epochs=200, batch_size=batch_size, verbose=2, shuffle=False, validation_data=[X_valid, y_valid, valid_weight], callbacks=[Earlystop, checkpoint], sample_weight=train_weight)
  </code></pre>