{"action":"created","issue":{"url":"https://api.github.com/repos/keras-team/keras/issues/8011","repository_url":"https://api.github.com/repos/keras-team/keras","labels_url":"https://api.github.com/repos/keras-team/keras/issues/8011/labels{/name}","comments_url":"https://api.github.com/repos/keras-team/keras/issues/8011/comments","events_url":"https://api.github.com/repos/keras-team/keras/issues/8011/events","html_url":"https://github.com/keras-team/keras/issues/8011","id":261453020,"node_id":"MDU6SXNzdWUyNjE0NTMwMjA=","number":8011,"title":"Multi label classification with binary_crossentropy and class_weight - must contain all classes error","user":{"login":"cinneesol","id":12005966,"node_id":"MDQ6VXNlcjEyMDA1OTY2","avatar_url":"https://avatars0.githubusercontent.com/u/12005966?v=4","gravatar_id":"","url":"https://api.github.com/users/cinneesol","html_url":"https://github.com/cinneesol","followers_url":"https://api.github.com/users/cinneesol/followers","following_url":"https://api.github.com/users/cinneesol/following{/other_user}","gists_url":"https://api.github.com/users/cinneesol/gists{/gist_id}","starred_url":"https://api.github.com/users/cinneesol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cinneesol/subscriptions","organizations_url":"https://api.github.com/users/cinneesol/orgs","repos_url":"https://api.github.com/users/cinneesol/repos","events_url":"https://api.github.com/users/cinneesol/events{/privacy}","received_events_url":"https://api.github.com/users/cinneesol/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-09-28T20:37:30Z","updated_at":"2018-06-13T08:56:21Z","closed_at":null,"author_association":"NONE","body":"Hello, \r\n\r\nI have run into an issue with class_weights and binary_crossentropy for multi label outputs.\r\n\r\nIn essence:\r\nOutput is an ndarray Y with Y.shape[1] the number of label outputs, e.g.:\r\n[[0,0,1,0,1,1,0],\r\n[0,0,1,0,1,1,1],\r\n....\r\n[0,0,0,0,1,0,0]]\r\n\r\n\r\nI have defined the output layer as follows (method A):\r\n```\r\n...\r\nout_dense=Dense(Y.shape[1], \r\n                 activation='sigmoid', name='OUTPUT_BINARY')(intermediate_dense)\r\n\r\n```\r\nand defined the model:\r\n\r\n```\r\nmodel = Model(inputs=[... some inputs...], outputs=out_dense)\r\nmodel.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\r\n```\r\n\r\nweight dictionary defined as follows:\r\n`W = {0.0: 0.54356142358280057, 1.0: 6.2390227278685089}\r\n`\r\nNow, when I fit:\r\n`model.fit([...some inputs...], Y, batch_size=64, epochs=1, validation_split=0.25, class_weight={'OUTPUT_BINARY':W}, sample_weight=None)\r\n`\r\nI get the error:\r\n\r\n```\r\n/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)\r\n   1521             check_batch_axis=False,\r\n   1522             batch_size=batch_size)\r\n-> 1523         # Prepare validation data.\r\n   1524         do_validation = False\r\n   1525         if validation_data:\r\n\r\n/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_axis, batch_size)\r\n   1388                           for (ref, sw, cw, mode)\r\n   1389                           in zip(y, sample_weights, class_weights, self._feed_sample_weight_modes)]\r\n-> 1390         _check_array_lengths(x, y, sample_weights)\r\n   1391         _check_loss_and_target_compatibility(y,\r\n   1392                                              self._feed_loss_fns,\r\n\r\n/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in <listcomp>(.0)\r\n   1387         sample_weights = [_standardize_weights(ref, sw, cw, mode)\r\n   1388                           for (ref, sw, cw, mode)\r\n-> 1389                           in zip(y, sample_weights, class_weights, self._feed_sample_weight_modes)]\r\n   1390         _check_array_lengths(x, y, sample_weights)\r\n   1391         _check_loss_and_target_compatibility(y,\r\n\r\n/usr/local/lib/python3.5/dist-packages/keras/engine/training.py in _standardize_weights(y, sample_weight, class_weight, sample_weight_mode)\r\n    556                              '`class_weight`.'\r\n    557                              % (existing_classes - existing_class_weight))\r\n--> 558         return weights\r\n    559     else:\r\n    560         if sample_weight_mode is None:\r\n\r\nValueError: `class_weight` must contain all classes in the data. The classes {2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 17, 19, 20, 21, 28, 29, 31, 34, 36, 37, 38, 41, 43, 44, 45, 47} exist in the data but not in `class_weight`.\r\n\r\n```\r\nI have traced it to :\r\n\r\n```\r\ndef _standardize_weights(y, sample_weight=None, class_weight=None,\r\n                         sample_weight_mode=None):\r\n...\r\n        if y.shape[1] > 1:\r\n            y_classes = y.argmax(axis=1)\r\n        elif y.shape[1] == 1:\r\n            y_classes = np.reshape(y, y.shape[0])\r\n        else:\r\n            y_classes = y\r\n\r\n```\r\n\r\nin training.py.\r\n\r\nIt seems that here keras sees the output as a softmax type one hot encoded and returns the index of the maximum for each row.\r\n\r\n\r\nThe only way I saw to fix this was to create multiple single outputs by using a for loop (method B):\r\n\r\n```\r\noutputs=[]\r\nl_weights=[]\r\nfor i in range(y_in_service_indicator.shape[1]):\r\n    out_dense=Dense(1, activation='sigmoid', name='O_'+str(1+i))(intermediate_dense)\r\n    outputs.append(out_dense)\r\n    l_weights.append(1.0)\r\n\r\nmodel = Model([...some inputs...], outputs=outputs)\r\nmodel.compile(loss='binary_crossentropy', loss_weights=l_weights,\r\n              optimizer='adam', metrics=['accuracy'])\r\n\r\n```\r\n(I appreciate may not need the loss weights)\r\nand define the outputs as :\r\n\r\n```\r\ny_s_binary = []\r\nfor i in list(range(Y.shape[1])):\r\n    y_s_binary.append(y_in_service_indicator[:,i])\r\n\r\n```\r\nand now fit:\r\n\r\n```\r\nmodel.fit(X_intervals, y_s_binary, batch_size=64, epochs=1, validation_split=0.25,\r\n          class_weight={'OUTPUT_BINARY':W}, sample_weight=None)\r\n\r\n```\r\nThis works, but it feels to me like I am missing somehting - would appreciate input on how to make Keras work with single defined Y ndarray and model output as  in method A.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/keras-team/keras/issues/comments/396865806","html_url":"https://github.com/keras-team/keras/issues/8011#issuecomment-396865806","issue_url":"https://api.github.com/repos/keras-team/keras/issues/8011","id":396865806,"node_id":"MDEyOklzc3VlQ29tbWVudDM5Njg2NTgwNg==","user":{"login":"maxim5","id":2580604,"node_id":"MDQ6VXNlcjI1ODA2MDQ=","avatar_url":"https://avatars0.githubusercontent.com/u/2580604?v=4","gravatar_id":"","url":"https://api.github.com/users/maxim5","html_url":"https://github.com/maxim5","followers_url":"https://api.github.com/users/maxim5/followers","following_url":"https://api.github.com/users/maxim5/following{/other_user}","gists_url":"https://api.github.com/users/maxim5/gists{/gist_id}","starred_url":"https://api.github.com/users/maxim5/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maxim5/subscriptions","organizations_url":"https://api.github.com/users/maxim5/orgs","repos_url":"https://api.github.com/users/maxim5/repos","events_url":"https://api.github.com/users/maxim5/events{/privacy}","received_events_url":"https://api.github.com/users/maxim5/received_events","type":"User","site_admin":false},"created_at":"2018-06-13T08:56:21Z","updated_at":"2018-06-13T08:56:21Z","author_association":"NONE","body":"@sharifza Of course. That's why this bug is open."}}
{"actor":{"display_login":"maxim5"}}