{"action":"created","issue":{"url":"https://api.github.com/repos/deepchem/deepchem/issues/1092","repository_url":"https://api.github.com/repos/deepchem/deepchem","labels_url":"https://api.github.com/repos/deepchem/deepchem/issues/1092/labels{/name}","comments_url":"https://api.github.com/repos/deepchem/deepchem/issues/1092/comments","events_url":"https://api.github.com/repos/deepchem/deepchem/issues/1092/events","html_url":"https://github.com/deepchem/deepchem/issues/1092","id":297972942,"number":1092,"title":"Do the batch normalization functions implemented differentiate between training and evaluation?","user":{"login":"simonfqy","id":10753120,"avatar_url":"https://avatars1.githubusercontent.com/u/10753120?v=4","gravatar_id":"","url":"https://api.github.com/users/simonfqy","html_url":"https://github.com/simonfqy","followers_url":"https://api.github.com/users/simonfqy/followers","following_url":"https://api.github.com/users/simonfqy/following{/other_user}","gists_url":"https://api.github.com/users/simonfqy/gists{/gist_id}","starred_url":"https://api.github.com/users/simonfqy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simonfqy/subscriptions","organizations_url":"https://api.github.com/users/simonfqy/orgs","repos_url":"https://api.github.com/users/simonfqy/repos","events_url":"https://api.github.com/users/simonfqy/events{/privacy}","received_events_url":"https://api.github.com/users/simonfqy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-17T02:56:54Z","updated_at":"2018-02-19T22:32:01Z","closed_at":null,"author_association":"NONE","body":"In the script `deepchem/models/tensorgraph/layers.py`, there are two classes for batch normalization, `BatchNorm` and `BatchNormalization`, which you can see from here.\r\n https://github.com/deepchem/deepchem/blob/6f86377d9bff4541ff50ec014743d840a3e230f4/deepchem/models/tensorgraph/layers.py#L2870 \r\n\r\nHowever, I think that in the `create_tensor()` functions of both classes, the cases between training and evaluating are not differentiated. Especially in the `create_tensor` function of the `BatchNorm` class which calls `tf.layers.batch_normalization()`, the \"training\" flag of this `TensorFlow` function is not specified. So I suppose that it will not work entirely properly.\r\n\r\nI tried including the flag myself. I inserted one line of code (the second line below) in `create_tensor()` function of the `BatchNorm` class:\r\n```Python\r\nparent_tensor = inputs[0]\r\nself.training = tf.equal(tf.constant(1.0, dtype=tf.float32), kwargs['training']) # Newly inserted\r\nout_tensor = tf.layers.batch_normalization(parent_tensor)\r\n```\r\n\r\nin which the `kwargs['training']` should be `self._training_placeholder` of TensorGraph class, which should have a value of 1.0 for training case and 0.0 for evaluation/testing case. However this line of code does not work and causes the program to stop with error, I don't know why. This makes me wonder how to incorporate the training flag into the functions.\r\n\r\nPlease correct me if I am wrong. Thanks!"},"comment":{"url":"https://api.github.com/repos/deepchem/deepchem/issues/comments/366818112","html_url":"https://github.com/deepchem/deepchem/issues/1092#issuecomment-366818112","issue_url":"https://api.github.com/repos/deepchem/deepchem/issues/1092","id":366818112,"user":{"login":"simonfqy","id":10753120,"avatar_url":"https://avatars1.githubusercontent.com/u/10753120?v=4","gravatar_id":"","url":"https://api.github.com/users/simonfqy","html_url":"https://github.com/simonfqy","followers_url":"https://api.github.com/users/simonfqy/followers","following_url":"https://api.github.com/users/simonfqy/following{/other_user}","gists_url":"https://api.github.com/users/simonfqy/gists{/gist_id}","starred_url":"https://api.github.com/users/simonfqy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simonfqy/subscriptions","organizations_url":"https://api.github.com/users/simonfqy/orgs","repos_url":"https://api.github.com/users/simonfqy/repos","events_url":"https://api.github.com/users/simonfqy/events{/privacy}","received_events_url":"https://api.github.com/users/simonfqy/received_events","type":"User","site_admin":false},"created_at":"2018-02-19T22:32:00Z","updated_at":"2018-02-19T22:32:00Z","author_association":"NONE","body":"@miaecle Thanks for your reply.\r\nCurrently the `self._training_placeholder` is used as input to `layer.create_tensor()` in the `build()` function of `TensorGraph` class, as in this link:\r\nhttps://github.com/deepchem/deepchem/blob/6f86377d9bff4541ff50ec014743d840a3e230f4/deepchem/models/tensorgraph/tensor_graph.py#L473\r\n\r\nAre you suggesting that I can simply change that line to \r\n```Python\r\nlayer.create_tensor(in_layers=self._training_placeholder)\r\n```\r\nand also change the line in `layers.py` to\r\n```Python\r\nout_tensor = tf.layers.batch_normalization(training=parent_tensor)\r\n```\r\nwhere the `parent_tensor` is `self._get_input_tensors(in_layers)[0]`, which would be `self._training_placeholder` of `TensorGraph` class.   \r\n\r\nIf it is what you mean, I can go with this. However, even if it works for me now, I recommend you have a better way of handling it, because treating `self._training_placeholder` as `parent_tensor` might not be what you initially planned and could possibly break some code.\r\nNevertheless, I will try implementing it later today."}}
{"actor":{"display_login":"simonfqy"}}