{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/1358","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/1358/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/1358/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/1358/events","html_url":"https://github.com/tensorflow/tensorflow/pull/1358","id":137881842,"number":1358,"title":"Mean of cross entropy vs sum in summaries tutorial","user":{"login":"basveeling","id":536975,"avatar_url":"https://avatars.githubusercontent.com/u/536975?v=3","gravatar_id":"","url":"https://api.github.com/users/basveeling","html_url":"https://github.com/basveeling","followers_url":"https://api.github.com/users/basveeling/followers","following_url":"https://api.github.com/users/basveeling/following{/other_user}","gists_url":"https://api.github.com/users/basveeling/gists{/gist_id}","starred_url":"https://api.github.com/users/basveeling/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basveeling/subscriptions","organizations_url":"https://api.github.com/users/basveeling/orgs","repos_url":"https://api.github.com/users/basveeling/repos","events_url":"https://api.github.com/users/basveeling/events{/privacy}","received_events_url":"https://api.github.com/users/basveeling/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/cla:%20yes","name":"cla: yes","color":"009800"}],"state":"open","locked":false,"assignee":{"login":"Sohl-Dickstein","id":498544,"avatar_url":"https://avatars.githubusercontent.com/u/498544?v=3","gravatar_id":"","url":"https://api.github.com/users/Sohl-Dickstein","html_url":"https://github.com/Sohl-Dickstein","followers_url":"https://api.github.com/users/Sohl-Dickstein/followers","following_url":"https://api.github.com/users/Sohl-Dickstein/following{/other_user}","gists_url":"https://api.github.com/users/Sohl-Dickstein/gists{/gist_id}","starred_url":"https://api.github.com/users/Sohl-Dickstein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Sohl-Dickstein/subscriptions","organizations_url":"https://api.github.com/users/Sohl-Dickstein/orgs","repos_url":"https://api.github.com/users/Sohl-Dickstein/repos","events_url":"https://api.github.com/users/Sohl-Dickstein/events{/privacy}","received_events_url":"https://api.github.com/users/Sohl-Dickstein/received_events","type":"User","site_admin":false},"milestone":null,"comments":8,"created_at":"2016-03-02T14:05:02Z","updated_at":"2016-04-02T07:09:58Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/tensorflow/tensorflow/pulls/1358","html_url":"https://github.com/tensorflow/tensorflow/pull/1358","diff_url":"https://github.com/tensorflow/tensorflow/pull/1358.diff","patch_url":"https://github.com/tensorflow/tensorflow/pull/1358.patch"},"body":"The `mnist_with_summaries.py` and `mnist_softmax.py` tutorials compute the sum of the cross_entropy over the data points in the batch. This seems to deviate from the more common approach of taking the mean, such as in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/mnist/mnist.py#L99 \r\nThe current form works as a loss function, but might confuse newcomers who are figuring out how gradients are aggregated in TensorFlow (as experienced by me, see http://stackoverflow.com/questions/35731506/unaggregated-gradients-gradients-per-example-in-tensorflow :-))"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/204663758","html_url":"https://github.com/tensorflow/tensorflow/pull/1358#issuecomment-204663758","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/1358","id":204663758,"user":{"login":"basveeling","id":536975,"avatar_url":"https://avatars.githubusercontent.com/u/536975?v=3","gravatar_id":"","url":"https://api.github.com/users/basveeling","html_url":"https://github.com/basveeling","followers_url":"https://api.github.com/users/basveeling/followers","following_url":"https://api.github.com/users/basveeling/following{/other_user}","gists_url":"https://api.github.com/users/basveeling/gists{/gist_id}","starred_url":"https://api.github.com/users/basveeling/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basveeling/subscriptions","organizations_url":"https://api.github.com/users/basveeling/orgs","repos_url":"https://api.github.com/users/basveeling/repos","events_url":"https://api.github.com/users/basveeling/events{/privacy}","received_events_url":"https://api.github.com/users/basveeling/received_events","type":"User","site_admin":false},"created_at":"2016-04-02T07:09:58Z","updated_at":"2016-04-02T07:09:58Z","body":"@Sohl-Dickstein: done. I determined the new learning rate for the softmax tutorial by cross validation:\r\nhttps://gist.github.com/basveeling/0b43d5d9fe533e9aaf710d13356931f4\r\nI haven't validated the results for the convolutional mnist network in the pros tutorial due to lack of gpu, but someone can verify this with https://gist.github.com/basveeling/80b11bf657891c0c8f253c899a4be99d ."}}