{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/3767","repository_url":"https://api.github.com/repos/dmlc/mxnet","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/3767/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/3767/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/3767/events","html_url":"https://github.com/dmlc/mxnet/issues/3767","id":188023838,"number":3767,"title":"Problems using csviter","user":{"login":"wsstriving","id":9416457,"avatar_url":"https://avatars.githubusercontent.com/u/9416457?v=3","gravatar_id":"","url":"https://api.github.com/users/wsstriving","html_url":"https://github.com/wsstriving","followers_url":"https://api.github.com/users/wsstriving/followers","following_url":"https://api.github.com/users/wsstriving/following{/other_user}","gists_url":"https://api.github.com/users/wsstriving/gists{/gist_id}","starred_url":"https://api.github.com/users/wsstriving/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wsstriving/subscriptions","organizations_url":"https://api.github.com/users/wsstriving/orgs","repos_url":"https://api.github.com/users/wsstriving/repos","events_url":"https://api.github.com/users/wsstriving/events{/privacy}","received_events_url":"https://api.github.com/users/wsstriving/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-11-08T15:36:24Z","updated_at":"2016-11-09T02:21:25Z","closed_at":null,"body":"In the example/module/mnist_mlp.py, validation  accuracy is computed as follows:\r\n```python\r\n# a dummy call just to test if the API works for merge_batches=True\r\n# preds = mod.predict(val_dataiter)\r\n\r\n# perform prediction and calculate accuracy manually\r\npreds = mod.predict(val_dataiter, merge_batches=False)\r\nval_dataiter.reset()\r\nacc_sum = 0.0; acc_cnt = 0\r\nfor i, batch in enumerate(val_dataiter):\r\n    print i\r\n    pred_label = preds[i][0].asnumpy().argmax(axis=1)\r\n    label = batch.label[0].asnumpy().astype('int32')\r\n    acc_sum += (label == pred_label).sum()\r\n    acc_cnt += len(pred_label)\r\nprint('validation Accuracy: %.3f' % (acc_sum / acc_cnt))\r\n```\r\n\r\nI borrowed this part in my own training scripts, which read in csv data using csviter.\r\n```python\r\n    data_val = mx.io.CSVIter(data_csv = valdatacsv,\r\n                                    data_shape = (400, ),\r\n                                    label_csv = vallabelcsv,\r\n                                    label_shape = (1,),\r\n                                    batch_size = batchsize,\r\n                                    shuffle = True,\r\n                                    seed = 10)\r\n    #preds =  mod.predict(data_val, merge_batches=False)\r\n    preds =  mod.predict(data_val, merge_batches=False)\r\n    data_val.reset()\r\n    acc_sum = 0.0\r\n    acc_cnt = 0\r\n    for i, batch in enumerate(data_val):\r\n        print i\r\n        pred_label = preds[i][0].asnumpy().argmax(axis=1)\r\n        label = batch.label[0].asnumpy().astype('int32')\r\n        acc_sum += (label==pred_label).sum()\r\n        acc_cnt += len(pred_label)\r\n    print('Validation accuracy = %.3f' % (acc_sum/acc_cnt))\r\n```\r\nHowever, this will throw an error as \r\n```\r\nTraceback (most recent call last):\r\n  File \"train_model.py\", line 93, in <module>\r\n    pred_label = preds[i][0].asnumpy().argmax(axis=1)\r\nIndexError: list index out of range\r\n```\r\nThe validation sample number is 9534 and the batchsize is 128; 9534/128=74.48, which means there should be 74 full batch and the last batch is not full. This error happens because the the preds abandons the last batch? I don't know how to fix this (MnistIter doesn't have this problem). Then a strange thing happens, if we uncomment the commented line, which means we compute the preds twice(which seems to me overwriting, which makes no difference); no error will be reported, but the computed validation accuracy is very low (wrong);  I think it should because of the last batch causes the label shift, but can not understand why it happens and how to fix it. Could anyone give some explanation? "},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/259318996","html_url":"https://github.com/dmlc/mxnet/issues/3767#issuecomment-259318996","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/3767","id":259318996,"user":{"login":"wsstriving","id":9416457,"avatar_url":"https://avatars.githubusercontent.com/u/9416457?v=3","gravatar_id":"","url":"https://api.github.com/users/wsstriving","html_url":"https://github.com/wsstriving","followers_url":"https://api.github.com/users/wsstriving/followers","following_url":"https://api.github.com/users/wsstriving/following{/other_user}","gists_url":"https://api.github.com/users/wsstriving/gists{/gist_id}","starred_url":"https://api.github.com/users/wsstriving/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wsstriving/subscriptions","organizations_url":"https://api.github.com/users/wsstriving/orgs","repos_url":"https://api.github.com/users/wsstriving/repos","events_url":"https://api.github.com/users/wsstriving/events{/privacy}","received_events_url":"https://api.github.com/users/wsstriving/received_events","type":"User","site_admin":false},"created_at":"2016-11-09T02:21:25Z","updated_at":"2016-11-09T02:21:25Z","body":"@shuokay Nope, this doesn't since preds[i] is a list"}}
{"actor":{"display_login":"wsstriving"}}