{"action":"created","issue":{"url":"https://api.github.com/repos/maxpumperla/hyperas/issues/163","repository_url":"https://api.github.com/repos/maxpumperla/hyperas","labels_url":"https://api.github.com/repos/maxpumperla/hyperas/issues/163/labels{/name}","comments_url":"https://api.github.com/repos/maxpumperla/hyperas/issues/163/comments","events_url":"https://api.github.com/repos/maxpumperla/hyperas/issues/163/events","html_url":"https://github.com/maxpumperla/hyperas/issues/163","id":316721207,"node_id":"MDU6SXNzdWUzMTY3MjEyMDc=","number":163,"title":"Different evaluation results during tuning and after tuning?","user":{"login":"Kirayue","id":13309212,"node_id":"MDQ6VXNlcjEzMzA5MjEy","avatar_url":"https://avatars2.githubusercontent.com/u/13309212?v=4","gravatar_id":"","url":"https://api.github.com/users/Kirayue","html_url":"https://github.com/Kirayue","followers_url":"https://api.github.com/users/Kirayue/followers","following_url":"https://api.github.com/users/Kirayue/following{/other_user}","gists_url":"https://api.github.com/users/Kirayue/gists{/gist_id}","starred_url":"https://api.github.com/users/Kirayue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kirayue/subscriptions","organizations_url":"https://api.github.com/users/Kirayue/orgs","repos_url":"https://api.github.com/users/Kirayue/repos","events_url":"https://api.github.com/users/Kirayue/events{/privacy}","received_events_url":"https://api.github.com/users/Kirayue/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-04-23T08:58:36Z","updated_at":"2018-06-13T14:39:31Z","closed_at":"2018-06-13T09:26:47Z","author_association":"NONE","body":"Hello,\r\nI randomly split data five times, and tuned the model on each split.\r\nI found that model.evaluate results are different except the first split. I print l2 value, (train_loss, train_acc), (val_loss, val_acc) and model each tuning round.\r\nAfter tuning, I use best_model that hyperas returned to evaluate the same train/val data, the results is different from turning. The model's memory location are the same. I have no idea what's happening.\r\nThank you.\r\n\r\n![image](https://user-images.githubusercontent.com/13309212/39116646-8dc6e9f6-4717-11e8-9593-eef3f4fe350b.png)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/maxpumperla/hyperas/issues/comments/396961664","html_url":"https://github.com/maxpumperla/hyperas/issues/163#issuecomment-396961664","issue_url":"https://api.github.com/repos/maxpumperla/hyperas/issues/163","id":396961664,"node_id":"MDEyOklzc3VlQ29tbWVudDM5Njk2MTY2NA==","user":{"login":"Kirayue","id":13309212,"node_id":"MDQ6VXNlcjEzMzA5MjEy","avatar_url":"https://avatars2.githubusercontent.com/u/13309212?v=4","gravatar_id":"","url":"https://api.github.com/users/Kirayue","html_url":"https://github.com/Kirayue","followers_url":"https://api.github.com/users/Kirayue/followers","following_url":"https://api.github.com/users/Kirayue/following{/other_user}","gists_url":"https://api.github.com/users/Kirayue/gists{/gist_id}","starred_url":"https://api.github.com/users/Kirayue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kirayue/subscriptions","organizations_url":"https://api.github.com/users/Kirayue/orgs","repos_url":"https://api.github.com/users/Kirayue/repos","events_url":"https://api.github.com/users/Kirayue/events{/privacy}","received_events_url":"https://api.github.com/users/Kirayue/received_events","type":"User","site_admin":false},"created_at":"2018-06-13T14:39:31Z","updated_at":"2018-06-13T14:39:31Z","author_association":"NONE","body":"I override x_train by overriding ` .npy `  files\r\n```\r\nnp.save('tmp/x_train.npy', x_train)\r\nnp.save('tmp/y_train.npy', y_train)\r\nnp.save('tmp/x_val.npy', x_val)\r\nnp.save('tmp/y_val.npy', y_val)\r\n```\r\nthanks for your reply"}}
{"actor":{"display_login":"Kirayue"}}