{"action":"closed","issue":{"url":"https://api.github.com/repos/keras-team/keras/issues/11119","repository_url":"https://api.github.com/repos/keras-team/keras","labels_url":"https://api.github.com/repos/keras-team/keras/issues/11119/labels{/name}","comments_url":"https://api.github.com/repos/keras-team/keras/issues/11119/comments","events_url":"https://api.github.com/repos/keras-team/keras/issues/11119/events","html_url":"https://github.com/keras-team/keras/issues/11119","id":359274723,"node_id":"MDU6SXNzdWUzNTkyNzQ3MjM=","number":11119,"title":"Progressively slower model save/load times","user":{"login":"abaxi","id":26046328,"node_id":"MDQ6VXNlcjI2MDQ2MzI4","avatar_url":"https://avatars0.githubusercontent.com/u/26046328?v=4","gravatar_id":"","url":"https://api.github.com/users/abaxi","html_url":"https://github.com/abaxi","followers_url":"https://api.github.com/users/abaxi/followers","following_url":"https://api.github.com/users/abaxi/following{/other_user}","gists_url":"https://api.github.com/users/abaxi/gists{/gist_id}","starred_url":"https://api.github.com/users/abaxi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/abaxi/subscriptions","organizations_url":"https://api.github.com/users/abaxi/orgs","repos_url":"https://api.github.com/users/abaxi/repos","events_url":"https://api.github.com/users/abaxi/events{/privacy}","received_events_url":"https://api.github.com/users/abaxi/received_events","type":"User","site_admin":false},"labels":[{"id":585833447,"node_id":"MDU6TGFiZWw1ODU4MzM0NDc=","url":"https://api.github.com/repos/keras-team/keras/labels/to%20investigate","name":"to investigate","color":"f9d0c4","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-09-12T00:47:35Z","updated_at":"2018-09-12T09:38:49Z","closed_at":"2018-09-12T09:38:49Z","author_association":"NONE","body":"I've noticed that the process of saving and loading saved models slows down as I generate more models. To clarify, say I have to load 10 models from hdf5 files. The 1st model I load will be loaded quickly, but every successive load will be progressively slower. The models are of the same size, and memory is not an issue. I've attached a python script to reproduce the issue. \r\n\r\nOS: Ubuntu 16.04\r\nBackend: Tensorflow\r\nUsing CPU only\r\n\r\nThank you\r\n\r\n### Reproducing code example:\r\n\r\n```\r\nimport time\r\nimport numpy as np\r\nimport keras\r\n\r\n\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Dense, Activation\r\n\r\n\r\ndef get_model(X):\r\n    metric = \"mse\"\r\n    loss_fn = \"mean_squared_error\"\r\n    opt_meth = \"rmsprop\"\r\n    \r\n    layers = [X.shape[1], 10, X.shape[1]]\r\n    act_fns = [\"elu\", \"elu\", \"elu\"]\r\n    \r\n    model = Sequential()\r\n\r\n    for i in range(len(layers)):\r\n        model.add(Dense(units=layers[i]))\r\n        model.add(Activation(act_fns[i]))\r\n\r\n    model.compile(optimizer=opt_meth, loss=loss_fn, metrics=[metric])\r\n    model.fit(X, X, verbose=0)\r\n    \r\n    return model\r\n\r\nif __name__ == '__main__':\r\n    N = 10\r\n    X = np.random.randn(1000,30)\r\n    for i in range(N):\r\n        fname = str(i)+\".hdf5\"\r\n        m = get_model(X)   \r\n        t = time.time()\r\n        m.save(fname)\r\n        t = round(time.time() - t, 3)\r\n        del m\r\n        print \"iter\",i,\" model saved in time:\", t\r\n\r\n    print \"loading models:\"\r\n    for i in range(N):\r\n        fname = str(i)+\".hdf5\"\r\n        t = time.time()\r\n        m = keras.models.load_model(fname)\r\n        t = round(time.time()-t,3)\r\n        del m\r\n        print \"iter:\",i, \"time taken:\", t     \r\n\r\n```\r\n\r\n### Example output:\r\n```\r\nUsing TensorFlow backend.\r\n2018-09-11 20:36:51.754719: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports\r\n instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\r\niter 0  model saved in time: 0.027\r\niter 1  model saved in time: 0.039\r\niter 2  model saved in time: 0.054\r\niter 3  model saved in time: 0.065\r\niter 4  model saved in time: 0.073\r\niter 5  model saved in time: 0.088\r\niter 6  model saved in time: 0.101\r\niter 7  model saved in time: 0.108\r\niter 8  model saved in time: 0.125\r\niter 9  model saved in time: 0.139\r\nloading models:\r\niter: 0 time taken: 0.522\r\niter: 1 time taken: 0.559\r\niter: 2 time taken: 0.592\r\niter: 3 time taken: 0.691\r\niter: 4 time taken: 0.674\r\niter: 5 time taken: 0.711\r\niter: 6 time taken: 0.751\r\niter: 7 time taken: 0.788\r\niter: 8 time taken: 0.824\r\niter: 9 time taken: 0.874\r\n\r\n```\r\n\r\n"}}
{"actor":{"display_login":"gabrieldemarmiesse"}}