{"action":"created","issue":{"url":"https://api.github.com/repos/ucbrise/clipper/issues/549","repository_url":"https://api.github.com/repos/ucbrise/clipper","labels_url":"https://api.github.com/repos/ucbrise/clipper/issues/549/labels{/name}","comments_url":"https://api.github.com/repos/ucbrise/clipper/issues/549/comments","events_url":"https://api.github.com/repos/ucbrise/clipper/issues/549/events","html_url":"https://github.com/ucbrise/clipper/issues/549","id":350258454,"node_id":"MDU6SXNzdWUzNTAyNTg0NTQ=","number":549,"title":"[Bug] Memory leak occurred in QueryFrontend when all the user requests met SLO","user":{"login":"withsmilo","id":3915543,"node_id":"MDQ6VXNlcjM5MTU1NDM=","avatar_url":"https://avatars2.githubusercontent.com/u/3915543?v=4","gravatar_id":"","url":"https://api.github.com/users/withsmilo","html_url":"https://github.com/withsmilo","followers_url":"https://api.github.com/users/withsmilo/followers","following_url":"https://api.github.com/users/withsmilo/following{/other_user}","gists_url":"https://api.github.com/users/withsmilo/gists{/gist_id}","starred_url":"https://api.github.com/users/withsmilo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/withsmilo/subscriptions","organizations_url":"https://api.github.com/users/withsmilo/orgs","repos_url":"https://api.github.com/users/withsmilo/repos","events_url":"https://api.github.com/users/withsmilo/events{/privacy}","received_events_url":"https://api.github.com/users/withsmilo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-08-14T02:26:55Z","updated_at":"2018-08-15T00:09:29Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"To reproduce this bug, try below steps.\r\n\r\n### Step1 : Start hello-world sample (cache_size=0, slo_micros=100)\r\n```python\r\nfrom clipper_admin import ClipperConnection, DockerContainerManager\r\nfrom clipper_admin.deployers import python as python_deployer\r\n\r\ndef feature_sum(xs):\r\n    return [str(sum(x)) for x in xs]\r\n\r\nclipper_conn = ClipperConnection(DockerContainerManager())\r\nclipper_conn.start_clipper(cache_size=0)  # Disable PredictionCache\r\nclipper_conn.register_application(name=\"hello-world\", input_type=\"doubles\", default_output=\"-1.0\", slo_micros=100)\r\npython_deployer.deploy_python_closure(clipper_conn, name=\"sum-model\", version=1, input_type=\"doubles\", func=feature_sum)\r\nclipper_conn.link_model_to_app(app_name=\"hello-world\", model_name=\"sum-model\")\r\n```\r\n\r\n### Step2 : Login to QueryFrontend\r\n```\r\n$ docker exec -it <QueryFrontend_Container_ID> /bin/bash\r\n```\r\n\r\n### Step3 : Check current memory usage. (Approx. 19 MiB)\r\n```\r\nroot@6ddeb1430e8b:/# cat /sys/fs/cgroup/memory/memory.stat | grep -w rss\r\nrss 20230144\r\n```\r\n\r\n### Step4 : Send many requests to the QueryFrontend\r\n```python\r\nimport requests, json, numpy as np\r\nheaders = {\"Content-type\": \"application/json\"}\r\nfor i in range(10000):\r\n    requests.post(\"http://localhost:1337/hello-world/predict\", headers=headers, data=json.dumps({\"input\": list(np.random.random(10))})).json()\r\n```\r\n\r\n### Step5 : Check that all the requests met SLO\r\n```\r\n.....\r\n{u'use_default': True, u'output': -1.0, u'query_id': 9996, u'default_explanation': u'Failed to retrieve a prediction response within the specified latency SLO'}\r\n{u'use_default': True, u'output': -1.0, u'query_id': 9997, u'default_explanation': u'Failed to retrieve a prediction response within the specified latency SLO'}\r\n{u'use_default': True, u'output': -1.0, u'query_id': 9998, u'default_explanation': u'Failed to retrieve a prediction response within the specified latency SLO'}\r\n{u'use_default': True, u'output': -1.0, u'query_id': 9999, u'default_explanation': u'Failed to retrieve a prediction response within the specified latency SLO'}\r\n>>>\r\n```\r\n\r\n### Step5 : Check current memory usage. (Approx. 39 MiB)\r\n```\r\nroot@6ddeb1430e8b:/# cat /sys/fs/cgroup/memory/memory.stat | grep -w rss\r\nrss 41529344\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/ucbrise/clipper/issues/comments/413054682","html_url":"https://github.com/ucbrise/clipper/issues/549#issuecomment-413054682","issue_url":"https://api.github.com/repos/ucbrise/clipper/issues/549","id":413054682,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzA1NDY4Mg==","user":{"login":"simon-mo","id":21118851,"node_id":"MDQ6VXNlcjIxMTE4ODUx","avatar_url":"https://avatars3.githubusercontent.com/u/21118851?v=4","gravatar_id":"","url":"https://api.github.com/users/simon-mo","html_url":"https://github.com/simon-mo","followers_url":"https://api.github.com/users/simon-mo/followers","following_url":"https://api.github.com/users/simon-mo/following{/other_user}","gists_url":"https://api.github.com/users/simon-mo/gists{/gist_id}","starred_url":"https://api.github.com/users/simon-mo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simon-mo/subscriptions","organizations_url":"https://api.github.com/users/simon-mo/orgs","repos_url":"https://api.github.com/users/simon-mo/repos","events_url":"https://api.github.com/users/simon-mo/events{/privacy}","received_events_url":"https://api.github.com/users/simon-mo/received_events","type":"User","site_admin":false},"created_at":"2018-08-15T00:09:29Z","updated_at":"2018-08-15T00:09:29Z","author_association":"CONTRIBUTOR","body":"Evict entry should be called on `put` when the cache buffer is full. But the cache size of 0. \r\n\r\nhttps://github.com/ucbrise/clipper/blob/80c97d27a38d60caaebb2a1ae6a995dd7ff1c82d/src/libclipper/src/task_executor.cpp#L78\r\n\r\nI’ll investigate it. "}}
{"actor":{"display_login":"simon-mo"}}