{"action":"closed","issue":{"url":"https://api.github.com/repos/gluster/glusterd2/issues/1079","repository_url":"https://api.github.com/repos/gluster/glusterd2","labels_url":"https://api.github.com/repos/gluster/glusterd2/issues/1079/labels{/name}","comments_url":"https://api.github.com/repos/gluster/glusterd2/issues/1079/comments","events_url":"https://api.github.com/repos/gluster/glusterd2/issues/1079/events","html_url":"https://github.com/gluster/glusterd2/issues/1079","id":345095417,"node_id":"MDU6SXNzdWUzNDUwOTU0MTc=","number":1079,"title":"Batch storing volume info in etcd as single etcd txn","user":{"login":"prashanthpai","id":411294,"node_id":"MDQ6VXNlcjQxMTI5NA==","avatar_url":"https://avatars1.githubusercontent.com/u/411294?v=4","gravatar_id":"","url":"https://api.github.com/users/prashanthpai","html_url":"https://github.com/prashanthpai","followers_url":"https://api.github.com/users/prashanthpai/followers","following_url":"https://api.github.com/users/prashanthpai/following{/other_user}","gists_url":"https://api.github.com/users/prashanthpai/gists{/gist_id}","starred_url":"https://api.github.com/users/prashanthpai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prashanthpai/subscriptions","organizations_url":"https://api.github.com/users/prashanthpai/orgs","repos_url":"https://api.github.com/users/prashanthpai/repos","events_url":"https://api.github.com/users/prashanthpai/events{/privacy}","received_events_url":"https://api.github.com/users/prashanthpai/received_events","type":"User","site_admin":false},"labels":[{"id":1005825846,"node_id":"MDU6TGFiZWwxMDA1ODI1ODQ2","url":"https://api.github.com/repos/gluster/glusterd2/labels/FA:%20observability","name":"FA: observability","color":"bfdadc","default":false},{"id":507041753,"node_id":"MDU6TGFiZWw1MDcwNDE3NTM=","url":"https://api.github.com/repos/gluster/glusterd2/labels/FW:%20Central%20Store","name":"FW: Central Store","color":"fbca04","default":false},{"id":507041838,"node_id":"MDU6TGFiZWw1MDcwNDE4Mzg=","url":"https://api.github.com/repos/gluster/glusterd2/labels/FW:%20Transaction","name":"FW: Transaction","color":"fbca04","default":false},{"id":546077723,"node_id":"MDU6TGFiZWw1NDYwNzc3MjM=","url":"https://api.github.com/repos/gluster/glusterd2/labels/FW:%20Volume%20Management","name":"FW: Volume Management","color":"fbca04","default":false},{"id":971990657,"node_id":"MDU6TGFiZWw5NzE5OTA2NTc=","url":"https://api.github.com/repos/gluster/glusterd2/labels/GCS/1.0-Blocker","name":"GCS/1.0-Blocker","color":"c96a0c","default":false},{"id":763218947,"node_id":"MDU6TGFiZWw3NjMyMTg5NDc=","url":"https://api.github.com/repos/gluster/glusterd2/labels/priority:%20high","name":"priority: high","color":"96152e","default":false},{"id":937176238,"node_id":"MDU6TGFiZWw5MzcxNzYyMzg=","url":"https://api.github.com/repos/gluster/glusterd2/labels/usability","name":"usability","color":"280377","default":false}],"state":"closed","locked":false,"assignee":{"login":"prashanthpai","id":411294,"node_id":"MDQ6VXNlcjQxMTI5NA==","avatar_url":"https://avatars1.githubusercontent.com/u/411294?v=4","gravatar_id":"","url":"https://api.github.com/users/prashanthpai","html_url":"https://github.com/prashanthpai","followers_url":"https://api.github.com/users/prashanthpai/followers","following_url":"https://api.github.com/users/prashanthpai/following{/other_user}","gists_url":"https://api.github.com/users/prashanthpai/gists{/gist_id}","starred_url":"https://api.github.com/users/prashanthpai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prashanthpai/subscriptions","organizations_url":"https://api.github.com/users/prashanthpai/orgs","repos_url":"https://api.github.com/users/prashanthpai/repos","events_url":"https://api.github.com/users/prashanthpai/events{/privacy}","received_events_url":"https://api.github.com/users/prashanthpai/received_events","type":"User","site_admin":false},"assignees":[{"login":"prashanthpai","id":411294,"node_id":"MDQ6VXNlcjQxMTI5NA==","avatar_url":"https://avatars1.githubusercontent.com/u/411294?v=4","gravatar_id":"","url":"https://api.github.com/users/prashanthpai","html_url":"https://github.com/prashanthpai","followers_url":"https://api.github.com/users/prashanthpai/followers","following_url":"https://api.github.com/users/prashanthpai/following{/other_user}","gists_url":"https://api.github.com/users/prashanthpai/gists{/gist_id}","starred_url":"https://api.github.com/users/prashanthpai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prashanthpai/subscriptions","organizations_url":"https://api.github.com/users/prashanthpai/orgs","repos_url":"https://api.github.com/users/prashanthpai/repos","events_url":"https://api.github.com/users/prashanthpai/events{/privacy}","received_events_url":"https://api.github.com/users/prashanthpai/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2018-07-27T05:29:27Z","updated_at":"2018-12-12T09:55:10Z","closed_at":"2018-12-12T09:55:10Z","author_association":"CONTRIBUTOR","body":"A glance at the following screenshot by @sseshasa reveals that `storeVolinfo` takes a lot of time:\r\n\r\n![screenshot_20180725_141008](https://user-images.githubusercontent.com/36185324/43245169-f7c88aa6-90ca-11e8-976a-b76c82d9cb26.png)\r\n\r\nThe originator node txn step `storeVolinfo` took **68% of time taken** to complete entire volume operation.\r\n\r\n```go\r\n// storeVolInfo uses to store the volinfo based on key and to generate client volfile\r\nfunc storeVolInfo(c transaction.TxnCtx, key string) error {\r\n        var volinfo volume.Volinfo\r\n        if err := c.Get(key, &volinfo); err != nil {\r\n                c.Logger().WithError(err).WithField(\r\n                        \"key\", \"volinfo\").Debug(\"Failed to get key from store\")\r\n                return err\r\n        }\r\n\r\n        if err := volume.AddOrUpdateVolumeFunc(&volinfo); err != nil {\r\n                c.Logger().WithError(err).WithField(\r\n                        \"volume\", volinfo.Name).Debug(\"failed to store volume info\")\r\n                return err\r\n        }\r\n\r\n        if err := volgen.Generate(); err != nil {\r\n                c.Logger().WithError(err).WithField(\r\n                        \"volume\", volinfo.Name).Debug(\"failed to generate volfiles\")\r\n                return err\r\n        }\r\n\r\n        return nil\r\n}\r\n```\r\n\r\nWe can batch all PUTs (including those from volgen) originating from `storeVolinfo` to etcd as a single etcd txn. May require some code changes there but totally worth it considering the disproportionate amount of time it takes."}}
{"actor":{"display_login":"aravindavk"}}