{"action":"closed","number":107,"pull_request":{"url":"https://api.github.com/repos/openzfs/openzfs/pulls/107","id":70966941,"html_url":"https://github.com/openzfs/openzfs/pull/107","diff_url":"https://github.com/openzfs/openzfs/pull/107.diff","patch_url":"https://github.com/openzfs/openzfs/pull/107.patch","issue_url":"https://api.github.com/repos/openzfs/openzfs/issues/107","number":107,"state":"closed","locked":false,"title":"6988 spa_sync() spends half its time in dmu_objset_do_userquota_updates","user":{"login":"ahrens","id":799124,"avatar_url":"https://avatars.githubusercontent.com/u/799124?v=3","gravatar_id":"","url":"https://api.github.com/users/ahrens","html_url":"https://github.com/ahrens","followers_url":"https://api.github.com/users/ahrens/followers","following_url":"https://api.github.com/users/ahrens/following{/other_user}","gists_url":"https://api.github.com/users/ahrens/gists{/gist_id}","starred_url":"https://api.github.com/users/ahrens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahrens/subscriptions","organizations_url":"https://api.github.com/users/ahrens/orgs","repos_url":"https://api.github.com/users/ahrens/repos","events_url":"https://api.github.com/users/ahrens/events{/privacy}","received_events_url":"https://api.github.com/users/ahrens/received_events","type":"User","site_admin":false},"body":"Using a benchmark which creates 2 million files in one TXG, I observe\r\nthat the thread running spa_sync() is on CPU almost the entire time we\r\nare syncing, and therefore can be a performance bottleneck. About 50% of\r\nthe time in spa_sync() is in dmu_objset_do_userquota_updates().\r\n\r\nThe problem is that dmu_objset_do_userquota_updates() calls\r\nzap_increment_int(DMU_USERUSED_OBJECT) once for every file that was\r\nmodified (or created). In this benchmark, all the files are owned by the\r\nsame user/group, so all 2 million calls to zap_increment_int() are\r\nmodifying the same entry in the zap. The same issue exists for the\r\nDMU_GROUPUSED_OBJECT.\r\n\r\nWe should keep an in-memory map from user to space delta while we are\r\nsyncing, and when we finish, iterate over the in-memory map and modify\r\nthe ZAP once per entry. This reduces the number of calls to\r\nzap_increment_int() from \"number of objects modified\" to \"number of\r\nowners/groups of modified files\".\r\n\r\nThis reduced the time spent in spa_sync() in the file create benchmark\r\nby ~33%, from 11 seconds to 7 seconds.\r\n\r\nChanges sponsored by Intel Corp.\r\n\r\nUpstream bugs: DLPX-44799\r\nLinux bug: https://github.com/zfsonlinux/zfs/issues/4642","created_at":"2016-05-22T03:37:41Z","updated_at":"2016-09-28T18:00:44Z","closed_at":"2016-09-28T18:00:44Z","merged_at":null,"merge_commit_sha":"fa3a2246f0bf8a0053371ccb64e22d0681685b53","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/openzfs/openzfs/pulls/107/commits","review_comments_url":"https://api.github.com/repos/openzfs/openzfs/pulls/107/comments","review_comment_url":"https://api.github.com/repos/openzfs/openzfs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/openzfs/openzfs/issues/107/comments","statuses_url":"https://api.github.com/repos/openzfs/openzfs/statuses/30f26df367b5876949923d40b3b73ce477a093a5","head":{"label":"ahrens:DLPX-44799-spa_sync-spends-half-its-time-in-dmu_objset_do_userquota_updates","ref":"DLPX-44799-spa_sync-spends-half-its-time-in-dmu_objset_do_userquota_updates","sha":"30f26df367b5876949923d40b3b73ce477a093a5","user":{"login":"ahrens","id":799124,"avatar_url":"https://avatars.githubusercontent.com/u/799124?v=3","gravatar_id":"","url":"https://api.github.com/users/ahrens","html_url":"https://github.com/ahrens","followers_url":"https://api.github.com/users/ahrens/followers","following_url":"https://api.github.com/users/ahrens/following{/other_user}","gists_url":"https://api.github.com/users/ahrens/gists{/gist_id}","starred_url":"https://api.github.com/users/ahrens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahrens/subscriptions","organizations_url":"https://api.github.com/users/ahrens/orgs","repos_url":"https://api.github.com/users/ahrens/repos","events_url":"https://api.github.com/users/ahrens/events{/privacy}","received_events_url":"https://api.github.com/users/ahrens/received_events","type":"User","site_admin":false},"repo":{"id":10103472,"name":"illumos","full_name":"ahrens/illumos","owner":{"login":"ahrens","id":799124,"avatar_url":"https://avatars.githubusercontent.com/u/799124?v=3","gravatar_id":"","url":"https://api.github.com/users/ahrens","html_url":"https://github.com/ahrens","followers_url":"https://api.github.com/users/ahrens/followers","following_url":"https://api.github.com/users/ahrens/following{/other_user}","gists_url":"https://api.github.com/users/ahrens/gists{/gist_id}","starred_url":"https://api.github.com/users/ahrens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahrens/subscriptions","organizations_url":"https://api.github.com/users/ahrens/orgs","repos_url":"https://api.github.com/users/ahrens/repos","events_url":"https://api.github.com/users/ahrens/events{/privacy}","received_events_url":"https://api.github.com/users/ahrens/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/ahrens/illumos","description":"works in progress based on illumos","fork":true,"url":"https://api.github.com/repos/ahrens/illumos","forks_url":"https://api.github.com/repos/ahrens/illumos/forks","keys_url":"https://api.github.com/repos/ahrens/illumos/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ahrens/illumos/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ahrens/illumos/teams","hooks_url":"https://api.github.com/repos/ahrens/illumos/hooks","issue_events_url":"https://api.github.com/repos/ahrens/illumos/issues/events{/number}","events_url":"https://api.github.com/repos/ahrens/illumos/events","assignees_url":"https://api.github.com/repos/ahrens/illumos/assignees{/user}","branches_url":"https://api.github.com/repos/ahrens/illumos/branches{/branch}","tags_url":"https://api.github.com/repos/ahrens/illumos/tags","blobs_url":"https://api.github.com/repos/ahrens/illumos/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ahrens/illumos/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ahrens/illumos/git/refs{/sha}","trees_url":"https://api.github.com/repos/ahrens/illumos/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ahrens/illumos/statuses/{sha}","languages_url":"https://api.github.com/repos/ahrens/illumos/languages","stargazers_url":"https://api.github.com/repos/ahrens/illumos/stargazers","contributors_url":"https://api.github.com/repos/ahrens/illumos/contributors","subscribers_url":"https://api.github.com/repos/ahrens/illumos/subscribers","subscription_url":"https://api.github.com/repos/ahrens/illumos/subscription","commits_url":"https://api.github.com/repos/ahrens/illumos/commits{/sha}","git_commits_url":"https://api.github.com/repos/ahrens/illumos/git/commits{/sha}","comments_url":"https://api.github.com/repos/ahrens/illumos/comments{/number}","issue_comment_url":"https://api.github.com/repos/ahrens/illumos/issues/comments{/number}","contents_url":"https://api.github.com/repos/ahrens/illumos/contents/{+path}","compare_url":"https://api.github.com/repos/ahrens/illumos/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ahrens/illumos/merges","archive_url":"https://api.github.com/repos/ahrens/illumos/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ahrens/illumos/downloads","issues_url":"https://api.github.com/repos/ahrens/illumos/issues{/number}","pulls_url":"https://api.github.com/repos/ahrens/illumos/pulls{/number}","milestones_url":"https://api.github.com/repos/ahrens/illumos/milestones{/number}","notifications_url":"https://api.github.com/repos/ahrens/illumos/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ahrens/illumos/labels{/name}","releases_url":"https://api.github.com/repos/ahrens/illumos/releases{/id}","deployments_url":"https://api.github.com/repos/ahrens/illumos/deployments","created_at":"2013-05-16T14:50:32Z","updated_at":"2016-04-22T02:33:59Z","pushed_at":"2016-09-24T17:51:25Z","git_url":"git://github.com/ahrens/illumos.git","ssh_url":"git@github.com:ahrens/illumos.git","clone_url":"https://github.com/ahrens/illumos.git","svn_url":"https://github.com/ahrens/illumos","homepage":"","size":273706,"stargazers_count":2,"watchers_count":2,"language":"C","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":2,"default_branch":"dpp"}},"base":{"label":"openzfs:master","ref":"master","sha":"ec740b054b8de7985649ab4bd36c32cd0ebd6d6f","user":{"login":"openzfs","id":4510897,"avatar_url":"https://avatars.githubusercontent.com/u/4510897?v=3","gravatar_id":"","url":"https://api.github.com/users/openzfs","html_url":"https://github.com/openzfs","followers_url":"https://api.github.com/users/openzfs/followers","following_url":"https://api.github.com/users/openzfs/following{/other_user}","gists_url":"https://api.github.com/users/openzfs/gists{/gist_id}","starred_url":"https://api.github.com/users/openzfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/openzfs/subscriptions","organizations_url":"https://api.github.com/users/openzfs/orgs","repos_url":"https://api.github.com/users/openzfs/repos","events_url":"https://api.github.com/users/openzfs/events{/privacy}","received_events_url":"https://api.github.com/users/openzfs/received_events","type":"Organization","site_admin":false},"repo":{"id":44220580,"name":"openzfs","full_name":"openzfs/openzfs","owner":{"login":"openzfs","id":4510897,"avatar_url":"https://avatars.githubusercontent.com/u/4510897?v=3","gravatar_id":"","url":"https://api.github.com/users/openzfs","html_url":"https://github.com/openzfs","followers_url":"https://api.github.com/users/openzfs/followers","following_url":"https://api.github.com/users/openzfs/following{/other_user}","gists_url":"https://api.github.com/users/openzfs/gists{/gist_id}","starred_url":"https://api.github.com/users/openzfs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/openzfs/subscriptions","organizations_url":"https://api.github.com/users/openzfs/orgs","repos_url":"https://api.github.com/users/openzfs/repos","events_url":"https://api.github.com/users/openzfs/events{/privacy}","received_events_url":"https://api.github.com/users/openzfs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/openzfs/openzfs","description":"OpenZFS is a storage platform that encompasses the functionality of traditional filesystems and volume managers, delivering enterprise reliability, modern functionality, and consistent performance in an easy to administer package on several operating system platforms.","fork":true,"url":"https://api.github.com/repos/openzfs/openzfs","forks_url":"https://api.github.com/repos/openzfs/openzfs/forks","keys_url":"https://api.github.com/repos/openzfs/openzfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/openzfs/openzfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/openzfs/openzfs/teams","hooks_url":"https://api.github.com/repos/openzfs/openzfs/hooks","issue_events_url":"https://api.github.com/repos/openzfs/openzfs/issues/events{/number}","events_url":"https://api.github.com/repos/openzfs/openzfs/events","assignees_url":"https://api.github.com/repos/openzfs/openzfs/assignees{/user}","branches_url":"https://api.github.com/repos/openzfs/openzfs/branches{/branch}","tags_url":"https://api.github.com/repos/openzfs/openzfs/tags","blobs_url":"https://api.github.com/repos/openzfs/openzfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/openzfs/openzfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/openzfs/openzfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/openzfs/openzfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/openzfs/openzfs/statuses/{sha}","languages_url":"https://api.github.com/repos/openzfs/openzfs/languages","stargazers_url":"https://api.github.com/repos/openzfs/openzfs/stargazers","contributors_url":"https://api.github.com/repos/openzfs/openzfs/contributors","subscribers_url":"https://api.github.com/repos/openzfs/openzfs/subscribers","subscription_url":"https://api.github.com/repos/openzfs/openzfs/subscription","commits_url":"https://api.github.com/repos/openzfs/openzfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/openzfs/openzfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/openzfs/openzfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/openzfs/openzfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/openzfs/openzfs/contents/{+path}","compare_url":"https://api.github.com/repos/openzfs/openzfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/openzfs/openzfs/merges","archive_url":"https://api.github.com/repos/openzfs/openzfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/openzfs/openzfs/downloads","issues_url":"https://api.github.com/repos/openzfs/openzfs/issues{/number}","pulls_url":"https://api.github.com/repos/openzfs/openzfs/pulls{/number}","milestones_url":"https://api.github.com/repos/openzfs/openzfs/milestones{/number}","notifications_url":"https://api.github.com/repos/openzfs/openzfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/openzfs/openzfs/labels{/name}","releases_url":"https://api.github.com/repos/openzfs/openzfs/releases{/id}","deployments_url":"https://api.github.com/repos/openzfs/openzfs/deployments","created_at":"2015-10-14T03:03:50Z","updated_at":"2016-09-27T18:24:20Z","pushed_at":"2016-09-28T18:00:39Z","git_url":"git://github.com/openzfs/openzfs.git","ssh_url":"git@github.com:openzfs/openzfs.git","clone_url":"https://github.com/openzfs/openzfs.git","svn_url":"https://github.com/openzfs/openzfs","homepage":"http://open-zfs.org","size":258593,"stargazers_count":91,"watchers_count":91,"language":"C","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":34,"mirror_url":null,"open_issues_count":38,"forks":34,"open_issues":38,"watchers":91,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/openzfs/openzfs/pulls/107"},"html":{"href":"https://github.com/openzfs/openzfs/pull/107"},"issue":{"href":"https://api.github.com/repos/openzfs/openzfs/issues/107"},"comments":{"href":"https://api.github.com/repos/openzfs/openzfs/issues/107/comments"},"review_comments":{"href":"https://api.github.com/repos/openzfs/openzfs/pulls/107/comments"},"review_comment":{"href":"https://api.github.com/repos/openzfs/openzfs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/openzfs/openzfs/pulls/107/commits"},"statuses":{"href":"https://api.github.com/repos/openzfs/openzfs/statuses/30f26df367b5876949923d40b3b73ce477a093a5"}},"merged":false,"mergeable":false,"mergeable_state":"dirty","merged_by":null,"comments":3,"review_comments":0,"commits":2,"additions":86,"deletions":20,"changed_files":1}}
{"actor":{"display_login":"ahrens"}}