{"action":"created","comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments/180960129","pull_request_review_id":111473892,"id":180960129,"diff_hunk":"@@ -0,0 +1,74 @@\n+#! /bin/ksh -p\n+#\n+# CDDL HEADER START\n+#\n+# The contents of this file are subject to the terms of the\n+# Common Development and Distribution License (the \"License\").\n+# You may not use this file except in compliance with the License.\n+#\n+# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE\n+# or http://www.opensolaris.org/os/licensing.\n+# See the License for the specific language governing permissions\n+# and limitations under the License.\n+#\n+# When distributing Covered Code, include this CDDL HEADER in each\n+# file and include the License file at usr/src/OPENSOLARIS.LICENSE.\n+# If applicable, add the following below this CDDL HEADER, with the\n+# fields enclosed by brackets \"[]\" replaced with your own identifying\n+# information: Portions Copyright [yyyy] [name of copyright owner]\n+#\n+# CDDL HEADER END\n+#\n+\n+#\n+# Copyright (c) 2018 by Nutanix. All rights reserved.\n+#\n+\n+. $STF_SUITE/include/libtest.shlib\n+\n+#\n+# DESCRIPTION:\n+# Copy a large number of files between 2 directories\n+# within a zfs filesystem works without errors.\n+# This make sure zap upgrading and expanding works.\n+#\n+# STRATEGY:\n+#\n+# 1. Create NR_FILES files in directory src\n+# 2. Check the number of files is correct\n+# 3. Copy files from src to dst in readdir order\n+# 4. Check the number of files is correct\n+#\n+\n+verify_runnable \"global\"\n+\n+function cleanup\n+{\n+\trm -rf $TESTDIR/src $TESTDIR/dst\n+}\n+\n+log_assert \"Copy a large number of files between 2 directories\" \\\n+\t\"within a zfs filesystem works without errors\"\n+\n+log_onexit cleanup\n+\n+NR_FILES=60000\n+BATCH=1000\n+\n+log_must mkdir $TESTDIR/src\n+log_must mkdir $TESTDIR/dst\n+\n+WD=$(pwd)\n+cd $TESTDIR/src\n+# create NR_FILES in BATCH at a time to prevent overlowing argument buffer\n+for i in $(seq $(($NR_FILES/$BATCH))); do touch $(seq $((($i-1)*$BATCH+1)) $(($i*$BATCH))); done\n+cd $WD\n+\n+log_must test $NR_FILES -eq $(ls -U $TESTDIR/src | wc -l)\n+\n+# copy files from src to dst, use cp_files to make sure readdir order","path":"tests/zfs-tests/tests/functional/cp_files/cp_files_001_pos.ksh","position":69,"original_position":69,"commit_id":"2ac564614434456d3f85c40da0647cae52c9194d","original_commit_id":"2ac564614434456d3f85c40da0647cae52c9194d","user":{"login":"bunder2015","id":12985665,"avatar_url":"https://avatars3.githubusercontent.com/u/12985665?v=4","gravatar_id":"","url":"https://api.github.com/users/bunder2015","html_url":"https://github.com/bunder2015","followers_url":"https://api.github.com/users/bunder2015/followers","following_url":"https://api.github.com/users/bunder2015/following{/other_user}","gists_url":"https://api.github.com/users/bunder2015/gists{/gist_id}","starred_url":"https://api.github.com/users/bunder2015/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bunder2015/subscriptions","organizations_url":"https://api.github.com/users/bunder2015/orgs","repos_url":"https://api.github.com/users/bunder2015/repos","events_url":"https://api.github.com/users/bunder2015/events{/privacy}","received_events_url":"https://api.github.com/users/bunder2015/received_events","type":"User","site_admin":false},"body":"nit: s/make sure/ensure","created_at":"2018-04-12T04:04:37Z","updated_at":"2018-04-12T04:04:50Z","html_url":"https://github.com/zfsonlinux/zfs/pull/7421#discussion_r180960129","pull_request_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments/180960129"},"html":{"href":"https://github.com/zfsonlinux/zfs/pull/7421#discussion_r180960129"},"pull_request":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421"}}},"pull_request":{"url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421","id":180672949,"html_url":"https://github.com/zfsonlinux/zfs/pull/7421","diff_url":"https://github.com/zfsonlinux/zfs/pull/7421.diff","patch_url":"https://github.com/zfsonlinux/zfs/pull/7421.patch","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/7421","number":7421,"state":"open","locked":false,"title":"Fix ENOSPC in \"Handle zap_add() failures in mixed ...\"","user":{"login":"tuxoko","id":119672,"avatar_url":"https://avatars3.githubusercontent.com/u/119672?v=4","gravatar_id":"","url":"https://api.github.com/users/tuxoko","html_url":"https://github.com/tuxoko","followers_url":"https://api.github.com/users/tuxoko/followers","following_url":"https://api.github.com/users/tuxoko/following{/other_user}","gists_url":"https://api.github.com/users/tuxoko/gists{/gist_id}","starred_url":"https://api.github.com/users/tuxoko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tuxoko/subscriptions","organizations_url":"https://api.github.com/users/tuxoko/orgs","repos_url":"https://api.github.com/users/tuxoko/repos","events_url":"https://api.github.com/users/tuxoko/events{/privacy}","received_events_url":"https://api.github.com/users/tuxoko/received_events","type":"User","site_admin":false},"body":"Commit cc63068 caused ENOSPC error when copy a large amount of files\r\nbetween two directories. The reason is that the patch limits zap leaf\r\nexpansion to 2 retries, and return ENOSPC when failed.\r\n\r\nThe intent for limiting retries is to prevent pointlessly growing table\r\nto max size when adding a block full of entries with same name in\r\ndifferent case in mixed mode. However, it turns out we cannot use any\r\nlimit on the retry. When we copy files from one directory in readdir\r\norder, we are copying in hash order, one leaf block at a time. Which\r\nmeans that if the leaf block in source directory has expanded 6 times,\r\nand you copy those entries in that block, by the time you need to expand\r\nthe leaf in destination directory, you need to expand it 6 times in one\r\ngo. So any limit on the retry will result in error where it shouldn't.\r\n\r\nNote that while we do use different salt for different directories, it\r\nseems that the salt/hash function doesn't provide enough randomization\r\nto the hash distance to prevent this from happening.\r\n\r\nSince cc63068 has already been reverted. This patch adds it back and\r\nremoves the retry limit.\r\n\r\nAlso, as it turn out, failing on zap_add() has a serious side effect for\r\nmzap_upgrade(). When upgrading from micro zap to fat zap, it will\r\ncall zap_add() to transfer entries one at a time. If it hit any error\r\nhalfway through, the remaining entries will be lost, causing those files\r\nto become orphan. This patch add a VERIFY to catch it.\r\n\r\nSigned-off-by: Chunwei Chen <david.chen@nutanix.com>\r\n\r\n<!--- Provide a general summary of your changes in the Title above -->\r\n\r\n<!---\r\nDocumentation on ZFS Buildbot options can be found at\r\nhttps://github.com/zfsonlinux/zfs/wiki/Buildbot-Options\r\n-->\r\n\r\n### Description\r\n<!--- Describe your changes in detail -->\r\n\r\n### Motivation and Context\r\n#7401\r\n\r\n### How Has This Been Tested?\r\n<!--- Please describe in detail how you tested your changes. -->\r\n<!--- Include details of your testing environment, and the tests you ran to -->\r\n<!--- see how your change affects other areas of the code, etc. -->\r\n<!--- If your change is a performance enhancement, please provide benchmarks here. -->\r\n\r\n### Types of changes\r\n<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->\r\n- [x] Bug fix (non-breaking change which fixes an issue)\r\n- [ ] New feature (non-breaking change which adds functionality)\r\n- [ ] Performance enhancement (non-breaking change which improves efficiency)\r\n- [ ] Code cleanup (non-breaking change which makes code smaller or more readable)\r\n- [ ] Breaking change (fix or feature that would cause existing functionality to change)\r\n- [ ] Documentation (a change to man pages or other documentation)\r\n\r\n### Checklist:\r\n<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->\r\n<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->\r\n- [ ] My code follows the ZFS on Linux code style requirements.\r\n- [ ] I have updated the documentation accordingly.\r\n- [ ] I have read the **CONTRIBUTING** document.\r\n- [ ] I have added tests to cover my changes.\r\n- [ ] All new and existing tests passed.\r\n- [ ] All commit messages are properly formatted and contain `Signed-off-by`.\r\n- [ ] Change has been approved by a ZFS on Linux member.\r\n","created_at":"2018-04-10T17:21:33Z","updated_at":"2018-04-12T04:04:50Z","closed_at":null,"merged_at":null,"merge_commit_sha":"94012242134bfdadca10d7b924ed7a9a2887d0d0","assignee":null,"assignees":[],"requested_reviewers":[{"login":"ahrens","id":799124,"avatar_url":"https://avatars1.githubusercontent.com/u/799124?v=4","gravatar_id":"","url":"https://api.github.com/users/ahrens","html_url":"https://github.com/ahrens","followers_url":"https://api.github.com/users/ahrens/followers","following_url":"https://api.github.com/users/ahrens/following{/other_user}","gists_url":"https://api.github.com/users/ahrens/gists{/gist_id}","starred_url":"https://api.github.com/users/ahrens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ahrens/subscriptions","organizations_url":"https://api.github.com/users/ahrens/orgs","repos_url":"https://api.github.com/users/ahrens/repos","events_url":"https://api.github.com/users/ahrens/events{/privacy}","received_events_url":"https://api.github.com/users/ahrens/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421/commits","review_comments_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421/comments","review_comment_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/7421/comments","statuses_url":"https://api.github.com/repos/zfsonlinux/zfs/statuses/2ac564614434456d3f85c40da0647cae52c9194d","head":{"label":"tuxoko:i7401","ref":"i7401","sha":"2ac564614434456d3f85c40da0647cae52c9194d","user":{"login":"tuxoko","id":119672,"avatar_url":"https://avatars3.githubusercontent.com/u/119672?v=4","gravatar_id":"","url":"https://api.github.com/users/tuxoko","html_url":"https://github.com/tuxoko","followers_url":"https://api.github.com/users/tuxoko/followers","following_url":"https://api.github.com/users/tuxoko/following{/other_user}","gists_url":"https://api.github.com/users/tuxoko/gists{/gist_id}","starred_url":"https://api.github.com/users/tuxoko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tuxoko/subscriptions","organizations_url":"https://api.github.com/users/tuxoko/orgs","repos_url":"https://api.github.com/users/tuxoko/repos","events_url":"https://api.github.com/users/tuxoko/events{/privacy}","received_events_url":"https://api.github.com/users/tuxoko/received_events","type":"User","site_admin":false},"repo":{"id":15040965,"name":"zfs","full_name":"tuxoko/zfs","owner":{"login":"tuxoko","id":119672,"avatar_url":"https://avatars3.githubusercontent.com/u/119672?v=4","gravatar_id":"","url":"https://api.github.com/users/tuxoko","html_url":"https://github.com/tuxoko","followers_url":"https://api.github.com/users/tuxoko/followers","following_url":"https://api.github.com/users/tuxoko/following{/other_user}","gists_url":"https://api.github.com/users/tuxoko/gists{/gist_id}","starred_url":"https://api.github.com/users/tuxoko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tuxoko/subscriptions","organizations_url":"https://api.github.com/users/tuxoko/orgs","repos_url":"https://api.github.com/users/tuxoko/repos","events_url":"https://api.github.com/users/tuxoko/events{/privacy}","received_events_url":"https://api.github.com/users/tuxoko/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/tuxoko/zfs","description":"Native ZFS for Linux","fork":true,"url":"https://api.github.com/repos/tuxoko/zfs","forks_url":"https://api.github.com/repos/tuxoko/zfs/forks","keys_url":"https://api.github.com/repos/tuxoko/zfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/tuxoko/zfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/tuxoko/zfs/teams","hooks_url":"https://api.github.com/repos/tuxoko/zfs/hooks","issue_events_url":"https://api.github.com/repos/tuxoko/zfs/issues/events{/number}","events_url":"https://api.github.com/repos/tuxoko/zfs/events","assignees_url":"https://api.github.com/repos/tuxoko/zfs/assignees{/user}","branches_url":"https://api.github.com/repos/tuxoko/zfs/branches{/branch}","tags_url":"https://api.github.com/repos/tuxoko/zfs/tags","blobs_url":"https://api.github.com/repos/tuxoko/zfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/tuxoko/zfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/tuxoko/zfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/tuxoko/zfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/tuxoko/zfs/statuses/{sha}","languages_url":"https://api.github.com/repos/tuxoko/zfs/languages","stargazers_url":"https://api.github.com/repos/tuxoko/zfs/stargazers","contributors_url":"https://api.github.com/repos/tuxoko/zfs/contributors","subscribers_url":"https://api.github.com/repos/tuxoko/zfs/subscribers","subscription_url":"https://api.github.com/repos/tuxoko/zfs/subscription","commits_url":"https://api.github.com/repos/tuxoko/zfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/tuxoko/zfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/tuxoko/zfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/tuxoko/zfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/tuxoko/zfs/contents/{+path}","compare_url":"https://api.github.com/repos/tuxoko/zfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/tuxoko/zfs/merges","archive_url":"https://api.github.com/repos/tuxoko/zfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/tuxoko/zfs/downloads","issues_url":"https://api.github.com/repos/tuxoko/zfs/issues{/number}","pulls_url":"https://api.github.com/repos/tuxoko/zfs/pulls{/number}","milestones_url":"https://api.github.com/repos/tuxoko/zfs/milestones{/number}","notifications_url":"https://api.github.com/repos/tuxoko/zfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/tuxoko/zfs/labels{/name}","releases_url":"https://api.github.com/repos/tuxoko/zfs/releases{/id}","deployments_url":"https://api.github.com/repos/tuxoko/zfs/deployments","created_at":"2013-12-09T07:30:34Z","updated_at":"2017-04-17T20:50:55Z","pushed_at":"2018-04-11T21:14:26Z","git_url":"git://github.com/tuxoko/zfs.git","ssh_url":"git@github.com:tuxoko/zfs.git","clone_url":"https://github.com/tuxoko/zfs.git","svn_url":"https://github.com/tuxoko/zfs","homepage":"http://zfsonlinux.org/","size":46782,"stargazers_count":2,"watchers_count":2,"language":"C","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":0,"open_issues":0,"watchers":2,"default_branch":"master"}},"base":{"label":"zfsonlinux:master","ref":"master","sha":"dfb1ad027f23f65bfe47493e195c18565bce4148","user":{"login":"zfsonlinux","id":575083,"avatar_url":"https://avatars3.githubusercontent.com/u/575083?v=4","gravatar_id":"","url":"https://api.github.com/users/zfsonlinux","html_url":"https://github.com/zfsonlinux","followers_url":"https://api.github.com/users/zfsonlinux/followers","following_url":"https://api.github.com/users/zfsonlinux/following{/other_user}","gists_url":"https://api.github.com/users/zfsonlinux/gists{/gist_id}","starred_url":"https://api.github.com/users/zfsonlinux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zfsonlinux/subscriptions","organizations_url":"https://api.github.com/users/zfsonlinux/orgs","repos_url":"https://api.github.com/users/zfsonlinux/repos","events_url":"https://api.github.com/users/zfsonlinux/events{/privacy}","received_events_url":"https://api.github.com/users/zfsonlinux/received_events","type":"Organization","site_admin":false},"repo":{"id":437011,"name":"zfs","full_name":"zfsonlinux/zfs","owner":{"login":"zfsonlinux","id":575083,"avatar_url":"https://avatars3.githubusercontent.com/u/575083?v=4","gravatar_id":"","url":"https://api.github.com/users/zfsonlinux","html_url":"https://github.com/zfsonlinux","followers_url":"https://api.github.com/users/zfsonlinux/followers","following_url":"https://api.github.com/users/zfsonlinux/following{/other_user}","gists_url":"https://api.github.com/users/zfsonlinux/gists{/gist_id}","starred_url":"https://api.github.com/users/zfsonlinux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zfsonlinux/subscriptions","organizations_url":"https://api.github.com/users/zfsonlinux/orgs","repos_url":"https://api.github.com/users/zfsonlinux/repos","events_url":"https://api.github.com/users/zfsonlinux/events{/privacy}","received_events_url":"https://api.github.com/users/zfsonlinux/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/zfsonlinux/zfs","description":"ZFS on Linux - the official OpenZFS implementation for Linux.","fork":false,"url":"https://api.github.com/repos/zfsonlinux/zfs","forks_url":"https://api.github.com/repos/zfsonlinux/zfs/forks","keys_url":"https://api.github.com/repos/zfsonlinux/zfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zfsonlinux/zfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zfsonlinux/zfs/teams","hooks_url":"https://api.github.com/repos/zfsonlinux/zfs/hooks","issue_events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/events{/number}","events_url":"https://api.github.com/repos/zfsonlinux/zfs/events","assignees_url":"https://api.github.com/repos/zfsonlinux/zfs/assignees{/user}","branches_url":"https://api.github.com/repos/zfsonlinux/zfs/branches{/branch}","tags_url":"https://api.github.com/repos/zfsonlinux/zfs/tags","blobs_url":"https://api.github.com/repos/zfsonlinux/zfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zfsonlinux/zfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zfsonlinux/zfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/zfsonlinux/zfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zfsonlinux/zfs/statuses/{sha}","languages_url":"https://api.github.com/repos/zfsonlinux/zfs/languages","stargazers_url":"https://api.github.com/repos/zfsonlinux/zfs/stargazers","contributors_url":"https://api.github.com/repos/zfsonlinux/zfs/contributors","subscribers_url":"https://api.github.com/repos/zfsonlinux/zfs/subscribers","subscription_url":"https://api.github.com/repos/zfsonlinux/zfs/subscription","commits_url":"https://api.github.com/repos/zfsonlinux/zfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/zfsonlinux/zfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/zfsonlinux/zfs/contents/{+path}","compare_url":"https://api.github.com/repos/zfsonlinux/zfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zfsonlinux/zfs/merges","archive_url":"https://api.github.com/repos/zfsonlinux/zfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zfsonlinux/zfs/downloads","issues_url":"https://api.github.com/repos/zfsonlinux/zfs/issues{/number}","pulls_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls{/number}","milestones_url":"https://api.github.com/repos/zfsonlinux/zfs/milestones{/number}","notifications_url":"https://api.github.com/repos/zfsonlinux/zfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/labels{/name}","releases_url":"https://api.github.com/repos/zfsonlinux/zfs/releases{/id}","deployments_url":"https://api.github.com/repos/zfsonlinux/zfs/deployments","created_at":"2009-12-14T20:20:34Z","updated_at":"2018-04-12T01:52:39Z","pushed_at":"2018-04-12T03:50:51Z","git_url":"git://github.com/zfsonlinux/zfs.git","ssh_url":"git@github.com:zfsonlinux/zfs.git","clone_url":"https://github.com/zfsonlinux/zfs.git","svn_url":"https://github.com/zfsonlinux/zfs","homepage":"http://zfsonlinux.org/","size":45863,"stargazers_count":3193,"watchers_count":3193,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":744,"mirror_url":null,"archived":false,"open_issues_count":934,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":744,"open_issues":934,"watchers":3193,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421"},"html":{"href":"https://github.com/zfsonlinux/zfs/pull/7421"},"issue":{"href":"https://api.github.com/repos/zfsonlinux/zfs/issues/7421"},"comments":{"href":"https://api.github.com/repos/zfsonlinux/zfs/issues/7421/comments"},"review_comments":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421/comments"},"review_comment":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7421/commits"},"statuses":{"href":"https://api.github.com/repos/zfsonlinux/zfs/statuses/2ac564614434456d3f85c40da0647cae52c9194d"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"bunder2015"}}