{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/840","repository_url":"https://api.github.com/repos/zfsonlinux/zfs","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/840/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/840/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/840/events","html_url":"https://github.com/zfsonlinux/zfs/issues/840","id":5705666,"number":840,"title":"Resilver restarting?","user":{"login":"chrisrd","id":1211482,"avatar_url":"https://avatars1.githubusercontent.com/u/1211482?v=4","gravatar_id":"","url":"https://api.github.com/users/chrisrd","html_url":"https://github.com/chrisrd","followers_url":"https://api.github.com/users/chrisrd/followers","following_url":"https://api.github.com/users/chrisrd/following{/other_user}","gists_url":"https://api.github.com/users/chrisrd/gists{/gist_id}","starred_url":"https://api.github.com/users/chrisrd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chrisrd/subscriptions","organizations_url":"https://api.github.com/users/chrisrd/orgs","repos_url":"https://api.github.com/users/chrisrd/repos","events_url":"https://api.github.com/users/chrisrd/events{/privacy}","received_events_url":"https://api.github.com/users/chrisrd/received_events","type":"User","site_admin":false},"labels":[{"id":123008,"url":"https://api.github.com/repos/zfsonlinux/zfs/labels/Documentation","name":"Documentation","color":"0b02e1","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":52,"created_at":"2012-07-19T05:31:15Z","updated_at":"2018-05-03T21:36:26Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I used `zpool replace ${pool} ${guid} ${dev}` to replace one dead and one dying disk, which started a resilver. I've been monitoring the progress and have seen it apparently restart, e.g.:\n\n```\nwhile true; do ts=$(date +\"%F-%T\"); echo -e \"$ts \\c\"; zpool status pool2 | egrep scanned; sleep 30; done\n...\n2012-07-19-13:55:09     1.59T scanned out of 29.9T at 286M/s, 28h49m to go\n2012-07-19-13:55:39     1.60T scanned out of 29.9T at 286M/s, 28h48m to go\n2012-07-19-13:56:09     53.7M scanned out of 29.9T at 1.99M/s, (scan is slow, no estimated time)\n2012-07-19-13:56:39     89.4M scanned out of 29.9T at 1.57M/s, (scan is slow, no estimated time)\n...\n```\n\nIn conjunction with the apparent restart there's a disk error reported in the `kern.log`:\n\n```\nJul 19 13:55:35 b5 kernel: [81414.117130] mpt2sas0: log_info(0x31080000): originator(PL), code(0x08), sub_code(0x0000)\nJul 19 13:55:35 b5 kernel: [81414.117194] sd 0:0:57:0: [sdbc] Unhandled sense code\nJul 19 13:55:35 b5 kernel: [81414.117223] sd 0:0:57:0: [sdbc]  Result: hostbyte=invalid driverbyte=DRIVER_SENSE\nJul 19 13:55:35 b5 kernel: [81414.117276] sd 0:0:57:0: [sdbc]  Sense Key : Medium Error [current] \nJul 19 13:55:35 b5 kernel: [81414.117311] Info fld=0x5066f2a4\nJul 19 13:55:35 b5 kernel: [81414.117333] sd 0:0:57:0: [sdbc]  Add. Sense: Unrecovered read error\nJul 19 13:55:35 b5 kernel: [81414.117366] sd 0:0:57:0: [sdbc] CDB: Read(10): 28 00 50 66 f2 6c 00 00 39 00\nJul 19 13:55:35 b5 kernel: [81414.117422] end_request: critical target error, dev sdbc, sector 1348924012\nJul 19 13:55:35 b5 kernel: [81414.117455] ZFS: zio error=121 type=1 offset=688647362560 size=29184 flags=20668 delay=7000\n```\n\n...where `sdbc` is one of the drives being replaced.\n\nThis has happened at least 4 or 5 times since the replaces were originally done some 24 hours ago, and the 3 times I've actually watched it happen have all been with errors on `sdbc` (at different sectors).  \n\nOn the other hand, there've been other disk errors, including (I think) on `sdbc`, that haven't caused the resilver to apparently restart.\n\nIs this normal, and/or can I expect the resilver to actually finish, or is it stuck in a loop?\n\nOh, I've just realised I have a few dumps of the complete `zfs status` taken during all this, which show that as far as the `in progress since...` message is concerned the resilver really is restarting. E.g. from this morning vs now:\n\n```\nscan: resilver in progress since Thu Jul 19 05:08:45 2012\nscan: resilver in progress since Thu Jul 19 14:47:10 2012\n```\n\nThe complete current `zpool status`, with the /dev/mapper devices annotated with their matching /dev device:\n\n```\n  pool: pool2\n state: DEGRADED\nstatus: One or more devices is currently being resilvered.  The pool will\n    continue to function, possibly in a degraded state.\naction: Wait for the resilver to complete.\n scan: resilver in progress since Thu Jul 19 14:47:10 2012\n    653G scanned out of 29.9T at 392M/s, 21h43m to go\n    39.4G resilvered, 2.13% done\nconfig:\n\n    NAME                                        STATE     READ WRITE CKSUM\n    pool2                                  DEGRADED     0     0     0\n      raidz2-0                                  ONLINE       0     0     0\n        sdbk  Hitachi_HDS5C30_ML0220F30E2DMD    ONLINE       0     0     0\n        sdbi  Hitachi_HDS5C30_ML0221F30B7ZND    ONLINE       0     0     0\n        sdac  Hitachi_HDS7230_MN1240F33XBYYD    ONLINE       0     0     0\n        sdad  Hitachi_HDS7230_MN1240FA03E4HD    ONLINE       0     0     0\n        sdab  Hitachi_HDS7230_MN1240FA0HUZAD    ONLINE       0     0     0\n        sdaa  Hitachi_HDS7230_MN1240FA0JXU2D    ONLINE       0     0     0\n        sdag  Hitachi_HDS7230_MN1240FA0K07YD    ONLINE       0     0     0\n        sdaf  Hitachi_HDS7230_MN1240FA0KP7YD    ONLINE       0     0     0\n        sdah  Hitachi_HDS7230_MN1240FA0LA36D    ONLINE       0     0     0\n        sdae  Hitachi_HDS7230_MN1240FA0LMP9D    ONLINE       0     0     0\n        sdz   Hitachi_HDS7230_MN1270FA0SV8ED    ONLINE       0     0     0\n      raidz2-1                                  ONLINE       0     0     0\n        sdl   ST2000DL003-9VT_5YD1V6ZL          ONLINE       0     0     0\n        sdaj  ST2000DL003-9VT_5YD1W04H          ONLINE       0     0     0\n        sdn   ST2000DL003-9VT_5YD1XEDS          ONLINE       0     0     0\n        sdi   ST2000DL003-9VT_5YD1XF0K          ONLINE       0     0     0\n        sdak  ST2000DL003-9VT_5YD1XQ15          ONLINE       0     0     0\n        sdo   ST2000DL003-9VT_5YD1Z9BZ          ONLINE       0     0     0\n        sdbf  WDC_WD20EARX-00_WD-WMAZA5589913   ONLINE       0     0  816K\n        sdai  ST2000DL003-9VT_5YD563JW          ONLINE       0     0     0\n        sds   ST2000DL003-9VT_5YD56MEZ          ONLINE       0     0     0\n        sdy   ST2000DL003-9VT_5YD5FWQ6          ONLINE       0     0     0\n        sdbs  WDC_WD20EARX-00_WD-WMAZA5567696   ONLINE       0     0     0\n      raidz2-2                             DEGRADED     3     0     0\n        sdbe  WDC_WD20EADS-00_WD-WCAVY1129416   ONLINE     110     0     1  (resilvering)\n        sdaz  WDC_WD20EADS-00_WD-WCAVY1162803   ONLINE     120     0     2\n              replacing-2                       ONLINE       0     0    39\n          sdbc  WDC_WD20EADS-00_WD-WCAVY1184604 ONLINE      70     0     0  (resilvering)\n          sdbj  WDC_WD20EARX-00_WD-WMAZA5603921 ONLINE       0     0     0  (resilvering)\n        sdbh  WDC_WD20EADS-00_WD-WCAVY1188798   ONLINE     158     0     0\n              replacing-4                       UNAVAIL      0     0     0\n                17433600781902467301            UNAVAIL      0     0     0  was /dev/mapper/WDC_WD20EADS-00_WD-WCAVY1188828\n          sdba  Hitachi_HDS7230_MN1240FA0LHJLD  ONLINE       0     0     0  (resilvering)\n        sdbr  WDC_WD20EADS-00_WD-WCAVY1188933   ONLINE     153     0     2\n        sdbt  WDC_WD20EADS-00_WD-WCAVY7404957   ONLINE       0     0     0\n        sdbg  WDC_WD20EARX-00_WD-WCAZA9871198   ONLINE       0     0     0\n        sdbq  WDC_WD20EARX-00_WD-WMAZA5346235   ONLINE       0     0     0\n        sdbp  WDC_WD20EARX-00_WD-WMAZA5534735   ONLINE       0     0     0\n        sdbl  WDC_WD20EARX-00_WD-WMAZA5561260   ONLINE       0     0     0\n\nerrors: No known data errors\n```\n"},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/386445038","html_url":"https://github.com/zfsonlinux/zfs/issues/840#issuecomment-386445038","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/840","id":386445038,"user":{"login":"StefanCohen","id":33824565,"avatar_url":"https://avatars3.githubusercontent.com/u/33824565?v=4","gravatar_id":"","url":"https://api.github.com/users/StefanCohen","html_url":"https://github.com/StefanCohen","followers_url":"https://api.github.com/users/StefanCohen/followers","following_url":"https://api.github.com/users/StefanCohen/following{/other_user}","gists_url":"https://api.github.com/users/StefanCohen/gists{/gist_id}","starred_url":"https://api.github.com/users/StefanCohen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StefanCohen/subscriptions","organizations_url":"https://api.github.com/users/StefanCohen/orgs","repos_url":"https://api.github.com/users/StefanCohen/repos","events_url":"https://api.github.com/users/StefanCohen/events{/privacy}","received_events_url":"https://api.github.com/users/StefanCohen/received_events","type":"User","site_admin":false},"created_at":"2018-05-03T21:36:26Z","updated_at":"2018-05-03T21:36:26Z","author_association":"NONE","body":"@4oo4 Yes. That is what I do."}}
{"actor":{"display_login":"StefanCohen"}}