{"action":"closed","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/30432","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/30432/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/30432/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/30432/events","html_url":"https://github.com/elastic/elasticsearch/issues/30432","id":320895934,"number":30432,"title":"[CI] SplitIndexIT.testSplitIndexPrimaryTerm Failure on Windows","user":{"login":"jdconrad","id":2126764,"avatar_url":"https://avatars2.githubusercontent.com/u/2126764?v=4","gravatar_id":"","url":"https://api.github.com/users/jdconrad","html_url":"https://github.com/jdconrad","followers_url":"https://api.github.com/users/jdconrad/followers","following_url":"https://api.github.com/users/jdconrad/following{/other_user}","gists_url":"https://api.github.com/users/jdconrad/gists{/gist_id}","starred_url":"https://api.github.com/users/jdconrad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jdconrad/subscriptions","organizations_url":"https://api.github.com/users/jdconrad/orgs","repos_url":"https://api.github.com/users/jdconrad/repos","events_url":"https://api.github.com/users/jdconrad/events{/privacy}","received_events_url":"https://api.github.com/users/jdconrad/received_events","type":"User","site_admin":false},"labels":[{"id":163824881,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Core/Index%20APIs","name":":Core/Index APIs","color":"0e8a16","default":false},{"id":148612629,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/jenkins","name":"jenkins","color":"207de5","default":false},{"id":912911731,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v6.4.0","name":"v6.4.0","color":"DDDDDD","default":false},{"id":484855464,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v7.0.0","name":"v7.0.0","color":"dddddd","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-05-07T17:55:22Z","updated_at":"2018-05-09T13:59:30Z","closed_at":"2018-05-09T13:59:30Z","author_association":"CONTRIBUTOR","body":"Unable to reproduce the failure locally on Arch.  It looks like it may be a Windows file system issue based on the console log.\r\n\r\nhttps://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-windows-compatibility/1612/console\r\n\r\nReproduce:\r\n```\r\ngradlew :server:integTest \\\r\n  -Dtests.seed=64C5D7D608CE1D96 \\\r\n  -Dtests.class=org.elasticsearch.action.admin.indices.create.SplitIndexIT \\\r\n  -Dtests.method=\"testSplitIndexPrimaryTerm\" \\\r\n  -Dtests.security.manager=true \\\r\n  -Dtests.locale=be \\\r\n  -Dtests.timezone=Indian/Chagos\r\n```\r\n\r\nStack:\r\n```\r\n17:00:40   1> java.nio.file.NoSuchFileException: C:\\Users\\jenkins\\workspace\\elastic+elasticsearch+master+multijob-windows-compatibility\\server\\build\\testrun\\integTest\\J1\\temp\\org.elasticsearch.action.admin.indices.create.SplitIndexIT_64C5D7D608CE1D96-001\\tempDir-002\\data\\nodes\\1\\indices\\xKokK_BuT-apELfAZW8bIQ\\4\\index\\_0.cfs\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) ~[?:?]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.DisableFsyncFS.newFileChannel(DisableFsyncFS.java:45) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:166) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:166) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat java.nio.channels.FileChannel.open(FileChannel.java:287) ~[?:1.8.0_162]\r\n17:00:40   1> \tat java.nio.channels.FileChannel.open(FileChannel.java:335) ~[?:1.8.0_162]\r\n17:00:40   1> \tat org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238) ~[lucene-core-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:13]\r\n17:00:40   1> \tat org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:99) ~[lucene-core-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:13]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.checksumFromLuceneFile(Store.java:912) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.loadMetadata(Store.java:878) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.<init>(Store.java:797) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.getMetadata(Store.java:293) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.getMetadata(Store.java:258) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.cleanupAndVerify(Store.java:689) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.RecoveryTarget.cleanFiles(RecoveryTarget.java:440) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.PeerRecoveryTargetService$CleanFilesRequestHandler.messageReceived(PeerRecoveryTargetService.java:556) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.PeerRecoveryTargetService$CleanFilesRequestHandler.messageReceived(PeerRecoveryTargetService.java:550) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1603) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:724) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [main/:?]\r\n17:00:40   1> \tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]\r\n17:00:40   1> \tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]\r\n17:00:40   1> \tat java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]\r\n17:00:40   1> [2018-05-07T23:00:27,722][WARN ][o.e.i.c.IndicesClusterStateService] [node_sd1] [[second_split][4]] marking and sending shard failed due to [failed recovery]\r\n17:00:40 \r\n17:00:40   1> org.elasticsearch.indices.recovery.RecoveryFailedException: [second_split][4]: Recovery failed from {node_sd3}{NLTLyjrDQwuBMTwWDThvbQ}{b3m9isxqQZ29FSiO0_Bk3Q}{127.0.0.1}{127.0.0.1:65365} into {node_sd1}{TwB1E1dsTOqzEmdemTSSFA}{ZGobC3lTRlmButqR-U53Eg}{127.0.0.1}{127.0.0.1:62751} (failed to clean after recovery)\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.RecoveryTarget.cleanFiles(RecoveryTarget.java:468) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.PeerRecoveryTargetService$CleanFilesRequestHandler.messageReceived(PeerRecoveryTargetService.java:556) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.PeerRecoveryTargetService$CleanFilesRequestHandler.messageReceived(PeerRecoveryTargetService.java:550) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:30) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1603) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:724) [main/:?]\r\n17:00:40   1> \tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [main/:?]\r\n17:00:40   1> \tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]\r\n17:00:40   1> \tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]\r\n17:00:40   1> \tat java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]\r\n17:00:40   1> Caused by: java.nio.file.NoSuchFileException: C:\\Users\\jenkins\\workspace\\elastic+elasticsearch+master+multijob-windows-compatibility\\server\\build\\testrun\\integTest\\J1\\temp\\org.elasticsearch.action.admin.indices.create.SplitIndexIT_64C5D7D608CE1D96-001\\tempDir-002\\data\\nodes\\1\\indices\\xKokK_BuT-apELfAZW8bIQ\\4\\index\\_0.cfs\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) ~[?:?]\r\n17:00:40   1> \tat sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115) ~[?:?]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.DisableFsyncFS.newFileChannel(DisableFsyncFS.java:45) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:166) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:166) ~[lucene-test-framework-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:16]\r\n17:00:40   1> \tat java.nio.channels.FileChannel.open(FileChannel.java:287) ~[?:1.8.0_162]\r\n17:00:40   1> \tat java.nio.channels.FileChannel.open(FileChannel.java:335) ~[?:1.8.0_162]\r\n17:00:40   1> \tat org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238) ~[lucene-core-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:13]\r\n17:00:40   1> \tat org.apache.lucene.store.FilterDirectory.openInput(FilterDirectory.java:99) ~[lucene-core-7.4.0-snapshot-1ed95c097b.jar:7.4.0-snapshot-1ed95c097b 1ed95c097b82ee5f175e93f3fe62572abe064da6 - jimczi - 2018-05-03 16:16:13]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.checksumFromLuceneFile(Store.java:912) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.loadMetadata(Store.java:878) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store$MetadataSnapshot.<init>(Store.java:797) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.getMetadata(Store.java:293) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.getMetadata(Store.java:258) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.index.store.Store.cleanupAndVerify(Store.java:689) ~[main/:?]\r\n17:00:40   1> \tat org.elasticsearch.indices.recovery.RecoveryTarget.cleanFiles(RecoveryTarget.java:440) [main/:?]\r\n17:00:40   1> \t... 10 more\r\n```"}}
{"actor":{"display_login":"bleskes"}}