title: Kafka - Broker fails because all log dirs have failed
body: <p>I am attempting a simple kafka config on Windows os.  My zookeeper and kafka installations use default configs (except for data and log dir paths).  I can start kafka and produce/consume messages without issue; however, when the broker attempts to delete old messages (I set log retention to 100 ms), I get the following error:</p>
  
  <pre><code>    java.nio.file.FileSystemException: C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log -&gt; C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log.deleted: The process cannot access the file because it is being used by another process.
  
          at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
          at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
          at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:387)
          at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
          at java.nio.file.Files.move(Files.java:1395)
          at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:697)
          at org.apache.kafka.common.record.FileRecords.renameTo(FileRecords.java:212)
          at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:415)
          at kafka.log.Log.kafka$log$Log$$asyncDeleteSegment(Log.scala:1601)
          at kafka.log.Log.kafka$log$Log$$deleteSegment(Log.scala:1588)
          at kafka.log.Log$$anonfun$deleteSegments$1$$anonfun$apply$mcI$sp$1.apply(Log.scala:1170)
          at kafka.log.Log$$anonfun$deleteSegments$1$$anonfun$apply$mcI$sp$1.apply(Log.scala:1170)
          at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
          at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
          at kafka.log.Log$$anonfun$deleteSegments$1.apply$mcI$sp(Log.scala:1170)
          at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
          at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:1161)
          at kafka.log.Log.maybeHandleIOException(Log.scala:1678)
          at kafka.log.Log.deleteSegments(Log.scala:1161)
          at kafka.log.Log.deleteOldSegments(Log.scala:1156)
          at kafka.log.Log.deleteRetentionMsBreachedSegments(Log.scala:1228)
          at kafka.log.Log.deleteOldSegments(Log.scala:1222)
          at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:854)
          at kafka.log.LogManager$$anonfun$cleanupLogs$3.apply(LogManager.scala:852)
          at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
          at scala.collection.immutable.List.foreach(List.scala:392)
          at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
          at kafka.log.LogManager.cleanupLogs(LogManager.scala:852)
          at kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:385)
          at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
          at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:62)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:748)
          Suppressed: java.nio.file.FileSystemException: C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log -&gt; C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log.deleted: The process cannot access the file because it is being used by another process.
  
                  at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
                  at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
                  at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
                  at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
                  at java.nio.file.Files.move(Files.java:1395)
                  at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:694)
                  ... 32 more
  
  And then...
  
  [2018-08-01 18:14:01,479] INFO [ReplicaManager broker=0] Stopping serving replicas in dir C:\Workspace\kafka_2.11-1.1.0\kafka-logs (kafka.server.ReplicaManager)
  [2018-08-01 18:14:01,480] ERROR Uncaught exception in scheduled task 'kafka-log-retention' (kafka.utils.KafkaScheduler)
  org.apache.kafka.common.errors.KafkaStorageException: Error while deleting segments for discrete-topic-0 in dir C:\Workspace\kafka_2.11-1.1.0\kafka-logs
  Caused by: java.nio.file.FileSystemException: C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log -&gt; C:\Workspace\kafka_2.11-1.1.0\kafka-logs\discrete-topic-0\00000000000000000000.log.deleted: The process cannot access the file because it is being used by another process.
  [2018-08-01 18:14:01,504] INFO Stopping serving logs in dir C:\Workspace\kafka_2.11-1.1.0\kafka-logs (kafka.log.LogManager)
  [2018-08-01 18:14:01,508] ERROR Shutdown broker because all log dirs in C:\Workspace\kafka_2.11-1.1.0\kafka-logs have failed (kafka.log.LogManager)
  </code></pre>
  
  <p>This seems to be this issue:  <a href="https://issues.apache.org/jira/browse/KAFKA-6188" rel="noreferrer">https://issues.apache.org/jira/browse/KAFKA-6188</a> </p>
  
  <p>kafka version: kafka_2.11-1.1.0
  Windows 10
  Zookeeper version: 3.4.12</p>
  
  <p>I have a few questions:</p>
  
  <ol>
  <li>Has anyone seen this issue and is there a workaround?</li>
  <li>Is running a kafka broker on Windows a viable options at this point?  I know it is not supported for Windows and there seem to be multiple critical issues.</li>
  </ol>
  
  <p>Thanks</p>