<p dir="auto">I'm seeing NPE's occur on random shards while performing a snapshot. Cluster Health throughout the process is green with very little CPU usage and heap. I usually try and run another snapshot after it finishes and the previous failed shards will complete and a new set will fail.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="host                 ip        heap.percent ram.percent load node.role master name
&lt;redacted&gt;    127.0.1.1           55          59 0.38 d         *      Caldari
&lt;redacted&gt; 127.0.1.1           40          56 0.01 d         m      Amarr
&lt;redacted&gt; 127.0.1.1           44          56 0.01 d         m      Minmatar
&lt;redacted&gt; 127.0.1.1           30          66 0.00 d         m      Gallente"><pre class="notranslate"><code class="notranslate">host                 ip        heap.percent ram.percent load node.role master name
&lt;redacted&gt;    127.0.1.1           55          59 0.38 d         *      Caldari
&lt;redacted&gt; 127.0.1.1           40          56 0.01 d         m      Amarr
&lt;redacted&gt; 127.0.1.1           44          56 0.01 d         m      Minmatar
&lt;redacted&gt; 127.0.1.1           30          66 0.00 d         m      Gallente
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2014-10-31 15:35:31,771][WARN ][snapshots                ] [Amarr] [[&lt;idx_name&gt;][2]] [s3_backups:1414767797] failed to create snapshot
org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException: [&lt;idx_name&gt;][2] null
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:141)
    at org.elasticsearch.index.snapshots.IndexShardSnapshotAndRestoreService.snapshot(IndexShardSnapshotAndRestoreService.java:86)
    at org.elasticsearch.snapshots.SnapshotsService$6.run(SnapshotsService.java:829)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:456)
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:131)
    ... 5 more"><pre class="notranslate"><code class="notranslate">[2014-10-31 15:35:31,771][WARN ][snapshots                ] [Amarr] [[&lt;idx_name&gt;][2]] [s3_backups:1414767797] failed to create snapshot
org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException: [&lt;idx_name&gt;][2] null
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:141)
    at org.elasticsearch.index.snapshots.IndexShardSnapshotAndRestoreService.snapshot(IndexShardSnapshotAndRestoreService.java:86)
    at org.elasticsearch.snapshots.SnapshotsService$6.run(SnapshotsService.java:829)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NullPointerException
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:456)
    at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:131)
    ... 5 more
</code></pre></div>
<p dir="auto">A snapshot run failed with <code class="notranslate">IndexShardSnapshotFailedException</code> caused by an NPE for a lot of shards:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2014-07-30 18:11:56,063][WARN ][snapshots                ] [node.name] [[index.name][274]] [node.name:index.name] failed to create snapshot
org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException: [index.name][274] null
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:141)
        at org.elasticsearch.index.snapshots.IndexShardSnapshotAndRestoreService.snapshot(IndexShardSnapshotAndRestoreService.java:86)
        at org.elasticsearch.snapshots.SnapshotsService$6.run(SnapshotsService.java:829)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:456)
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:131)
        ... 5 more"><pre class="notranslate"><code class="notranslate">[2014-07-30 18:11:56,063][WARN ][snapshots                ] [node.name] [[index.name][274]] [node.name:index.name] failed to create snapshot
org.elasticsearch.index.snapshots.IndexShardSnapshotFailedException: [index.name][274] null
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:141)
        at org.elasticsearch.index.snapshots.IndexShardSnapshotAndRestoreService.snapshot(IndexShardSnapshotAndRestoreService.java:86)
        at org.elasticsearch.snapshots.SnapshotsService$6.run(SnapshotsService.java:829)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository$SnapshotContext.snapshot(BlobStoreIndexShardRepository.java:456)
        at org.elasticsearch.index.snapshots.blobstore.BlobStoreIndexShardRepository.snapshot(BlobStoreIndexShardRepository.java:131)
        ... 5 more
</code></pre></div>
<p dir="auto">This is an index with 500 shards, snapshotting to an NFS mount.</p>