{"push_id":1461506010,"size":5,"distinct_size":5,"ref":"refs/heads/master","head":"58d73bae74fd41e8a588e2ea0ab8b66adf072d9f","before":"9e5cedae2385750e3f01b85e9bc83a25a865195c","commits":[{"sha":"e32c7f1d72ff8eca8b1529c5bfc7a83aff8c6c9b","author":{"name":"David Pilato","email":"d8099b0f90e53c29d29eb12a7a3caa5af4f12681@pilato.fr"},"message":"Explain how to use bulk processor in a test context\n\nWhen using a bulk processor in test, you might write something like:\n\n```java\nBulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {\n    @Override public void beforeBulk(long executionId, BulkRequest request) {}\n    @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {}\n    @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) {}\n})\n        .setBulkActions(10000)\n        .setFlushInterval(TimeValue.timeValueSeconds(10))\n        .build();\n\nfor (int i = 0; i < 10000; i++) {\n    bulkProcessor.add(new IndexRequest(\"foo\", \"bar\", \"doc_\" + i)\n            .source(jsonBuilder().startObject().field(\"foo\", \"bar\").endObject()\n    ));\n}\n\nbulkProcessor.flush();\nclient.admin().indices().prepareRefresh(\"foo\").get();\nSearchResponse response = client.prepareSearch(\"foo\").get();\n// response does not contain any hit\n```\n\nThe problem is that by default bulkProcessor defines the number of concurrent requests to 1 which is using behind the scene an Async BulkRequestHandler.\nWhen you call `flush()` in a test, you expect it to flush all the content of the bulk so you can search for your docs.\nBut because of the async handling, there is a great chance that none of the documents has been indexed yet when you call the `refresh` method.\n\nWe should advice in our Java guide to explicitly set concurrent requests to `0` so users will use behind the scene the Sync BulkRequestHandler.\n\n```java\nBulkProcessor bulkProcessor = BulkProcessor.builder(client, new BulkProcessor.Listener() {\n    @Override public void beforeBulk(long executionId, BulkRequest request) {}\n    @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {}\n    @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) {}\n})\n        .setBulkActions(5000)\n        .setFlushInterval(TimeValue.timeValueSeconds(10))\n        .setConcurrentRequests(0)\n        .build();\n```\n\nCloses #22158.","distinct":true,"url":"https://api.github.com/repos/gfyoung/elasticsearch/commits/e32c7f1d72ff8eca8b1529c5bfc7a83aff8c6c9b"},{"sha":"efe5a75d263e5f4f32326d1ff079b81ac1dc23f1","author":{"name":"David Pilato","email":"d8099b0f90e53c29d29eb12a7a3caa5af4f12681@pilato.fr"},"message":"Merge remote-tracking branch 'origin/master'","distinct":true,"url":"https://api.github.com/repos/gfyoung/elasticsearch/commits/efe5a75d263e5f4f32326d1ff079b81ac1dc23f1"},{"sha":"0b338bf52394e49c6f4e653e8a6b7479dc71b561","author":{"name":"Simon Willnauer","email":"8718b1fe0e1a7f45303dfdcdba36c82b0922f90c@elasticsearch.com"},"message":"Cleanup random stats serialization code (#22223)\n\nSome of our stats serialization code duplicates complicated seriazliation logic\r\nor could use existing building blocks from StreamOutput/Input. This commit\r\ncleans up some of the serialization code.","distinct":true,"url":"https://api.github.com/repos/gfyoung/elasticsearch/commits/0b338bf52394e49c6f4e653e8a6b7479dc71b561"},{"sha":"1f3eb068d54f6752e0dac155f8137b6eabc725bb","author":{"name":"Simon Willnauer","email":"8718b1fe0e1a7f45303dfdcdba36c82b0922f90c@elasticsearch.com"},"message":"Add infrastructure to manage network connections outside of Transport/TransportService (#22194)\n\nSome expert users like UnicastZenPing today establishes real connections to nodes during it's ping\r\nphase that can be used by other parts of the system. Yet, this is potentially dangerous\r\nand undesirable unless the nodes have been fully verified and should be connected to in the\r\ncase of a cluster state update or if we join a newly elected master. For use-cases like this, this change adds the infrastructure to manually handle connections that are not publicly available on the node ie. should not be managed by `Transport`/`TransportSerivce`","distinct":true,"url":"https://api.github.com/repos/gfyoung/elasticsearch/commits/1f3eb068d54f6752e0dac155f8137b6eabc725bb"},{"sha":"58d73bae74fd41e8a588e2ea0ab8b66adf072d9f","author":{"name":"Jason Tedor","email":"158e602a35675d2a0b685478824aa5995557a235@tedor.me"},"message":"Tighten sequence numbers recovery\n\nThis commit touches addresses issues related to recovery and sequence numbers:\r\n - A sequence number can be assigned and a Lucene commit created with a\r\n   maximum sequence number at least as large as that sequence number,\r\n   yet the operation corresponding to that sequence number can be\r\n   missing from both the Lucene commit and the translog. This means that\r\n   upon recovery the local checkpoint will be stuck at or below this\r\n   missing sequence number. To address this, we force the local\r\n   checkpoint to the maximum sequence number in the Lucene commit when\r\n   opening the engine. Note that there can still be gaps in the history\r\n   in the translog but we do not address those here.\r\n - The global checkpoint is transferred to the target shard at the end\r\n   of peer recovery.\r\n - Additionally, we reenable the relocation integration tests.\r\n\r\nLastly, this work uncovered some bugs in the assignment of sequence\r\nnumbers on replica operations:\r\n - setting the sequence number on replica write requests was missing,\r\n   very likely introduced as a result of resolving merge conflicts\r\n - handling operations that arrive out of order on a replica and have a\r\n   version conflict with a previous operation were never marked as\r\n   processed\r\n\r\nRelates #22212","distinct":true,"url":"https://api.github.com/repos/gfyoung/elasticsearch/commits/58d73bae74fd41e8a588e2ea0ab8b66adf072d9f"}]}
{"actor":{"display_login":"gfyoung"}}