{"action":"created","issue":{"url":"https://api.github.com/repos/olivere/elastic/issues/766","repository_url":"https://api.github.com/repos/olivere/elastic","labels_url":"https://api.github.com/repos/olivere/elastic/issues/766/labels{/name}","comments_url":"https://api.github.com/repos/olivere/elastic/issues/766/comments","events_url":"https://api.github.com/repos/olivere/elastic/issues/766/events","html_url":"https://github.com/olivere/elastic/issues/766","id":319053134,"number":766,"title":"Reset Bulk Service when body cannot be serialized","user":{"login":"rwynn","id":260672,"avatar_url":"https://avatars2.githubusercontent.com/u/260672?v=4","gravatar_id":"","url":"https://api.github.com/users/rwynn","html_url":"https://github.com/rwynn","followers_url":"https://api.github.com/users/rwynn/followers","following_url":"https://api.github.com/users/rwynn/following{/other_user}","gists_url":"https://api.github.com/users/rwynn/gists{/gist_id}","starred_url":"https://api.github.com/users/rwynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rwynn/subscriptions","organizations_url":"https://api.github.com/users/rwynn/orgs","repos_url":"https://api.github.com/users/rwynn/repos","events_url":"https://api.github.com/users/rwynn/events{/privacy}","received_events_url":"https://api.github.com/users/rwynn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-04-30T22:10:20Z","updated_at":"2018-05-01T21:02:03Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"### Which version of Elastic are you using?\r\n\r\n[ ] elastic.v2 (for Elasticsearch 1.x)\r\n[ ] elastic.v3 (for Elasticsearch 2.x)\r\n[X ] elastic.v5 (for Elasticsearch 5.x)\r\n[X ] elastic.v6 (for Elasticsearch 6.x)\r\n\r\n### Please describe the expected behavior\r\nThe bulk service should be reset when it encounters an error serializing its body. E.g. when the input contains a value of NaN, +/- Infinity, or a time.Time with a year outside of [0, 9999].\r\n\r\nhttps://github.com/olivere/elastic/blob/release-branch.v6/bulk.go#L202\r\n\r\nOtherwise, the service (when used in conjunction with a bulk processor) gets stuck in an endless error loop since it is only reset after a request is completed.  The request will never happen if the input is invalid.\r\n\r\nI'm not sure if it should be the responsibility of the bulk service or the bulk processor to break out of this loop.\r\n\r\n### Please describe the actual behavior\r\n\r\nWhen the bulk service is used with a bulk processor, once a request is encountered that will not serialize, the bulk processor will not be able to reset the bulk service and continually get errors back.  \r\n\r\n### Any steps to reproduce the behavior?\r\n\r\nA document with a float64 that passes `math.IsNaN()` or `math.IsInf(var, 0)` or a time.Time with a year outside of `[0,9999]` will trigger this error loop since those values trigger a JSON serialization error.  \r\n"},"comment":{"url":"https://api.github.com/repos/olivere/elastic/issues/comments/385789200","html_url":"https://github.com/olivere/elastic/issues/766#issuecomment-385789200","issue_url":"https://api.github.com/repos/olivere/elastic/issues/766","id":385789200,"user":{"login":"rwynn","id":260672,"avatar_url":"https://avatars2.githubusercontent.com/u/260672?v=4","gravatar_id":"","url":"https://api.github.com/users/rwynn","html_url":"https://github.com/rwynn","followers_url":"https://api.github.com/users/rwynn/followers","following_url":"https://api.github.com/users/rwynn/following{/other_user}","gists_url":"https://api.github.com/users/rwynn/gists{/gist_id}","starred_url":"https://api.github.com/users/rwynn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rwynn/subscriptions","organizations_url":"https://api.github.com/users/rwynn/orgs","repos_url":"https://api.github.com/users/rwynn/repos","events_url":"https://api.github.com/users/rwynn/events{/privacy}","received_events_url":"https://api.github.com/users/rwynn/received_events","type":"User","site_admin":false},"created_at":"2018-05-01T21:02:02Z","updated_at":"2018-05-01T21:02:02Z","author_association":"CONTRIBUTOR","body":"@olivere I would need to verify on the backoff stategy but I think the infinite loop would still exist.  The bulk processor reuses the configured number of worker/service pairs created when the processor is started.  The only path to reset the requests slice for those services is forward: to be able to form request, transmit, and parse response.\r\n\r\nEven if auto flush, **retry**, etc are completely turned off in the processor, the requests slice in the service will still continue to grow after this condition proportionate to the user calling Add().\r\n\r\nThe only remediation the user has in this case is to `Stop` and `Start` the processor which re-creates the worker/service pairs.\r\n\r\nI get why the request actions are not throw away, but maybe the default should be to throw them away in `Do` unless a specific type of error is received - like a net error...\r\n\r\nhttps://github.com/olivere/elastic/blob/release-branch.v6/bulk_processor.go#L480\r\n\r\nBecause of all the errors that can happen, a communication error is the only I can think of that would be transient in this case. \r\n    "}}
{"actor":{"display_login":"rwynn"}}