{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/9442","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/9442/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/9442/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/9442/events","html_url":"https://github.com/elastic/elasticsearch/issues/9442","id":55623415,"node_id":"MDU6SXNzdWU1NTYyMzQxNQ==","number":9442,"title":"Highlighting sometimes returns too much text (orders of magnitude larger than fragment_size) ","user":{"login":"scharron","id":400720,"node_id":"MDQ6VXNlcjQwMDcyMA==","avatar_url":"https://avatars0.githubusercontent.com/u/400720?v=4","gravatar_id":"","url":"https://api.github.com/users/scharron","html_url":"https://github.com/scharron","followers_url":"https://api.github.com/users/scharron/followers","following_url":"https://api.github.com/users/scharron/following{/other_user}","gists_url":"https://api.github.com/users/scharron/gists{/gist_id}","starred_url":"https://api.github.com/users/scharron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/scharron/subscriptions","organizations_url":"https://api.github.com/users/scharron/orgs","repos_url":"https://api.github.com/users/scharron/repos","events_url":"https://api.github.com/users/scharron/events{/privacy}","received_events_url":"https://api.github.com/users/scharron/received_events","type":"User","site_admin":false},"labels":[{"id":111549183,"node_id":"MDU6TGFiZWwxMTE1NDkxODM=","url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Search/Highlighting","name":":Search/Highlighting","color":"0e8a16","default":false},{"id":23173,"node_id":"MDU6TGFiZWwyMzE3Mw==","url":"https://api.github.com/repos/elastic/elasticsearch/labels/%3Ebug","name":">bug","color":"b60205","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":15,"created_at":"2015-01-27T14:23:17Z","updated_at":"2019-06-28T05:50:41Z","closed_at":"2016-11-25T16:35:09Z","author_association":"NONE","body":"Index mapping : \n\n``` json\n{\n    \"mappings\" : {\n        \"test\": {\n            \"properties\": {\n                \"content\" : {\n                    \"type\" : \"string\",\n                    \"analyzer\" : \"french\"\n                }\n            }\n        }\n    }\n}\n```\n\nDocument inserted : \nhttps://gist.github.com/scharron/684a4fbab85135c203ee\n\nQuery : \n\n``` json\n{\n  \"query\": {\n    \"filtered\": {\n      \"query\": {\n        \"query_string\": {\n          \"query\": \"\\\"Vlaams Brabant\\\"\",\n          \"fields\": [\n            \"content\"\n          ]\n        }\n      }\n    }\n  },\n  \"highlight\": {\n    \"fragment_size\": 100,\n    \"fields\": {\n      \"content\": {}\n    }\n  }\n}\n```\n\nI would expect to get about 200 characters in 2 fragments, but instead I get 12KB of data in only one fragment.\nI also tried the `postings` and the `fast vector` highlighters with the same result.\nThe behaviour is the same for both an ES 1.4.2 and ES 1.3.4.\n\nIt may be related to other curious behaviours : http://stackoverflow.com/questions/28167990/curious-behaviour-of-fragment-size-in-elasticsearch-highlighting?noredirect=1\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/506610347","html_url":"https://github.com/elastic/elasticsearch/issues/9442#issuecomment-506610347","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/9442","id":506610347,"node_id":"MDEyOklzc3VlQ29tbWVudDUwNjYxMDM0Nw==","user":{"login":"choco-love","id":52056939,"node_id":"MDQ6VXNlcjUyMDU2OTM5","avatar_url":"https://avatars1.githubusercontent.com/u/52056939?v=4","gravatar_id":"","url":"https://api.github.com/users/choco-love","html_url":"https://github.com/choco-love","followers_url":"https://api.github.com/users/choco-love/followers","following_url":"https://api.github.com/users/choco-love/following{/other_user}","gists_url":"https://api.github.com/users/choco-love/gists{/gist_id}","starred_url":"https://api.github.com/users/choco-love/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/choco-love/subscriptions","organizations_url":"https://api.github.com/users/choco-love/orgs","repos_url":"https://api.github.com/users/choco-love/repos","events_url":"https://api.github.com/users/choco-love/events{/privacy}","received_events_url":"https://api.github.com/users/choco-love/received_events","type":"User","site_admin":false},"created_at":"2019-06-28T05:50:41Z","updated_at":"2019-06-28T05:50:41Z","author_association":"NONE","body":"> I had the same issue and was not able to work around.\r\n> \r\n> * Plain highlighter has this bug, sometimes returns too much.\r\n> * Postings highlighter doesn't support fragment_size.\r\n> * FVH [doesn't work on nested fields](https://github.com/elastic/elasticsearch/issues/19265) ¯\\⁠_(ツ)_/⁠¯\r\n> \r\n> So basically `fragment_size` is unusable. I had to set `number_of_fragments: 0` to get all the text, then implement my own highlight-aware truncating in code.\r\n\r\nhow did you fix your problem???"}}
{"actor":{"display_login":"choco-love"}}