{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/21970","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/21970/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/21970/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/21970/events","html_url":"https://github.com/elastic/elasticsearch/issues/21970","id":193469662,"number":21970,"title":"Using stored script faces “Too many dynamic script compilations within one minute” error.","user":{"login":"xxx111x1","id":13516071,"avatar_url":"https://avatars.githubusercontent.com/u/13516071?v=3","gravatar_id":"","url":"https://api.github.com/users/xxx111x1","html_url":"https://github.com/xxx111x1","followers_url":"https://api.github.com/users/xxx111x1/followers","following_url":"https://api.github.com/users/xxx111x1/following{/other_user}","gists_url":"https://api.github.com/users/xxx111x1/gists{/gist_id}","starred_url":"https://api.github.com/users/xxx111x1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xxx111x1/subscriptions","organizations_url":"https://api.github.com/users/xxx111x1/orgs","repos_url":"https://api.github.com/users/xxx111x1/repos","events_url":"https://api.github.com/users/xxx111x1/events{/privacy}","received_events_url":"https://api.github.com/users/xxx111x1/received_events","type":"User","site_admin":false},"labels":[{"id":146834791,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Scripting","name":":Scripting","color":"009800","default":false},{"id":111624690,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/feedback_needed","name":"feedback_needed","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":{"login":"dakrone","id":19060,"avatar_url":"https://avatars.githubusercontent.com/u/19060?v=3","gravatar_id":"","url":"https://api.github.com/users/dakrone","html_url":"https://github.com/dakrone","followers_url":"https://api.github.com/users/dakrone/followers","following_url":"https://api.github.com/users/dakrone/following{/other_user}","gists_url":"https://api.github.com/users/dakrone/gists{/gist_id}","starred_url":"https://api.github.com/users/dakrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dakrone/subscriptions","organizations_url":"https://api.github.com/users/dakrone/orgs","repos_url":"https://api.github.com/users/dakrone/repos","events_url":"https://api.github.com/users/dakrone/events{/privacy}","received_events_url":"https://api.github.com/users/dakrone/received_events","type":"User","site_admin":false},"assignees":[{"login":"dakrone","id":19060,"avatar_url":"https://avatars.githubusercontent.com/u/19060?v=3","gravatar_id":"","url":"https://api.github.com/users/dakrone","html_url":"https://github.com/dakrone","followers_url":"https://api.github.com/users/dakrone/followers","following_url":"https://api.github.com/users/dakrone/following{/other_user}","gists_url":"https://api.github.com/users/dakrone/gists{/gist_id}","starred_url":"https://api.github.com/users/dakrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dakrone/subscriptions","organizations_url":"https://api.github.com/users/dakrone/orgs","repos_url":"https://api.github.com/users/dakrone/repos","events_url":"https://api.github.com/users/dakrone/events{/privacy}","received_events_url":"https://api.github.com/users/dakrone/received_events","type":"User","site_admin":false}],"milestone":null,"comments":6,"created_at":"2016-12-05T11:11:50Z","updated_at":"2016-12-07T22:12:09Z","closed_at":null,"body":"**Elasticsearch version**: 5.0\r\n**Plugins installed**: [X-pack]\r\n**JVM version**: 1.8.0_111\r\n**OS version**: Ubuntu 14.04\r\n\r\n**Description of the problem including expected versus actual behavior**:\r\nHi Team,\r\nWe are facing \"on_failure_message=[script] Too many dynamic script compilations within one minute, max: [32/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute\" error. and we are using stored script. as you can see below. \r\n\r\nWe also change the \"script.max_compilations_per_minute\" to have a try, but it seems not work. Our index rate is around 15k/s, and the failed is around 20%.\r\n\r\n**Configuration**: \r\n```\r\n{\r\n  \"logstashLogs\": {\r\n    \"description\": \"Get device id and user id\",\r\n    \"processors\": [\r\n      {\r\n        \"script\": {\r\n          \"lang\": \"painless\",\r\n          \"id\": \"index-device-and-app\"\r\n        }\r\n      },\r\n      {\r\n        \"script\": {\r\n          \"lang\": \"painless\",\r\n          \"id\": \"extract-app-stat\"\r\n        }\r\n      }\r\n    ],\r\n    \"on_failure\": [\r\n      {\r\n        \"set\": {\r\n          \"field\": \"_index\",\r\n          \"value\": \"failed-{{ _index }}\"\r\n        }\r\n      },\r\n      {\r\n        \"set\": {\r\n          \"field\": \"_error\",\r\n          \"value\": \"{{ _ingest }}\"\r\n        }\r\n      }\r\n    ]\r\n  }\r\n}\r\n```\r\n**Scripts**:\r\nindex-device-and-app.painless:\r\n```\r\nif (ctx._index.startsWith('logstash-raw-')) {\r\n    if (ctx.tag == 'gBbJrY' || ctx.tag == 'tThBgH') {\r\n        def re = /^\\d+\\/\\d+\\/\\d+\\/(?<source>[^\\/]+)\\/(?<uid>\\d+)\\/(?<did>\\d+)_\\d+\\.\\S+$/;\r\n        def m = re.matcher(ctx.parameters[0]);\r\n        if (m.matches()) {\r\n            ctx._index = ctx._index.replace('logstash-raw-', 'logstash-' + m.namedGroup('source') + '-');\r\n            ctx.deviceId = m.namedGroup('did');\r\n            ctx.userId = m.namedGroup('uid');\r\n        }\r\n    }\r\n}\r\n```\r\n\r\nextract-app-stat.painless:\r\n```\r\nif (ctx._index.startsWith('logstash-app-')) {\r\n    def re = /^\\[(?<logtime>.*?)\\] \\[INFO\\] \\[STAT\\] (?<measure>.*)$/;\r\n    def m = re.matcher(ctx.parameters[1]);\r\n    if (m.matches()) {\r\n        def measureValues = /,/.split(m.namedGroup('measure'));\r\n        def name = measureValues[0].replace(' ', '');\r\n        def value = null;\r\n\r\n        if (measureValues.length > 1) {\r\n            value = measureValues[1].trim();\r\n        }\r\n\r\n        if (value == 'on' || value == 'off' || value == 'auto') {\r\n            name = name + value;\r\n            value = null;\r\n        }\r\n\r\n        if (value == null) {\r\n            ctx[name + '_count'] = 1;\r\n        } else {\r\n            if (value.startsWith('time = ')) {\r\n                value = Float.parseFloat(value.substring(7));\r\n            }\r\n            ctx[name] = value;\r\n            ctx[name + '_count'] = 1;\r\n        }\r\n        ctx.stat = true;\r\n    }\r\n    re = /^\\[(?<logtime>.*?)\\].*$/;\r\n    m = re.matcher(ctx.parameters[1]);\r\n    if (m.matches()) {\r\n        def f = new SimpleDateFormat(\"yyyy-MM-dd'T'HH:mm:ss.SSSZ\");\r\n        def p = new SimpleDateFormat('yy-MM-dd hh:mm:ss');\r\n        f.setTimeZone(TimeZone.getTimeZone('Asia/Shanghai'));\r\n        p.setTimeZone(TimeZone.getTimeZone('Asia/Shanghai'));\r\n        ctx.logtime = f.format(p.parse(m.namedGroup('logtime')));\r\n    }\r\n}\r\n```\r\n\r\n\r\n\r\n\r\n**Provide logs (if relevant)**:\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/265589822","html_url":"https://github.com/elastic/elasticsearch/issues/21970#issuecomment-265589822","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/21970","id":265589822,"user":{"login":"dakrone","id":19060,"avatar_url":"https://avatars.githubusercontent.com/u/19060?v=3","gravatar_id":"","url":"https://api.github.com/users/dakrone","html_url":"https://github.com/dakrone","followers_url":"https://api.github.com/users/dakrone/followers","following_url":"https://api.github.com/users/dakrone/following{/other_user}","gists_url":"https://api.github.com/users/dakrone/gists{/gist_id}","starred_url":"https://api.github.com/users/dakrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dakrone/subscriptions","organizations_url":"https://api.github.com/users/dakrone/orgs","repos_url":"https://api.github.com/users/dakrone/repos","events_url":"https://api.github.com/users/dakrone/events{/privacy}","received_events_url":"https://api.github.com/users/dakrone/received_events","type":"User","site_admin":false},"created_at":"2016-12-07T22:12:09Z","updated_at":"2016-12-07T22:12:09Z","body":"Can you include the entire exception and stacktrace from the logs when this occurs next? I'd like to see where it came from.\r\n\r\nAre you doing any other scripting in this cluster other than this pipeline during indexing?"}}
{"actor":{"display_login":"dakrone"}}