{"action":"opened","issue":{"url":"https://api.github.com/repos/Yelp/elastalert/issues/514","repository_url":"https://api.github.com/repos/Yelp/elastalert","labels_url":"https://api.github.com/repos/Yelp/elastalert/issues/514/labels{/name}","comments_url":"https://api.github.com/repos/Yelp/elastalert/issues/514/comments","events_url":"https://api.github.com/repos/Yelp/elastalert/issues/514/events","html_url":"https://github.com/Yelp/elastalert/issues/514","id":153244810,"number":514,"title":"Examples do not pass the test.","user":{"login":"magmax","id":584026,"avatar_url":"https://avatars.githubusercontent.com/u/584026?v=3","gravatar_id":"","url":"https://api.github.com/users/magmax","html_url":"https://github.com/magmax","followers_url":"https://api.github.com/users/magmax/followers","following_url":"https://api.github.com/users/magmax/following{/other_user}","gists_url":"https://api.github.com/users/magmax/gists{/gist_id}","starred_url":"https://api.github.com/users/magmax/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/magmax/subscriptions","organizations_url":"https://api.github.com/users/magmax/orgs","repos_url":"https://api.github.com/users/magmax/repos","events_url":"https://api.github.com/users/magmax/events{/privacy}","received_events_url":"https://api.github.com/users/magmax/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-05-05T14:33:35Z","updated_at":"2016-05-05T14:33:35Z","closed_at":null,"body":"When running `elastalert-test-rule example_rules/example_new_term.yaml  --schema-only`, the result is:\r\n```\r\nelastalert-test-rule example_rules/example_new_term.yaml  --schema-only\r\nTraceback (most recent call last):\r\n  File \"/root/tmp/elastalert/venv/bin/elastalert-test-rule\", line 9, in <module>\r\n    load_entry_point('elastalert==0.0.80', 'console_scripts', 'elastalert-test-rule')()\r\n  File \"/root/tmp/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.0.80-py2.7.egg/elastalert/test_rule.py\", line 287, in main\r\n    test_instance.run_rule_test()\r\n  File \"/root/tmp/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.0.80-py2.7.egg/elastalert/test_rule.py\", line 275, in run_rule_test\r\n    hits = self.test_file(copy.deepcopy(rule_yaml), args)\r\n  File \"/root/tmp/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.0.80-py2.7.egg/elastalert/test_rule.py\", line 46, in test_file\r\n    load_options(conf, {})\r\n  File \"/root/tmp/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.0.80-py2.7.egg/elastalert/config.py\", line 102, in load_options\r\n    raise EAException(\"Invalid Rule: %s\\n%s\" % (rule.get('name'), e))\r\nelastalert.util.EAException: Invalid Rule: Example rule\r\n{'index': 'logstash-*', 'terms_window_size': {'days': 90}, 'fields': ['ip_address'], 'alert': ['email'], 'filter': [{'term': {'_type': 'login_event'}}, {'term': {'username': 'admin'}}], 'type': 'new_term', 'email': ['elastalert@ex\r\nample.com'], 'name': 'Example rule'} is not valid under any of the given schemas\r\n\r\nFailed validating 'oneOf' in schema:\r\n    {'$schema': 'http://json-schema.org/draft-04/schema#',\r\n     'definitions': {'arrayOfStrings': {'items': {'type': 'string'},\r\n                                        'type': ['string', 'array']},\r\n                     'arrayOfStringsOrOtherArrays': {'items': {'type': 'array'},\r\n                                                     'type': ['string',\r\n                                                              'array']},\r\n                     'filter': {},\r\n                     'timeFrame': {'additionalProperties': False,\r\n                                   'properties': {'days': {'type': 'number'},\r\n                                                  'hours': {'type': 'number'},\r\n                                                  'milliseconds': {'type': 'number'},\r\n                                                  'minutes': {'type': 'number'},\r\n                                                  'schedule': {'type': 'string'},\r\n                                                  'seconds': {'type': 'number'},\r\n                                                  'weeks': {'type': 'number'}},\r\n                                   'type': 'object'}},\r\n     'oneOf': [{'properties': {'type': {'enum': ['any']}},\r\n                'title': 'Any'},\r\n               {'properties': {'blacklist': {'items': {'type': 'string'},\r\n                                             'type': 'array'},\r\n                               'compare_key': {'type': 'string'},\r\n                               'type': {'enum': ['blacklist']}},\r\n                'required': ['blacklist', 'compare_key'],\r\n                'title': 'Blacklist'},\r\n               {'properties': {'compare_key': {'type': 'string'},\r\n                               'ignore_null': {'type': 'boolean'},\r\n                               'type': {'enum': ['whitelist']},\r\n                               'whitelist': {'items': {'type': 'string'},\r\n                                             'type': 'array'}},\r\n                'required': ['whitelist', 'compare_key', 'ignore_null'],\r\n                'title': 'Whitelist'},\r\n               {'properties': {'compare_key': {'type': 'string'},\r\n                               'ignore_null': {'type': 'boolean'},\r\n                               'timeframe': {'additionalProperties': False,\r\n                                             'properties': {'days': {'type': 'number'},\r\n                                                            'hours': {'type': 'number'},\r\n                                                            'milliseconds': {'type': 'number'},\r\n                                                            'minutes': {'type': 'number'},\r\n                                                            'schedule': {'type': 'string'},\r\n                                                            'seconds': {'type': 'number'},\r\n                                                            'weeks': {'type': 'number'}},\r\n                                             'type': 'object'},\r\n                               'type': {'enum': ['change']}},\r\n                'required': ['query_key', 'compare_key', 'ignore_null'],\r\n                'title': 'Change'},\r\n               {'properties': {'attach_related': {'type': 'boolean'},\r\n                               'doc_type': {'type': 'string'},\r\n                               'num_events': {'type': 'integer'},\r\n                               'terms_size': {'type': 'integer'},\r\n                               'timeframe': {'additionalProperties': False,\r\n                                             'properties': {'days': {'type': 'number'},\r\n                                                            'hours': {'type': 'number'},\r\n                                                            'milliseconds': {'type': 'number'},\r\n                                                            'minutes': {'type': 'number'},\r\n                                                            'schedule': {'type': 'string'},\r\n                                                            'seconds': {'type': 'number'},\r\n                                                            'weeks': {'type': 'number'}},\r\n                                             'type': 'object'},\r\n                               'type': {'enum': ['frequency']},\r\n                               'use_count_query': {'type': 'boolean'},\r\n                               'use_terms_query': {'type': 'boolean'}},\r\n                'required': ['num_events', 'timeframe'],\r\n                'title': 'Frequency'},\r\n               {'properties': {'alert_on_new_data': {'type': 'boolean'},\r\n                               'doc_type': {'type': 'string'},\r\n                               'spike_height': {'type': 'number'},\r\n                               'spike_type': {'enum': ['up',\r\n                                                       'down',\r\n                                                       'both']},\r\n                               'terms_size': {'type': 'integer'},\r\n                               'threshold_cur': {'type': 'integer'},\r\n                               'threshold_ref': {'type': 'integer'},\r\n                               'timeframe': {'additionalProperties': False,\r\n                                             'properties': {'days': {'type': 'number'},\r\n                                                            'hours': {'type': 'number'},\r\n                                                            'milliseconds': {'type': 'number'},\r\n                                                            'minutes': {'type': 'number'},\r\n                                                            'schedule': {'type': 'string'},\r\n                                                            'seconds': {'type': 'number'},\r\n                                                            'weeks': {'type': 'number'}},\r\n                                             'type': 'object'},\r\n                               'type': {'enum': ['spike']},\r\n                               'use_count_query': {'type': 'boolean'},\r\n                               'use_terms_query': {'type': 'boolean'}},\r\n                'required': ['spike_height', 'spike_type', 'timeframe'],\r\n                'title': 'Spike'},\r\n               {'properties': {'doc_type': {'type': 'string'},\r\n                               'threshold': {'type': 'integer'},\r\n                               'timeframe': {'additionalProperties': False,\r\n                                             'properties': {'days': {'type': 'number'},\r\n                                                            'hours': {'type': 'number'},\r\n                                                            'milliseconds': {'type': 'number'},\r\n                                                            'minutes': {'type': 'number'},\r\n                                                            'schedule': {'type': 'string'},\r\n                                                            'seconds': {'type': 'number'},\r\n                                                            'weeks': {'type': 'number'}},\r\n                                             'type': 'object'},\r\n                               'type': {'enum': ['flatline']},\r\n                               'use_count_query': {'type': 'boolean'}},\r\n                'required': ['threshold', 'timeframe'],\r\n                'title': 'Flatline'},\r\n               {'properties': {'alert_on_missing_field': {'type': 'boolean'},\r\n                               'fields': {'items': {'type': 'array'},\r\n                                          'type': ['string', 'array']},\r\n                               'terms_size': {'type': 'integer'},\r\n                               'terms_window_size': {'additionalProperties': False,\r\n                                                     'properties': {'days': {'type': 'number'},\r\n                                                                    'hours': {'type': 'number'},\r\n                                                                    'milliseconds': {'type': 'number'},\r\n                                                                    'minutes': {'type': 'number'},\r\n                                                                    'schedule': {'type': 'string'},\r\n                                                                    'seconds': {'type': 'number'},\r\n                                                                    'weeks': {'type': 'number'}},\r\n                                                     'type': 'object'},\r\n                               'type': {'enum': ['new_term']},\r\n                               'use_terms_query': {'type': 'boolean'}},\r\n                'required': ['fields'],\r\n                'title': 'New Term'},\r\n               {'properties': {'cardinality_field': {'type': 'string'},\r\n                               'max_cardinality': {'type': 'integer'},\r\n                               'min_cardinality': {'type': 'integer'},\r\n                               'timeframe': {'additionalProperties': False,\r\n                                             'properties': {'days': {'type': 'number'},\r\n                                                            'hours': {'type': 'number'},\r\n                                                            'milliseconds': {'type': 'number'},\r\n                                                            'minutes': {'type': 'number'},\r\n                                                            'schedule': {'type': 'string'},\r\n                                                            'seconds': {'type': 'number'},\r\n                                                            'weeks': {'type': 'number'}},\r\n                                             'type': 'object'},\r\n                               'type': {'enum': ['cardinality']}},\r\n                'required': ['cardinality_field', 'timeframe'],\r\n                'title': 'Cardinality'},\r\n               {'properties': {'type': {'pattern': '[.]'}},\r\n                'title': 'Custom Rule from Module'}],\r\n     'properties': {'aggregation': {'additionalProperties': False,\r\n                                    'properties': {'days': {'type': 'number'},\r\n                                                   'hours': {'type': 'number'},\r\n                                                   'milliseconds': {'type': 'number'},\r\n                                                   'minutes': {'type': 'number'},\r\n                                                   'schedule': {'type': 'string'},\r\n                                                   'seconds': {'type': 'number'},\r\n                                                   'weeks': {'type': 'number'}},\r\n                                    'type': 'object'},\r\n                    'alert_text': {'type': 'string'},\r\n                    'alert_text_args': {'items': {'type': 'string'},\r\n                                        'type': 'array'},\r\n                    'alert_text_kw': {'type': 'object'},\r\n                    'alert_text_type': {'enum': ['alert_text_only',\r\n                                                 'exclude_fields']},\r\n                    'buffer_time': {'additionalProperties': False,\r\n                                    'properties': {'days': {'type': 'number'},\r\n                                                   'hours': {'type': 'number'},\r\n                                                   'milliseconds': {'type': 'number'},\r\n                                                   'minutes': {'type': 'number'},\r\n                                                   'schedule': {'type': 'string'},\r\n                                                   'seconds': {'type': 'number'},\r\n                                                   'weeks': {'type': 'number'}},\r\n                                    'type': 'object'},\r\n                    'command': {'items': {'type': 'string'},\r\n                                'type': ['string', 'array']},\r\n                    'email': {'items': {'type': 'string'},\r\n                              'type': ['string', 'array']},\r\n                    'email_reply_to': {'type': 'string'},\r\n                    'es_host': {'type': 'string'},\r\n                    'es_password': {'type': 'string'},\r\n                    'es_port': {'type': 'integer'},\r\n                    'es_username': {'type': 'string'},\r\n                    'exponential_realert': {'additionalProperties': False,\r\n                                            'properties': {'days': {'type': 'number'},\r\n                                                           'hours': {'type': 'number'},\r\n                                                           'milliseconds': {'type': 'number'},\r\n                                                           'minutes': {'type': 'number'},\r\n                                                           'schedule': {'type': 'string'},\r\n                                                           'seconds': {'type': 'number'},\r\n                                                           'weeks': {'type': 'number'}},\r\n                                            'type': 'object'},\r\n                    'field': {},\r\n                    'filter': {'items': {}, 'type': 'array'},\r\n                    'from_addr': {'type': 'string'},\r\n                    'generate_kibana_link': {'type': 'boolean'},\r\n                    'gitter_msg_level': {'enum': ['info', 'error']},\r\n                    'gitter_proxy': {'type': 'string'},\r\n                    'gitter_webhook_url': {'type': 'string'},\r\n                    'hipchat_auth_token': {'type': 'string'},\r\n                    'hipchat_domain': {'type': 'string'},\r\n                    'hipchat_ignore_ssl_errors': {'type': 'boolean'},\r\n                    'hipchat_room_id': {'type': 'number'},\r\n                    'include': {'items': {'type': 'string'},\r\n                                'type': 'array'},\r\n                    'index': {'type': 'string'},\r\n                    'jira_account_file': {'type': 'string'},\r\n                    'jira_bump_tickets': {'type': 'boolean'},\r\n                    'jira_component': {'type': 'string'},\r\n                    'jira_issuetype': {'type': 'string'},\r\n                    'jira_label': {'type': 'string'},\r\n                    'jira_max_age': {'type': 'number'},\r\n                    'jira_project': {'type': 'string'},\r\n                    'jira_server': {'type': 'string'},\r\n                    'kibana_dashboard': {'type': 'string'},\r\n                    'match_enhancements': {'items': {'type': 'string'},\r\n                                           'type': 'array'},\r\n                    'max_query_size': {'type': 'integer'},\r\n                    'name': {'type': 'string'},\r\n                    'notify_email': {'items': {'type': 'string'},\r\n                                     'type': ['string', 'array']},\r\n                    'pagerduty_client_name': {'type': 'string'},\r\n                    'pagerduty_service_key': {'type': 'string'},\r\n                    'pipe_match_json': {'type': 'boolean'},\r\n                    'query_delay': {'additionalProperties': False,\r\n                                    'properties': {'days': {'type': 'number'},\r\n                                                   'hours': {'type': 'number'},\r\n                                                   'milliseconds': {'type': 'number'},\r\n                                                   'minutes': {'type': 'number'},\r\n                                                   'schedule': {'type': 'string'},\r\n                                                   'seconds': {'type': 'number'},\r\n                                                   'weeks': {'type': 'number'}},\r\n                                    'type': 'object'},\r\n                    'query_key': {'items': {'type': 'string'},\r\n                                  'type': ['string', 'array']},\r\n                    'raw_count_keys': {'type': 'boolean'},\r\n                    'realert': {'additionalProperties': False,\r\n                                'properties': {'days': {'type': 'number'},\r\n                                               'hours': {'type': 'number'},\r\n                                               'milliseconds': {'type': 'number'},\r\n                                               'minutes': {'type': 'number'},\r\n                                               'schedule': {'type': 'string'},\r\n                                               'seconds': {'type': 'number'},\r\n                                               'weeks': {'type': 'number'}},\r\n                                'type': 'object'},\r\n                    'slack_emoji_override': {'type': 'string'},\r\n                    'slack_msg_color': {'enum': ['good',\r\n                                                 'warning',\r\n                                                 'danger']},\r\n                    'slack_username_override': {'type': 'string'},\r\n                    'slack_webhook_url': {'type': 'string'},\r\n                    'smtp_host': {'type': 'string'},\r\n                    'telegram_api_url': {'type': 'string'},\r\n                    'telegram_bot_token': {'type': 'string'},\r\n                    'telegram_room_id': {'type': 'string'},\r\n                    'timestamp_field': {'type': 'string'},\r\n                    'top_count_keys': {'items': {'type': 'string'},\r\n                                       'type': 'array'},\r\n                    'top_count_number': {'type': 'integer'},\r\n                    'use_kibana_dashboard': {'type': 'string'},\r\n                    'use_local_time': {'type': 'boolean'},\r\n                    'use_ssl': {'type': 'boolean'},\r\n                    'use_strftime_index': {'type': 'boolean'},\r\n                    'victorops_api_key': {'type': 'string'},\r\n                    'victorops_entity_display_name': {'type': 'string'},\r\n                    'victorops_message_type': {'enum': ['INFO',\r\n                                                        'WARNING',\r\n                                                        'ACKNOWLEDGEMENT',\r\n                                                        'CRITICAL',\r\n                                                        'RECOVERY']},\r\n                    'victorops_routing_key': {'type': 'string'}},\r\n     'required': ['name', 'type', 'index', 'alert'],\r\n     'type': 'object'}\r\n\r\nOn instance:\r\n    {'alert': ['email'],\r\n     'email': ['elastalert@example.com'],\r\n     'fields': ['ip_address'],\r\n     'filter': [{'term': {'_type': 'login_event'}},\r\n                {'term': {'username': 'admin'}}],\r\n     'index': 'logstash-*',\r\n     'name': 'Example rule',\r\n     'terms_window_size': {'days': 90},\r\n     'type': 'new_term'}\r\n```\r\n\r\nExamples should be tested in travis in order to ensure their validity (at least the schema)"}}