{"action":"created","issue":{"url":"https://api.github.com/repos/wizzie-io/enricher/issues/16","repository_url":"https://api.github.com/repos/wizzie-io/enricher","labels_url":"https://api.github.com/repos/wizzie-io/enricher/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/wizzie-io/enricher/issues/16/comments","events_url":"https://api.github.com/repos/wizzie-io/enricher/issues/16/events","html_url":"https://github.com/wizzie-io/enricher/issues/16","id":378703674,"node_id":"MDU6SXNzdWUzNzg3MDM2NzQ=","number":16,"title":"SimpleStreamEnrich documentation","user":{"login":"ST2Labs","id":9781565,"node_id":"MDQ6VXNlcjk3ODE1NjU=","avatar_url":"https://avatars3.githubusercontent.com/u/9781565?v=4","gravatar_id":"","url":"https://api.github.com/users/ST2Labs","html_url":"https://github.com/ST2Labs","followers_url":"https://api.github.com/users/ST2Labs/followers","following_url":"https://api.github.com/users/ST2Labs/following{/other_user}","gists_url":"https://api.github.com/users/ST2Labs/gists{/gist_id}","starred_url":"https://api.github.com/users/ST2Labs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ST2Labs/subscriptions","organizations_url":"https://api.github.com/users/ST2Labs/orgs","repos_url":"https://api.github.com/users/ST2Labs/repos","events_url":"https://api.github.com/users/ST2Labs/events{/privacy}","received_events_url":"https://api.github.com/users/ST2Labs/received_events","type":"User","site_admin":false},"labels":[{"id":990402410,"node_id":"MDU6TGFiZWw5OTA0MDI0MTA=","url":"https://api.github.com/repos/wizzie-io/enricher/labels/documentation","name":"documentation","color":"a2eeef","default":false},{"id":1120577252,"node_id":"MDU6TGFiZWwxMTIwNTc3MjUy","url":"https://api.github.com/repos/wizzie-io/enricher/labels/question","name":"question","color":"e2cd8c","default":true}],"state":"open","locked":false,"assignee":{"login":"andresgomezfrr","id":4025771,"node_id":"MDQ6VXNlcjQwMjU3NzE=","avatar_url":"https://avatars1.githubusercontent.com/u/4025771?v=4","gravatar_id":"","url":"https://api.github.com/users/andresgomezfrr","html_url":"https://github.com/andresgomezfrr","followers_url":"https://api.github.com/users/andresgomezfrr/followers","following_url":"https://api.github.com/users/andresgomezfrr/following{/other_user}","gists_url":"https://api.github.com/users/andresgomezfrr/gists{/gist_id}","starred_url":"https://api.github.com/users/andresgomezfrr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andresgomezfrr/subscriptions","organizations_url":"https://api.github.com/users/andresgomezfrr/orgs","repos_url":"https://api.github.com/users/andresgomezfrr/repos","events_url":"https://api.github.com/users/andresgomezfrr/events{/privacy}","received_events_url":"https://api.github.com/users/andresgomezfrr/received_events","type":"User","site_admin":false},"assignees":[{"login":"andresgomezfrr","id":4025771,"node_id":"MDQ6VXNlcjQwMjU3NzE=","avatar_url":"https://avatars1.githubusercontent.com/u/4025771?v=4","gravatar_id":"","url":"https://api.github.com/users/andresgomezfrr","html_url":"https://github.com/andresgomezfrr","followers_url":"https://api.github.com/users/andresgomezfrr/followers","following_url":"https://api.github.com/users/andresgomezfrr/following{/other_user}","gists_url":"https://api.github.com/users/andresgomezfrr/gists{/gist_id}","starred_url":"https://api.github.com/users/andresgomezfrr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andresgomezfrr/subscriptions","organizations_url":"https://api.github.com/users/andresgomezfrr/orgs","repos_url":"https://api.github.com/users/andresgomezfrr/repos","events_url":"https://api.github.com/users/andresgomezfrr/events{/privacy}","received_events_url":"https://api.github.com/users/andresgomezfrr/received_events","type":"User","site_admin":false},{"login":"pablocasares","id":4056828,"node_id":"MDQ6VXNlcjQwNTY4Mjg=","avatar_url":"https://avatars0.githubusercontent.com/u/4056828?v=4","gravatar_id":"","url":"https://api.github.com/users/pablocasares","html_url":"https://github.com/pablocasares","followers_url":"https://api.github.com/users/pablocasares/followers","following_url":"https://api.github.com/users/pablocasares/following{/other_user}","gists_url":"https://api.github.com/users/pablocasares/gists{/gist_id}","starred_url":"https://api.github.com/users/pablocasares/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pablocasares/subscriptions","organizations_url":"https://api.github.com/users/pablocasares/orgs","repos_url":"https://api.github.com/users/pablocasares/repos","events_url":"https://api.github.com/users/pablocasares/events{/privacy}","received_events_url":"https://api.github.com/users/pablocasares/received_events","type":"User","site_admin":false}],"milestone":null,"comments":15,"created_at":"2018-11-08T12:08:45Z","updated_at":"2018-11-08T16:38:26Z","closed_at":null,"author_association":"NONE","body":"Hi, \r\n\r\nI'm try to use **simpleStreamEnrich**, and I think what I have to define enricher like this:\r\n\r\nLink: https://wizzie-io.github.io/enricher/configuration/eql_syntax#\r\n\r\n```\r\n{\r\n\"enrichers\": [\r\n{\r\n  \"name\": \"test-simple\",\r\n  \"className\": \"io.wizzie.enricher.enrichment.simple\",\r\n  \"properties\": {\r\n  -...-\r\n  }\r\n}\r\n],\r\n\"queries\": {\r\n.... <<< could I use moren than one query ??>>>\r\n}\r\n}\r\n``` \r\nIt' ok--- I can't find API Reference or any docs over that.\r\n\r\nThanks in advance"},"comment":{"url":"https://api.github.com/repos/wizzie-io/enricher/issues/comments/437064105","html_url":"https://github.com/wizzie-io/enricher/issues/16#issuecomment-437064105","issue_url":"https://api.github.com/repos/wizzie-io/enricher/issues/16","id":437064105,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzA2NDEwNQ==","user":{"login":"pablocasares","id":4056828,"node_id":"MDQ6VXNlcjQwNTY4Mjg=","avatar_url":"https://avatars0.githubusercontent.com/u/4056828?v=4","gravatar_id":"","url":"https://api.github.com/users/pablocasares","html_url":"https://github.com/pablocasares","followers_url":"https://api.github.com/users/pablocasares/followers","following_url":"https://api.github.com/users/pablocasares/following{/other_user}","gists_url":"https://api.github.com/users/pablocasares/gists{/gist_id}","starred_url":"https://api.github.com/users/pablocasares/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pablocasares/subscriptions","organizations_url":"https://api.github.com/users/pablocasares/orgs","repos_url":"https://api.github.com/users/pablocasares/repos","events_url":"https://api.github.com/users/pablocasares/events{/privacy}","received_events_url":"https://api.github.com/users/pablocasares/received_events","type":"User","site_admin":false},"created_at":"2018-11-08T16:38:26Z","updated_at":"2018-11-08T16:38:26Z","author_association":"CONTRIBUTOR","body":"You shouldn't do a loop with enricher. You shouldn't select an input topic as an output topic (because enricher would be reading and sending events to itself). \r\n\r\nSo, to do what you want, you need 3 topics.\r\n\r\n1. SyslogTopicInput --> partitioned by IP.\r\n2. OTX-ReputationTopicInput --> partitioned by IP.\r\n3. SyslogWithOTXOutput --> will be generated partitioned by IP.\r\n\r\nSo, as an example, you could try (assuming you already have Syslog with IP partitioning)\r\n```\r\n{\r\n  \"joiners\":[\r\n    {\"name\":\"streamPreferred\", \"className\":\"io.wizzie.enricher.enrichment.join.impl.queryable.StreamPreferredJoiner\"}\r\n  ],\r\n  \"queries\": {\r\n    \"query1\": \"SELECT * FROM STREAM SyslogTopicInput JOIN SELECT * FROM TABLE OTX-ReputationTopicInput USING streamPreferred INSERT INTO STREAM SyslogWithOTXOutput\"\r\n   }\r\n}\r\n```\r\n\r\nBut, if you haven't Syslog topic partitioned by IP, you could use this plan in order to first partition it and then process it.\r\n\r\n```\r\n{\r\n  \"joiners\":[\r\n    {\"name\":\"streamPreferred\", \"className\":\"io.wizzie.enricher.enrichment.join.impl.queryable.StreamPreferredJoiner\"}\r\n  ],\r\n  \"queries\": {\r\n   \"query1\": \"SELECT * FROM STREAM Syslog INSERT INTO SyslogIPPartitioned PARTITION BY myIPfieldname\"\r\n    \"query1\": \"SELECT * FROM STREAM SyslogIPPartitioned JOIN SELECT * FROM TABLE OTX-ReputationTopicInput USING streamPreferred INSERT INTO STREAM SyslogWithOTXOutput\"\r\n   }\r\n}\r\n```\r\n\r\nPARTITION BY syntax is only available at master branch. This has been recently added to this project but this should work.\r\n\r\nAt SyslogWithOTXOutput topic you will have something like:\r\n```\r\nX.X.X.X, { \"ip\":\"X.X.X.X\", reputation_field, url\": \"yyy\", \"code\": \"x3203\" }\r\n```\r\nAlso, I recommend you to look at our Normalizer:\r\n\r\nhttps://github.com/wizzie-io/normalizer\r\n\r\nMaybe it will help you to do Regex transformations for your data fields:\r\n\r\nhttps://wizzie-io.github.io/normalizer/functions/mapper_functions#regexmapper"}}
{"actor":{"display_login":"pablocasares"}}