{"action":"created","issue":{"url":"https://api.github.com/repos/balabit/syslog-ng/issues/2196","repository_url":"https://api.github.com/repos/balabit/syslog-ng","labels_url":"https://api.github.com/repos/balabit/syslog-ng/issues/2196/labels{/name}","comments_url":"https://api.github.com/repos/balabit/syslog-ng/issues/2196/comments","events_url":"https://api.github.com/repos/balabit/syslog-ng/issues/2196/events","html_url":"https://github.com/balabit/syslog-ng/pull/2196","id":343060725,"node_id":"MDExOlB1bGxSZXF1ZXN0MjAyODEyNzA5","number":2196,"title":"python-destination: template support","user":{"login":"furiel","id":11007226,"node_id":"MDQ6VXNlcjExMDA3MjI2","avatar_url":"https://avatars0.githubusercontent.com/u/11007226?v=4","gravatar_id":"","url":"https://api.github.com/users/furiel","html_url":"https://github.com/furiel","followers_url":"https://api.github.com/users/furiel/followers","following_url":"https://api.github.com/users/furiel/following{/other_user}","gists_url":"https://api.github.com/users/furiel/gists{/gist_id}","starred_url":"https://api.github.com/users/furiel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/furiel/subscriptions","organizations_url":"https://api.github.com/users/furiel/orgs","repos_url":"https://api.github.com/users/furiel/repos","events_url":"https://api.github.com/users/furiel/events{/privacy}","received_events_url":"https://api.github.com/users/furiel/received_events","type":"User","site_admin":false},"labels":[{"id":122658708,"node_id":"MDU6TGFiZWwxMjI2NTg3MDg=","url":"https://api.github.com/repos/balabit/syslog-ng/labels/needs-review","name":"needs-review","color":"bfe5bf","default":false},{"id":988239098,"node_id":"MDU6TGFiZWw5ODgyMzkwOTg=","url":"https://api.github.com/repos/balabit/syslog-ng/labels/user-visible-feature","name":"user-visible-feature","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/balabit/syslog-ng/milestones/16","html_url":"https://github.com/balabit/syslog-ng/milestone/16","labels_url":"https://api.github.com/repos/balabit/syslog-ng/milestones/16/labels","id":3555169,"node_id":"MDk6TWlsZXN0b25lMzU1NTE2OQ==","number":16,"title":"OSE 3.18","description":"","creator":{"login":"lbudai","id":5908786,"node_id":"MDQ6VXNlcjU5MDg3ODY=","avatar_url":"https://avatars1.githubusercontent.com/u/5908786?v=4","gravatar_id":"","url":"https://api.github.com/users/lbudai","html_url":"https://github.com/lbudai","followers_url":"https://api.github.com/users/lbudai/followers","following_url":"https://api.github.com/users/lbudai/following{/other_user}","gists_url":"https://api.github.com/users/lbudai/gists{/gist_id}","starred_url":"https://api.github.com/users/lbudai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lbudai/subscriptions","organizations_url":"https://api.github.com/users/lbudai/orgs","repos_url":"https://api.github.com/users/lbudai/repos","events_url":"https://api.github.com/users/lbudai/events{/privacy}","received_events_url":"https://api.github.com/users/lbudai/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":28,"state":"open","created_at":"2018-08-08T10:01:49Z","updated_at":"2018-09-20T07:37:03Z","due_on":"2018-10-07T07:00:00Z","closed_at":null},"comments":41,"created_at":"2018-07-20T10:44:16Z","updated_at":"2018-09-20T14:49:58Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/balabit/syslog-ng/pulls/2196","html_url":"https://github.com/balabit/syslog-ng/pull/2196","diff_url":"https://github.com/balabit/syslog-ng/pull/2196.diff","patch_url":"https://github.com/balabit/syslog-ng/pull/2196.patch"},"body":"Fixes: https://github.com/balabit/syslog-ng/issues/2172\r\n\r\nThis patchset adds support for template formatting in python destination. All necessary parameters are exposed to the user code: LTZ_SEND, LTZ_LOCAL, sequence number, and LogTemplateOptions instance of the destination. This patchset does not support construction of LogTemplateOptions, nor their fields are exposed to the user, only that is generated by the grammar is available to the user. Though such functionality might be useful, and can be added later.\r\n\r\nLTZ_SEND, LTZ_LOCAL is exposed in a newly created syslog-ng module. LogTemplateOptions and sequence number is exposed through self (the class, provided by the user).\r\n\r\nExamples:\r\n(Don't forget to add `import syslogng`)\r\n\r\n* Creating a new LogTemplate object \r\n```\r\nself.log_template = syslogng.LogTemplate(\"${S_STAMP}\")\r\nself.log_template = syslogng.LogTemplate(\"${S_STAMP}\", self.template_options)\r\n```\r\n\r\n* LogTemplateException\r\n\r\nIn case a malformed template is provided, an exception will be raised. For that sake, a new exception class is added: syslogng.LogTemplateException:\r\n```\r\ntry:\r\n    syslogng.LogTemplate(\"$(incomplete\");\r\nexcept syslogng.LogTemplateException as e:\r\n     print(e)\r\n```\r\n\r\n\r\n* Formatting template:\r\nTemplates can be formatted using the format method. \r\nReturn value: with python3, the return value is unicode (the standard string representation) if the result is unicode or covertible to unicode, else PyBytes. With python2, the return value is PyBytes (the standart string representation).\r\n```\r\nself.log_template.format(msg)\r\nself.log_template.format(msg, self.template_options)\r\nself.log_template.format(msg, self.template_options, syslogng.LTZ_SEND)\r\nself.log_template.format(msg, self.template_options, syslogng.LTZ_SEND, self.seqnum)\r\n```\r\nOnly msg and is mandatory for the `format` method.\r\n\r\n* Formatting template works with keyword arguments as well:\r\n```\r\nself.log_template.format(msg=msg, options=self.template_options, tz=syslogng.LTZ_SEND, seqnum=self.seqnum))\r\n```\r\n\r\nTemplate options must be either provided in the constructor or in the `format` method. Otherwise syslog-ng will raise RuntimeError during in the `format` method.\r\n\r\nNotes to developers:\r\n- In practice some macros could work without any LogTemplate options provided (NULL passed to log_template_format). Though in case of some macros: like STAMP, syslog-ng would crash, which I wanted to avoid in a language binding. That's why I am enforcing that LogTemplateOptions must be passed either in constructor or in the format method.\r\n\r\n- I was trying hard to keep reference counts at bay. Though there is one place (somewhat hidden hidden :( )\r\nthat I know there is a reference leak: PY_LTZ_LOCAL, PY_LTZ_SEND. So that I could unref these, there should be a pair for python_module_init, like deinit, but syslog-ng currently does not support deinit for modules (so files). Or some kind of garbage collection mechanism. I created an internal jira ticket about it. As for this reference leak, this does **not** result in memory leak. Interestingly enough, it seems many (all?) PyObject numbers share the same object, so by the time I created these objects,  the reference count was already 433 for PY_LTZ_SEND (0), which means the zero-object was already reffed many times, and most probably will not disappear when syslog-ng exits. We cannot unload the zero object. So this is a cosmetic reference leak."},"comment":{"url":"https://api.github.com/repos/balabit/syslog-ng/issues/comments/423212407","html_url":"https://github.com/balabit/syslog-ng/pull/2196#issuecomment-423212407","issue_url":"https://api.github.com/repos/balabit/syslog-ng/issues/2196","id":423212407,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMzIxMjQwNw==","user":{"login":"furiel","id":11007226,"node_id":"MDQ6VXNlcjExMDA3MjI2","avatar_url":"https://avatars0.githubusercontent.com/u/11007226?v=4","gravatar_id":"","url":"https://api.github.com/users/furiel","html_url":"https://github.com/furiel","followers_url":"https://api.github.com/users/furiel/followers","following_url":"https://api.github.com/users/furiel/following{/other_user}","gists_url":"https://api.github.com/users/furiel/gists{/gist_id}","starred_url":"https://api.github.com/users/furiel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/furiel/subscriptions","organizations_url":"https://api.github.com/users/furiel/orgs","repos_url":"https://api.github.com/users/furiel/repos","events_url":"https://api.github.com/users/furiel/events{/privacy}","received_events_url":"https://api.github.com/users/furiel/received_events","type":"User","site_admin":false},"created_at":"2018-09-20T14:49:58Z","updated_at":"2018-09-20T14:49:58Z","author_association":"CONTRIBUTOR","body":"Thanks for the thorough review. Yes I think that would be useful for python parser too, or maybe even for python source. Though I do not not plan to work on it in the near future."}}
{"actor":{"display_login":"furiel"}}