{"action":"closed","number":305,"pull_request":{"url":"https://api.github.com/repos/dagster-io/dagster/pulls/305","id":234122534,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM0MTIyNTM0","html_url":"https://github.com/dagster-io/dagster/pull/305","diff_url":"https://github.com/dagster-io/dagster/pull/305.diff","patch_url":"https://github.com/dagster-io/dagster/pull/305.patch","issue_url":"https://api.github.com/repos/dagster-io/dagster/issues/305","number":305,"state":"closed","locked":false,"title":"Initial stab at structured logging","user":{"login":"schrockn","id":28738937,"node_id":"MDQ6VXNlcjI4NzM4OTM3","avatar_url":"https://avatars2.githubusercontent.com/u/28738937?v=4","gravatar_id":"","url":"https://api.github.com/users/schrockn","html_url":"https://github.com/schrockn","followers_url":"https://api.github.com/users/schrockn/followers","following_url":"https://api.github.com/users/schrockn/following{/other_user}","gists_url":"https://api.github.com/users/schrockn/gists{/gist_id}","starred_url":"https://api.github.com/users/schrockn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schrockn/subscriptions","organizations_url":"https://api.github.com/users/schrockn/orgs","repos_url":"https://api.github.com/users/schrockn/repos","events_url":"https://api.github.com/users/schrockn/events{/privacy}","received_events_url":"https://api.github.com/users/schrockn/received_events","type":"User","site_admin":false},"body":"I'm not totally happy with this, but it's a start. In particular I think naming could be substantially better.\r\n\r\nA few changes here.\r\n\r\n1. For all dagster integrations with the python logging framework, we now stash all the dagster-domain metadata (e.g. the context stack) in a special property in the logging.LogRecord object. Previously (I didn't really understand this) all of those properties were unconditionally monkeypatched on the LogRecord object within the python logging module (yeah seriously) so this was risky and confusing. This change also required a horrible hack to be applied to the JsonFileHandler in order to maintain backward compatibility. Superconductive projects currently have a dependency on this.\r\n2. There is a new base level logger handler which just passes through all the logging information we have to a provided callback. This is the StructuredLoggerHandler.\r\n3. The Structured logger handler is used to create a logger that will emit a stream of typed, structured \"Event Records\" so a callback. This is what the graphql subscriptions would hook into. These would be directly translated to graphql types. It uses the \"event_type\" property set on the log message to decide what structured event to create. We have no formal schema to manage this process really.\r\n4. There is a new \"events\" api on the context. This is mostly for organizational purposes.\r\n\r\nThis structure makes it so that these events appear as normal log messages in, for example, the console logging use case, but if you add a structured event logger you can also hook into the stream of typed log messages.\r\n\r\nI've only added the pipeline start, success, and failure events as an example. There will be a lot more once we agree on an API.\r\n\r\n","created_at":"2018-11-28T02:00:43Z","updated_at":"2018-11-28T16:17:52Z","closed_at":"2018-11-28T16:17:52Z","merged_at":"2018-11-28T16:17:52Z","merge_commit_sha":"f2a6b74f4d575ad292cb1e81f658d56770c15dd4","assignee":null,"assignees":[],"requested_reviewers":[{"login":"mgasner","id":427016,"node_id":"MDQ6VXNlcjQyNzAxNg==","avatar_url":"https://avatars3.githubusercontent.com/u/427016?v=4","gravatar_id":"","url":"https://api.github.com/users/mgasner","html_url":"https://github.com/mgasner","followers_url":"https://api.github.com/users/mgasner/followers","following_url":"https://api.github.com/users/mgasner/following{/other_user}","gists_url":"https://api.github.com/users/mgasner/gists{/gist_id}","starred_url":"https://api.github.com/users/mgasner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mgasner/subscriptions","organizations_url":"https://api.github.com/users/mgasner/orgs","repos_url":"https://api.github.com/users/mgasner/repos","events_url":"https://api.github.com/users/mgasner/events{/privacy}","received_events_url":"https://api.github.com/users/mgasner/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/dagster-io/dagster/pulls/305/commits","review_comments_url":"https://api.github.com/repos/dagster-io/dagster/pulls/305/comments","review_comment_url":"https://api.github.com/repos/dagster-io/dagster/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dagster-io/dagster/issues/305/comments","statuses_url":"https://api.github.com/repos/dagster-io/dagster/statuses/13efc04fb49ac6a0608e99cbffabd4aae9ea3460","head":{"label":"dagster-io:schrockn/structured-logging","ref":"schrockn/structured-logging","sha":"13efc04fb49ac6a0608e99cbffabd4aae9ea3460","user":{"login":"dagster-io","id":40032576,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQwMDMyNTc2","avatar_url":"https://avatars0.githubusercontent.com/u/40032576?v=4","gravatar_id":"","url":"https://api.github.com/users/dagster-io","html_url":"https://github.com/dagster-io","followers_url":"https://api.github.com/users/dagster-io/followers","following_url":"https://api.github.com/users/dagster-io/following{/other_user}","gists_url":"https://api.github.com/users/dagster-io/gists{/gist_id}","starred_url":"https://api.github.com/users/dagster-io/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dagster-io/subscriptions","organizations_url":"https://api.github.com/users/dagster-io/orgs","repos_url":"https://api.github.com/users/dagster-io/repos","events_url":"https://api.github.com/users/dagster-io/events{/privacy}","received_events_url":"https://api.github.com/users/dagster-io/received_events","type":"Organization","site_admin":false},"repo":{"id":131619646,"node_id":"MDEwOlJlcG9zaXRvcnkxMzE2MTk2NDY=","name":"dagster","full_name":"dagster-io/dagster","private":false,"owner":{"login":"dagster-io","id":40032576,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQwMDMyNTc2","avatar_url":"https://avatars0.githubusercontent.com/u/40032576?v=4","gravatar_id":"","url":"https://api.github.com/users/dagster-io","html_url":"https://github.com/dagster-io","followers_url":"https://api.github.com/users/dagster-io/followers","following_url":"https://api.github.com/users/dagster-io/following{/other_user}","gists_url":"https://api.github.com/users/dagster-io/gists{/gist_id}","starred_url":"https://api.github.com/users/dagster-io/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dagster-io/subscriptions","organizations_url":"https://api.github.com/users/dagster-io/orgs","repos_url":"https://api.github.com/users/dagster-io/repos","events_url":"https://api.github.com/users/dagster-io/events{/privacy}","received_events_url":"https://api.github.com/users/dagster-io/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/dagster-io/dagster","description":" Dagster is an opinionated system and programming model for data pipelines.","fork":false,"url":"https://api.github.com/repos/dagster-io/dagster","forks_url":"https://api.github.com/repos/dagster-io/dagster/forks","keys_url":"https://api.github.com/repos/dagster-io/dagster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dagster-io/dagster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dagster-io/dagster/teams","hooks_url":"https://api.github.com/repos/dagster-io/dagster/hooks","issue_events_url":"https://api.github.com/repos/dagster-io/dagster/issues/events{/number}","events_url":"https://api.github.com/repos/dagster-io/dagster/events","assignees_url":"https://api.github.com/repos/dagster-io/dagster/assignees{/user}","branches_url":"https://api.github.com/repos/dagster-io/dagster/branches{/branch}","tags_url":"https://api.github.com/repos/dagster-io/dagster/tags","blobs_url":"https://api.github.com/repos/dagster-io/dagster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dagster-io/dagster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dagster-io/dagster/git/refs{/sha}","trees_url":"https://api.github.com/repos/dagster-io/dagster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dagster-io/dagster/statuses/{sha}","languages_url":"https://api.github.com/repos/dagster-io/dagster/languages","stargazers_url":"https://api.github.com/repos/dagster-io/dagster/stargazers","contributors_url":"https://api.github.com/repos/dagster-io/dagster/contributors","subscribers_url":"https://api.github.com/repos/dagster-io/dagster/subscribers","subscription_url":"https://api.github.com/repos/dagster-io/dagster/subscription","commits_url":"https://api.github.com/repos/dagster-io/dagster/commits{/sha}","git_commits_url":"https://api.github.com/repos/dagster-io/dagster/git/commits{/sha}","comments_url":"https://api.github.com/repos/dagster-io/dagster/comments{/number}","issue_comment_url":"https://api.github.com/repos/dagster-io/dagster/issues/comments{/number}","contents_url":"https://api.github.com/repos/dagster-io/dagster/contents/{+path}","compare_url":"https://api.github.com/repos/dagster-io/dagster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dagster-io/dagster/merges","archive_url":"https://api.github.com/repos/dagster-io/dagster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dagster-io/dagster/downloads","issues_url":"https://api.github.com/repos/dagster-io/dagster/issues{/number}","pulls_url":"https://api.github.com/repos/dagster-io/dagster/pulls{/number}","milestones_url":"https://api.github.com/repos/dagster-io/dagster/milestones{/number}","notifications_url":"https://api.github.com/repos/dagster-io/dagster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dagster-io/dagster/labels{/name}","releases_url":"https://api.github.com/repos/dagster-io/dagster/releases{/id}","deployments_url":"https://api.github.com/repos/dagster-io/dagster/deployments","created_at":"2018-04-30T16:30:04Z","updated_at":"2018-11-28T15:13:38Z","pushed_at":"2018-11-28T16:17:52Z","git_url":"git://github.com/dagster-io/dagster.git","ssh_url":"git@github.com:dagster-io/dagster.git","clone_url":"https://github.com/dagster-io/dagster.git","svn_url":"https://github.com/dagster-io/dagster","homepage":"","size":2085,"stargazers_count":45,"watchers_count":45,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"archived":false,"open_issues_count":28,"license":null,"forks":3,"open_issues":28,"watchers":45,"default_branch":"master"}},"base":{"label":"dagster-io:master","ref":"master","sha":"6e685d80ce93c9902f45450004c66daba5633641","user":{"login":"dagster-io","id":40032576,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQwMDMyNTc2","avatar_url":"https://avatars0.githubusercontent.com/u/40032576?v=4","gravatar_id":"","url":"https://api.github.com/users/dagster-io","html_url":"https://github.com/dagster-io","followers_url":"https://api.github.com/users/dagster-io/followers","following_url":"https://api.github.com/users/dagster-io/following{/other_user}","gists_url":"https://api.github.com/users/dagster-io/gists{/gist_id}","starred_url":"https://api.github.com/users/dagster-io/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dagster-io/subscriptions","organizations_url":"https://api.github.com/users/dagster-io/orgs","repos_url":"https://api.github.com/users/dagster-io/repos","events_url":"https://api.github.com/users/dagster-io/events{/privacy}","received_events_url":"https://api.github.com/users/dagster-io/received_events","type":"Organization","site_admin":false},"repo":{"id":131619646,"node_id":"MDEwOlJlcG9zaXRvcnkxMzE2MTk2NDY=","name":"dagster","full_name":"dagster-io/dagster","private":false,"owner":{"login":"dagster-io","id":40032576,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQwMDMyNTc2","avatar_url":"https://avatars0.githubusercontent.com/u/40032576?v=4","gravatar_id":"","url":"https://api.github.com/users/dagster-io","html_url":"https://github.com/dagster-io","followers_url":"https://api.github.com/users/dagster-io/followers","following_url":"https://api.github.com/users/dagster-io/following{/other_user}","gists_url":"https://api.github.com/users/dagster-io/gists{/gist_id}","starred_url":"https://api.github.com/users/dagster-io/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dagster-io/subscriptions","organizations_url":"https://api.github.com/users/dagster-io/orgs","repos_url":"https://api.github.com/users/dagster-io/repos","events_url":"https://api.github.com/users/dagster-io/events{/privacy}","received_events_url":"https://api.github.com/users/dagster-io/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/dagster-io/dagster","description":" Dagster is an opinionated system and programming model for data pipelines.","fork":false,"url":"https://api.github.com/repos/dagster-io/dagster","forks_url":"https://api.github.com/repos/dagster-io/dagster/forks","keys_url":"https://api.github.com/repos/dagster-io/dagster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dagster-io/dagster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dagster-io/dagster/teams","hooks_url":"https://api.github.com/repos/dagster-io/dagster/hooks","issue_events_url":"https://api.github.com/repos/dagster-io/dagster/issues/events{/number}","events_url":"https://api.github.com/repos/dagster-io/dagster/events","assignees_url":"https://api.github.com/repos/dagster-io/dagster/assignees{/user}","branches_url":"https://api.github.com/repos/dagster-io/dagster/branches{/branch}","tags_url":"https://api.github.com/repos/dagster-io/dagster/tags","blobs_url":"https://api.github.com/repos/dagster-io/dagster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dagster-io/dagster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dagster-io/dagster/git/refs{/sha}","trees_url":"https://api.github.com/repos/dagster-io/dagster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dagster-io/dagster/statuses/{sha}","languages_url":"https://api.github.com/repos/dagster-io/dagster/languages","stargazers_url":"https://api.github.com/repos/dagster-io/dagster/stargazers","contributors_url":"https://api.github.com/repos/dagster-io/dagster/contributors","subscribers_url":"https://api.github.com/repos/dagster-io/dagster/subscribers","subscription_url":"https://api.github.com/repos/dagster-io/dagster/subscription","commits_url":"https://api.github.com/repos/dagster-io/dagster/commits{/sha}","git_commits_url":"https://api.github.com/repos/dagster-io/dagster/git/commits{/sha}","comments_url":"https://api.github.com/repos/dagster-io/dagster/comments{/number}","issue_comment_url":"https://api.github.com/repos/dagster-io/dagster/issues/comments{/number}","contents_url":"https://api.github.com/repos/dagster-io/dagster/contents/{+path}","compare_url":"https://api.github.com/repos/dagster-io/dagster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dagster-io/dagster/merges","archive_url":"https://api.github.com/repos/dagster-io/dagster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dagster-io/dagster/downloads","issues_url":"https://api.github.com/repos/dagster-io/dagster/issues{/number}","pulls_url":"https://api.github.com/repos/dagster-io/dagster/pulls{/number}","milestones_url":"https://api.github.com/repos/dagster-io/dagster/milestones{/number}","notifications_url":"https://api.github.com/repos/dagster-io/dagster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dagster-io/dagster/labels{/name}","releases_url":"https://api.github.com/repos/dagster-io/dagster/releases{/id}","deployments_url":"https://api.github.com/repos/dagster-io/dagster/deployments","created_at":"2018-04-30T16:30:04Z","updated_at":"2018-11-28T15:13:38Z","pushed_at":"2018-11-28T16:17:52Z","git_url":"git://github.com/dagster-io/dagster.git","ssh_url":"git@github.com:dagster-io/dagster.git","clone_url":"https://github.com/dagster-io/dagster.git","svn_url":"https://github.com/dagster-io/dagster","homepage":"","size":2085,"stargazers_count":45,"watchers_count":45,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"archived":false,"open_issues_count":28,"license":null,"forks":3,"open_issues":28,"watchers":45,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dagster-io/dagster/pulls/305"},"html":{"href":"https://github.com/dagster-io/dagster/pull/305"},"issue":{"href":"https://api.github.com/repos/dagster-io/dagster/issues/305"},"comments":{"href":"https://api.github.com/repos/dagster-io/dagster/issues/305/comments"},"review_comments":{"href":"https://api.github.com/repos/dagster-io/dagster/pulls/305/comments"},"review_comment":{"href":"https://api.github.com/repos/dagster-io/dagster/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dagster-io/dagster/pulls/305/commits"},"statuses":{"href":"https://api.github.com/repos/dagster-io/dagster/statuses/13efc04fb49ac6a0608e99cbffabd4aae9ea3460"}},"author_association":"COLLABORATOR","merged":true,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":{"login":"schrockn","id":28738937,"node_id":"MDQ6VXNlcjI4NzM4OTM3","avatar_url":"https://avatars2.githubusercontent.com/u/28738937?v=4","gravatar_id":"","url":"https://api.github.com/users/schrockn","html_url":"https://github.com/schrockn","followers_url":"https://api.github.com/users/schrockn/followers","following_url":"https://api.github.com/users/schrockn/following{/other_user}","gists_url":"https://api.github.com/users/schrockn/gists{/gist_id}","starred_url":"https://api.github.com/users/schrockn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schrockn/subscriptions","organizations_url":"https://api.github.com/users/schrockn/orgs","repos_url":"https://api.github.com/users/schrockn/repos","events_url":"https://api.github.com/users/schrockn/events{/privacy}","received_events_url":"https://api.github.com/users/schrockn/received_events","type":"User","site_admin":false},"comments":0,"review_comments":0,"maintainer_can_modify":false,"commits":6,"additions":387,"deletions":39,"changed_files":8}}
{"actor":{"display_login":"schrockn"}}