{"action":"created","comment":{"url":"https://api.github.com/repos/nasa/cumulus/pulls/comments/234009740","pull_request_review_id":175558255,"id":234009740,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzNDAwOTc0MA==","diff_hunk":"@@ -7,12 +7,16 @@ hide_title: true\n # Workflow Triggers\n For a workflow to run, it needs to be associated with a rule (see [rule configuration](data-cookbooks/setup.md#rules). The rule configuration determines how and when a workflow execution is triggered. Rules can be triggered one time, on a schedule, or by new data written to a kinesis stream.\n \n-There are three lambda functions in the API package responsible for scheduling and starting workflows: `SF scheduler`, `kinesis consumer`, `and SF starter`. Each Cumulus instance comes with a Start SF [SQS queue](https://aws.amazon.com/sqs/).\n+There are three lambda functions in the API package responsible for scheduling and starting workflows: `SF scheduler`, `message consumer`, `and SF starter`. Each Cumulus instance comes with a Start SF [SQS queue](https://aws.amazon.com/sqs/).\n \n The `SF scheduler` lambda puts a message onto the `start SF` queue. This message is picked up the `Start SF` lambda and an execution is started with the body of the message as the input.\n \n When a one time rule is created, the `schedule SF` lambda is triggered. Rules that are not one time are associated with a [CloudWatch event](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/WhatIsCloudWatchEvents.html) which will manage the trigger of the lambdas that trigger the workflows.\n \n-For a scheduled rule, the Cloudwatch event is triggered on the given schedule which calls directly to the `schedule SF` lambda. For a kinesis rule, when data is added to the kinesis stream, the Cloudwatch event is triggered, which calls the `kinesis consumer` lambda. The `kinesis consumer` lambda parses the kinesis message and finds all of the rules associated with that message. For each rule (which corresponds to one workflow), the `schedule SF` lambda is triggered to queue a message to start the workflow.\n+For a scheduled rule, the Cloudwatch event is triggered on the given schedule which calls directly to the `schedule SF` lambda. \n+\n+For a kinesis rule, when data is added to the kinesis stream, the Cloudwatch event is triggered, which calls the `message consumer` lambda. The `message consumer` lambda parses the kinesis message and finds all of the rules associated with that message. For each rule (which corresponds to one workflow), the `schedule SF` lambda is triggered to queue a message to start the workflow.\n+\n+For an sns rule, when a message is published to the SNS topic, the `message consumer` receives the SNS message (JSON expected), parses it into an object, starts a new execution of the workflow associated with the rule and passes the object in the `payload` field of the Cumulus message.","path":"docs/workflows/workflow-triggers.md","position":16,"original_position":16,"commit_id":"135278e6399fba0416db986fbd2ecc8ed2154f07","original_commit_id":"02abbfa4565ce128f44ff11c2b67f24a7eba5a0d","user":{"login":"mennovandiermen","id":14986053,"node_id":"MDQ6VXNlcjE0OTg2MDUz","avatar_url":"https://avatars3.githubusercontent.com/u/14986053?v=4","gravatar_id":"","url":"https://api.github.com/users/mennovandiermen","html_url":"https://github.com/mennovandiermen","followers_url":"https://api.github.com/users/mennovandiermen/followers","following_url":"https://api.github.com/users/mennovandiermen/following{/other_user}","gists_url":"https://api.github.com/users/mennovandiermen/gists{/gist_id}","starred_url":"https://api.github.com/users/mennovandiermen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mennovandiermen/subscriptions","organizations_url":"https://api.github.com/users/mennovandiermen/orgs","repos_url":"https://api.github.com/users/mennovandiermen/repos","events_url":"https://api.github.com/users/mennovandiermen/events{/privacy}","received_events_url":"https://api.github.com/users/mennovandiermen/received_events","type":"User","site_admin":false},"body":"I left that one lowercase because the value in the rule definition must be `sns`. Maybe I should backtick it, same for the other types of rules above?","created_at":"2018-11-15T21:12:54Z","updated_at":"2018-11-15T21:12:54Z","html_url":"https://github.com/nasa/cumulus/pull/697#discussion_r234009740","pull_request_url":"https://api.github.com/repos/nasa/cumulus/pulls/697","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/comments/234009740"},"html":{"href":"https://github.com/nasa/cumulus/pull/697#discussion_r234009740"},"pull_request":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/697"}},"in_reply_to_id":233957921},"pull_request":{"url":"https://api.github.com/repos/nasa/cumulus/pulls/697","id":229844077,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI5ODQ0MDc3","html_url":"https://github.com/nasa/cumulus/pull/697","diff_url":"https://github.com/nasa/cumulus/pull/697.diff","patch_url":"https://github.com/nasa/cumulus/pull/697.patch","issue_url":"https://api.github.com/repos/nasa/cumulus/issues/697","number":697,"state":"open","locked":false,"title":"CUMULUS-977: Trigger workflow via SNS","user":{"login":"mennovandiermen","id":14986053,"node_id":"MDQ6VXNlcjE0OTg2MDUz","avatar_url":"https://avatars3.githubusercontent.com/u/14986053?v=4","gravatar_id":"","url":"https://api.github.com/users/mennovandiermen","html_url":"https://github.com/mennovandiermen","followers_url":"https://api.github.com/users/mennovandiermen/followers","following_url":"https://api.github.com/users/mennovandiermen/following{/other_user}","gists_url":"https://api.github.com/users/mennovandiermen/gists{/gist_id}","starred_url":"https://api.github.com/users/mennovandiermen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mennovandiermen/subscriptions","organizations_url":"https://api.github.com/users/mennovandiermen/orgs","repos_url":"https://api.github.com/users/mennovandiermen/repos","events_url":"https://api.github.com/users/mennovandiermen/events{/privacy}","received_events_url":"https://api.github.com/users/mennovandiermen/received_events","type":"User","site_admin":false},"body":"**Summary:** Summary of changes\r\n\r\nAddresses [CUMULUS-977: SNS-triggered workflows](https://bugs.earthdata.nasa.gov/browse/CUMULUS-977)\r\n\r\n## Changes\r\n\r\n  - Rename `kinesisConsumer` to `messageConsumer` as it handles both Kinesis streams and SNS topics as of this version.\r\n  - Add `sns`-type rule support. These rules create a subscription between an SNS topic and the `messageConsumer`. When a message is received, `messageConsumer` is triggered and passes the SNS message (JSON format expected) in its entirety to the workflow in the `payload` field of the Cumulus message.\r\n\r\n## Test Plan\r\nThings that should succeed before merging.\r\n\r\n- [x] Unit tests\r\n- [x] Integration tests\r\n- [x] Adhoc testing\r\n- [x] Update CHANGELOG\r\n","created_at":"2018-11-09T23:06:51Z","updated_at":"2018-11-15T21:12:54Z","closed_at":null,"merged_at":null,"merge_commit_sha":"18dddc7f8550069d3691bd54ca0925469dce5279","assignee":{"login":"laurenfrederick","id":25331898,"node_id":"MDQ6VXNlcjI1MzMxODk4","avatar_url":"https://avatars3.githubusercontent.com/u/25331898?v=4","gravatar_id":"","url":"https://api.github.com/users/laurenfrederick","html_url":"https://github.com/laurenfrederick","followers_url":"https://api.github.com/users/laurenfrederick/followers","following_url":"https://api.github.com/users/laurenfrederick/following{/other_user}","gists_url":"https://api.github.com/users/laurenfrederick/gists{/gist_id}","starred_url":"https://api.github.com/users/laurenfrederick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laurenfrederick/subscriptions","organizations_url":"https://api.github.com/users/laurenfrederick/orgs","repos_url":"https://api.github.com/users/laurenfrederick/repos","events_url":"https://api.github.com/users/laurenfrederick/events{/privacy}","received_events_url":"https://api.github.com/users/laurenfrederick/received_events","type":"User","site_admin":false},"assignees":[{"login":"laurenfrederick","id":25331898,"node_id":"MDQ6VXNlcjI1MzMxODk4","avatar_url":"https://avatars3.githubusercontent.com/u/25331898?v=4","gravatar_id":"","url":"https://api.github.com/users/laurenfrederick","html_url":"https://github.com/laurenfrederick","followers_url":"https://api.github.com/users/laurenfrederick/followers","following_url":"https://api.github.com/users/laurenfrederick/following{/other_user}","gists_url":"https://api.github.com/users/laurenfrederick/gists{/gist_id}","starred_url":"https://api.github.com/users/laurenfrederick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laurenfrederick/subscriptions","organizations_url":"https://api.github.com/users/laurenfrederick/orgs","repos_url":"https://api.github.com/users/laurenfrederick/repos","events_url":"https://api.github.com/users/laurenfrederick/events{/privacy}","received_events_url":"https://api.github.com/users/laurenfrederick/received_events","type":"User","site_admin":false}],"requested_reviewers":[{"login":"Jkovarik","id":3083296,"node_id":"MDQ6VXNlcjMwODMyOTY=","avatar_url":"https://avatars0.githubusercontent.com/u/3083296?v=4","gravatar_id":"","url":"https://api.github.com/users/Jkovarik","html_url":"https://github.com/Jkovarik","followers_url":"https://api.github.com/users/Jkovarik/followers","following_url":"https://api.github.com/users/Jkovarik/following{/other_user}","gists_url":"https://api.github.com/users/Jkovarik/gists{/gist_id}","starred_url":"https://api.github.com/users/Jkovarik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Jkovarik/subscriptions","organizations_url":"https://api.github.com/users/Jkovarik/orgs","repos_url":"https://api.github.com/users/Jkovarik/repos","events_url":"https://api.github.com/users/Jkovarik/events{/privacy}","received_events_url":"https://api.github.com/users/Jkovarik/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[{"id":989762822,"node_id":"MDU6TGFiZWw5ODk3NjI4MjI=","url":"https://api.github.com/repos/nasa/cumulus/labels/PR%20feedback","name":"PR feedback","color":"b7e8f4","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/nasa/cumulus/pulls/697/commits","review_comments_url":"https://api.github.com/repos/nasa/cumulus/pulls/697/comments","review_comment_url":"https://api.github.com/repos/nasa/cumulus/pulls/comments{/number}","comments_url":"https://api.github.com/repos/nasa/cumulus/issues/697/comments","statuses_url":"https://api.github.com/repos/nasa/cumulus/statuses/135278e6399fba0416db986fbd2ecc8ed2154f07","head":{"label":"nasa:CUMULUS-977-sns-consumer","ref":"CUMULUS-977-sns-consumer","sha":"135278e6399fba0416db986fbd2ecc8ed2154f07","user":{"login":"nasa","id":848102,"node_id":"MDEyOk9yZ2FuaXphdGlvbjg0ODEwMg==","avatar_url":"https://avatars2.githubusercontent.com/u/848102?v=4","gravatar_id":"","url":"https://api.github.com/users/nasa","html_url":"https://github.com/nasa","followers_url":"https://api.github.com/users/nasa/followers","following_url":"https://api.github.com/users/nasa/following{/other_user}","gists_url":"https://api.github.com/users/nasa/gists{/gist_id}","starred_url":"https://api.github.com/users/nasa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nasa/subscriptions","organizations_url":"https://api.github.com/users/nasa/orgs","repos_url":"https://api.github.com/users/nasa/repos","events_url":"https://api.github.com/users/nasa/events{/privacy}","received_events_url":"https://api.github.com/users/nasa/received_events","type":"Organization","site_admin":false},"repo":{"id":95879196,"node_id":"MDEwOlJlcG9zaXRvcnk5NTg3OTE5Ng==","name":"cumulus","full_name":"nasa/cumulus","private":false,"owner":{"login":"nasa","id":848102,"node_id":"MDEyOk9yZ2FuaXphdGlvbjg0ODEwMg==","avatar_url":"https://avatars2.githubusercontent.com/u/848102?v=4","gravatar_id":"","url":"https://api.github.com/users/nasa","html_url":"https://github.com/nasa","followers_url":"https://api.github.com/users/nasa/followers","following_url":"https://api.github.com/users/nasa/following{/other_user}","gists_url":"https://api.github.com/users/nasa/gists{/gist_id}","starred_url":"https://api.github.com/users/nasa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nasa/subscriptions","organizations_url":"https://api.github.com/users/nasa/orgs","repos_url":"https://api.github.com/users/nasa/repos","events_url":"https://api.github.com/users/nasa/events{/privacy}","received_events_url":"https://api.github.com/users/nasa/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/nasa/cumulus","description":"Cumulus Framework + Cumulus API","fork":false,"url":"https://api.github.com/repos/nasa/cumulus","forks_url":"https://api.github.com/repos/nasa/cumulus/forks","keys_url":"https://api.github.com/repos/nasa/cumulus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nasa/cumulus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nasa/cumulus/teams","hooks_url":"https://api.github.com/repos/nasa/cumulus/hooks","issue_events_url":"https://api.github.com/repos/nasa/cumulus/issues/events{/number}","events_url":"https://api.github.com/repos/nasa/cumulus/events","assignees_url":"https://api.github.com/repos/nasa/cumulus/assignees{/user}","branches_url":"https://api.github.com/repos/nasa/cumulus/branches{/branch}","tags_url":"https://api.github.com/repos/nasa/cumulus/tags","blobs_url":"https://api.github.com/repos/nasa/cumulus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nasa/cumulus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nasa/cumulus/git/refs{/sha}","trees_url":"https://api.github.com/repos/nasa/cumulus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nasa/cumulus/statuses/{sha}","languages_url":"https://api.github.com/repos/nasa/cumulus/languages","stargazers_url":"https://api.github.com/repos/nasa/cumulus/stargazers","contributors_url":"https://api.github.com/repos/nasa/cumulus/contributors","subscribers_url":"https://api.github.com/repos/nasa/cumulus/subscribers","subscription_url":"https://api.github.com/repos/nasa/cumulus/subscription","commits_url":"https://api.github.com/repos/nasa/cumulus/commits{/sha}","git_commits_url":"https://api.github.com/repos/nasa/cumulus/git/commits{/sha}","comments_url":"https://api.github.com/repos/nasa/cumulus/comments{/number}","issue_comment_url":"https://api.github.com/repos/nasa/cumulus/issues/comments{/number}","contents_url":"https://api.github.com/repos/nasa/cumulus/contents/{+path}","compare_url":"https://api.github.com/repos/nasa/cumulus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nasa/cumulus/merges","archive_url":"https://api.github.com/repos/nasa/cumulus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nasa/cumulus/downloads","issues_url":"https://api.github.com/repos/nasa/cumulus/issues{/number}","pulls_url":"https://api.github.com/repos/nasa/cumulus/pulls{/number}","milestones_url":"https://api.github.com/repos/nasa/cumulus/milestones{/number}","notifications_url":"https://api.github.com/repos/nasa/cumulus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nasa/cumulus/labels{/name}","releases_url":"https://api.github.com/repos/nasa/cumulus/releases{/id}","deployments_url":"https://api.github.com/repos/nasa/cumulus/deployments","created_at":"2017-06-30T10:52:28Z","updated_at":"2018-11-15T15:10:23Z","pushed_at":"2018-11-15T21:06:00Z","git_url":"git://github.com/nasa/cumulus.git","ssh_url":"git@github.com:nasa/cumulus.git","clone_url":"https://github.com/nasa/cumulus.git","svn_url":"https://github.com/nasa/cumulus","homepage":"","size":63287,"stargazers_count":26,"watchers_count":26,"language":"JavaScript","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":18,"mirror_url":null,"archived":false,"open_issues_count":8,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":18,"open_issues":8,"watchers":26,"default_branch":"master"}},"base":{"label":"nasa:master","ref":"master","sha":"05d2b73aaf441ba7b65a653d18da87251a2835a8","user":{"login":"nasa","id":848102,"node_id":"MDEyOk9yZ2FuaXphdGlvbjg0ODEwMg==","avatar_url":"https://avatars2.githubusercontent.com/u/848102?v=4","gravatar_id":"","url":"https://api.github.com/users/nasa","html_url":"https://github.com/nasa","followers_url":"https://api.github.com/users/nasa/followers","following_url":"https://api.github.com/users/nasa/following{/other_user}","gists_url":"https://api.github.com/users/nasa/gists{/gist_id}","starred_url":"https://api.github.com/users/nasa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nasa/subscriptions","organizations_url":"https://api.github.com/users/nasa/orgs","repos_url":"https://api.github.com/users/nasa/repos","events_url":"https://api.github.com/users/nasa/events{/privacy}","received_events_url":"https://api.github.com/users/nasa/received_events","type":"Organization","site_admin":false},"repo":{"id":95879196,"node_id":"MDEwOlJlcG9zaXRvcnk5NTg3OTE5Ng==","name":"cumulus","full_name":"nasa/cumulus","private":false,"owner":{"login":"nasa","id":848102,"node_id":"MDEyOk9yZ2FuaXphdGlvbjg0ODEwMg==","avatar_url":"https://avatars2.githubusercontent.com/u/848102?v=4","gravatar_id":"","url":"https://api.github.com/users/nasa","html_url":"https://github.com/nasa","followers_url":"https://api.github.com/users/nasa/followers","following_url":"https://api.github.com/users/nasa/following{/other_user}","gists_url":"https://api.github.com/users/nasa/gists{/gist_id}","starred_url":"https://api.github.com/users/nasa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nasa/subscriptions","organizations_url":"https://api.github.com/users/nasa/orgs","repos_url":"https://api.github.com/users/nasa/repos","events_url":"https://api.github.com/users/nasa/events{/privacy}","received_events_url":"https://api.github.com/users/nasa/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/nasa/cumulus","description":"Cumulus Framework + Cumulus API","fork":false,"url":"https://api.github.com/repos/nasa/cumulus","forks_url":"https://api.github.com/repos/nasa/cumulus/forks","keys_url":"https://api.github.com/repos/nasa/cumulus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nasa/cumulus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nasa/cumulus/teams","hooks_url":"https://api.github.com/repos/nasa/cumulus/hooks","issue_events_url":"https://api.github.com/repos/nasa/cumulus/issues/events{/number}","events_url":"https://api.github.com/repos/nasa/cumulus/events","assignees_url":"https://api.github.com/repos/nasa/cumulus/assignees{/user}","branches_url":"https://api.github.com/repos/nasa/cumulus/branches{/branch}","tags_url":"https://api.github.com/repos/nasa/cumulus/tags","blobs_url":"https://api.github.com/repos/nasa/cumulus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nasa/cumulus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nasa/cumulus/git/refs{/sha}","trees_url":"https://api.github.com/repos/nasa/cumulus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nasa/cumulus/statuses/{sha}","languages_url":"https://api.github.com/repos/nasa/cumulus/languages","stargazers_url":"https://api.github.com/repos/nasa/cumulus/stargazers","contributors_url":"https://api.github.com/repos/nasa/cumulus/contributors","subscribers_url":"https://api.github.com/repos/nasa/cumulus/subscribers","subscription_url":"https://api.github.com/repos/nasa/cumulus/subscription","commits_url":"https://api.github.com/repos/nasa/cumulus/commits{/sha}","git_commits_url":"https://api.github.com/repos/nasa/cumulus/git/commits{/sha}","comments_url":"https://api.github.com/repos/nasa/cumulus/comments{/number}","issue_comment_url":"https://api.github.com/repos/nasa/cumulus/issues/comments{/number}","contents_url":"https://api.github.com/repos/nasa/cumulus/contents/{+path}","compare_url":"https://api.github.com/repos/nasa/cumulus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nasa/cumulus/merges","archive_url":"https://api.github.com/repos/nasa/cumulus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nasa/cumulus/downloads","issues_url":"https://api.github.com/repos/nasa/cumulus/issues{/number}","pulls_url":"https://api.github.com/repos/nasa/cumulus/pulls{/number}","milestones_url":"https://api.github.com/repos/nasa/cumulus/milestones{/number}","notifications_url":"https://api.github.com/repos/nasa/cumulus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nasa/cumulus/labels{/name}","releases_url":"https://api.github.com/repos/nasa/cumulus/releases{/id}","deployments_url":"https://api.github.com/repos/nasa/cumulus/deployments","created_at":"2017-06-30T10:52:28Z","updated_at":"2018-11-15T15:10:23Z","pushed_at":"2018-11-15T21:06:00Z","git_url":"git://github.com/nasa/cumulus.git","ssh_url":"git@github.com:nasa/cumulus.git","clone_url":"https://github.com/nasa/cumulus.git","svn_url":"https://github.com/nasa/cumulus","homepage":"","size":63287,"stargazers_count":26,"watchers_count":26,"language":"JavaScript","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":18,"mirror_url":null,"archived":false,"open_issues_count":8,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":18,"open_issues":8,"watchers":26,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/697"},"html":{"href":"https://github.com/nasa/cumulus/pull/697"},"issue":{"href":"https://api.github.com/repos/nasa/cumulus/issues/697"},"comments":{"href":"https://api.github.com/repos/nasa/cumulus/issues/697/comments"},"review_comments":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/697/comments"},"review_comment":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/nasa/cumulus/pulls/697/commits"},"statuses":{"href":"https://api.github.com/repos/nasa/cumulus/statuses/135278e6399fba0416db986fbd2ecc8ed2154f07"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"mennovandiermen"}}