{"action":"created","issue":{"url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/162","repository_url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader","labels_url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/162/labels{/name}","comments_url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/162/comments","events_url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/162/events","html_url":"https://github.com/awslabs/aws-lambda-redshift-loader/issues/162","id":339649250,"node_id":"MDU6SXNzdWUzMzk2NDkyNTA=","number":162,"title":"reprocessBatch.js not triggering lambda","user":{"login":"kdelga","id":31020192,"node_id":"MDQ6VXNlcjMxMDIwMTky","avatar_url":"https://avatars2.githubusercontent.com/u/31020192?v=4","gravatar_id":"","url":"https://api.github.com/users/kdelga","html_url":"https://github.com/kdelga","followers_url":"https://api.github.com/users/kdelga/followers","following_url":"https://api.github.com/users/kdelga/following{/other_user}","gists_url":"https://api.github.com/users/kdelga/gists{/gist_id}","starred_url":"https://api.github.com/users/kdelga/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kdelga/subscriptions","organizations_url":"https://api.github.com/users/kdelga/orgs","repos_url":"https://api.github.com/users/kdelga/repos","events_url":"https://api.github.com/users/kdelga/events{/privacy}","received_events_url":"https://api.github.com/users/kdelga/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":29,"created_at":"2018-07-10T00:11:28Z","updated_at":"2018-08-08T10:47:14Z","closed_at":null,"author_association":"NONE","body":"Been using the lambda for a few months now and it's been working great! \r\nWe had originally been ignoring the reprocessing of failed loads (which have been happening due to redshift timeouts), but now that we want to address this issue we are running into a problem running the `reprocessBatch.js` script. \r\n\r\nWe can successfully run the script and see it print out `Submitted reprocess request for s3_bucket/our_file_1` for each file. We can see in dynamo that the batch has been updated to `reprocessed` state. We can also see that the last updated field of the file on S3 is modified to the current time. It looks like everything is working as expected but it simply isn't loaded by the lambda loader. We spot checked individual records within the files to confirm they are not making it to RS and nothing in the current open batches indicate that these reprocessed files have been added.\r\n\r\nOur initial thought was that the \"in place copy\" command that the script runs was not triggering the lambda, but we have no way of verifying this hypothesis one way or the other. Neither the lambda UI nor the RS query UI indicate that anything has changed with this batch. Any thoughts on what's going on here would be greatly appreciated."},"comment":{"url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/comments/411365077","html_url":"https://github.com/awslabs/aws-lambda-redshift-loader/issues/162#issuecomment-411365077","issue_url":"https://api.github.com/repos/awslabs/aws-lambda-redshift-loader/issues/162","id":411365077,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMTM2NTA3Nw==","user":{"login":"IanMeyers","id":6339696,"node_id":"MDQ6VXNlcjYzMzk2OTY=","avatar_url":"https://avatars3.githubusercontent.com/u/6339696?v=4","gravatar_id":"","url":"https://api.github.com/users/IanMeyers","html_url":"https://github.com/IanMeyers","followers_url":"https://api.github.com/users/IanMeyers/followers","following_url":"https://api.github.com/users/IanMeyers/following{/other_user}","gists_url":"https://api.github.com/users/IanMeyers/gists{/gist_id}","starred_url":"https://api.github.com/users/IanMeyers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/IanMeyers/subscriptions","organizations_url":"https://api.github.com/users/IanMeyers/orgs","repos_url":"https://api.github.com/users/IanMeyers/repos","events_url":"https://api.github.com/users/IanMeyers/events{/privacy}","received_events_url":"https://api.github.com/users/IanMeyers/received_events","type":"User","site_admin":false},"created_at":"2018-08-08T10:47:14Z","updated_at":"2018-08-08T10:47:14Z","author_association":"CONTRIBUTOR","body":"I've fixed what I believe to be the root cause of this in 51169b3. The file reprocessing function was incorrectly constructed and not being run. I have retested and see a previously processed file rotated onto a newly opened batch as expected."}}
{"actor":{"display_login":"IanMeyers"}}