{"action":"closed","number":1119,"pull_request":{"url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119","id":174846569,"html_url":"https://github.com/fedspendingtransparency/usaspending-api/pull/1119","diff_url":"https://github.com/fedspendingtransparency/usaspending-api/pull/1119.diff","patch_url":"https://github.com/fedspendingtransparency/usaspending-api/pull/1119.patch","issue_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/1119","number":1119,"state":"closed","locked":false,"title":"Hotfix historical fabs nightly","user":{"login":"nmonga91","id":16244012,"avatar_url":"https://avatars2.githubusercontent.com/u/16244012?v=4","gravatar_id":"","url":"https://api.github.com/users/nmonga91","html_url":"https://github.com/nmonga91","followers_url":"https://api.github.com/users/nmonga91/followers","following_url":"https://api.github.com/users/nmonga91/following{/other_user}","gists_url":"https://api.github.com/users/nmonga91/gists{/gist_id}","starred_url":"https://api.github.com/users/nmonga91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nmonga91/subscriptions","organizations_url":"https://api.github.com/users/nmonga91/orgs","repos_url":"https://api.github.com/users/nmonga91/repos","events_url":"https://api.github.com/users/nmonga91/events{/privacy}","received_events_url":"https://api.github.com/users/nmonga91/received_events","type":"User","site_admin":false},"body":"**High level description:**\r\nUpdating the nightly loader to consume historical FABS data from the Broker that should've been active, but were not marked as such.\r\n\r\n**Technical details:**\r\nUpdating the nightly loader to consume historical FABS data that should have been active. The loader uses a historical flag, as well as date and fiscal year arguments. Date will be used to compare to an exact match on `updated_at` since that is a manually set field for these records in question on the Broker. Fiscal year will be the year at which to start loading that data. If there is no data to load for the given Fiscal Year, the loader will go backwards in increments of 1 until it reaches < 1900. \r\n\r\nThe load will happen in batches of `500,000` records.\r\n\r\nHistorical records were not marked as active if their `modified_at` field was null and did not have another record replace it where the latter record's `modified_at` field was not null. The aforementioned records were reactivated with the following changes:\r\n1. `modified_at` = `action_date`\r\n2. `updated_at` = `2018-03-13 00:00:00`\r\n3. `is_active` = `TRUE`\r\n4. Only the most recent value is activated, where sorting is based on action date. If the action dates are identical, the Broker DB's primary key for the table is used instead.\r\n\r\n**Link to JIRA Ticket:**\r\n[DEV-544](https://federal-spending-transparency.atlassian.net/browse/DEV-544)\r\n\r\n**The following are ALL required for the PR to be merged:**\r\n- [ ] Backend review completed\r\n- [ ] Matview impact assessment completed\r\n    - Will increase data volume for matviews\r\n- [x] Frontend impact assessment completed (N/A)\r\n- [x] Data validation completed (N/A)\r\n- [ ] API Performance evaluation completed and present:\r\n    ```    \r\n    Data volume changes:\r\n        - will be adding ~3.3 million transactions\r\n    ```","created_at":"2018-03-14T03:38:42Z","updated_at":"2018-03-21T12:38:17Z","closed_at":"2018-03-21T12:38:17Z","merged_at":null,"merge_commit_sha":"febf5ea238a7d7796a68d1c950fe849bb9ca68da","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":690683541,"url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/labels/do%20not%20merge","name":"do not merge","color":"ee0701","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119/commits","review_comments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119/comments","review_comment_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/comments{/number}","comments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/1119/comments","statuses_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/statuses/c2cf2c6bd183d60d3b4c2669a2bc336112dd2007","head":{"label":"fedspendingtransparency:hotfix-historical-fabs-nightly","ref":"hotfix-historical-fabs-nightly","sha":"c2cf2c6bd183d60d3b4c2669a2bc336112dd2007","user":{"login":"fedspendingtransparency","id":8397762,"avatar_url":"https://avatars0.githubusercontent.com/u/8397762?v=4","gravatar_id":"","url":"https://api.github.com/users/fedspendingtransparency","html_url":"https://github.com/fedspendingtransparency","followers_url":"https://api.github.com/users/fedspendingtransparency/followers","following_url":"https://api.github.com/users/fedspendingtransparency/following{/other_user}","gists_url":"https://api.github.com/users/fedspendingtransparency/gists{/gist_id}","starred_url":"https://api.github.com/users/fedspendingtransparency/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fedspendingtransparency/subscriptions","organizations_url":"https://api.github.com/users/fedspendingtransparency/orgs","repos_url":"https://api.github.com/users/fedspendingtransparency/repos","events_url":"https://api.github.com/users/fedspendingtransparency/events{/privacy}","received_events_url":"https://api.github.com/users/fedspendingtransparency/received_events","type":"Organization","site_admin":false},"repo":{"id":65394827,"name":"usaspending-api","full_name":"fedspendingtransparency/usaspending-api","owner":{"login":"fedspendingtransparency","id":8397762,"avatar_url":"https://avatars0.githubusercontent.com/u/8397762?v=4","gravatar_id":"","url":"https://api.github.com/users/fedspendingtransparency","html_url":"https://github.com/fedspendingtransparency","followers_url":"https://api.github.com/users/fedspendingtransparency/followers","following_url":"https://api.github.com/users/fedspendingtransparency/following{/other_user}","gists_url":"https://api.github.com/users/fedspendingtransparency/gists{/gist_id}","starred_url":"https://api.github.com/users/fedspendingtransparency/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fedspendingtransparency/subscriptions","organizations_url":"https://api.github.com/users/fedspendingtransparency/orgs","repos_url":"https://api.github.com/users/fedspendingtransparency/repos","events_url":"https://api.github.com/users/fedspendingtransparency/events{/privacy}","received_events_url":"https://api.github.com/users/fedspendingtransparency/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/fedspendingtransparency/usaspending-api","description":"An RESTful API for U.S. federal spending data.","fork":false,"url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api","forks_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/forks","keys_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/keys{/key_id}","collaborators_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/teams","hooks_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/hooks","issue_events_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/events{/number}","events_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/events","assignees_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/assignees{/user}","branches_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/branches{/branch}","tags_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/tags","blobs_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/refs{/sha}","trees_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/trees{/sha}","statuses_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/statuses/{sha}","languages_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/languages","stargazers_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/stargazers","contributors_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/contributors","subscribers_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/subscribers","subscription_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/subscription","commits_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/commits{/sha}","git_commits_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/commits{/sha}","comments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/comments{/number}","issue_comment_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/comments{/number}","contents_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/contents/{+path}","compare_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/compare/{base}...{head}","merges_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/merges","archive_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/downloads","issues_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues{/number}","pulls_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls{/number}","milestones_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/milestones{/number}","notifications_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/labels{/name}","releases_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/releases{/id}","deployments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/deployments","created_at":"2016-08-10T15:39:45Z","updated_at":"2018-03-19T17:44:31Z","pushed_at":"2018-03-21T12:35:41Z","git_url":"git://github.com/fedspendingtransparency/usaspending-api.git","ssh_url":"git@github.com:fedspendingtransparency/usaspending-api.git","clone_url":"https://github.com/fedspendingtransparency/usaspending-api.git","svn_url":"https://github.com/fedspendingtransparency/usaspending-api","homepage":"","size":49103,"stargazers_count":57,"watchers_count":57,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":18,"mirror_url":null,"archived":false,"open_issues_count":9,"license":null,"forks":18,"open_issues":9,"watchers":57,"default_branch":"dev"}},"base":{"label":"fedspendingtransparency:stg","ref":"stg","sha":"13977d774caf7cb80d06ac2b1f153cac8b99458b","user":{"login":"fedspendingtransparency","id":8397762,"avatar_url":"https://avatars0.githubusercontent.com/u/8397762?v=4","gravatar_id":"","url":"https://api.github.com/users/fedspendingtransparency","html_url":"https://github.com/fedspendingtransparency","followers_url":"https://api.github.com/users/fedspendingtransparency/followers","following_url":"https://api.github.com/users/fedspendingtransparency/following{/other_user}","gists_url":"https://api.github.com/users/fedspendingtransparency/gists{/gist_id}","starred_url":"https://api.github.com/users/fedspendingtransparency/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fedspendingtransparency/subscriptions","organizations_url":"https://api.github.com/users/fedspendingtransparency/orgs","repos_url":"https://api.github.com/users/fedspendingtransparency/repos","events_url":"https://api.github.com/users/fedspendingtransparency/events{/privacy}","received_events_url":"https://api.github.com/users/fedspendingtransparency/received_events","type":"Organization","site_admin":false},"repo":{"id":65394827,"name":"usaspending-api","full_name":"fedspendingtransparency/usaspending-api","owner":{"login":"fedspendingtransparency","id":8397762,"avatar_url":"https://avatars0.githubusercontent.com/u/8397762?v=4","gravatar_id":"","url":"https://api.github.com/users/fedspendingtransparency","html_url":"https://github.com/fedspendingtransparency","followers_url":"https://api.github.com/users/fedspendingtransparency/followers","following_url":"https://api.github.com/users/fedspendingtransparency/following{/other_user}","gists_url":"https://api.github.com/users/fedspendingtransparency/gists{/gist_id}","starred_url":"https://api.github.com/users/fedspendingtransparency/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fedspendingtransparency/subscriptions","organizations_url":"https://api.github.com/users/fedspendingtransparency/orgs","repos_url":"https://api.github.com/users/fedspendingtransparency/repos","events_url":"https://api.github.com/users/fedspendingtransparency/events{/privacy}","received_events_url":"https://api.github.com/users/fedspendingtransparency/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/fedspendingtransparency/usaspending-api","description":"An RESTful API for U.S. federal spending data.","fork":false,"url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api","forks_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/forks","keys_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/keys{/key_id}","collaborators_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/teams","hooks_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/hooks","issue_events_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/events{/number}","events_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/events","assignees_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/assignees{/user}","branches_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/branches{/branch}","tags_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/tags","blobs_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/refs{/sha}","trees_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/trees{/sha}","statuses_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/statuses/{sha}","languages_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/languages","stargazers_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/stargazers","contributors_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/contributors","subscribers_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/subscribers","subscription_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/subscription","commits_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/commits{/sha}","git_commits_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/git/commits{/sha}","comments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/comments{/number}","issue_comment_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/comments{/number}","contents_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/contents/{+path}","compare_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/compare/{base}...{head}","merges_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/merges","archive_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/downloads","issues_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues{/number}","pulls_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls{/number}","milestones_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/milestones{/number}","notifications_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/labels{/name}","releases_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/releases{/id}","deployments_url":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/deployments","created_at":"2016-08-10T15:39:45Z","updated_at":"2018-03-19T17:44:31Z","pushed_at":"2018-03-21T12:35:41Z","git_url":"git://github.com/fedspendingtransparency/usaspending-api.git","ssh_url":"git@github.com:fedspendingtransparency/usaspending-api.git","clone_url":"https://github.com/fedspendingtransparency/usaspending-api.git","svn_url":"https://github.com/fedspendingtransparency/usaspending-api","homepage":"","size":49103,"stargazers_count":57,"watchers_count":57,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":18,"mirror_url":null,"archived":false,"open_issues_count":9,"license":null,"forks":18,"open_issues":9,"watchers":57,"default_branch":"dev"}},"_links":{"self":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119"},"html":{"href":"https://github.com/fedspendingtransparency/usaspending-api/pull/1119"},"issue":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/1119"},"comments":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/issues/1119/comments"},"review_comments":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119/comments"},"review_comment":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/pulls/1119/commits"},"statuses":{"href":"https://api.github.com/repos/fedspendingtransparency/usaspending-api/statuses/c2cf2c6bd183d60d3b4c2669a2bc336112dd2007"}},"author_association":"CONTRIBUTOR","merged":false,"mergeable":true,"rebaseable":true,"mergeable_state":"unstable","merged_by":null,"comments":0,"review_comments":0,"maintainer_can_modify":false,"commits":7,"additions":71,"deletions":1,"changed_files":1}}
{"actor":{"display_login":"nmonga91"}}