{"action":"created","comment":{"url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/comments/245337599","pull_request_review_id":189418942,"id":245337599,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0NTMzNzU5OQ==","diff_hunk":"@@ -0,0 +1,163 @@\n+# Housing Counselor Data Processing\n+\n+The [Find a Housing Counselor tool](https://www.consumerfinance.gov/find-a-housing-counselor/) gets its data from the US Department of Housing and Urban Development. HUD provides an API to their list of approved counseling agencies.\n+\n+We generate the JSON data for the Find a Housing Counselor tool using the `hud_generate_json` management command. This page describes how that command works to transform HUD data to the JSON file format we use for the FAHC tool, including the following steps:\n+  1. fetch agency listings from HUD\n+  2. clean the results\n+  3. fill in any missing latitude and longitude values\n+  4. create files of the 10 nearest results for each US ZIP code\n+\n+\n+## Fetch agency listings\n+\n+(_in `fetcher.py`_)\n+\n+Request every housing counselor in HUD's database with this request:\n+\n+```\n+https://data.hud.gov/Housing_Counselor/searchByLocation?Lat=38.8951&Long=-77.0367&Distance=5000\n+```\n+\n+It returns thousands of results like this:\n+\n+```json\n+    {\n+        \"services\": \"DFC,FBC,PPC,RHC\",\n+        \"languages\": \"ENG,SPA\",\n+        \"agc_STATUS\": \"A\",\n+        \"agc_SRC_CD\": \"HUD\",\n+        \"counslg_METHOD\": \"Face to Face Counseling,Group Counseling,Internet Counseling,Phone Counseling\",\n+        \"agcid\": \"80790\",\n+        \"adr1\": \"24331 W. Eight Mile Road\",","path":"docs/housing-counselor-data.md","position":32,"original_position":32,"commit_id":"3f63675712043d7f16601c8baf2494a858facc7e","original_commit_id":"3f63675712043d7f16601c8baf2494a858facc7e","user":{"login":"chosak","id":654645,"node_id":"MDQ6VXNlcjY1NDY0NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/654645?v=4","gravatar_id":"","url":"https://api.github.com/users/chosak","html_url":"https://github.com/chosak","followers_url":"https://api.github.com/users/chosak/followers","following_url":"https://api.github.com/users/chosak/following{/other_user}","gists_url":"https://api.github.com/users/chosak/gists{/gist_id}","starred_url":"https://api.github.com/users/chosak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chosak/subscriptions","organizations_url":"https://api.github.com/users/chosak/orgs","repos_url":"https://api.github.com/users/chosak/repos","events_url":"https://api.github.com/users/chosak/events{/privacy}","received_events_url":"https://api.github.com/users/chosak/received_events","type":"User","site_admin":false},"body":"It occurs to me that it might be a good idea to modify this data (both here and below) into a made-up counselor instead of including specific details of a real one. That way in future if this real one changes its data or is closed, we won't have out-of-date data in our docs. We also wouldn't want to be seen as specifically endorsing or highlighting one counselor.","created_at":"2019-01-04T15:53:36Z","updated_at":"2019-01-04T15:58:17Z","html_url":"https://github.com/cfpb/cfgov-refresh/pull/4728#discussion_r245337599","pull_request_url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/comments/245337599"},"html":{"href":"https://github.com/cfpb/cfgov-refresh/pull/4728#discussion_r245337599"},"pull_request":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728"}}},"pull_request":{"url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728","id":241354910,"node_id":"MDExOlB1bGxSZXF1ZXN0MjQxMzU0OTEw","html_url":"https://github.com/cfpb/cfgov-refresh/pull/4728","diff_url":"https://github.com/cfpb/cfgov-refresh/pull/4728.diff","patch_url":"https://github.com/cfpb/cfgov-refresh/pull/4728.patch","issue_url":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/4728","number":4728,"state":"open","locked":false,"title":"Documentation for hud_generate_json management command","user":{"login":"schbetsy","id":4295388,"node_id":"MDQ6VXNlcjQyOTUzODg=","avatar_url":"https://avatars2.githubusercontent.com/u/4295388?v=4","gravatar_id":"","url":"https://api.github.com/users/schbetsy","html_url":"https://github.com/schbetsy","followers_url":"https://api.github.com/users/schbetsy/followers","following_url":"https://api.github.com/users/schbetsy/following{/other_user}","gists_url":"https://api.github.com/users/schbetsy/gists{/gist_id}","starred_url":"https://api.github.com/users/schbetsy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schbetsy/subscriptions","organizations_url":"https://api.github.com/users/schbetsy/orgs","repos_url":"https://api.github.com/users/schbetsy/repos","events_url":"https://api.github.com/users/schbetsy/events{/privacy}","received_events_url":"https://api.github.com/users/schbetsy/received_events","type":"User","site_admin":false},"body":"An overview of how we generate the JSON files that power the Find a Housing Counselor tool.\r\n\r\n## Additions\r\n\r\n- a documentation page under the \"Back End Guidelines\" category, called \"Housing Counselor Data Processing\" \r\n\r\n## Notes\r\n\r\n- I'm not up to speed on our documentation standards. Feedback welcome on where this should live, its organization, formatting, and content, and anything else.\r\n\r\n## Checklist\r\n\r\n- [x] PR has an informative and human-readable title\r\n- [x] Changes are limited to a single goal (no scope creep)\r\n- [x] Code can be automatically merged (no conflicts)\r\n- [ ] Code follows the standards laid out in the [development playbook](https://github.com/cfpb/development)\r\n- [x] Passes all existing automated tests\r\n- [x] Any _change_ in functionality is tested\r\n- [x] New functions are documented (with a description, list of inputs, and expected output)\r\n- [x] Placeholder code is flagged / future todos are captured in comments\r\n- [ ] Visually tested in supported browsers and devices (see checklist below :point_down:)\r\n- [x] Project documentation has been updated\r\n- [ ] Reviewers requested with the [Reviewers tool](https://help.github.com/articles/requesting-a-pull-request-review/) :arrow_right:","created_at":"2018-12-28T15:50:25Z","updated_at":"2019-01-04T15:58:17Z","closed_at":null,"merged_at":null,"merge_commit_sha":"30b9faaee40727b38b0a7bd40fe007f6d08e9e53","assignee":null,"assignees":[],"requested_reviewers":[{"login":"cfarm","id":702526,"node_id":"MDQ6VXNlcjcwMjUyNg==","avatar_url":"https://avatars3.githubusercontent.com/u/702526?v=4","gravatar_id":"","url":"https://api.github.com/users/cfarm","html_url":"https://github.com/cfarm","followers_url":"https://api.github.com/users/cfarm/followers","following_url":"https://api.github.com/users/cfarm/following{/other_user}","gists_url":"https://api.github.com/users/cfarm/gists{/gist_id}","starred_url":"https://api.github.com/users/cfarm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cfarm/subscriptions","organizations_url":"https://api.github.com/users/cfarm/orgs","repos_url":"https://api.github.com/users/cfarm/repos","events_url":"https://api.github.com/users/cfarm/events{/privacy}","received_events_url":"https://api.github.com/users/cfarm/received_events","type":"User","site_admin":false},{"login":"willbarton","id":10562538,"node_id":"MDQ6VXNlcjEwNTYyNTM4","avatar_url":"https://avatars0.githubusercontent.com/u/10562538?v=4","gravatar_id":"","url":"https://api.github.com/users/willbarton","html_url":"https://github.com/willbarton","followers_url":"https://api.github.com/users/willbarton/followers","following_url":"https://api.github.com/users/willbarton/following{/other_user}","gists_url":"https://api.github.com/users/willbarton/gists{/gist_id}","starred_url":"https://api.github.com/users/willbarton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willbarton/subscriptions","organizations_url":"https://api.github.com/users/willbarton/orgs","repos_url":"https://api.github.com/users/willbarton/repos","events_url":"https://api.github.com/users/willbarton/events{/privacy}","received_events_url":"https://api.github.com/users/willbarton/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728/commits","review_comments_url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728/comments","review_comment_url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/comments{/number}","comments_url":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/4728/comments","statuses_url":"https://api.github.com/repos/cfpb/cfgov-refresh/statuses/3f63675712043d7f16601c8baf2494a858facc7e","head":{"label":"schbetsy:hud_docs","ref":"hud_docs","sha":"3f63675712043d7f16601c8baf2494a858facc7e","user":{"login":"schbetsy","id":4295388,"node_id":"MDQ6VXNlcjQyOTUzODg=","avatar_url":"https://avatars2.githubusercontent.com/u/4295388?v=4","gravatar_id":"","url":"https://api.github.com/users/schbetsy","html_url":"https://github.com/schbetsy","followers_url":"https://api.github.com/users/schbetsy/followers","following_url":"https://api.github.com/users/schbetsy/following{/other_user}","gists_url":"https://api.github.com/users/schbetsy/gists{/gist_id}","starred_url":"https://api.github.com/users/schbetsy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schbetsy/subscriptions","organizations_url":"https://api.github.com/users/schbetsy/orgs","repos_url":"https://api.github.com/users/schbetsy/repos","events_url":"https://api.github.com/users/schbetsy/events{/privacy}","received_events_url":"https://api.github.com/users/schbetsy/received_events","type":"User","site_admin":false},"repo":{"id":143042436,"node_id":"MDEwOlJlcG9zaXRvcnkxNDMwNDI0MzY=","name":"cfgov-refresh","full_name":"schbetsy/cfgov-refresh","private":false,"owner":{"login":"schbetsy","id":4295388,"node_id":"MDQ6VXNlcjQyOTUzODg=","avatar_url":"https://avatars2.githubusercontent.com/u/4295388?v=4","gravatar_id":"","url":"https://api.github.com/users/schbetsy","html_url":"https://github.com/schbetsy","followers_url":"https://api.github.com/users/schbetsy/followers","following_url":"https://api.github.com/users/schbetsy/following{/other_user}","gists_url":"https://api.github.com/users/schbetsy/gists{/gist_id}","starred_url":"https://api.github.com/users/schbetsy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schbetsy/subscriptions","organizations_url":"https://api.github.com/users/schbetsy/orgs","repos_url":"https://api.github.com/users/schbetsy/repos","events_url":"https://api.github.com/users/schbetsy/events{/privacy}","received_events_url":"https://api.github.com/users/schbetsy/received_events","type":"User","site_admin":false},"html_url":"https://github.com/schbetsy/cfgov-refresh","description":"Django project protecting American consumers","fork":true,"url":"https://api.github.com/repos/schbetsy/cfgov-refresh","forks_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/forks","keys_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/keys{/key_id}","collaborators_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/teams","hooks_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/hooks","issue_events_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/issues/events{/number}","events_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/events","assignees_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/assignees{/user}","branches_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/branches{/branch}","tags_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/tags","blobs_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/git/refs{/sha}","trees_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/git/trees{/sha}","statuses_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/statuses/{sha}","languages_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/languages","stargazers_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/stargazers","contributors_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/contributors","subscribers_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/subscribers","subscription_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/subscription","commits_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/commits{/sha}","git_commits_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/git/commits{/sha}","comments_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/comments{/number}","issue_comment_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/issues/comments{/number}","contents_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/contents/{+path}","compare_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/compare/{base}...{head}","merges_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/merges","archive_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/downloads","issues_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/issues{/number}","pulls_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/pulls{/number}","milestones_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/milestones{/number}","notifications_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/labels{/name}","releases_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/releases{/id}","deployments_url":"https://api.github.com/repos/schbetsy/cfgov-refresh/deployments","created_at":"2018-07-31T16:55:21Z","updated_at":"2019-01-03T15:46:35Z","pushed_at":"2019-01-03T15:46:32Z","git_url":"git://github.com/schbetsy/cfgov-refresh.git","ssh_url":"git@github.com:schbetsy/cfgov-refresh.git","clone_url":"https://github.com/schbetsy/cfgov-refresh.git","svn_url":"https://github.com/schbetsy/cfgov-refresh","homepage":"https://www.consumerfinance.gov/","size":226130,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"cfpb:master","ref":"master","sha":"88d4f5337f2dc5a8c04c493217617213ed1bc9ae","user":{"login":"cfpb","id":1071563,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwNzE1NjM=","avatar_url":"https://avatars2.githubusercontent.com/u/1071563?v=4","gravatar_id":"","url":"https://api.github.com/users/cfpb","html_url":"https://github.com/cfpb","followers_url":"https://api.github.com/users/cfpb/followers","following_url":"https://api.github.com/users/cfpb/following{/other_user}","gists_url":"https://api.github.com/users/cfpb/gists{/gist_id}","starred_url":"https://api.github.com/users/cfpb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cfpb/subscriptions","organizations_url":"https://api.github.com/users/cfpb/orgs","repos_url":"https://api.github.com/users/cfpb/repos","events_url":"https://api.github.com/users/cfpb/events{/privacy}","received_events_url":"https://api.github.com/users/cfpb/received_events","type":"Organization","site_admin":false},"repo":{"id":18852566,"node_id":"MDEwOlJlcG9zaXRvcnkxODg1MjU2Ng==","name":"cfgov-refresh","full_name":"cfpb/cfgov-refresh","private":false,"owner":{"login":"cfpb","id":1071563,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwNzE1NjM=","avatar_url":"https://avatars2.githubusercontent.com/u/1071563?v=4","gravatar_id":"","url":"https://api.github.com/users/cfpb","html_url":"https://github.com/cfpb","followers_url":"https://api.github.com/users/cfpb/followers","following_url":"https://api.github.com/users/cfpb/following{/other_user}","gists_url":"https://api.github.com/users/cfpb/gists{/gist_id}","starred_url":"https://api.github.com/users/cfpb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cfpb/subscriptions","organizations_url":"https://api.github.com/users/cfpb/orgs","repos_url":"https://api.github.com/users/cfpb/repos","events_url":"https://api.github.com/users/cfpb/events{/privacy}","received_events_url":"https://api.github.com/users/cfpb/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/cfpb/cfgov-refresh","description":"Django project protecting American consumers","fork":false,"url":"https://api.github.com/repos/cfpb/cfgov-refresh","forks_url":"https://api.github.com/repos/cfpb/cfgov-refresh/forks","keys_url":"https://api.github.com/repos/cfpb/cfgov-refresh/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cfpb/cfgov-refresh/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cfpb/cfgov-refresh/teams","hooks_url":"https://api.github.com/repos/cfpb/cfgov-refresh/hooks","issue_events_url":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/events{/number}","events_url":"https://api.github.com/repos/cfpb/cfgov-refresh/events","assignees_url":"https://api.github.com/repos/cfpb/cfgov-refresh/assignees{/user}","branches_url":"https://api.github.com/repos/cfpb/cfgov-refresh/branches{/branch}","tags_url":"https://api.github.com/repos/cfpb/cfgov-refresh/tags","blobs_url":"https://api.github.com/repos/cfpb/cfgov-refresh/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cfpb/cfgov-refresh/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cfpb/cfgov-refresh/git/refs{/sha}","trees_url":"https://api.github.com/repos/cfpb/cfgov-refresh/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cfpb/cfgov-refresh/statuses/{sha}","languages_url":"https://api.github.com/repos/cfpb/cfgov-refresh/languages","stargazers_url":"https://api.github.com/repos/cfpb/cfgov-refresh/stargazers","contributors_url":"https://api.github.com/repos/cfpb/cfgov-refresh/contributors","subscribers_url":"https://api.github.com/repos/cfpb/cfgov-refresh/subscribers","subscription_url":"https://api.github.com/repos/cfpb/cfgov-refresh/subscription","commits_url":"https://api.github.com/repos/cfpb/cfgov-refresh/commits{/sha}","git_commits_url":"https://api.github.com/repos/cfpb/cfgov-refresh/git/commits{/sha}","comments_url":"https://api.github.com/repos/cfpb/cfgov-refresh/comments{/number}","issue_comment_url":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/comments{/number}","contents_url":"https://api.github.com/repos/cfpb/cfgov-refresh/contents/{+path}","compare_url":"https://api.github.com/repos/cfpb/cfgov-refresh/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cfpb/cfgov-refresh/merges","archive_url":"https://api.github.com/repos/cfpb/cfgov-refresh/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cfpb/cfgov-refresh/downloads","issues_url":"https://api.github.com/repos/cfpb/cfgov-refresh/issues{/number}","pulls_url":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls{/number}","milestones_url":"https://api.github.com/repos/cfpb/cfgov-refresh/milestones{/number}","notifications_url":"https://api.github.com/repos/cfpb/cfgov-refresh/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cfpb/cfgov-refresh/labels{/name}","releases_url":"https://api.github.com/repos/cfpb/cfgov-refresh/releases{/id}","deployments_url":"https://api.github.com/repos/cfpb/cfgov-refresh/deployments","created_at":"2014-04-16T19:22:03Z","updated_at":"2019-01-04T15:46:59Z","pushed_at":"2019-01-04T15:49:12Z","git_url":"git://github.com/cfpb/cfgov-refresh.git","ssh_url":"git@github.com:cfpb/cfgov-refresh.git","clone_url":"https://github.com/cfpb/cfgov-refresh.git","svn_url":"https://github.com/cfpb/cfgov-refresh","homepage":"https://www.consumerfinance.gov/","size":252435,"stargazers_count":106,"watchers_count":106,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":66,"mirror_url":null,"archived":false,"open_issues_count":69,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":66,"open_issues":69,"watchers":106,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728"},"html":{"href":"https://github.com/cfpb/cfgov-refresh/pull/4728"},"issue":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/4728"},"comments":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/issues/4728/comments"},"review_comments":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728/comments"},"review_comment":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/pulls/4728/commits"},"statuses":{"href":"https://api.github.com/repos/cfpb/cfgov-refresh/statuses/3f63675712043d7f16601c8baf2494a858facc7e"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"chosak"}}