{"action":"created","issue":{"url":"https://api.github.com/repos/HDI-Project/ATM/issues/82","repository_url":"https://api.github.com/repos/HDI-Project/ATM","labels_url":"https://api.github.com/repos/HDI-Project/ATM/issues/82/labels{/name}","comments_url":"https://api.github.com/repos/HDI-Project/ATM/issues/82/comments","events_url":"https://api.github.com/repos/HDI-Project/ATM/issues/82/events","html_url":"https://github.com/HDI-Project/ATM/issues/82","id":297151377,"node_id":"MDU6SXNzdWUyOTcxNTEzNzc=","number":82,"title":"REST API","user":{"login":"jobliz","id":935934,"node_id":"MDQ6VXNlcjkzNTkzNA==","avatar_url":"https://avatars1.githubusercontent.com/u/935934?v=4","gravatar_id":"","url":"https://api.github.com/users/jobliz","html_url":"https://github.com/jobliz","followers_url":"https://api.github.com/users/jobliz/followers","following_url":"https://api.github.com/users/jobliz/following{/other_user}","gists_url":"https://api.github.com/users/jobliz/gists{/gist_id}","starred_url":"https://api.github.com/users/jobliz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jobliz/subscriptions","organizations_url":"https://api.github.com/users/jobliz/orgs","repos_url":"https://api.github.com/users/jobliz/repos","events_url":"https://api.github.com/users/jobliz/events{/privacy}","received_events_url":"https://api.github.com/users/jobliz/received_events","type":"User","site_admin":false},"labels":[{"id":829352843,"node_id":"MDU6TGFiZWw4MjkzNTI4NDM=","url":"https://api.github.com/repos/HDI-Project/ATM/labels/feature","name":"feature","color":"5edb9e","default":false}],"state":"open","locked":false,"assignee":{"login":"RogerTangos","id":1609475,"node_id":"MDQ6VXNlcjE2MDk0NzU=","avatar_url":"https://avatars0.githubusercontent.com/u/1609475?v=4","gravatar_id":"","url":"https://api.github.com/users/RogerTangos","html_url":"https://github.com/RogerTangos","followers_url":"https://api.github.com/users/RogerTangos/followers","following_url":"https://api.github.com/users/RogerTangos/following{/other_user}","gists_url":"https://api.github.com/users/RogerTangos/gists{/gist_id}","starred_url":"https://api.github.com/users/RogerTangos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RogerTangos/subscriptions","organizations_url":"https://api.github.com/users/RogerTangos/orgs","repos_url":"https://api.github.com/users/RogerTangos/repos","events_url":"https://api.github.com/users/RogerTangos/events{/privacy}","received_events_url":"https://api.github.com/users/RogerTangos/received_events","type":"User","site_admin":false},"assignees":[{"login":"RogerTangos","id":1609475,"node_id":"MDQ6VXNlcjE2MDk0NzU=","avatar_url":"https://avatars0.githubusercontent.com/u/1609475?v=4","gravatar_id":"","url":"https://api.github.com/users/RogerTangos","html_url":"https://github.com/RogerTangos","followers_url":"https://api.github.com/users/RogerTangos/followers","following_url":"https://api.github.com/users/RogerTangos/following{/other_user}","gists_url":"https://api.github.com/users/RogerTangos/gists{/gist_id}","starred_url":"https://api.github.com/users/RogerTangos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RogerTangos/subscriptions","organizations_url":"https://api.github.com/users/RogerTangos/orgs","repos_url":"https://api.github.com/users/RogerTangos/repos","events_url":"https://api.github.com/users/RogerTangos/events{/privacy}","received_events_url":"https://api.github.com/users/RogerTangos/received_events","type":"User","site_admin":false}],"milestone":null,"comments":13,"created_at":"2018-02-14T16:05:48Z","updated_at":"2018-10-10T19:08:23Z","closed_at":null,"author_association":"NONE","body":"A REST API would be useful to access ATM from any other language and device through the web, and also to illustrate how the code is structured and might be extended. \r\n\r\nFrom the project's readme I get that the internal API's are going to change and that it consequently might be a bit early to develop a REST API, but I wanted to see what was possible to do with the current code. The API currently serves:\r\n\r\n- Various GET endpoints for reading data from the four entities that are currently present in the database\r\n- 1 GET endpoint to run the worker.py script inside the virtualenv as a subprocess and retrieve it's stdout and stderr\r\n- 1 POST endpoint to send a .csv file with the HTTP request, save the file to the atm/data directory and run enter_data on it.\r\n\r\nNo modifications were made outside of the rest_api_server.py file except to the requirements.txt file, adding flask and simplejson to the project dependencies.\r\n\r\n**TODO's / caveats:**\r\n\r\n- No AWS integration\r\n- api.py currently does not check if the uploaded filename is already present, so a CSV upload can rewrite a previously sent file with the same name. This needs fixing, but I thought that I should ask first if the atm/data directory is the right one to put new files in, and if storing UUID's is ok with the project's design before using them and doing a pull request.\r\n\r\n**Example api.py usage:**\r\n\r\nAfter following the readme's installation instructions and running python scripts/rest_api_server.py on a separate shell under the virtualenv:\r\n\r\n`curl localhost:5000/enter_data -F file=@/path/file.csv`\r\n\r\nIt should return:\r\n\r\n`{\"success\": true}`\r\n\r\nTo see the created dataset:\r\n\r\n`curl localhost:5000/datasets/1 | jq`\r\n\r\n```\r\n{\r\n  \"class_column\": \"class\",\r\n  \"description\": null,\r\n  \"size_kb\": 6,\r\n  \"test_path\": null,\r\n  \"k_classes\": 3,\r\n  \"majority\": 0.333333333,\r\n  \"d_features\": 6,\r\n  \"train_path\": \"FULLPATH/file.csv\",\r\n  \"id\": 1,\r\n  \"n_examples\": 150,\r\n  \"name\": \"file\"\r\n}\r\n```\r\n\r\nTo see the created datarun:\r\n\r\n`curl localhost:5000/dataruns/1 | jq`\r\n\r\n```\r\n{\r\n  \"status\": \"pending\",\r\n  \"start_time\": null,\r\n  \"description\": \"uniform__uniform\",\r\n  \"r_minimum\": 2,\r\n  \"metric\": \"f1\",\r\n  \"budget\": 100,\r\n  \"selector\": \"uniform\",\r\n  \"priority\": 1,\r\n  \"score_target\": \"cv_judgment_metric\",\r\n  \"deadline\": null,\r\n  \"budget_type\": \"classifier\",\r\n  \"id\": 1,\r\n  \"tuner\": \"uniform\",\r\n  \"dataset_id\": 1,\r\n  \"gridding\": 0,\r\n  \"k_window\": 3,\r\n  \"end_time\": null\r\n}\r\n```\r\n\r\nTo run the worker.py script once:\r\n\r\n`curl localhost:5000/simple_worker | jq`\r\n\r\n_after a while_\r\n\r\n```\r\n{\r\n  \"stderr\": \"\",\r\n  \"stdout\": \"huge stdout string with worker.py's output\"\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/HDI-Project/ATM/issues/comments/428694846","html_url":"https://github.com/HDI-Project/ATM/issues/82#issuecomment-428694846","issue_url":"https://api.github.com/repos/HDI-Project/ATM/issues/82","id":428694846,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODY5NDg0Ng==","user":{"login":"micahjsmith","id":5473606,"node_id":"MDQ6VXNlcjU0NzM2MDY=","avatar_url":"https://avatars3.githubusercontent.com/u/5473606?v=4","gravatar_id":"","url":"https://api.github.com/users/micahjsmith","html_url":"https://github.com/micahjsmith","followers_url":"https://api.github.com/users/micahjsmith/followers","following_url":"https://api.github.com/users/micahjsmith/following{/other_user}","gists_url":"https://api.github.com/users/micahjsmith/gists{/gist_id}","starred_url":"https://api.github.com/users/micahjsmith/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/micahjsmith/subscriptions","organizations_url":"https://api.github.com/users/micahjsmith/orgs","repos_url":"https://api.github.com/users/micahjsmith/repos","events_url":"https://api.github.com/users/micahjsmith/events{/privacy}","received_events_url":"https://api.github.com/users/micahjsmith/received_events","type":"User","site_admin":false},"created_at":"2018-10-10T19:08:23Z","updated_at":"2018-10-10T19:08:23Z","author_association":"MEMBER","body":"A couple questions:\r\n```\r\nGET info about one or more dataruns\r\nDELETE to remove a datarun\r\n/dataruns?<int: id>&<int: dataset_id>&<str: description>&<str: tuner>&<str: status>\r\n\r\nPATCH to stop a datarun\r\n/dataruns?<int: id>&<int: dataset_id>&<str: description>&<str: tuner>&<str: status>/status\r\n```\r\n\r\nWhy not just provide the datarun id? Why is tuner a query parameter? Would we stop dataruns with a tuner matching some string?\r\n\r\n```\r\nPUT to update or replace a worker\r\nPATCH to update/modify a worker\r\n```\r\nWhat is the difference here?\r\n\r\n```\r\nPATCH to activate or deactivate the worker\r\n/worker?<int: id>?<int: dataset_id>/status\r\n```\r\nThought we discussed that this is not currently possible or relevant"}}
{"actor":{"display_login":"micahjsmith"}}