{"action":"created","issue":{"url":"https://api.github.com/repos/rozap/transform/issues/9","repository_url":"https://api.github.com/repos/rozap/transform","labels_url":"https://api.github.com/repos/rozap/transform/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/rozap/transform/issues/9/comments","events_url":"https://api.github.com/repos/rozap/transform/issues/9/events","html_url":"https://github.com/rozap/transform/issues/9","id":135908032,"number":9,"title":"Upload from browser directly to S3","user":{"login":"vilterp","id":7341,"avatar_url":"https://avatars.githubusercontent.com/u/7341?v=3","gravatar_id":"","url":"https://api.github.com/users/vilterp","html_url":"https://github.com/vilterp","followers_url":"https://api.github.com/users/vilterp/followers","following_url":"https://api.github.com/users/vilterp/following{/other_user}","gists_url":"https://api.github.com/users/vilterp/gists{/gist_id}","starred_url":"https://api.github.com/users/vilterp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vilterp/subscriptions","organizations_url":"https://api.github.com/users/vilterp/orgs","repos_url":"https://api.github.com/users/vilterp/repos","events_url":"https://api.github.com/users/vilterp/events{/privacy}","received_events_url":"https://api.github.com/users/vilterp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-02-24T00:11:18Z","updated_at":"2016-02-24T07:02:32Z","closed_at":null,"body":"Using an approach like Brian's here: https://gist.github.com/boldfield/66561b4f875511c053f2\r\n\r\n1. Browser =(file size)=> our server\r\n2. our server =(...?)=> S3 API\r\n3. S3 API returns signed URLs\r\n4. our server returns them to the browser; one for each chunk\r\n5. browser =(chunk)=> S3 (PUTs to signed URLs)\r\n6. S3 =(notification)=> SQS => our server\r\n7. our server requests chunk from S3 and starts parsing it into basic table chunks\r\n"},"comment":{"url":"https://api.github.com/repos/rozap/transform/issues/comments/188109967","html_url":"https://github.com/rozap/transform/issues/9#issuecomment-188109967","issue_url":"https://api.github.com/repos/rozap/transform/issues/9","id":188109967,"user":{"login":"rozap","id":736051,"avatar_url":"https://avatars.githubusercontent.com/u/736051?v=3","gravatar_id":"","url":"https://api.github.com/users/rozap","html_url":"https://github.com/rozap","followers_url":"https://api.github.com/users/rozap/followers","following_url":"https://api.github.com/users/rozap/following{/other_user}","gists_url":"https://api.github.com/users/rozap/gists{/gist_id}","starred_url":"https://api.github.com/users/rozap/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rozap/subscriptions","organizations_url":"https://api.github.com/users/rozap/orgs","repos_url":"https://api.github.com/users/rozap/repos","events_url":"https://api.github.com/users/rozap/events{/privacy}","received_events_url":"https://api.github.com/users/rozap/received_events","type":"User","site_admin":false},"created_at":"2016-02-24T07:02:32Z","updated_at":"2016-02-24T07:02:32Z","body":"this won't work for a lot of files, because we can't chunk them until they're deserialized into a basic table. \r\n\r\ni think we should maybe aim for simple first, so uploading the whole thing to the extractor. also i worry about having complex logic in the client, as it would make exposing the API super hard down the line. initially it would be ideal to just have a simple `/upload` endpoint that people could post to. "}}