{"action":"created","issue":{"url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/2198","repository_url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End","labels_url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/2198/labels{/name}","comments_url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/2198/comments","events_url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/2198/events","html_url":"https://github.com/zooniverse/Panoptes-Front-End/issues/2198","id":132802858,"number":2198,"title":"Feature request: True CSV export / CSV flatfile?","user":{"login":"khufkens","id":1354258,"avatar_url":"https://avatars.githubusercontent.com/u/1354258?v=3","gravatar_id":"","url":"https://api.github.com/users/khufkens","html_url":"https://github.com/khufkens","followers_url":"https://api.github.com/users/khufkens/followers","following_url":"https://api.github.com/users/khufkens/following{/other_user}","gists_url":"https://api.github.com/users/khufkens/gists{/gist_id}","starred_url":"https://api.github.com/users/khufkens/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khufkens/subscriptions","organizations_url":"https://api.github.com/users/khufkens/orgs","repos_url":"https://api.github.com/users/khufkens/repos","events_url":"https://api.github.com/users/khufkens/events{/privacy}","received_events_url":"https://api.github.com/users/khufkens/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-02-10T20:04:37Z","updated_at":"2016-02-10T20:14:51Z","closed_at":null,"body":"Is there a way to get a true (or partially true) CSV export of the classification data?\r\n\r\nAlthough I understand the need for structured data some fields now are buried deep within the current comma separated columns. Mainly using environments not used to dictionaries and or JSON data (R, spreadsheets, ... ) are stuck with a complex workflow for even the most basic enquiries (e.g. which images have been retired?). I feel information such as \"subject ID\", filename, retired status, should be moved up and be displayed as a separate column rather than buried in Subject_Data. I'm not sure if you could expand the whole file to a flat file CSV, this would be even more convenient (but might blow up the size of the data export). \r\n\r\nFor now I'll look into python as it might be more suited for JSON work. But for compatibility reasons a more convenient CSV export would be welcome for sure.\r\n\r\nThis is the code to recover the filename of a subject in R:\r\n```R\r\nimage_name = unlist(lapply(subject_data,\r\nfunction(x)unlist(x)[grep(\"#Filename\",names(unlist(x)))]))\r\n```\r\n\r\nSimilar key information such at retired status can only be extracted using an equally complex command:\r\n```R\r\nretired_status = unlist(lapply(subject_data,function(x){\r\n  y = unlist(x)[grep(\"subject_id\",names(unlist(x)))];\r\n  if (is.null(y) | length(y)==0){\r\n    return(0) # no retired\r\n  }else{\r\n    return(1) # retired\r\n  }\r\n  }))\r\n```\r\nI haven't looked at the true annotation data but this might be even more challenging to fold out or collapse into an R ingestible data frame."},"comment":{"url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/comments/182560938","html_url":"https://github.com/zooniverse/Panoptes-Front-End/issues/2198#issuecomment-182560938","issue_url":"https://api.github.com/repos/zooniverse/Panoptes-Front-End/issues/2198","id":182560938,"user":{"login":"marten","id":277,"avatar_url":"https://avatars.githubusercontent.com/u/277?v=3","gravatar_id":"","url":"https://api.github.com/users/marten","html_url":"https://github.com/marten","followers_url":"https://api.github.com/users/marten/followers","following_url":"https://api.github.com/users/marten/following{/other_user}","gists_url":"https://api.github.com/users/marten/gists{/gist_id}","starred_url":"https://api.github.com/users/marten/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marten/subscriptions","organizations_url":"https://api.github.com/users/marten/orgs","repos_url":"https://api.github.com/users/marten/repos","events_url":"https://api.github.com/users/marten/events{/privacy}","received_events_url":"https://api.github.com/users/marten/received_events","type":"User","site_admin":false},"created_at":"2016-02-10T20:14:51Z","updated_at":"2016-02-10T20:14:51Z","body":"All I can say is work is already underway: https://github.com/zooniverse/Panoptes/pull/1553. Although I haven't been able to work on this for a bit (holidays and then other priorities including performance optimizing for being on live tv), I hope to wrap that feature up soon.\r\n\r\nBasically we'll be expanding what are now single rows into multiple rows, once for each item in the annotation. We're also going to make the subject information easier to deal with (as long as you only have one subject).\r\n\r\nThat last parenthetical is the reason it's so complicated right now: internally we're already prepared for dealing with annotations comparing or otherwise relating to multiple subjects at the same time. Which then meant that we had to be able to supply a list of subjects in `subject_data`, because each one will have their own retired status etc. We're not recognising that in doing so, we've made our exports hard to use for 99% of the projects, which only have one subject per classification. This will be fixed."}}