{"action":"created","issue":{"url":"https://api.github.com/repos/brendan-r/boxr/issues/86","repository_url":"https://api.github.com/repos/brendan-r/boxr","labels_url":"https://api.github.com/repos/brendan-r/boxr/issues/86/labels{/name}","comments_url":"https://api.github.com/repos/brendan-r/boxr/issues/86/comments","events_url":"https://api.github.com/repos/brendan-r/boxr/issues/86/events","html_url":"https://github.com/brendan-r/boxr/issues/86","id":386930932,"node_id":"MDU6SXNzdWUzODY5MzA5MzI=","number":86,"title":"Trouble for box_read() with JSON files","user":{"login":"nathancday","id":808225,"node_id":"MDQ6VXNlcjgwODIyNQ==","avatar_url":"https://avatars0.githubusercontent.com/u/808225?v=4","gravatar_id":"","url":"https://api.github.com/users/nathancday","html_url":"https://github.com/nathancday","followers_url":"https://api.github.com/users/nathancday/followers","following_url":"https://api.github.com/users/nathancday/following{/other_user}","gists_url":"https://api.github.com/users/nathancday/gists{/gist_id}","starred_url":"https://api.github.com/users/nathancday/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathancday/subscriptions","organizations_url":"https://api.github.com/users/nathancday/orgs","repos_url":"https://api.github.com/users/nathancday/repos","events_url":"https://api.github.com/users/nathancday/events{/privacy}","received_events_url":"https://api.github.com/users/nathancday/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-12-03T17:45:21Z","updated_at":"2018-12-04T04:39:02Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I noticed the JSON related tests failing for me locally and traced the error back to `rio::import()`. Would like to confirm with someone else though....\r\n\r\nThis code is lifted from `test_05`:\r\n\r\n```\r\n  # Write a little .json file\r\n  tf <- paste0(tempfile(), \".json\")\r\n  df <- data.frame(a = letters[1:5], b = 1:5, c = round(rnorm(5), 3), \r\n                   stringsAsFactors = FALSE)\r\n  l  <- list(a = 1:10, b = matrix(1, 3, 3), c = df)\r\n```\r\n\r\nAnd it `l` looks like this:\r\n\r\n```\r\nl\r\n$a\r\n [1]  1  2  3  4  5  6  7  8  9 10\r\n\r\n$b\r\n     [,1] [,2] [,3]\r\n[1,]    1    1    1\r\n[2,]    1    1    1\r\n[3,]    1    1    1\r\n\r\n$c\r\n  a b      c\r\n1 a 1  0.824\r\n2 b 2 -0.575\r\n3 c 3  0.448\r\n4 d 4 -1.343\r\n5 e 5  0.187\r\n```\r\n\r\nA nice three element list. But when we upload to Box and get the intermediate function `rio::import()` involved things get ugly. According to the `rio` docs `import()` \"only ever returns a single data frame\", and this is seems to be the root of the problem.\r\n\r\n```\r\nb <- box_ul(0, tf)\r\nl2 <- box_read(b$id)\r\n# normal read in message\r\nl2\r\nError in match.names(clabs, names(xi)) : \r\n  names do not match previous names\r\nIn addition: Warning message:\r\nIn format.data.frame(x, digits = digits, na.encode = FALSE) :\r\n  corrupt data frame: columns will be truncated or padded with NAs\r\n```\r\n\r\nYou can't even view the object due to the error in matching names.\r\n\r\nYou can get a viable (but gross) object if you adjust the `simplifyDataFrame = ` argument that is passed through to the `jsonlite::fromJSON()` call happening within `rio::import()`\r\n\r\n```\r\nl3 <- box_read(b$id, simplifyDataFrame = FALSE)\r\nl3\r\n    a  b.1  b.2  b.3            c\r\n1   1    1    1    1 a, 1, -0.643\r\n2   2    1    1    1 b, 2, -2.318\r\n3   3    1    1    1  c, 3, 0.244\r\n4   4 <NA> <NA> <NA>  d, 4, -1.01\r\n5   5 <NA> <NA> <NA>  e, 5, 0.186\r\n6   6 <NA> <NA> <NA>         <NA>\r\n7   7 <NA> <NA> <NA>         <NA>\r\n8   8 <NA> <NA> <NA>         <NA>\r\n9   9 <NA> <NA> <NA>         <NA>\r\n10 10 <NA> <NA> <NA>         <NA>\r\n```\r\n\r\nWhile this is view-able, it doesn't represent the original data well and would require additional cleaning steps by the end-user.\r\n\r\nTo prove this problem is coming from `rio::import()`, we get the same error if we go directly.\r\n\r\n```\r\nrio::import(tf)\r\nError in match.names(clabs, names(xi)) : \r\n  names do not match previous names\r\nIn addition: Warning message:\r\nIn format.data.frame(x, digits = digits, na.encode = FALSE) :\r\n  corrupt data frame: columns will be truncated or padded with NAs\r\n```\r\nEven the rio prescribed alternative `import_list()` fails with the same error.\r\n\r\nIMO I think the logic should be adjusted for `box_read()` so that it will bypass `rio` altogether for JSON formats and just call `jsonlite::fromJSON()` directly. Which will return the original object properly.\r\n\r\n```\r\njsonlite::fromJSON(tf)\r\n$a\r\n [1]  1  2  3  4  5  6  7  8  9 10\r\n\r\n$b\r\n     [,1] [,2] [,3]\r\n[1,]    1    1    1\r\n[2,]    1    1    1\r\n[3,]    1    1    1\r\n\r\n$c\r\n  a b      c\r\n1 a 1 -0.643\r\n2 b 2 -2.318\r\n3 c 3  0.244\r\n4 d 4 -1.010\r\n5 e 5  0.186\r\n```\r\n\r\nThis shouldn't interfere with any of the Box API logic, since the file is already downloaded locally via `boxr:::boxGet()` before reading it into R with `rio::import()`.\r\n"},"comment":{"url":"https://api.github.com/repos/brendan-r/boxr/issues/comments/443968726","html_url":"https://github.com/brendan-r/boxr/issues/86#issuecomment-443968726","issue_url":"https://api.github.com/repos/brendan-r/boxr/issues/86","id":443968726,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0Mzk2ODcyNg==","user":{"login":"ijlyttle","id":2800934,"node_id":"MDQ6VXNlcjI4MDA5MzQ=","avatar_url":"https://avatars3.githubusercontent.com/u/2800934?v=4","gravatar_id":"","url":"https://api.github.com/users/ijlyttle","html_url":"https://github.com/ijlyttle","followers_url":"https://api.github.com/users/ijlyttle/followers","following_url":"https://api.github.com/users/ijlyttle/following{/other_user}","gists_url":"https://api.github.com/users/ijlyttle/gists{/gist_id}","starred_url":"https://api.github.com/users/ijlyttle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ijlyttle/subscriptions","organizations_url":"https://api.github.com/users/ijlyttle/orgs","repos_url":"https://api.github.com/users/ijlyttle/repos","events_url":"https://api.github.com/users/ijlyttle/events{/privacy}","received_events_url":"https://api.github.com/users/ijlyttle/received_events","type":"User","site_admin":false},"created_at":"2018-12-04T04:39:02Z","updated_at":"2018-12-04T04:39:02Z","author_association":"COLLABORATOR","body":"I have not tried it yet, but I think using the `read_fun` might be the best way to go in the short term.\r\n\r\nHave you used the function `purrr::partial()`? This could be a useful tool here, as well. \r\n\r\nMy thought, for now, would be to look at `box_read_json()` to see it how it might be doing things differently than `box_read(...., read_fun = jsonlite::fromJSON)`."}}
{"actor":{"display_login":"ijlyttle"}}