{"action":"created","issue":{"url":"https://api.github.com/repos/hadley/haven/issues/132","repository_url":"https://api.github.com/repos/hadley/haven","labels_url":"https://api.github.com/repos/hadley/haven/issues/132/labels{/name}","comments_url":"https://api.github.com/repos/hadley/haven/issues/132/comments","events_url":"https://api.github.com/repos/hadley/haven/issues/132/events","html_url":"https://github.com/hadley/haven/issues/132","id":115873043,"number":132,"title":"write_dta needs to check for valid Stata variable names","user":{"login":"wbuchanan","id":5542790,"avatar_url":"https://avatars.githubusercontent.com/u/5542790?v=3","gravatar_id":"","url":"https://api.github.com/users/wbuchanan","html_url":"https://github.com/wbuchanan","followers_url":"https://api.github.com/users/wbuchanan/followers","following_url":"https://api.github.com/users/wbuchanan/following{/other_user}","gists_url":"https://api.github.com/users/wbuchanan/gists{/gist_id}","starred_url":"https://api.github.com/users/wbuchanan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wbuchanan/subscriptions","organizations_url":"https://api.github.com/users/wbuchanan/orgs","repos_url":"https://api.github.com/users/wbuchanan/repos","events_url":"https://api.github.com/users/wbuchanan/events{/privacy}","received_events_url":"https://api.github.com/users/wbuchanan/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-11-09T13:08:40Z","updated_at":"2016-05-30T19:29:35Z","closed_at":"2016-05-30T19:29:15Z","body":"```   \r\nd3urls <- list(\"Selections\" = \"https://github.com/mbostock/d3/wiki/Selections\",\r\n\t\t \"Transitions\" = \"https://github.com/mbostock/d3/wiki/Transitions\",\r\n\t\t \"Arrays\" = \"https://github.com/mbostock/d3/wiki/Arrays\",\r\n\t\t \"Requests\" = \"https://github.com/mbostock/d3/wiki/Requests\",\r\n\t\t \"Formatting\" = \"https://github.com/mbostock/d3/wiki/Formatting\",\r\n\t\t \"Localization\" = \"https://github.com/mbostock/d3/wiki/Localization\",\r\n\t\t \"Colors\" = \"https://github.com/mbostock/d3/wiki/Colors\",\r\n\t\t \"Namespaces\" = \"https://github.com/mbostock/d3/wiki/Namespaces\",\r\n\t\t \"Math\" = \"https://github.com/mbostock/d3/wiki/Math\", \r\n\t\t \"Internals\" = \"https://github.com/mbostock/d3/wiki/Internals\",\r\n\t\t \"Behaviors - Drag\" = \"https://github.com/mbostock/d3/wiki/Drag-Behavior\",\r\n\t\t \"Behaviors - Zoom\" = \"https://github.com/mbostock/d3/wiki/Zoom-Behavior\",\r\n\t\t \"Geo - Paths\" = \"https://github.com/mbostock/d3/wiki/Geo-Paths\", \r\n\t\t \"Geo - Projections\" = \"https://github.com/mbostock/d3/wiki/Geo-Projections\", \r\n\t\t \"Geo - Streams\" = \"https://github.com/mbostock/d3/wiki/Geo-Streams\",\r\n\t\t \"Geom - Voronoi\" = \"https://github.com/mbostock/d3/wiki/Voronoi-Geom\", \r\n\t\t \"Geom - Hull\" = \"https://github.com/mbostock/d3/wiki/Hull-Geom\",\r\n\t\t \"Geom - Polygon\" = \"https://github.com/mbostock/d3/wiki/Polygon-Geom\", \r\n\t\t \"Geom - Quadtree\" = \"https://github.com/mbostock/d3/wiki/Quadtree-Geom\", \r\n\t\t \"Layouts - Bundle\" = \"https://github.com/mbostock/d3/wiki/Bundle-Layout\", \r\n\t\t \"Layouts - Chord\" = \"https://github.com/mbostock/d3/wiki/Chord-Layout\", \r\n\t\t \"Layouts - Cluster\" = \"https://github.com/mbostock/d3/wiki/Cluster-Layout\", \r\n\t\t \"Layouts - Force\" = \"https://github.com/mbostock/d3/wiki/Force-Layout\", \r\n\t\t \"Layouts - Hierarchy\" = \"https://github.com/mbostock/d3/wiki/Hierarchy-Layout\", \r\n\t\t \"Layouts - Histogram\" = \"https://github.com/mbostock/d3/wiki/Histogram-Layout\", \r\n\t\t \"Layouts - Pack\" = \"https://github.com/mbostock/d3/wiki/Pack-Layout\", \r\n\t\t \"Layouts - Partition\" = \"https://github.com/mbostock/d3/wiki/Partition-Layout\", \r\n\t\t \"Layouts - Pie\" = \"https://github.com/mbostock/d3/wiki/Pie-Layout\", \r\n\t\t \"Layouts - Stack\" = \"https://github.com/mbostock/d3/wiki/Stack-Layout\", \r\n\t\t \"Layouts - Tree\" = \"https://github.com/mbostock/d3/wiki/Tree-Layout\", \r\n\t\t \"Layouts - Treemap\" = \"https://github.com/mbostock/d3/wiki/Treemap-Layout\", \r\n\t\t \"Scales - Quantitative\" = \"https://github.com/mbostock/d3/wiki/Quantitative-Scales\",\r\n\t\t \"Scales - Ordinal\" = \"https://github.com/mbostock/d3/wiki/Ordinal-Scales\",\r\n\t\t \"Scales - Timeseries\" = \"https://github.com/mbostock/d3/wiki/Time-Scales\",\r\n\t\t \"SVG - Shapes\" = \"https://github.com/mbostock/d3/wiki/SVG-Shapes\",\r\n\t\t \"SVG - Axes\" = \"https://github.com/mbostock/d3/wiki/SVG-Axes\",\r\n\t\t \"SVG - Controls\" = \"https://github.com/mbostock/d3/wiki/SVG-Controls\",\r\n\t\t \"Time - Formatting\" = \"https://github.com/mbostock/d3/wiki/Time-Formatting\",\r\n\t\t \"Time - Scales\" = \"https://github.com/mbostock/d3/wiki/Time-Scales\",\r\n\t\t \"Time - Intervals\" = \"https://github.com/mbostock/d3/wiki/Time-Intervals\") \r\n\r\nlibrary(magrittr)\r\ncolnm <- names(d3urls)\r\nd3x <- xml2::read_html(d3urls[[1]]) %>% rvest::html_nodes(\"p\") %>% rvest::html_text()\r\nd3x <- d3x[grepl(\"^#.*\", d3x)] \r\nd3x <- gsub(\"# \", \"\", d3x) \r\nr <- c(1:length(d3x))\r\nd3x <- as.data.frame(cbind(r, d3x), stringsAsFactors = FALSE)\r\nnames(d3x) <- c(\"id\", colnm[1])\r\n\r\nfor (i in c(2:40)) {\r\n\tx <- xml2::read_html(d3urls[[i]]) %>% rvest::html_nodes(\"p\") %>% rvest::html_text()\r\n\tx <- x[grepl(\"^#.*\", x)] \r\n\tx <- gsub(\"# \", \"\", x) \r\n\tr <- c(1:length(x))\r\n\tx <- as.data.frame(cbind(r, x), stringsAsFactors = FALSE)\r\n\tnames(x) <- c(\"id\", colnm[i])\r\n\td3x <- dplyr::full_join(d3x, x, by = \"id\")\t\r\n}\r\n\r\nrm(x, r)\r\n\r\nhaven::write_dta(d3x, \"~/Desktop/d3Methods.dta\")\r\n```   \r\n\r\nThen I load the file in Stata 14.1MP8 using:\r\n\r\n```\r\nuse ~/Desktop/d3Methods.dta, clear\r\n```\r\n\r\nThe problem occurs when using the Stata command 'compress', which is used to optimize storage on disk of the file (e.g., downcasts types to the smallest type possible without loosing precision so things like 1.00000000000000000000000 would be cast as a 1-byte integer value rather than a float/double).  In this case, I think there is a problem with the writing functions and how they insert binary zeros around the strings in the data frame (Stata uses binary zeros for padding a column so each record for a string column reserves the same number of bits for storage).  \r\n\r\nIf I write the same data out to a csv:\r\n\r\n```\r\nwrite.csv(d3x, \"~/Desktop/d3Methods.csv\", row.names = FALSE)\r\n```\r\n\r\nThen load the same data in Stata:\r\n\r\n```\r\n. import delimited using ~/Desktop/d3Methods.csv, delim(\",\") varn(1) clear \r\n(41 vars, 102 obs)\r\n\r\n. compress\r\n  (0 bytes saved)\r\n```\r\n\r\nThe issue goes away.  I couldn't capture the other error since it crashed Stata each time.  I can post the .dta files in version 13 and 14 if you'd like to compare it to the output from Haven.  "},"comment":{"url":"https://api.github.com/repos/hadley/haven/issues/comments/222544571","html_url":"https://github.com/hadley/haven/issues/132#issuecomment-222544571","issue_url":"https://api.github.com/repos/hadley/haven/issues/132","id":222544571,"user":{"login":"hadley","id":4196,"avatar_url":"https://avatars.githubusercontent.com/u/4196?v=3","gravatar_id":"","url":"https://api.github.com/users/hadley","html_url":"https://github.com/hadley","followers_url":"https://api.github.com/users/hadley/followers","following_url":"https://api.github.com/users/hadley/following{/other_user}","gists_url":"https://api.github.com/users/hadley/gists{/gist_id}","starred_url":"https://api.github.com/users/hadley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hadley/subscriptions","organizations_url":"https://api.github.com/users/hadley/orgs","repos_url":"https://api.github.com/users/hadley/repos","events_url":"https://api.github.com/users/hadley/events{/privacy}","received_events_url":"https://api.github.com/users/hadley/received_events","type":"User","site_admin":false},"created_at":"2016-05-30T19:29:35Z","updated_at":"2016-05-30T19:29:35Z","body":"I've been burnt too many times with R's helpful auto-renaming rules, so I've opted to be strict here and throw and error."}}
{"actor":{"display_login":"hadley"}}