{"action":"created","issue":{"url":"https://api.github.com/repos/hadley/dplyr/issues/1513","repository_url":"https://api.github.com/repos/hadley/dplyr","labels_url":"https://api.github.com/repos/hadley/dplyr/issues/1513/labels{/name}","comments_url":"https://api.github.com/repos/hadley/dplyr/issues/1513/comments","events_url":"https://api.github.com/repos/hadley/dplyr/issues/1513/events","html_url":"https://github.com/hadley/dplyr/issues/1513","id":115313955,"number":1513,"title":"Joining data frames with different internal character encodings (UTF-8 vs. latin1) fails","user":{"login":"huftis","id":100251,"avatar_url":"https://avatars.githubusercontent.com/u/100251?v=3","gravatar_id":"","url":"https://api.github.com/users/huftis","html_url":"https://github.com/huftis","followers_url":"https://api.github.com/users/huftis/followers","following_url":"https://api.github.com/users/huftis/following{/other_user}","gists_url":"https://api.github.com/users/huftis/gists{/gist_id}","starred_url":"https://api.github.com/users/huftis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huftis/subscriptions","organizations_url":"https://api.github.com/users/huftis/orgs","repos_url":"https://api.github.com/users/huftis/repos","events_url":"https://api.github.com/users/huftis/events{/privacy}","received_events_url":"https://api.github.com/users/huftis/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/hadley/dplyr/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/hadley/dplyr/labels/data%20frame","name":"data frame","color":"207de5"}],"state":"closed","locked":false,"assignee":{"login":"romainfrancois","id":2625526,"avatar_url":"https://avatars.githubusercontent.com/u/2625526?v=3","gravatar_id":"","url":"https://api.github.com/users/romainfrancois","html_url":"https://github.com/romainfrancois","followers_url":"https://api.github.com/users/romainfrancois/followers","following_url":"https://api.github.com/users/romainfrancois/following{/other_user}","gists_url":"https://api.github.com/users/romainfrancois/gists{/gist_id}","starred_url":"https://api.github.com/users/romainfrancois/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/romainfrancois/subscriptions","organizations_url":"https://api.github.com/users/romainfrancois/orgs","repos_url":"https://api.github.com/users/romainfrancois/repos","events_url":"https://api.github.com/users/romainfrancois/events{/privacy}","received_events_url":"https://api.github.com/users/romainfrancois/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/hadley/dplyr/milestones/8","html_url":"https://github.com/hadley/dplyr/milestones/0.5","labels_url":"https://api.github.com/repos/hadley/dplyr/milestones/8/labels","id":766580,"number":8,"title":"0.5","description":"","creator":{"login":"hadley","id":4196,"avatar_url":"https://avatars.githubusercontent.com/u/4196?v=3","gravatar_id":"","url":"https://api.github.com/users/hadley","html_url":"https://github.com/hadley","followers_url":"https://api.github.com/users/hadley/followers","following_url":"https://api.github.com/users/hadley/following{/other_user}","gists_url":"https://api.github.com/users/hadley/gists{/gist_id}","starred_url":"https://api.github.com/users/hadley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hadley/subscriptions","organizations_url":"https://api.github.com/users/hadley/orgs","repos_url":"https://api.github.com/users/hadley/repos","events_url":"https://api.github.com/users/hadley/events{/privacy}","received_events_url":"https://api.github.com/users/hadley/received_events","type":"User","site_admin":false},"open_issues":14,"closed_issues":239,"state":"open","created_at":"2014-08-26T20:16:29Z","updated_at":"2016-04-30T15:50:55Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2015-11-05T15:57:05Z","updated_at":"2016-05-01T07:04:44Z","closed_at":"2016-04-30T15:50:55Z","body":"When the join column(s) of two data frames has the same name but different internal encodings, the `dplyr` join functions fail. Example:\r\n\r\n    library(dplyr)\r\n    load(file(\"http://huftis.org/nedlasting/r/dplyr-encoding-join-problem.Rdata\"))\r\n\r\nThe two data frames can be joined by the ‘løpenummer’ column:\r\n  \r\n    > str(d1)\r\n    Classes ‘tbl_df’ and 'data.frame':\t6 obs. of  2 variables:\r\n     $ løpenummer: int  1 2 3 4 5 6\r\n     $ foo       : int  176 114 148 154 181 120\r\n    > str(d2)\r\n    Classes ‘tbl_df’ and 'data.frame':\t6 obs. of  2 variables:\r\n     $ løpenummer: int  4 5 6 7 8 9\r\n     $ baz       : int  24 46 45 10 35 16\r\n\r\nBut actually trying to join them fails with an error message:\r\n\r\n    > d1 %>% left_join(d2, by=\"løpenummer\")\r\n    Error: cannot join on columns 'løpenummer' x 'løpenummer': index out of bounds \r\n\r\nThe `left_join()` function does recognise that the data frames can be joined by ‘løpenummer’, but still fails:\r\n\r\n    > d1 %>% left_join(d2)\r\n    Joining by: \"løpenummer\"   \r\n    Error: cannot join on columns 'l�penummer' x 'l�penummer': index out of bounds \r\n\r\nNote that the ‘ø’ character is different in this error message. (On my system it’s shown as the ‘unknown character’ glyph.)\r\n\r\nThe two ‘løpenummer’ variables *do* have the exact same name, as confirmed by:\r\n\r\n    > names(d1)==names(d2)\r\n    [1]  TRUE FALSE\r\n\r\nBut the character encoding is different:\r\n\r\n    > Encoding(names(d1))\r\n    [1] \"UTF-8\"   \"unknown\"\r\n    > Encoding(names(d2))\r\n    [1] \"latin1\"  \"unknown\"\r\n\r\nI observe this bug on R 3.2.2, with the latest Git version (2015-11-05) of dplyr, on a 64-bits Linux system (though the problem is also present in the released 0.4.3 version in R 3.2.1 on a Windows system)."},"comment":{"url":"https://api.github.com/repos/hadley/dplyr/issues/comments/216021138","html_url":"https://github.com/hadley/dplyr/issues/1513#issuecomment-216021138","issue_url":"https://api.github.com/repos/hadley/dplyr/issues/1513","id":216021138,"user":{"login":"hadley","id":4196,"avatar_url":"https://avatars.githubusercontent.com/u/4196?v=3","gravatar_id":"","url":"https://api.github.com/users/hadley","html_url":"https://github.com/hadley","followers_url":"https://api.github.com/users/hadley/followers","following_url":"https://api.github.com/users/hadley/following{/other_user}","gists_url":"https://api.github.com/users/hadley/gists{/gist_id}","starred_url":"https://api.github.com/users/hadley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hadley/subscriptions","organizations_url":"https://api.github.com/users/hadley/orgs","repos_url":"https://api.github.com/users/hadley/repos","events_url":"https://api.github.com/users/hadley/events{/privacy}","received_events_url":"https://api.github.com/users/hadley/received_events","type":"User","site_admin":false},"created_at":"2016-05-01T07:04:44Z","updated_at":"2016-05-01T07:04:44Z","body":"You can use unicode escapes, which do work on windows:\r\n\r\n```R\r\n# stringi::stri_escape_unicode(\"løpenummer\")\r\ndf <- tibble::data_frame(x = 1:3)\r\nnames(df) <- \"l\\u00f8penummer\"\r\n```"}}