{"action":"created","issue":{"url":"https://api.github.com/repos/mojombo/bert/issues/24","repository_url":"https://api.github.com/repos/mojombo/bert","labels_url":"https://api.github.com/repos/mojombo/bert/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/mojombo/bert/issues/24/comments","events_url":"https://api.github.com/repos/mojombo/bert/issues/24/events","html_url":"https://github.com/mojombo/bert/pull/24","id":148224861,"number":24,"title":"Make the BERT protocol encoding aware","user":{"login":"tenderlove","id":3124,"avatar_url":"https://avatars.githubusercontent.com/u/3124?v=3","gravatar_id":"","url":"https://api.github.com/users/tenderlove","html_url":"https://github.com/tenderlove","followers_url":"https://api.github.com/users/tenderlove/followers","following_url":"https://api.github.com/users/tenderlove/following{/other_user}","gists_url":"https://api.github.com/users/tenderlove/gists{/gist_id}","starred_url":"https://api.github.com/users/tenderlove/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tenderlove/subscriptions","organizations_url":"https://api.github.com/users/tenderlove/orgs","repos_url":"https://api.github.com/users/tenderlove/repos","events_url":"https://api.github.com/users/tenderlove/events{/privacy}","received_events_url":"https://api.github.com/users/tenderlove/received_events","type":"User","site_admin":true},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-04-14T01:23:15Z","updated_at":"2016-04-14T18:16:44Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/mojombo/bert/pulls/24","html_url":"https://github.com/mojombo/bert/pull/24","diff_url":"https://github.com/mojombo/bert/pull/24.diff","patch_url":"https://github.com/mojombo/bert/pull/24.patch"},"body":"This PR makes the BERT protocol aware of Ruby encodings.  I have updated the Encoder and the Decoder as follows:\r\n\r\n## Decoder\r\n\r\nThe decoder now looks for a magic header and changes parsing styles based on the header it finds.  If the header is `MAGIC`, then it knows that the encoder didn't add any encoding information and uses the old decoder.  If the header is `VERSION2`, then it knows that the encoder added string encoding information and it adds the encoding information to the string.\r\n\r\n## Encoder\r\n\r\nThe encoder is split in to version 1 and version 2 encoding.  It defaults to version 1, and version 2 is enabled with a feature flag.  The reason I did this is so that we can deploy the updated bert gem to both the client and the server, then flip the feature flag on for the client, then flip it on for the server.  That way we can upgrade without downtime.\r\n\r\nHere is an IRB session to demonstrate the encoding support:\r\n\r\n```\r\n[aaron@TC bert (encoding)]$ irb -I lib:ext\r\nirb(main):001:0> require 'bert'\r\n=> true\r\nirb(main):002:0> BERT::Decoder.decode(BERT::Encoder.encode(\"日本語\")).encoding\r\n=> #<Encoding:ASCII-8BIT>\r\nirb(main):003:0> BERT::Decoder.decode(BERT::Encoder.encode(\"日本語\"))\r\n=> \"\\xE6\\x97\\xA5\\xE6\\x9C\\xAC\\xE8\\xAA\\x9E\"\r\nirb(main):004:0> BERT::Encode.version = :v2\r\n=> :v2\r\nirb(main):005:0> BERT::Decoder.decode(BERT::Encoder.encode(\"日本語\")).encoding\r\n=> #<Encoding:UTF-8>\r\nirb(main):006:0> BERT::Decoder.decode(BERT::Encoder.encode(\"日本語\"))\r\n=> \"日本語\"\r\nirb(main):007:0>\r\n```\r\n\r\n/cc @brianmario @tmm1 @charliesome @piki"},"comment":{"url":"https://api.github.com/repos/mojombo/bert/issues/comments/210084517","html_url":"https://github.com/mojombo/bert/pull/24#issuecomment-210084517","issue_url":"https://api.github.com/repos/mojombo/bert/issues/24","id":210084517,"user":{"login":"carlosmn","id":335443,"avatar_url":"https://avatars.githubusercontent.com/u/335443?v=3","gravatar_id":"","url":"https://api.github.com/users/carlosmn","html_url":"https://github.com/carlosmn","followers_url":"https://api.github.com/users/carlosmn/followers","following_url":"https://api.github.com/users/carlosmn/following{/other_user}","gists_url":"https://api.github.com/users/carlosmn/gists{/gist_id}","starred_url":"https://api.github.com/users/carlosmn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlosmn/subscriptions","organizations_url":"https://api.github.com/users/carlosmn/orgs","repos_url":"https://api.github.com/users/carlosmn/repos","events_url":"https://api.github.com/users/carlosmn/events{/privacy}","received_events_url":"https://api.github.com/users/carlosmn/received_events","type":"User","site_admin":true},"created_at":"2016-04-14T18:16:44Z","updated_at":"2016-04-14T18:16:44Z","body":"I would expect 90% of the cases to be utf8 strings, since that's the encoding we use (assume) when we extract Git data via rugged, or to be raw binary like from spawning a child process.\r\n\r\nHaving a way to specify both of these via a single int should help a lot with the smaller strings (and would still allow us to specify a different encoding if it's neither of these)."}}