{"action":"created","issue":{"url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/1","repository_url":"https://api.github.com/repos/cdegroc/warc-clueweb","labels_url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/1/comments","events_url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/1/events","html_url":"https://github.com/cdegroc/warc-clueweb/issues/1","id":155440622,"number":1,"title":"warc.py read data in 'rb' mode, occurring Unicode Error","user":{"login":"RominYue","id":7324409,"avatar_url":"https://avatars.githubusercontent.com/u/7324409?v=3","gravatar_id":"","url":"https://api.github.com/users/RominYue","html_url":"https://github.com/RominYue","followers_url":"https://api.github.com/users/RominYue/followers","following_url":"https://api.github.com/users/RominYue/following{/other_user}","gists_url":"https://api.github.com/users/RominYue/gists{/gist_id}","starred_url":"https://api.github.com/users/RominYue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RominYue/subscriptions","organizations_url":"https://api.github.com/users/RominYue/orgs","repos_url":"https://api.github.com/users/RominYue/repos","events_url":"https://api.github.com/users/RominYue/events{/privacy}","received_events_url":"https://api.github.com/users/RominYue/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-05-18T08:10:22Z","updated_at":"2016-05-19T19:21:40Z","closed_at":null,"body":"When I want to decode/encode the content that read from warc.gz file, there sometimes encounters “ ”'ascii' could not decode xxxx“ or \"utf8 could not decode xxxxx\"\r\n\r\nSeen from source file ,we read the warc.gz file in a 'rb' mode, so how to deal with decode/encode problem?"},"comment":{"url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/comments/220425889","html_url":"https://github.com/cdegroc/warc-clueweb/issues/1#issuecomment-220425889","issue_url":"https://api.github.com/repos/cdegroc/warc-clueweb/issues/1","id":220425889,"user":{"login":"cdegroc","id":1451981,"avatar_url":"https://avatars.githubusercontent.com/u/1451981?v=3","gravatar_id":"","url":"https://api.github.com/users/cdegroc","html_url":"https://github.com/cdegroc","followers_url":"https://api.github.com/users/cdegroc/followers","following_url":"https://api.github.com/users/cdegroc/following{/other_user}","gists_url":"https://api.github.com/users/cdegroc/gists{/gist_id}","starred_url":"https://api.github.com/users/cdegroc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdegroc/subscriptions","organizations_url":"https://api.github.com/users/cdegroc/orgs","repos_url":"https://api.github.com/users/cdegroc/repos","events_url":"https://api.github.com/users/cdegroc/events{/privacy}","received_events_url":"https://api.github.com/users/cdegroc/received_events","type":"User","site_admin":false},"created_at":"2016-05-19T19:21:40Z","updated_at":"2016-05-19T19:21:40Z","body":"Hi @RominYue,\r\n\r\nI haven't used this code for a while, but here are two suggestions:\r\n- WARCFile/WARCReader output raw strings that you will need to decode. The errors you receive could simply be due to an invalid WARC file.\r\n- Using 'rb' mode shouldn't hurt on linux/unix. But in any case, you can always override the mode from WARCFile constructor.\r\n\r\nHope this helps.\r\nClement"}}