{"action":"closed","issue":{"url":"https://api.github.com/repos/haskell-servant/servant/issues/186","labels_url":"https://api.github.com/repos/haskell-servant/servant/issues/186/labels{/name}","comments_url":"https://api.github.com/repos/haskell-servant/servant/issues/186/comments","events_url":"https://api.github.com/repos/haskell-servant/servant/issues/186/events","html_url":"https://github.com/haskell-servant/servant/issues/186","id":99891035,"number":186,"title":"Header value encoding (and ToByteString instances)","user":{"login":"berdario","id":752835,"avatar_url":"https://avatars.githubusercontent.com/u/752835?v=3","gravatar_id":"","url":"https://api.github.com/users/berdario","html_url":"https://github.com/berdario","followers_url":"https://api.github.com/users/berdario/followers","following_url":"https://api.github.com/users/berdario/following{/other_user}","gists_url":"https://api.github.com/users/berdario/gists{/gist_id}","starred_url":"https://api.github.com/users/berdario/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/berdario/subscriptions","organizations_url":"https://api.github.com/users/berdario/orgs","repos_url":"https://api.github.com/users/berdario/repos","events_url":"https://api.github.com/users/berdario/events{/privacy}","received_events_url":"https://api.github.com/users/berdario/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-09T12:19:15Z","updated_at":"2016-01-07T09:15:03Z","closed_at":"2016-01-07T09:15:03Z","body":"I found that `URI` doesn't have any `ToByteString` instance and upon looking into that I realized that both the instances for Text and String [blindly encode with utf-8](https://github.com/twittner/bytestring-conversion/blob/e8751a5b343666a77b0/src/Data/ByteString/Conversion/To.hs#L47)\r\n\r\nThis is probably not terribly important, since I expect browsers to ignore unknown bytes... but the correct thing to do would be to make sure that any header value is encoded in ASCII, as expected by the standard.\r\n\r\nRight now this situation might end up in (silent?) failures by clients to recognize the response. Making the ByteString conversion partial obviously sounds bad, but at least it would not fail silently. It would be even better to have a separate type from ByteString for HTTP Headers Values (since not all ByteStrings will be valid). Apparently ByteStrings are used inside Warp itself, though...\r\n\r\nmaybe something like...\r\n\r\n    newtype HeaderValue = HeaderValue ByteString\r\n    toHeaderValue :: (ToByteString s) => s -> Maybe HeaderValue\r\n\r\n\r\nFor my URI use case, the proper ToByteString instance would have to urlencode the utf-8 in the uri{Path,Query,Fragment} and puny-encode the idn. \r\n\r\nMoreover, by looking at the HTTP/2 spec, I realized that this might also potentially be a security issue:\r\n\r\nLet's assume that I define blindly a ToByteString instance for URI like above. And that one of the `Capture` receives a String/Text. If an user could be able to forge a resource identified by a String/Text containing CR,LF codepoints, this could result in having the rest of the uri [interpreted as another Header](https://www.owasp.org/index.php/HTTP_Response_Splitting) (there might be prevention for this inside Warp/Wai, so this might be moot... )\r\n\r\nReferences (You already know such stuff, but I'll leave them here for other's benefit and since it took me a while to search in the text):\r\n\r\n[HTTP/2 spec](https://httpwg.github.io/specs/rfc7540.html#rfc.section.10.3)\r\n\r\n> Similarly, HTTP/2 allows header field values that are not valid. While most of the values that can be encoded will not alter header field parsing, carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0) might be exploited by an attacker if they are translated verbatim. Any request or response that contains a character not permitted in a header field value MUST be treated as malformed (Section 8.1.2.6). Valid characters are defined by the field-content ABNF rule in Section 3.2 of [RFC7230].\r\n\r\nA [StackOverflow answer](http://stackoverflow.com/a/4410331/293735) that points to the same resource\r\n\r\n[HTTP/1.1 Syntax (RFC7230)](https://tools.ietf.org/html/rfc7230#section-3.2.4)\r\n\r\n> Historically, HTTP has allowed field content with text in the ISO-8859-1 charset [ISO-8859-1], supporting other charsets only through use of [RFC2047] encoding.  In practice, most HTTP header field values use only a subset of the US-ASCII charset [USASCII]. Newly defined header fields SHOULD limit their field values to US-ASCII octets.  A recipient SHOULD treat other octets in field content (obs-text) as opaque data."}}