{"action":"created","issue":{"url":"https://api.github.com/repos/rust-lang/rust/issues/30740","labels_url":"https://api.github.com/repos/rust-lang/rust/issues/30740/labels{/name}","comments_url":"https://api.github.com/repos/rust-lang/rust/issues/30740/comments","events_url":"https://api.github.com/repos/rust-lang/rust/issues/30740/events","html_url":"https://github.com/rust-lang/rust/pull/30740","id":125195108,"number":30740,"title":"Add fast path for ASCII in UTF-8 validation","user":{"login":"bluss","id":3209739,"avatar_url":"https://avatars.githubusercontent.com/u/3209739?v=3","gravatar_id":"","url":"https://api.github.com/users/bluss","html_url":"https://github.com/bluss","followers_url":"https://api.github.com/users/bluss/followers","following_url":"https://api.github.com/users/bluss/following{/other_user}","gists_url":"https://api.github.com/users/bluss/gists{/gist_id}","starred_url":"https://api.github.com/users/bluss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bluss/subscriptions","organizations_url":"https://api.github.com/users/bluss/orgs","repos_url":"https://api.github.com/users/bluss/repos","events_url":"https://api.github.com/users/bluss/events{/privacy}","received_events_url":"https://api.github.com/users/bluss/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rust-lang/rust/labels/relnotes","name":"relnotes","color":"fad8c7"}],"state":"closed","locked":false,"assignee":{"login":"brson","id":147214,"avatar_url":"https://avatars.githubusercontent.com/u/147214?v=3","gravatar_id":"","url":"https://api.github.com/users/brson","html_url":"https://github.com/brson","followers_url":"https://api.github.com/users/brson/followers","following_url":"https://api.github.com/users/brson/following{/other_user}","gists_url":"https://api.github.com/users/brson/gists{/gist_id}","starred_url":"https://api.github.com/users/brson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brson/subscriptions","organizations_url":"https://api.github.com/users/brson/orgs","repos_url":"https://api.github.com/users/brson/repos","events_url":"https://api.github.com/users/brson/events{/privacy}","received_events_url":"https://api.github.com/users/brson/received_events","type":"User","site_admin":false},"milestone":null,"comments":18,"created_at":"2016-01-06T15:16:27Z","updated_at":"2016-01-16T10:54:28Z","closed_at":"2016-01-16T03:11:45Z","pull_request":{"url":"https://api.github.com/repos/rust-lang/rust/pulls/30740","html_url":"https://github.com/rust-lang/rust/pull/30740","diff_url":"https://github.com/rust-lang/rust/pull/30740.diff","patch_url":"https://github.com/rust-lang/rust/pull/30740.patch"},"body":"Add fast path for ASCII in UTF-8 validation\r\n\r\nThis speeds up the ASCII case (and long stretches of ASCII in otherwise\r\nmixed UTF-8 data) when checking UTF-8 validity.\r\n\r\nBenchmark results suggest that on purely ASCII input, we can improve\r\nthroughput (megabytes verified / second) by a factor of 13 to 14 (smallish input).\r\nOn XML and mostly English language input (en.wikipedia XML dump),\r\nthroughput improves by a factor 7 (large input).\r\n\r\nOn mostly non-ASCII input, performance increases slightly or is the\r\nsame.\r\n\r\nThe UTF-8 validation is rewritten to use indexed access; since all\r\naccess is preceded by a (mandatory for validation) length check, bounds\r\nchecks are statically elided by LLVM and this formulation is in fact the best\r\nfor performance. A previous version had losses due to slice to iterator\r\nconversions.\r\n\r\nA large credit to Björn Steinbrink who improved this patch immensely,\r\nwriting this second version.\r\n\r\nBenchmark results on x86-64 (Sandy Bridge) compiled with -C opt-level=3.\r\n\r\nOld code is `regular`, this PR is called `fast`.\r\n\r\nDatasets:\r\n\r\n- `ascii` is just ASCII (2.5 kB)\r\n- `cyr` is cyrillic script with ascii spaces (5 kB)\r\n- `dewik10` is 10MB of a de.wikipedia XML dump\r\n- `enwik8` is 100MB of an en.wikipedia XML dump\r\n- `jawik10` is 10MB of a ja.wikipedia XML dump\r\n\r\n```\r\ntest from_utf8_ascii_fast        ... bench:         140 ns/iter (+/- 4) = 18221 MB/s\r\ntest from_utf8_ascii_regular     ... bench:       1,932 ns/iter (+/- 19) = 1320 MB/s\r\ntest from_utf8_cyr_fast          ... bench:      10,025 ns/iter (+/- 245) = 511 MB/s\r\ntest from_utf8_cyr_regular       ... bench:      10,944 ns/iter (+/- 795) = 468 MB/s\r\ntest from_utf8_dewik10_fast      ... bench:   6,017,909 ns/iter (+/- 105,755) = 1740 MB/s\r\ntest from_utf8_dewik10_regular   ... bench:  11,669,493 ns/iter (+/- 264,045) = 891 MB/s\r\ntest from_utf8_enwik8_fast       ... bench:  14,085,692 ns/iter (+/- 1,643,316) = 7000 MB/s\r\ntest from_utf8_enwik8_regular    ... bench:  93,657,410 ns/iter (+/- 5,353,353) = 1000 MB/s\r\ntest from_utf8_jawik10_fast      ... bench:  29,154,073 ns/iter (+/- 4,659,534) = 340 MB/s\r\ntest from_utf8_jawik10_regular   ... bench:  29,112,917 ns/iter (+/- 2,475,123) = 340 MB/s\r\n```\r\n\r\nCo-authored-by: Björn Steinbrink <bsteinbr@gmail.com>"},"comment":{"url":"https://api.github.com/repos/rust-lang/rust/issues/comments/172182616","html_url":"https://github.com/rust-lang/rust/pull/30740#issuecomment-172182616","issue_url":"https://api.github.com/repos/rust-lang/rust/issues/30740","id":172182616,"user":{"login":"bluss","id":3209739,"avatar_url":"https://avatars.githubusercontent.com/u/3209739?v=3","gravatar_id":"","url":"https://api.github.com/users/bluss","html_url":"https://github.com/bluss","followers_url":"https://api.github.com/users/bluss/followers","following_url":"https://api.github.com/users/bluss/following{/other_user}","gists_url":"https://api.github.com/users/bluss/gists{/gist_id}","starred_url":"https://api.github.com/users/bluss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bluss/subscriptions","organizations_url":"https://api.github.com/users/bluss/orgs","repos_url":"https://api.github.com/users/bluss/repos","events_url":"https://api.github.com/users/bluss/events{/privacy}","received_events_url":"https://api.github.com/users/bluss/received_events","type":"User","site_admin":false},"created_at":"2016-01-16T10:54:27Z","updated_at":"2016-01-16T10:54:27Z","body":"awesome. Thanks @brson and everyone."}}