{"action":"created","comment":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/comments/170899348","pull_request_review_id":99654719,"id":170899348,"diff_hunk":"@@ -0,0 +1,120 @@\n+# A specialized iterator for EachIndex of strings\n+struct EachStringIndex{T<:AbstractString}\n+    s::T\n+end\n+keys(s::AbstractString) = EachStringIndex(s)\n+\n+length(e::EachStringIndex) = length(e.s)\n+first(::EachStringIndex) = 1\n+last(e::EachStringIndex) = lastindex(e.s)\n+eltype(::Type{<:EachStringIndex}) = Int\n+\n+# Iteration over StringNext\n+#\n+# Any new subtype of AbstractString, should override\n+#\n+#   next(::StringNext{MyString}, state)\n+#\n+# to provide iteration over the string and its indices. All other iteration methods,\n+# including iteration over strings, iteration over pairs, indexing into string,\n+# iteration over indicies alone are derived from this method.\n+\n+const StringNext{T<:AbstractString} = Iterators.Next{T, EachStringIndex{T}}\n+StringNext(x::T) where {T<:AbstractString} = Next(x)\n+StringNext(x::T, idx) where {T<:AbstractString} = Next(x, idx)\n+StringNext(x::T, idx, itr) where {T<:AbstractString} = Next(x, idx, itr)\n+\n+start(sp::StringNext) = 1","path":"base/strings/iteration.jl","position":27,"original_position":27,"commit_id":"c2658e10e7d3fc14dfc436e8158a4dd64a9cc94a","original_commit_id":"c2658e10e7d3fc14dfc436e8158a4dd64a9cc94a","user":{"login":"StefanKarpinski","id":153596,"avatar_url":"https://avatars2.githubusercontent.com/u/153596?v=4","gravatar_id":"","url":"https://api.github.com/users/StefanKarpinski","html_url":"https://github.com/StefanKarpinski","followers_url":"https://api.github.com/users/StefanKarpinski/followers","following_url":"https://api.github.com/users/StefanKarpinski/following{/other_user}","gists_url":"https://api.github.com/users/StefanKarpinski/gists{/gist_id}","starred_url":"https://api.github.com/users/StefanKarpinski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StefanKarpinski/subscriptions","organizations_url":"https://api.github.com/users/StefanKarpinski/orgs","repos_url":"https://api.github.com/users/StefanKarpinski/repos","events_url":"https://api.github.com/users/StefanKarpinski/events{/privacy}","received_events_url":"https://api.github.com/users/StefanKarpinski/received_events","type":"User","site_admin":false},"body":"Part of the goal of this change is to separate the assumption currently baked into the string code that indices and itartion state of strings are the same thing. We want indices to _always_ be `1` through `ncodeunits(s)` but allow iteration state to be anything at all (even non-integers). So `firstindex(s)` and `start(s)` should be totally unrelated: the former is always `1` but the latter can be anything at all.","created_at":"2018-02-27T12:00:20Z","updated_at":"2018-02-27T12:13:03Z","html_url":"https://github.com/JuliaLang/julia/pull/26133#discussion_r170899348","pull_request_url":"https://api.github.com/repos/JuliaLang/julia/pulls/26133","author_association":"OWNER","_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/comments/170899348"},"html":{"href":"https://github.com/JuliaLang/julia/pull/26133#discussion_r170899348"},"pull_request":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/26133"}},"in_reply_to_id":170854235},"pull_request":{"url":"https://api.github.com/repos/JuliaLang/julia/pulls/26133","id":170305228,"html_url":"https://github.com/JuliaLang/julia/pull/26133","diff_url":"https://github.com/JuliaLang/julia/pull/26133.diff","patch_url":"https://github.com/JuliaLang/julia/pull/26133.patch","issue_url":"https://api.github.com/repos/JuliaLang/julia/issues/26133","number":26133,"state":"open","locked":false,"title":"Separate AbstractString interface from iteration protocol","user":{"login":"Keno","id":1291671,"avatar_url":"https://avatars1.githubusercontent.com/u/1291671?v=4","gravatar_id":"","url":"https://api.github.com/users/Keno","html_url":"https://github.com/Keno","followers_url":"https://api.github.com/users/Keno/followers","following_url":"https://api.github.com/users/Keno/following{/other_user}","gists_url":"https://api.github.com/users/Keno/gists{/gist_id}","starred_url":"https://api.github.com/users/Keno/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Keno/subscriptions","organizations_url":"https://api.github.com/users/Keno/orgs","repos_url":"https://api.github.com/users/Keno/repos","events_url":"https://api.github.com/users/Keno/events{/privacy}","received_events_url":"https://api.github.com/users/Keno/received_events","type":"User","site_admin":false},"body":"Up until now, the basic interface new AbstractStrings had to implement was:\r\n\r\n```\r\nstruct MyString; ...; end\r\nnext(::MyString, i::Int64)::Tuple{Char, Int64}\r\nisvalid(::MyString, i::Int64)::Bool\r\nncodeunits(::MyString)::Int64\r\n```\r\n\r\nIn this interface, the iteration state (i.e. the second tuple element returned\r\nfrom `next`) always had to be the next valid string index. This is inconvenient\r\nfor several reasons:\r\n1. The iteration protocol will change, breaking every use of this API\r\n2. Some strings may want iteration states other than linear indicies for efficiency\r\n   reasons (e.g. RopeStrings)\r\n3. Strings implementors can no longer assume that the second argument they receive\r\n   was necessarily produced by them, so may need to do various validation of the\r\n   iteration sate on every iteration.\r\n\r\nThis PR attempts to remedy this, by introducing a new generic `LeadIndPairs` iterator.\r\nThis iterator behaves similarly to `Pairs`, except that instead of the index of an\r\nelement, it gives the index of the next element (in other words, the indicies lead\r\nthe values by one element). The astute reader will note that the elements of this\r\niterator are precisely the elements of the tuple currently returned by next.\r\nThus, this PR changes, the requisite method to implement from:\r\n\r\n```\r\nnext(::MyString, i::Int64)::Tuple{Char, Int64}\r\n```\r\n\r\nto\r\n\r\n```\r\nnext(::StringLIPairs{MyString}, state::Any)::Tuple{Pair{Int, Char}, Int64}\r\n```\r\n\r\nwhere `StringLIPairs{T} = LeadIndPairs{Char, Int, EachIndexString{T}, T}`\r\n\r\nEfficient implementations of iteration over strings, the indicies as well as `Pairs`\r\ncan be derived from this iterator. The reason this iterator is useful is perhaps best\r\nunderstood by considering strings to be variable-length encodings of character arrays.\r\nIn a variable-length encoding, one generally decodes the value and the length (i.e. the\r\nindex of the next element) at the same time, so it makes sense to base the API\r\non the implementation of an iterator with these semantics.\r\n\r\nTo demonstrate the use and test the new abstract implementations based on this iterator,\r\nthere are three string types in the test suite:\r\n- CharString, as before, which simply wraps an array of `Chars` with direct indexing. The\r\n  only change to this iterator is to change the signature of the `next` method.\r\n- RopeString, which strings together several Strings, and more importantly does not have\r\n  efficient linear iteration state.\r\n- DecodeString, which decodes escape sequences on the fly as part of iteration. This string\r\n  type demonstrates one string type wrapping another string type to test the interface from\r\n  both sides\r\n\r\nThis bootstraps and many of the abstract string operations work on the three test string types I mentioned. Some more cleanup and testing is required, but this seemed like the right time to stop for feedback.","created_at":"2018-02-20T20:53:11Z","updated_at":"2018-02-27T12:13:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bfeff3548d2e9bc8db0cdbc5ce8a98cdcd0859b0","assignee":null,"assignees":[],"requested_reviewers":[{"login":"JeffBezanson","id":744556,"avatar_url":"https://avatars0.githubusercontent.com/u/744556?v=4","gravatar_id":"","url":"https://api.github.com/users/JeffBezanson","html_url":"https://github.com/JeffBezanson","followers_url":"https://api.github.com/users/JeffBezanson/followers","following_url":"https://api.github.com/users/JeffBezanson/following{/other_user}","gists_url":"https://api.github.com/users/JeffBezanson/gists{/gist_id}","starred_url":"https://api.github.com/users/JeffBezanson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JeffBezanson/subscriptions","organizations_url":"https://api.github.com/users/JeffBezanson/orgs","repos_url":"https://api.github.com/users/JeffBezanson/repos","events_url":"https://api.github.com/users/JeffBezanson/events{/privacy}","received_events_url":"https://api.github.com/users/JeffBezanson/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":{"url":"https://api.github.com/repos/JuliaLang/julia/milestones/4","html_url":"https://github.com/JuliaLang/julia/milestone/4","labels_url":"https://api.github.com/repos/JuliaLang/julia/milestones/4/labels","id":96266,"number":4,"title":"1.0","description":"","creator":{"login":"StefanKarpinski","id":153596,"avatar_url":"https://avatars2.githubusercontent.com/u/153596?v=4","gravatar_id":"","url":"https://api.github.com/users/StefanKarpinski","html_url":"https://github.com/StefanKarpinski","followers_url":"https://api.github.com/users/StefanKarpinski/followers","following_url":"https://api.github.com/users/StefanKarpinski/following{/other_user}","gists_url":"https://api.github.com/users/StefanKarpinski/gists{/gist_id}","starred_url":"https://api.github.com/users/StefanKarpinski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/StefanKarpinski/subscriptions","organizations_url":"https://api.github.com/users/StefanKarpinski/orgs","repos_url":"https://api.github.com/users/StefanKarpinski/repos","events_url":"https://api.github.com/users/StefanKarpinski/events{/privacy}","received_events_url":"https://api.github.com/users/StefanKarpinski/received_events","type":"User","site_admin":false},"open_issues":14,"closed_issues":434,"state":"open","created_at":"2012-03-15T04:57:43Z","updated_at":"2018-02-27T11:44:19Z","due_on":"2017-12-15T08:00:00Z","closed_at":null},"commits_url":"https://api.github.com/repos/JuliaLang/julia/pulls/26133/commits","review_comments_url":"https://api.github.com/repos/JuliaLang/julia/pulls/26133/comments","review_comment_url":"https://api.github.com/repos/JuliaLang/julia/pulls/comments{/number}","comments_url":"https://api.github.com/repos/JuliaLang/julia/issues/26133/comments","statuses_url":"https://api.github.com/repos/JuliaLang/julia/statuses/c2658e10e7d3fc14dfc436e8158a4dd64a9cc94a","head":{"label":"JuliaLang:kf/abstractstringit","ref":"kf/abstractstringit","sha":"c2658e10e7d3fc14dfc436e8158a4dd64a9cc94a","user":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars0.githubusercontent.com/u/743164?v=4","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"repo":{"id":1644196,"name":"julia","full_name":"JuliaLang/julia","owner":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars0.githubusercontent.com/u/743164?v=4","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/JuliaLang/julia","description":"The Julia Language: A fresh approach to technical computing.","fork":false,"url":"https://api.github.com/repos/JuliaLang/julia","forks_url":"https://api.github.com/repos/JuliaLang/julia/forks","keys_url":"https://api.github.com/repos/JuliaLang/julia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JuliaLang/julia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JuliaLang/julia/teams","hooks_url":"https://api.github.com/repos/JuliaLang/julia/hooks","issue_events_url":"https://api.github.com/repos/JuliaLang/julia/issues/events{/number}","events_url":"https://api.github.com/repos/JuliaLang/julia/events","assignees_url":"https://api.github.com/repos/JuliaLang/julia/assignees{/user}","branches_url":"https://api.github.com/repos/JuliaLang/julia/branches{/branch}","tags_url":"https://api.github.com/repos/JuliaLang/julia/tags","blobs_url":"https://api.github.com/repos/JuliaLang/julia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JuliaLang/julia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JuliaLang/julia/git/refs{/sha}","trees_url":"https://api.github.com/repos/JuliaLang/julia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JuliaLang/julia/statuses/{sha}","languages_url":"https://api.github.com/repos/JuliaLang/julia/languages","stargazers_url":"https://api.github.com/repos/JuliaLang/julia/stargazers","contributors_url":"https://api.github.com/repos/JuliaLang/julia/contributors","subscribers_url":"https://api.github.com/repos/JuliaLang/julia/subscribers","subscription_url":"https://api.github.com/repos/JuliaLang/julia/subscription","commits_url":"https://api.github.com/repos/JuliaLang/julia/commits{/sha}","git_commits_url":"https://api.github.com/repos/JuliaLang/julia/git/commits{/sha}","comments_url":"https://api.github.com/repos/JuliaLang/julia/comments{/number}","issue_comment_url":"https://api.github.com/repos/JuliaLang/julia/issues/comments{/number}","contents_url":"https://api.github.com/repos/JuliaLang/julia/contents/{+path}","compare_url":"https://api.github.com/repos/JuliaLang/julia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JuliaLang/julia/merges","archive_url":"https://api.github.com/repos/JuliaLang/julia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JuliaLang/julia/downloads","issues_url":"https://api.github.com/repos/JuliaLang/julia/issues{/number}","pulls_url":"https://api.github.com/repos/JuliaLang/julia/pulls{/number}","milestones_url":"https://api.github.com/repos/JuliaLang/julia/milestones{/number}","notifications_url":"https://api.github.com/repos/JuliaLang/julia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JuliaLang/julia/labels{/name}","releases_url":"https://api.github.com/repos/JuliaLang/julia/releases{/id}","deployments_url":"https://api.github.com/repos/JuliaLang/julia/deployments","created_at":"2011-04-21T07:01:50Z","updated_at":"2018-02-27T11:54:20Z","pushed_at":"2018-02-27T11:51:21Z","git_url":"git://github.com/JuliaLang/julia.git","ssh_url":"git@github.com:JuliaLang/julia.git","clone_url":"https://github.com/JuliaLang/julia.git","svn_url":"https://github.com/JuliaLang/julia","homepage":"https://julialang.org/","size":348365,"stargazers_count":10241,"watchers_count":10241,"language":"Julia","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":2280,"mirror_url":null,"archived":false,"open_issues_count":2519,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":2280,"open_issues":2519,"watchers":10241,"default_branch":"master"}},"base":{"label":"JuliaLang:master","ref":"master","sha":"d0723bfba6167a4e90a642e3ab7c8da258060793","user":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars0.githubusercontent.com/u/743164?v=4","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"repo":{"id":1644196,"name":"julia","full_name":"JuliaLang/julia","owner":{"login":"JuliaLang","id":743164,"avatar_url":"https://avatars0.githubusercontent.com/u/743164?v=4","gravatar_id":"","url":"https://api.github.com/users/JuliaLang","html_url":"https://github.com/JuliaLang","followers_url":"https://api.github.com/users/JuliaLang/followers","following_url":"https://api.github.com/users/JuliaLang/following{/other_user}","gists_url":"https://api.github.com/users/JuliaLang/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaLang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaLang/subscriptions","organizations_url":"https://api.github.com/users/JuliaLang/orgs","repos_url":"https://api.github.com/users/JuliaLang/repos","events_url":"https://api.github.com/users/JuliaLang/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaLang/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/JuliaLang/julia","description":"The Julia Language: A fresh approach to technical computing.","fork":false,"url":"https://api.github.com/repos/JuliaLang/julia","forks_url":"https://api.github.com/repos/JuliaLang/julia/forks","keys_url":"https://api.github.com/repos/JuliaLang/julia/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JuliaLang/julia/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JuliaLang/julia/teams","hooks_url":"https://api.github.com/repos/JuliaLang/julia/hooks","issue_events_url":"https://api.github.com/repos/JuliaLang/julia/issues/events{/number}","events_url":"https://api.github.com/repos/JuliaLang/julia/events","assignees_url":"https://api.github.com/repos/JuliaLang/julia/assignees{/user}","branches_url":"https://api.github.com/repos/JuliaLang/julia/branches{/branch}","tags_url":"https://api.github.com/repos/JuliaLang/julia/tags","blobs_url":"https://api.github.com/repos/JuliaLang/julia/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JuliaLang/julia/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JuliaLang/julia/git/refs{/sha}","trees_url":"https://api.github.com/repos/JuliaLang/julia/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JuliaLang/julia/statuses/{sha}","languages_url":"https://api.github.com/repos/JuliaLang/julia/languages","stargazers_url":"https://api.github.com/repos/JuliaLang/julia/stargazers","contributors_url":"https://api.github.com/repos/JuliaLang/julia/contributors","subscribers_url":"https://api.github.com/repos/JuliaLang/julia/subscribers","subscription_url":"https://api.github.com/repos/JuliaLang/julia/subscription","commits_url":"https://api.github.com/repos/JuliaLang/julia/commits{/sha}","git_commits_url":"https://api.github.com/repos/JuliaLang/julia/git/commits{/sha}","comments_url":"https://api.github.com/repos/JuliaLang/julia/comments{/number}","issue_comment_url":"https://api.github.com/repos/JuliaLang/julia/issues/comments{/number}","contents_url":"https://api.github.com/repos/JuliaLang/julia/contents/{+path}","compare_url":"https://api.github.com/repos/JuliaLang/julia/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JuliaLang/julia/merges","archive_url":"https://api.github.com/repos/JuliaLang/julia/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JuliaLang/julia/downloads","issues_url":"https://api.github.com/repos/JuliaLang/julia/issues{/number}","pulls_url":"https://api.github.com/repos/JuliaLang/julia/pulls{/number}","milestones_url":"https://api.github.com/repos/JuliaLang/julia/milestones{/number}","notifications_url":"https://api.github.com/repos/JuliaLang/julia/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JuliaLang/julia/labels{/name}","releases_url":"https://api.github.com/repos/JuliaLang/julia/releases{/id}","deployments_url":"https://api.github.com/repos/JuliaLang/julia/deployments","created_at":"2011-04-21T07:01:50Z","updated_at":"2018-02-27T11:54:20Z","pushed_at":"2018-02-27T11:51:21Z","git_url":"git://github.com/JuliaLang/julia.git","ssh_url":"git@github.com:JuliaLang/julia.git","clone_url":"https://github.com/JuliaLang/julia.git","svn_url":"https://github.com/JuliaLang/julia","homepage":"https://julialang.org/","size":348365,"stargazers_count":10241,"watchers_count":10241,"language":"Julia","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":2280,"mirror_url":null,"archived":false,"open_issues_count":2519,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":2280,"open_issues":2519,"watchers":10241,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/26133"},"html":{"href":"https://github.com/JuliaLang/julia/pull/26133"},"issue":{"href":"https://api.github.com/repos/JuliaLang/julia/issues/26133"},"comments":{"href":"https://api.github.com/repos/JuliaLang/julia/issues/26133/comments"},"review_comments":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/26133/comments"},"review_comment":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/JuliaLang/julia/pulls/26133/commits"},"statuses":{"href":"https://api.github.com/repos/JuliaLang/julia/statuses/c2658e10e7d3fc14dfc436e8158a4dd64a9cc94a"}},"author_association":"OWNER"}}
{"actor":{"display_login":"StefanKarpinski"}}