{"action":"created","issue":{"url":"https://api.github.com/repos/crystal-lang/crystal/issues/3734","repository_url":"https://api.github.com/repos/crystal-lang/crystal","labels_url":"https://api.github.com/repos/crystal-lang/crystal/issues/3734/labels{/name}","comments_url":"https://api.github.com/repos/crystal-lang/crystal/issues/3734/comments","events_url":"https://api.github.com/repos/crystal-lang/crystal/issues/3734/events","html_url":"https://github.com/crystal-lang/crystal/pull/3734","id":196437119,"number":3734,"title":"Array/Enumerable/Iterator reuse flag","user":{"login":"asterite","id":209371,"avatar_url":"https://avatars.githubusercontent.com/u/209371?v=3","gravatar_id":"","url":"https://api.github.com/users/asterite","html_url":"https://github.com/asterite","followers_url":"https://api.github.com/users/asterite/followers","following_url":"https://api.github.com/users/asterite/following{/other_user}","gists_url":"https://api.github.com/users/asterite/gists{/gist_id}","starred_url":"https://api.github.com/users/asterite/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asterite/subscriptions","organizations_url":"https://api.github.com/users/asterite/orgs","repos_url":"https://api.github.com/users/asterite/repos","events_url":"https://api.github.com/users/asterite/events{/privacy}","received_events_url":"https://api.github.com/users/asterite/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-12-19T15:12:39Z","updated_at":"2016-12-19T17:23:04Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/crystal-lang/crystal/pulls/3734","html_url":"https://github.com/crystal-lang/crystal/pull/3734","diff_url":"https://github.com/crystal-lang/crystal/pull/3734.diff","patch_url":"https://github.com/crystal-lang/crystal/pull/3734.patch"},"body":"This adds an optional argument `reuse` to some Array, Enumerable and Iterator methods.\r\n\r\nCrystal, like Ruby, provides many methods to iterate through chunks of an enumerable/array/iterator. For example:\r\n\r\n```cr\r\nr = (1..10)\r\n\r\n# Compute sums of each consecutive slice of 2 elements\r\np r.each_slice(2).map { |s| s.sum }.to_a  # => [3, 7, 11, 15, 19]\r\n```\r\n\r\nWhile I really like these methods, they aren't particularly efficient: a new array is created each time the block is executed (in this case, every `s` is a different array). Because of this, I sometimes choose not to use these methods and instead do something more manual, so performance is better, less memory is allocated and there's less pressure on the GC... specially in tight loops or bottlenecks.\r\n\r\nHowever, I'd say that in most cases we use these \"slices\" or chunks of data as read-only data. In the above case we are interested in the sum of each slice. A single array could be created and reused for every slice. This is what the `reuse` optional argument is about. We can rewrite the above like this:\r\n\r\n```cr\r\nr = (1..10)\r\np r.each_slice(2, reuse: true).map { |s| s.sum }.to_a  # => [3, 7, 11, 15, 19]\r\n```\r\n\r\nWe get the same result, but only a single array is created: `s` will be the same array in every iteration, but with different contents each time.\r\n\r\nTo show how performance is improved, I made this program:\r\n\r\n```cr\r\n# foo.cr\r\nr = (1..10_000_000)\r\n\r\nreuse = ENV[\"REUSE\"]? == \"1\"\r\n\r\ntime = Time.now\r\ns = 0\r\nr.chunk(reuse: reuse, &.itself).each do |x, a|\r\n  s += a.sum\r\nend\r\nputs \"chunk: #{Time.now - time} (#{s})\"\r\n\r\ntime = Time.now\r\ns = 0\r\nr.each_slice(3, reuse: reuse) do |a|\r\n  s += a.sum\r\nend\r\nputs \"each_slice: #{Time.now - time} (#{s})\"\r\n\r\na = (1..50).to_a\r\n\r\ntime = Time.now\r\ns = 0\r\na.each_permutation(3, reuse: reuse) do |a|\r\n  s += a.sum\r\nend\r\nputs \"each_permutation: #{Time.now - time} (#{s})\"\r\n\r\ntime = Time.now\r\ns = 0\r\na.each_combination(3, reuse: reuse) do |a|\r\n  s += a.sum\r\nend\r\nputs \"each_combination: #{Time.now - time} (#{s})\"\r\n\r\ntime = Time.now\r\ns = 0\r\na.each_repeated_combination(3, reuse: reuse) do |a|\r\n  s += a.sum\r\nend\r\nputs \"each_repeated_combination: #{Time.now - time} (#{s})\"\r\n\r\ntime = Time.now\r\ns = 0\r\nr.each_cons(3, reuse: reuse) do |a|\r\n  s += a.sum\r\nend\r\nputs \"each_cons: #{Time.now - time} (#{s})\"\r\n```\r\n\r\nWe compile it with `--release` and then run it with REUSE=0 and REUSE=1:\r\n\r\n```\r\n$ time REUSE=0 ./foo\r\nchunk: 00:00:00.9652130 (-2004260032)\r\neach_slice: 00:00:00.3240950 (-2004260032)\r\neach_permutation: 00:00:00.0137700 (8996400)\r\neach_combination: 00:00:00.0022050 (1499400)\r\neach_repeated_combination: 00:00:00.0025910 (1690650)\r\neach_cons: 00:00:01.5072000 (-1747812803)\r\n\r\nreal  0m2.822s\r\nuser  0m1.751s\r\nsys 0m2.645s\r\n\r\n$ time REUSE=1 ./foo\r\nchunk: 00:00:00.1730330 (-2004260032)\r\neach_slice: 00:00:00.0424320 (-2004260032)\r\neach_permutation: 00:00:00.0028600 (8996400)\r\neach_combination: 00:00:00.0004450 (1499400)\r\neach_repeated_combination: 00:00:00.0004990 (1690650)\r\neach_cons: 00:00:00.1141390 (-1747812803)\r\n\r\nreal  0m0.340s\r\nuser  0m0.334s\r\nsys 0m0.004s\r\n```\r\n\r\nIn some cases we get more than a 10x performance improvement. \r\n"},"comment":{"url":"https://api.github.com/repos/crystal-lang/crystal/issues/comments/268023960","html_url":"https://github.com/crystal-lang/crystal/pull/3734#issuecomment-268023960","issue_url":"https://api.github.com/repos/crystal-lang/crystal/issues/3734","id":268023960,"user":{"login":"asterite","id":209371,"avatar_url":"https://avatars.githubusercontent.com/u/209371?v=3","gravatar_id":"","url":"https://api.github.com/users/asterite","html_url":"https://github.com/asterite","followers_url":"https://api.github.com/users/asterite/followers","following_url":"https://api.github.com/users/asterite/following{/other_user}","gists_url":"https://api.github.com/users/asterite/gists{/gist_id}","starred_url":"https://api.github.com/users/asterite/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asterite/subscriptions","organizations_url":"https://api.github.com/users/asterite/orgs","repos_url":"https://api.github.com/users/asterite/repos","events_url":"https://api.github.com/users/asterite/events{/privacy}","received_events_url":"https://api.github.com/users/asterite/received_events","type":"User","site_admin":false},"created_at":"2016-12-19T17:23:04Z","updated_at":"2016-12-19T17:23:04Z","body":"@RX14 I don't understand. We do `ImmutableArray.new(other_array)`. But then `other_array` will change in the next iteration, so the immutable array will change.\r\n\r\n@yxhuvud It would be less intuitive:\r\n\r\n```cr\r\n(1..4).each_slice(2).to_a # => [[3, 4], [3, 4]]\r\n```"}}
{"actor":{"display_login":"asterite"}}