{"action":"opened","issue":{"url":"https://api.github.com/repos/golang/go/issues/28311","repository_url":"https://api.github.com/repos/golang/go","labels_url":"https://api.github.com/repos/golang/go/issues/28311/labels{/name}","comments_url":"https://api.github.com/repos/golang/go/issues/28311/comments","events_url":"https://api.github.com/repos/golang/go/issues/28311/events","html_url":"https://github.com/golang/go/issues/28311","id":372529787,"node_id":"MDU6SXNzdWUzNzI1Mjk3ODc=","number":28311,"title":"io: optimize WriteString with a pool of buffer","user":{"login":"pierrre","id":131535,"node_id":"MDQ6VXNlcjEzMTUzNQ==","avatar_url":"https://avatars3.githubusercontent.com/u/131535?v=4","gravatar_id":"","url":"https://api.github.com/users/pierrre","html_url":"https://github.com/pierrre","followers_url":"https://api.github.com/users/pierrre/followers","following_url":"https://api.github.com/users/pierrre/following{/other_user}","gists_url":"https://api.github.com/users/pierrre/gists{/gist_id}","starred_url":"https://api.github.com/users/pierrre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pierrre/subscriptions","organizations_url":"https://api.github.com/users/pierrre/orgs","repos_url":"https://api.github.com/users/pierrre/repos","events_url":"https://api.github.com/users/pierrre/events{/privacy}","received_events_url":"https://api.github.com/users/pierrre/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-22T13:50:30Z","updated_at":"2018-10-22T13:50:30Z","closed_at":null,"author_association":"NONE","body":"The current implementation of `io.WriteString` allocates a `[]byte` if the writer doesn't implement `StringWriter`.\r\nI think we can avoid this memory allocation by using a pool of buffer.\r\n\r\nI wrote a proof of concept:\r\n```go\r\npackage writestring\r\n\r\nimport (\r\n\t\"bytes\"\r\n\t\"io\"\r\n\t\"sync\"\r\n)\r\n\r\nfunc WriteString(w io.Writer, s string) (n int, err error) {\r\n\tif sw, ok := w.(stringWriter); ok {\r\n\t\treturn sw.WriteString(s)\r\n\t}\r\n\tif len(s) > maxBufSize {\r\n\t\treturn w.Write([]byte(s))\r\n\t}\r\n\tbuf := getBuf()\r\n\t_, _ = buf.WriteString(s)\r\n\tn, err = w.Write(buf.Bytes())\r\n\tputBuf(buf)\r\n\treturn n, err\r\n}\r\n\r\ntype stringWriter interface {\r\n\tWriteString(s string) (n int, err error)\r\n}\r\n\r\nconst maxBufSize = 1 << 16\r\n\r\nvar bufPool = &sync.Pool{\r\n\tNew: func() interface{} {\r\n\t\treturn new(bytes.Buffer)\r\n\t},\r\n}\r\n\r\nfunc getBuf() *bytes.Buffer {\r\n\tbuf := bufPool.Get().(*bytes.Buffer)\r\n\tbuf.Reset()\r\n\treturn buf\r\n}\r\n\r\nfunc putBuf(buf *bytes.Buffer) {\r\n\tif buf.Cap() <= maxBufSize {\r\n\t\tbufPool.Put(buf)\r\n\t}\r\n}\r\n```\r\n\r\nA benchmark:\r\n```go\r\npackage writestring\r\n\r\nimport (\r\n\t\"io\"\r\n\t\"strconv\"\r\n\t\"strings\"\r\n\t\"testing\"\r\n)\r\n\r\nvar nw io.Writer = new(nopWriter)\r\n\r\nfunc BenchmarkWriteString(b *testing.B) {\r\n\tfor p := 0; p <= 20; p++ {\r\n\t\tsize := 1 << uint(p)\r\n\t\ts := strings.Repeat(\"a\", size)\r\n\t\tb.Run(strconv.FormatInt(int64(size), 10), func(b *testing.B) {\r\n\t\t\tb.Run(\"Old\", func(b *testing.B) {\r\n\t\t\t\tfor i := 0; i < b.N; i++ {\r\n\t\t\t\t\t_, _ = io.WriteString(nw, s)\r\n\t\t\t\t}\r\n\t\t\t})\r\n\t\t\tb.Run(\"New\", func(b *testing.B) {\r\n\t\t\t\tfor i := 0; i < b.N; i++ {\r\n\t\t\t\t\t_, _ = WriteString(nw, s)\r\n\t\t\t\t}\r\n\t\t\t})\r\n\t\t})\r\n\t}\r\n}\r\n\r\ntype nopWriter struct{}\r\n\r\nfunc (w *nopWriter) Write(b []byte) (n int, err error) {\r\n\treturn len(b), nil\r\n}\r\n```\r\n\r\nThe benchmark result:\r\n```\r\n➜  writestring go test -bench=. -benchmem\r\ngoos: linux\r\ngoarch: amd64\r\npkg: _test/writestring\r\nBenchmarkWriteString/1/Old-12         \t50000000\t        31.1 ns/op\t       8 B/op\t       1 allocs/op\r\nBenchmarkWriteString/1/New-12         \t50000000\t        33.5 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/2/Old-12         \t50000000\t        31.4 ns/op\t       8 B/op\t       1 allocs/op\r\nBenchmarkWriteString/2/New-12         \t50000000\t        34.7 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/4/Old-12         \t50000000\t        30.1 ns/op\t       8 B/op\t       1 allocs/op\r\nBenchmarkWriteString/4/New-12         \t50000000\t        36.0 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/8/Old-12         \t50000000\t        28.8 ns/op\t       8 B/op\t       1 allocs/op\r\nBenchmarkWriteString/8/New-12         \t50000000\t        32.9 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/16/Old-12        \t50000000\t        49.7 ns/op\t      16 B/op\t       1 allocs/op\r\nBenchmarkWriteString/16/New-12        \t50000000\t        33.4 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/32/Old-12        \t50000000\t        70.6 ns/op\t      32 B/op\t       1 allocs/op\r\nBenchmarkWriteString/32/New-12        \t30000000\t        33.5 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/64/Old-12        \t30000000\t       110 ns/op\t      64 B/op\t       1 allocs/op\r\nBenchmarkWriteString/64/New-12        \t50000000\t        34.3 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/128/Old-12       \t30000000\t       144 ns/op\t     128 B/op\t       1 allocs/op\r\nBenchmarkWriteString/128/New-12       \t50000000\t        36.2 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/256/Old-12       \t20000000\t       198 ns/op\t     256 B/op\t       1 allocs/op\r\nBenchmarkWriteString/256/New-12       \t50000000\t        39.3 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/512/Old-12       \t10000000\t       291 ns/op\t     512 B/op\t       1 allocs/op\r\nBenchmarkWriteString/512/New-12       \t30000000\t        42.0 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/1024/Old-12      \t 5000000\t       490 ns/op\t    1024 B/op\t       1 allocs/op\r\nBenchmarkWriteString/1024/New-12      \t30000000\t        45.6 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/2048/Old-12      \t 3000000\t       500 ns/op\t    2048 B/op\t       1 allocs/op\r\nBenchmarkWriteString/2048/New-12      \t20000000\t        52.3 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/4096/Old-12      \t 2000000\t      1091 ns/op\t    4096 B/op\t       1 allocs/op\r\nBenchmarkWriteString/4096/New-12      \t20000000\t        68.3 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/8192/Old-12      \t 1000000\t      1709 ns/op\t    8192 B/op\t       1 allocs/op\r\nBenchmarkWriteString/8192/New-12      \t20000000\t       103 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/16384/Old-12     \t 1000000\t      3446 ns/op\t   16384 B/op\t       1 allocs/op\r\nBenchmarkWriteString/16384/New-12     \t 5000000\t       328 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/32768/Old-12     \t  500000\t      3281 ns/op\t   32768 B/op\t       1 allocs/op\r\nBenchmarkWriteString/32768/New-12     \t 2000000\t       653 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/65536/Old-12     \t  300000\t      5022 ns/op\t   65536 B/op\t       1 allocs/op\r\nBenchmarkWriteString/65536/New-12     \t 1000000\t      1566 ns/op\t       0 B/op\t       0 allocs/op\r\nBenchmarkWriteString/131072/Old-12    \t  100000\t     25294 ns/op\t  131072 B/op\t       1 allocs/op\r\nBenchmarkWriteString/131072/New-12    \t   50000\t     25332 ns/op\t  131072 B/op\t       1 allocs/op\r\nBenchmarkWriteString/262144/Old-12    \t   30000\t     50864 ns/op\t  262144 B/op\t       1 allocs/op\r\nBenchmarkWriteString/262144/New-12    \t   30000\t     50697 ns/op\t  262144 B/op\t       1 allocs/op\r\nBenchmarkWriteString/524288/Old-12    \t   10000\t    103603 ns/op\t  524288 B/op\t       1 allocs/op\r\nBenchmarkWriteString/524288/New-12    \t   10000\t    104370 ns/op\t  524288 B/op\t       1 allocs/op\r\nBenchmarkWriteString/1048576/Old-12   \t   10000\t    205753 ns/op\t 1048576 B/op\t       1 allocs/op\r\nBenchmarkWriteString/1048576/New-12   \t   10000\t    206086 ns/op\t 1048576 B/op\t       1 allocs/op\r\nPASS\r\nok  \t_test/writestring\t92.576s\r\n```\r\n\r\nI know we can't import `bytes` in `io`.\r\nA real implementation would require to copy a private implementation of `bytes.Buffer` to `io`."}}
{"actor":{"display_login":"pierrre"}}