{"action":"created","issue":{"url":"https://api.github.com/repos/esp8266/Arduino/issues/4584","repository_url":"https://api.github.com/repos/esp8266/Arduino","labels_url":"https://api.github.com/repos/esp8266/Arduino/issues/4584/labels{/name}","comments_url":"https://api.github.com/repos/esp8266/Arduino/issues/4584/comments","events_url":"https://api.github.com/repos/esp8266/Arduino/issues/4584/events","html_url":"https://github.com/esp8266/Arduino/pull/4584","id":309205503,"number":4584,"title":"Fix StreamString read O(n^2); improve range read performance","user":{"login":"Adam5Wu","id":2480998,"avatar_url":"https://avatars2.githubusercontent.com/u/2480998?v=4","gravatar_id":"","url":"https://api.github.com/users/Adam5Wu","html_url":"https://github.com/Adam5Wu","followers_url":"https://api.github.com/users/Adam5Wu/followers","following_url":"https://api.github.com/users/Adam5Wu/following{/other_user}","gists_url":"https://api.github.com/users/Adam5Wu/gists{/gist_id}","starred_url":"https://api.github.com/users/Adam5Wu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Adam5Wu/subscriptions","organizations_url":"https://api.github.com/users/Adam5Wu/orgs","repos_url":"https://api.github.com/users/Adam5Wu/repos","events_url":"https://api.github.com/users/Adam5Wu/events{/privacy}","received_events_url":"https://api.github.com/users/Adam5Wu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2018-03-28T03:10:06Z","updated_at":"2018-03-29T04:26:28Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/esp8266/Arduino/pulls/4584","html_url":"https://github.com/esp8266/Arduino/pull/4584","diff_url":"https://github.com/esp8266/Arduino/pull/4584.diff","patch_url":"https://github.com/esp8266/Arduino/pull/4584.patch"},"body":"I ran into a performance anomaly where decoding a base64 certificate is faster than reading the decoded DER blob from a StreamString.\r\n\r\nWhen I inspect the operation, I found StreamString is reading the string from head to tail, byte by byte, and with each byte read, it removes it from the head, which triggers memmove of the rest of the string content 1 byte forward... such algorithm is O(n^2)\r\nhttps://github.com/esp8266/Arduino/blob/29580e8166b6ec3562929cb9bdc4bcd741c12faf/cores/esp8266/StreamString.cpp#L48-L49\r\n\r\nThis PR changed the internals of StreamString to overcome the slow read operation:\r\n1. Instead of removing bytes from the head of string, an offset is used to keep track of read progress;\r\n2. Range read is further optimized with a single memcpy.\r\n\r\nThere is, however, a bit of caveat with this change:\r\n1. Any data consumed by read is not lost. It could be re-consumed by resetting the offset. I consider it a benefit, and have added a `reset()` method to do just that;\r\n2. String assignment needs to be accompanied by a `reset()` call. Because the offset is in derived class while String is the base class, assignment of new string content will not automatically trigger offset reset (unless a significant amount of refactoring is done in the String class)."},"comment":{"url":"https://api.github.com/repos/esp8266/Arduino/issues/comments/377117613","html_url":"https://github.com/esp8266/Arduino/pull/4584#issuecomment-377117613","issue_url":"https://api.github.com/repos/esp8266/Arduino/issues/4584","id":377117613,"user":{"login":"devyte","id":16824838,"avatar_url":"https://avatars0.githubusercontent.com/u/16824838?v=4","gravatar_id":"","url":"https://api.github.com/users/devyte","html_url":"https://github.com/devyte","followers_url":"https://api.github.com/users/devyte/followers","following_url":"https://api.github.com/users/devyte/following{/other_user}","gists_url":"https://api.github.com/users/devyte/gists{/gist_id}","starred_url":"https://api.github.com/users/devyte/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/devyte/subscriptions","organizations_url":"https://api.github.com/users/devyte/orgs","repos_url":"https://api.github.com/users/devyte/repos","events_url":"https://api.github.com/users/devyte/events{/privacy}","received_events_url":"https://api.github.com/users/devyte/received_events","type":"User","site_admin":false},"created_at":"2018-03-29T04:26:28Z","updated_at":"2018-03-29T04:26:28Z","author_association":"COLLABORATOR","body":"```\r\ntemplate<size_t sizeT>\r\nvoid parseAndInsertJson(StaticJsonBuffer<sizeT> &buffer, JsonObject &existing, String &fragment) {\r\n   ...\r\n}\r\n```"}}
{"actor":{"display_login":"devyte"}}