{"action":"created","issue":{"url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/198","repository_url":"https://api.github.com/repos/firebase/firebase-admin-python","labels_url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/198/labels{/name}","comments_url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/198/comments","events_url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/198/events","html_url":"https://github.com/firebase/firebase-admin-python/issues/198","id":361940488,"node_id":"MDU6SXNzdWUzNjE5NDA0ODg=","number":198,"title":"Performance issue and copyright violation in `_sseclient.py` (for `db.Reference.listen()`)","user":{"login":"daniel-ziegler","id":1620154,"node_id":"MDQ6VXNlcjE2MjAxNTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/1620154?v=4","gravatar_id":"","url":"https://api.github.com/users/daniel-ziegler","html_url":"https://github.com/daniel-ziegler","followers_url":"https://api.github.com/users/daniel-ziegler/followers","following_url":"https://api.github.com/users/daniel-ziegler/following{/other_user}","gists_url":"https://api.github.com/users/daniel-ziegler/gists{/gist_id}","starred_url":"https://api.github.com/users/daniel-ziegler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daniel-ziegler/subscriptions","organizations_url":"https://api.github.com/users/daniel-ziegler/orgs","repos_url":"https://api.github.com/users/daniel-ziegler/repos","events_url":"https://api.github.com/users/daniel-ziegler/events{/privacy}","received_events_url":"https://api.github.com/users/daniel-ziegler/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-09-19T21:47:26Z","updated_at":"2018-10-25T16:35:17Z","closed_at":null,"author_association":"NONE","body":"### [REQUIRED] Step 2: Describe your environment\r\n\r\n  * Operating System version: macOS 10.13.6\r\n  * Firebase SDK version: 2.13.0\r\n  * Firebase Product: database\r\n\r\n### [REQUIRED] Step 3: Describe the problem\r\n\r\n#### Steps to reproduce:\r\n\r\n1. Have some Firebase Realtime Database path containing a significant amount of data (>200KB, say).\r\n2. Call `db.reference('/path').listen(callback)`.\r\n\r\nAs the first update, the server will send the entire contents of that path. On my machine this pegs a CPU core for minutes before the callback is finally called. The issue is the following code: https://github.com/firebase/firebase-admin-python/blob/abd39805f83f906296d5c99b6c4943754154d02c/firebase_admin/_sseclient.py#L93-L96\r\nSince `self.resp_iterator` iterates through the stream one character at a time and `self._event_complete()` runs a regular expression over the entire buffer, this is an O(N^2) operation, which takes a long time when N is large.\r\n\r\n**Separately, there is a copyright violation here.** I discovered that `_sseclient.py, despite having a Google copyright header, is largely copied from https://github.com/btubbs/sseclient/blob/master/sseclient.py.\r\n\r\n#### Relevant Code:\r\n\r\nRun the following code to reproduce:\r\n```python\r\ndef listen_issue():\r\n    path = '/sandbox/listen'\r\n\r\n    print(\"Inserting data...\")\r\n    start = time.time()\r\n    for i in range(10):\r\n        long_data = ''.join(random.choice(string.ascii_uppercase) for _ in range(20000))\r\n        db.reference(os.path.join(path, str(i))).set(long_data)\r\n    end = time.time()\r\n    print(\"...done (took %f seconds)\" % (end - start))\r\n\r\n    done_event = threading.Event()\r\n    listener = db.reference(path).listen(lambda db_event: done_event.set())\r\n    start = time.time()\r\n    print(\"Waiting for first message from listen()...\")\r\n    done_event.wait()  # CPU usage is at 100% here\r\n    end = time.time()\r\n    print(\"...done (took %f seconds)\" % (end - start))\r\n    listener.close()\r\n```\r\n\r\nExample output on my machine:\r\n```\r\nInserting data...\r\n...done (took 1.678544 seconds)\r\nWaiting for first message from listen()...\r\n...done (took 109.180737 seconds)\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/comments/433120659","html_url":"https://github.com/firebase/firebase-admin-python/issues/198#issuecomment-433120659","issue_url":"https://api.github.com/repos/firebase/firebase-admin-python/issues/198","id":433120659,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMzEyMDY1OQ==","user":{"login":"daniel-ziegler","id":1620154,"node_id":"MDQ6VXNlcjE2MjAxNTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/1620154?v=4","gravatar_id":"","url":"https://api.github.com/users/daniel-ziegler","html_url":"https://github.com/daniel-ziegler","followers_url":"https://api.github.com/users/daniel-ziegler/followers","following_url":"https://api.github.com/users/daniel-ziegler/following{/other_user}","gists_url":"https://api.github.com/users/daniel-ziegler/gists{/gist_id}","starred_url":"https://api.github.com/users/daniel-ziegler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daniel-ziegler/subscriptions","organizations_url":"https://api.github.com/users/daniel-ziegler/orgs","repos_url":"https://api.github.com/users/daniel-ziegler/repos","events_url":"https://api.github.com/users/daniel-ziegler/events{/privacy}","received_events_url":"https://api.github.com/users/daniel-ziegler/received_events","type":"User","site_admin":false},"created_at":"2018-10-25T16:35:17Z","updated_at":"2018-10-25T16:35:17Z","author_association":"NONE","body":"Yes, I believe it does work, although there's some complexity like handling both PUT and PATCH events."}}
{"actor":{"display_login":"daniel-ziegler"}}