{"action":"created","issue":{"url":"https://api.github.com/repos/websockets/ws/issues/1369","repository_url":"https://api.github.com/repos/websockets/ws","labels_url":"https://api.github.com/repos/websockets/ws/issues/1369/labels{/name}","comments_url":"https://api.github.com/repos/websockets/ws/issues/1369/comments","events_url":"https://api.github.com/repos/websockets/ws/issues/1369/events","html_url":"https://github.com/websockets/ws/issues/1369","id":316866509,"number":1369,"title":"Zlib compression needs to be sync","user":{"login":"alexhultman","id":3397140,"avatar_url":"https://avatars1.githubusercontent.com/u/3397140?v=4","gravatar_id":"","url":"https://api.github.com/users/alexhultman","html_url":"https://github.com/alexhultman","followers_url":"https://api.github.com/users/alexhultman/followers","following_url":"https://api.github.com/users/alexhultman/following{/other_user}","gists_url":"https://api.github.com/users/alexhultman/gists{/gist_id}","starred_url":"https://api.github.com/users/alexhultman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexhultman/subscriptions","organizations_url":"https://api.github.com/users/alexhultman/orgs","repos_url":"https://api.github.com/users/alexhultman/repos","events_url":"https://api.github.com/users/alexhultman/events{/privacy}","received_events_url":"https://api.github.com/users/alexhultman/received_events","type":"User","site_admin":false},"labels":[{"id":911584111,"url":"https://api.github.com/repos/websockets/ws/labels/discussion","name":"discussion","color":"058479","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":18,"created_at":"2018-04-23T15:34:30Z","updated_at":"2018-04-24T14:49:29Z","closed_at":null,"author_association":"NONE","body":"I've read and followed the zlib \"memory fragmentation\" issue down to where people blaming the Linux kernel and glibc and whatnot.\r\n\r\nMy two cents:\r\n\r\n* There is nothing wrong with the Linux kernel\r\n* There is nothing wrong with glibc\r\n* There is nothing wrong with Node.js\r\n\r\nThe issue is with websockets/ws.\r\n\r\nDoing CPU-bound tasks \"async\" is nothing more than a delusion - there is no such thing as an async CPU-bound task. If it takes 1 second of CPU-time, it takes 1 second of CPU time no matter when you perform it. Passing a CPU bound task to some thread pool doesn't improve anything in fact it only adds more overhead:\r\n\r\n* Threading overhead\r\n* You need to copy the data one extra time (major overhead!)\r\n* You're basically just procrastinating what you need to do right now.\r\n\r\nIf you cannot keep up with what you buffer up, you're obviously going to blow the process up from too much memory usage since you produce more than you consume.\r\n\r\nYou have two solutions:\r\n\r\n1. Replace all zlib calls with sync calls and all problems go away and you will lower latencies and actually do work you need to do and not just fool yourself you're doing things \"in the background\".\r\n\r\nOR if you _really_ want to try and use multithreading here (which goes against how Node.js is built and deployed):\r\n\r\n2. Only buffer up NUMBER_OF_LOGICAL_CPU_CORES with the async version and after this do sync calls until the buffer contains less than NUMBER_OF_LOGICAL_CPU_CORES tasks. Aka - you NEED to block sometime because CPU work cannot be \"async\" like networking. This solution is more complex and goes against what Node.js is but you get to choose."},"comment":{"url":"https://api.github.com/repos/websockets/ws/issues/comments/383961028","html_url":"https://github.com/websockets/ws/issues/1369#issuecomment-383961028","issue_url":"https://api.github.com/repos/websockets/ws/issues/1369","id":383961028,"user":{"login":"alexhultman","id":3397140,"avatar_url":"https://avatars1.githubusercontent.com/u/3397140?v=4","gravatar_id":"","url":"https://api.github.com/users/alexhultman","html_url":"https://github.com/alexhultman","followers_url":"https://api.github.com/users/alexhultman/followers","following_url":"https://api.github.com/users/alexhultman/following{/other_user}","gists_url":"https://api.github.com/users/alexhultman/gists{/gist_id}","starred_url":"https://api.github.com/users/alexhultman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexhultman/subscriptions","organizations_url":"https://api.github.com/users/alexhultman/orgs","repos_url":"https://api.github.com/users/alexhultman/repos","events_url":"https://api.github.com/users/alexhultman/events{/privacy}","received_events_url":"https://api.github.com/users/alexhultman/received_events","type":"User","site_admin":false},"created_at":"2018-04-24T14:49:29Z","updated_at":"2018-04-24T14:49:29Z","author_association":"NONE","body":"It's not arrogant, it's just short and to the point. Remember: I'm here to _hint_ of an _improvement_ nothing else. It gets kind of tiresome just repeating myself over and over when I _know for a fact_ this to work.\r\n\r\n> The point is that it's not ws fault.\r\n\r\nWell okay then, ignore everything I've said in this thread and keep it broken."}}
{"actor":{"display_login":"alexhultman"}}