{"action":"created","issue":{"url":"https://api.github.com/repos/nodejs/node/issues/6456","repository_url":"https://api.github.com/repos/nodejs/node","labels_url":"https://api.github.com/repos/nodejs/node/issues/6456/labels{/name}","comments_url":"https://api.github.com/repos/nodejs/node/issues/6456/comments","events_url":"https://api.github.com/repos/nodejs/node/issues/6456/events","html_url":"https://github.com/nodejs/node/issues/6456","id":151760275,"number":6456,"title":"stdio buffered writes (chunked) issues & process.exit() truncation","user":{"login":"eljefedelrodeodeljefe","id":3899684,"avatar_url":"https://avatars.githubusercontent.com/u/3899684?v=3","gravatar_id":"","url":"https://api.github.com/users/eljefedelrodeodeljefe","html_url":"https://github.com/eljefedelrodeodeljefe","followers_url":"https://api.github.com/users/eljefedelrodeodeljefe/followers","following_url":"https://api.github.com/users/eljefedelrodeodeljefe/following{/other_user}","gists_url":"https://api.github.com/users/eljefedelrodeodeljefe/gists{/gist_id}","starred_url":"https://api.github.com/users/eljefedelrodeodeljefe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eljefedelrodeodeljefe/subscriptions","organizations_url":"https://api.github.com/users/eljefedelrodeodeljefe/orgs","repos_url":"https://api.github.com/users/eljefedelrodeodeljefe/repos","events_url":"https://api.github.com/users/eljefedelrodeodeljefe/events{/privacy}","received_events_url":"https://api.github.com/users/eljefedelrodeodeljefe/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nodejs/node/labels/confirmed-bug","name":"confirmed-bug","color":"fc2929"},{"url":"https://api.github.com/repos/nodejs/node/labels/process","name":"process","color":"00aaff"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":149,"created_at":"2016-04-28T22:47:27Z","updated_at":"2016-05-18T13:57:57Z","closed_at":null,"body":"**CTC Note (Fishrock123): We are working on resolving this. Unfortunately it's not the most trivial thing to investigate or fix well.**\r\n\r\nIf this is currently breaking your program, please use this **temporary fix**:\r\n\r\n```js\r\n[process.stdout, process.stderr].forEach((s) => {\r\n  s && s.isTTY && s._handle && s._handle.setBlocking &&\r\n    s._handle.setBlocking(true)\r\n})\r\n```\r\n\r\n---\r\n\r\n<!--\r\nThanks for wanting to report an issue you've found in Node.js. Please fill in\r\nthe template below by replacing the html comments with an appropriate answer.\r\nIf unsure about something, just do as best as you're able.\r\n\r\nversion: usually output of `node -v`\r\nplatform:  either `uname -a` output, or if Windows, version and 32 or 64-bit.\r\nsubsystem:  optional -- if known please specify affected core module name.\r\n\r\nIt will be much easier for us to fix the issue if a test case that reproduces\r\nthe problem is provided. Ideally this test case should not have any external\r\ndependencies. We understand that it is not always possible to reduce your code\r\nto a small test case, but we would appreciate to have as\r\nmuch data as possible.\r\n\r\nThank you!\r\n-->\r\n\r\n* **Version**: v6, (likely all and backportable)\r\n* **Platform**: all\r\n* **Subsystem**: `process`\r\n\r\nAs noted in #6297 async stdio will not be flushed upon immediate `process.exit()`. This may lay open general deficiencies around C `exit()` from C++ functions not being properly unwound and is probably not just introduced by latest `libuv` updates. It should be considered to add flushing, providing graceful exit and/or improving unwinding C++ stacks.\r\n\r\ncc @jasnell, @kzc, @Qix-, @bnoordhuis  \r\n\r\n#### Issues\r\n\r\nDiscussion has been already taking place at several places, e.g. #6297, #6456, #6379\r\n\r\n#### Summaries of Proposals\r\n\r\n> proposals are not exclusive and could lead to semantically unrelated contributions.\r\n\r\n- aid with `process.stdout.flush()`\r\n- `process.setBlocking(true)`\r\n- `node --blocking-stdio`\r\n- `longjmp()` towards main at exit in C++\r\n- move parts of `process.exit()` / `process.reallyExit()` to new method `os.exit()`\r\n- golang `panic()`- or c++ `throw`-like stack unwinding\r\n\r\n#### Discussions by Author (with content)\r\n\r\n***\r\n@ChALkeR \r\n_I tried to discuss this some time ago at IRC, but postponed it for quite a long time. Also I started the discussion of this in #1741, but I would like to extract the more specific discussion to a separate issue._\r\n\r\nI could miss some details, but will try to give a quick overview here.\r\n\r\nSeveral issues here:\r\n 1. Many calls to `console.log` (e.g. calling it in a loop) could chew up all the memory and die — #1741, #2970, #3171.\r\n 2. `console.log` has different behavior while printing to a terminal and being redirected to a file. — https://github.com/nodejs/node/issues/1741#issuecomment-105333932.\r\n 3. Output is sometimes truncated — #6297, there were other ones as far as I remember.\r\n 4. The behaviour seems to differ across platforms.\r\n\r\nAs I understand it — the output has an implicit write buffer (as it's non-blocking) of unlimited size.\r\n\r\nOne approach to fixing this would be to:\r\n 1. Introduce an explicit cyclic write buffer.\r\n 2. Make writes to that cyclic buffer blocking.\r\n 3. Make writes from the buffer to the actual output non blocking.\r\n 4. When the cyclic buffer reaches it's maximum size (e.g. 10 MiB) — block further writes to the buffer until a corresponding part of it is freed.\r\n 5. On (normal) exit, make sure the buffer is flushed.\r\n\r\nFor almost all cases, except for the ones that are currently broken, this would behave as a non-blocking buffer (because writes to the buffer are considerably faster than writes from the buffer to file/terminal).\r\n\r\nFor cases when the data is being piped to the output too quickly and when the output file/terminal does not manage to output it at the same rate — the write would turn into a blocking operation. It would also be blocking at the exit until all the data is written.\r\n\r\n--\r\n\r\nAnother approach would be to monitor (and limit) the size of data that is contained in the implicit buffer coming from the async queue, and make the operations block when that limit is reached."},"comment":{"url":"https://api.github.com/repos/nodejs/node/issues/comments/220034087","html_url":"https://github.com/nodejs/node/issues/6456#issuecomment-220034087","issue_url":"https://api.github.com/repos/nodejs/node/issues/6456","id":220034087,"user":{"login":"saghul","id":317464,"avatar_url":"https://avatars.githubusercontent.com/u/317464?v=3","gravatar_id":"","url":"https://api.github.com/users/saghul","html_url":"https://github.com/saghul","followers_url":"https://api.github.com/users/saghul/followers","following_url":"https://api.github.com/users/saghul/following{/other_user}","gists_url":"https://api.github.com/users/saghul/gists{/gist_id}","starred_url":"https://api.github.com/users/saghul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saghul/subscriptions","organizations_url":"https://api.github.com/users/saghul/orgs","repos_url":"https://api.github.com/users/saghul/repos","events_url":"https://api.github.com/users/saghul/events{/privacy}","received_events_url":"https://api.github.com/users/saghul/received_events","type":"User","site_admin":false},"created_at":"2016-05-18T13:57:57Z","updated_at":"2016-05-18T13:57:57Z","body":"@kzc here it is: https://github.com/nodejs/node/blob/master/src/stream_wrap.cc#L85 That should use `uv_fileno` to get the real fd. When the select trick is used the real fd is not there. It works on 6 because the select trick is no longer used."}}