{"action":"created","issue":{"url":"https://api.github.com/repos/ipfs/js-ipfs/issues/1325","repository_url":"https://api.github.com/repos/ipfs/js-ipfs","labels_url":"https://api.github.com/repos/ipfs/js-ipfs/issues/1325/labels{/name}","comments_url":"https://api.github.com/repos/ipfs/js-ipfs/issues/1325/comments","events_url":"https://api.github.com/repos/ipfs/js-ipfs/issues/1325/events","html_url":"https://github.com/ipfs/js-ipfs/issues/1325","id":317208909,"number":1325,"title":"Better error handling in daemon + gateway","user":{"login":"VictorBjelkholm","id":459764,"avatar_url":"https://avatars2.githubusercontent.com/u/459764?v=4","gravatar_id":"","url":"https://api.github.com/users/VictorBjelkholm","html_url":"https://github.com/VictorBjelkholm","followers_url":"https://api.github.com/users/VictorBjelkholm/followers","following_url":"https://api.github.com/users/VictorBjelkholm/following{/other_user}","gists_url":"https://api.github.com/users/VictorBjelkholm/gists{/gist_id}","starred_url":"https://api.github.com/users/VictorBjelkholm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VictorBjelkholm/subscriptions","organizations_url":"https://api.github.com/users/VictorBjelkholm/orgs","repos_url":"https://api.github.com/users/VictorBjelkholm/repos","events_url":"https://api.github.com/users/VictorBjelkholm/events{/privacy}","received_events_url":"https://api.github.com/users/VictorBjelkholm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"fsdiogo","id":33324750,"avatar_url":"https://avatars3.githubusercontent.com/u/33324750?v=4","gravatar_id":"","url":"https://api.github.com/users/fsdiogo","html_url":"https://github.com/fsdiogo","followers_url":"https://api.github.com/users/fsdiogo/followers","following_url":"https://api.github.com/users/fsdiogo/following{/other_user}","gists_url":"https://api.github.com/users/fsdiogo/gists{/gist_id}","starred_url":"https://api.github.com/users/fsdiogo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fsdiogo/subscriptions","organizations_url":"https://api.github.com/users/fsdiogo/orgs","repos_url":"https://api.github.com/users/fsdiogo/repos","events_url":"https://api.github.com/users/fsdiogo/events{/privacy}","received_events_url":"https://api.github.com/users/fsdiogo/received_events","type":"User","site_admin":false},"assignees":[{"login":"fsdiogo","id":33324750,"avatar_url":"https://avatars3.githubusercontent.com/u/33324750?v=4","gravatar_id":"","url":"https://api.github.com/users/fsdiogo","html_url":"https://github.com/fsdiogo","followers_url":"https://api.github.com/users/fsdiogo/followers","following_url":"https://api.github.com/users/fsdiogo/following{/other_user}","gists_url":"https://api.github.com/users/fsdiogo/gists{/gist_id}","starred_url":"https://api.github.com/users/fsdiogo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fsdiogo/subscriptions","organizations_url":"https://api.github.com/users/fsdiogo/orgs","repos_url":"https://api.github.com/users/fsdiogo/repos","events_url":"https://api.github.com/users/fsdiogo/events{/privacy}","received_events_url":"https://api.github.com/users/fsdiogo/received_events","type":"User","site_admin":false}],"milestone":null,"comments":19,"created_at":"2018-04-24T12:46:02Z","updated_at":"2018-05-25T11:21:36Z","closed_at":null,"author_association":"MEMBER","body":"Today, if there is a error in a stream or any other error within the daemon or the gateway, the entire process crashes. While not a problem for testing, it's a problem when deploying and trying to run a long-term daemon (for example, for the js-ipfs daemons).\r\n\r\nI raised this before in https://github.com/ipfs/js-ipfs/pull/1243\r\n\r\nBasically, we should be able to gracefully handle errors down the stack without having the daemon and gateway crash. \r\n\r\nCurrent issue is the following error that happens every ~10 minutes:\r\n\r\n```\r\n/usr/src/app/node_modules/pull-pair/index.js:11\r\n      throw new Error('already piped')\r\n      ^\r\n\r\nError: already piped\r\n    at sink (/usr/src/app/node_modules/pull-pair/index.js:11:13)\r\n    at consume (/usr/src/app/node_modules/pull-defer/sink.js:7:17)\r\n    at consume (/usr/src/app/node_modules/pull-defer/sink.js:7:17)\r\n    at Connection.consume (/usr/src/app/node_modules/pull-defer/sink.js:7:17)\r\n    at pull (/usr/src/app/node_modules/pull-stream/pull.js:43:9)\r\n    at Dialer.handle (/usr/src/app/node_modules/multistream-select/src/dialer/index.js:47:5)\r\n    at attemptMuxerUpgrade (/usr/src/app/node_modules/libp2p-switch/src/dial.js:164:10)\r\n    at gotWarmedUpConn (/usr/src/app/node_modules/libp2p-switch/src/dial.js:55:7)\r\n    at Switch.dial (/usr/src/app/node_modules/libp2p-switch/src/dial.js:41:9)\r\n    at _getPeerInfo (/usr/src/app/node_modules/libp2p/src/index.js:238:19)\r\n```\r\n\r\n**I'm not saying that we should (or should not) fix this particular error, but rather when this error (and similar) happens, the daemon and gateway should not die but continue running.**"},"comment":{"url":"https://api.github.com/repos/ipfs/js-ipfs/issues/comments/392022738","html_url":"https://github.com/ipfs/js-ipfs/issues/1325#issuecomment-392022738","issue_url":"https://api.github.com/repos/ipfs/js-ipfs/issues/1325","id":392022738,"user":{"login":"alanshaw","id":152863,"avatar_url":"https://avatars1.githubusercontent.com/u/152863?v=4","gravatar_id":"","url":"https://api.github.com/users/alanshaw","html_url":"https://github.com/alanshaw","followers_url":"https://api.github.com/users/alanshaw/followers","following_url":"https://api.github.com/users/alanshaw/following{/other_user}","gists_url":"https://api.github.com/users/alanshaw/gists{/gist_id}","starred_url":"https://api.github.com/users/alanshaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alanshaw/subscriptions","organizations_url":"https://api.github.com/users/alanshaw/orgs","repos_url":"https://api.github.com/users/alanshaw/repos","events_url":"https://api.github.com/users/alanshaw/events{/privacy}","received_events_url":"https://api.github.com/users/alanshaw/received_events","type":"User","site_admin":false},"created_at":"2018-05-25T11:21:36Z","updated_at":"2018-05-25T11:21:36Z","author_association":"MEMBER","body":"I have a couple more suggestions for your consideration:\r\n\r\n* When dealing with Node.js streams, use [`pump`](https://www.npmjs.com/package/pump) whenever possible to ensure errors are caught and handled\r\n* Add context to errors and allow our tests to assert on error messages we control in our modules by using [`explain-error`](https://www.npmjs.com/package/explain-error)\r\n\r\nTo address @VictorBjelkholm's original request:\r\n\r\nIn Node.js the best practice on uncaught exceptions is to crash the process. We need to track down and fix any issues that cause the process to crash. In Node.js you can't meaningfully reason about the state of the system in the case of an unhandled exception so you need to restart the process to avoid leaking memory, holding open connections unnecessarily, corrupting shared state etc. etc.\r\n\r\nIt's worth having a read of https://www.joyent.com/node-js/production/design/errors - relevant section:\r\n\r\n> The best way to recover from programmer errors is to crash immediately. \r\n>\r\n> The only downside to crashing on programmer errors is that connected clients may be temporarily disrupted, but remember:\r\n> * By definition, these errors are always bugs. We're not talking about legitimate system or network failures, but actual bugs in the program. They should be rare in production, and the top priority has to be to debug and fix them.\r\n> * For all the cases described above (and many more), the requests in flight are not necessarily going to complete successfully anyway. They may complete successfully, they may crash the server again, they may complete incorrectly in obvious ways, or they may complete wrongly in very subtle ways that are very hard to debug.\r\n> * In a reliable distributed system, clients must be able to deal with server failure by reconnecting and retrying requests. Network and system failure are a reality, whether or not the Node.js program itself is allowed to crash.\r\n> * If your production program is crashing so often that these disconnections are a problem, then the real problem is that the server is so buggy, not that it crashes in the case of a bug.\r\n> \r\n> If disconnecting clients is a frequently problem because a server crashes so often, you should focus on the bugs that cause the service to crash — and make those exceptional — rather than trying to avoid crashing in cases where the code is obviously wrong\r\n"}}
{"actor":{"display_login":"alanshaw"}}