{"action":"created","issue":{"url":"https://api.github.com/repos/sanack/node-jq/issues/51","repository_url":"https://api.github.com/repos/sanack/node-jq","labels_url":"https://api.github.com/repos/sanack/node-jq/issues/51/labels{/name}","comments_url":"https://api.github.com/repos/sanack/node-jq/issues/51/comments","events_url":"https://api.github.com/repos/sanack/node-jq/issues/51/events","html_url":"https://github.com/sanack/node-jq/issues/51","id":193255817,"number":51,"title":"node-jq throw E2BIG error when input json is too big","user":{"login":"qifuren1985","id":187371,"avatar_url":"https://avatars.githubusercontent.com/u/187371?v=3","gravatar_id":"","url":"https://api.github.com/users/qifuren1985","html_url":"https://github.com/qifuren1985","followers_url":"https://api.github.com/users/qifuren1985/followers","following_url":"https://api.github.com/users/qifuren1985/following{/other_user}","gists_url":"https://api.github.com/users/qifuren1985/gists{/gist_id}","starred_url":"https://api.github.com/users/qifuren1985/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qifuren1985/subscriptions","organizations_url":"https://api.github.com/users/qifuren1985/orgs","repos_url":"https://api.github.com/users/qifuren1985/repos","events_url":"https://api.github.com/users/qifuren1985/events{/privacy}","received_events_url":"https://api.github.com/users/qifuren1985/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-12-03T01:30:35Z","updated_at":"2016-12-03T16:16:57Z","closed_at":null,"body":"If input json is big array, it will throw E2BIG error.\r\nfor example, type this code in chrome console, it will generate a big array, and copy to clipboard.\r\n`function gen_data() { var ret = []; for(var i=0; i < 10000; i++) { ret.push({\"name\": i , \"kk\": i * 2}) } return ret; } copy(JSON.stringify(gen_data()));`\r\n\r\nwhen pass a array with 10000 elements to node-jq, it will throw E2BIG, due to a limitation of the size of the command line argument.\r\n\r\njq support read json string from stdin, so when call child_process to create process, we need pass json string to new process from stdin to fix this bug."},"comment":{"url":"https://api.github.com/repos/sanack/node-jq/issues/comments/264648661","html_url":"https://github.com/sanack/node-jq/issues/51#issuecomment-264648661","issue_url":"https://api.github.com/repos/sanack/node-jq/issues/51","id":264648661,"user":{"login":"davesnx","id":3763599,"avatar_url":"https://avatars.githubusercontent.com/u/3763599?v=3","gravatar_id":"","url":"https://api.github.com/users/davesnx","html_url":"https://github.com/davesnx","followers_url":"https://api.github.com/users/davesnx/followers","following_url":"https://api.github.com/users/davesnx/following{/other_user}","gists_url":"https://api.github.com/users/davesnx/gists{/gist_id}","starred_url":"https://api.github.com/users/davesnx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davesnx/subscriptions","organizations_url":"https://api.github.com/users/davesnx/orgs","repos_url":"https://api.github.com/users/davesnx/repos","events_url":"https://api.github.com/users/davesnx/events{/privacy}","received_events_url":"https://api.github.com/users/davesnx/received_events","type":"User","site_admin":false},"created_at":"2016-12-03T16:16:57Z","updated_at":"2016-12-03T16:16:57Z","body":"Hello @qifuren1985 \r\n\r\nI did a test and it's true there's a 'bug' arround E2BIG.\r\n\r\nFor reproduce I did this snippet:\r\n\r\n```js\r\nconst { run } = require('node-jq')\r\nconst fill = require('lodash.fill')\r\n\r\nconst filter = '. | map(select(.name > 1))'\r\nconst array2BIG = JSON.stringify(fill(Array(10000), { name: 2, lola: 'name' }))\r\nconst options = { input: 'string' }\r\n\r\nrun(filter, array2BIG, options)\r\n  .then((output) => {\r\n    console.log(output.length)\r\n  })\r\n  .catch((err) => {\r\n    console.error(err)\r\n  })\r\n```\r\n\r\nand the error that I get is:\r\n\r\n```bash\r\n{ Error: spawn E2BIG\r\n    at exports._errnoException (util.js:1036:11)\r\n    at ChildProcess.spawn (internal/child_process.js:313:11)\r\n    at exports.spawn (child_process.js:380:9)\r\n    at Object.exports.execFile (child_process.js:143:15)\r\n    at Object.exports.exec (child_process.js:103:18)\r\n    at /Users/davidsancho/dev/nodejqE2BIG/node_modules/node-jq/lib/exec.js:21:29\r\n    at exec (/Users/davidsancho/dev/nodejqE2BIG/node_modules/node-jq/lib/exec.js:20:10)\r\n    at /Users/davidsancho/dev/nodejqE2BIG/node_modules/node-jq/lib/jq.js:39:24\r\n    at run (/Users/davidsancho/dev/nodejqE2BIG/node_modules/node-jq/lib/jq.js:34:10)\r\n    at Object.<anonymous> (/Users/davidsancho/dev/nodejqE2BIG/index.js:8:1) code: 'E2BIG', errno: 'E2BIG', syscall: 'spawn' }\r\n```\r\n\r\nLet me explain what's happening here. The `E2BIG` means that the argument list is too long, as you point it out.\r\n\r\nDepend on which OS are you runnig node-jq, could you try to run `getconf ARG_MAX` in your terminal, will output the Maximum bytes per argument that your shell can run. It's pretty well explained [here](http://unix.stackexchange.com/questions/120642/what-defines-the-maximum-size-for-a-command-single-argument).\r\n\r\nIt's basically an error of child-process on node... [:sad:](http://stackoverflow.com/questions/36251711/node-child-process-throwing-e2big)"}}
{"actor":{"display_login":"davesnx"}}