{"action":"created","issue":{"url":"https://api.github.com/repos/mafintosh/csv-parser/issues/59","repository_url":"https://api.github.com/repos/mafintosh/csv-parser","labels_url":"https://api.github.com/repos/mafintosh/csv-parser/issues/59/labels{/name}","comments_url":"https://api.github.com/repos/mafintosh/csv-parser/issues/59/comments","events_url":"https://api.github.com/repos/mafintosh/csv-parser/issues/59/events","html_url":"https://github.com/mafintosh/csv-parser/issues/59","id":180836031,"number":59,"title":"Not processes big files","user":{"login":"DmitryGalyuk","id":10270114,"avatar_url":"https://avatars.githubusercontent.com/u/10270114?v=3","gravatar_id":"","url":"https://api.github.com/users/DmitryGalyuk","html_url":"https://github.com/DmitryGalyuk","followers_url":"https://api.github.com/users/DmitryGalyuk/followers","following_url":"https://api.github.com/users/DmitryGalyuk/following{/other_user}","gists_url":"https://api.github.com/users/DmitryGalyuk/gists{/gist_id}","starred_url":"https://api.github.com/users/DmitryGalyuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DmitryGalyuk/subscriptions","organizations_url":"https://api.github.com/users/DmitryGalyuk/orgs","repos_url":"https://api.github.com/users/DmitryGalyuk/repos","events_url":"https://api.github.com/users/DmitryGalyuk/events{/privacy}","received_events_url":"https://api.github.com/users/DmitryGalyuk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-10-04T08:44:36Z","updated_at":"2016-12-10T01:49:25Z","closed_at":null,"body":"I try to process big file (23K lines). Only around 13K lines are processed (i.e. \"data\" even is raised). And no error events are raised, no exceptions throw.\nFile is available by link (zipped): http://download.geonames.org/export/dump/cities15000.zip\nMy code:\n\n```\nfunction csvToMap(filename, columns, rowCallback) {\n    return new Promise(function(resolve, reject) {\n        try {\nconsole.log(filename);\n            var linesRead = 0;\n            fs.createReadStream(filename)\n                .pipe(csv({\n                    separator: '\\t',\n                    headers: columns\n                }))\n                .on('data', function(data) {\n                    try {\n                        rowCallback(data);\n                        linesRead++;\n                    } catch (e) {\n                        die(e, reject);\n                    }\n                })\n                .on('end', () => {\n                    console.log(filename + \": \" + linesRead + \" lines read\");\n                    resolve();\n                })\n                .on('error', (e) => {\n                    die(e, reject);\n                });\n\n        } catch (e) {\n            die(e, reject);\n        }\n    });\n}\n\nfunction die(msg, callback) {\n    console.log(\"error: \" + msg);\n    if (callback) {\n        callback(msg);\n    }\n    throw msg;\n}\n```\n"},"comment":{"url":"https://api.github.com/repos/mafintosh/csv-parser/issues/comments/266169938","html_url":"https://github.com/mafintosh/csv-parser/issues/59#issuecomment-266169938","issue_url":"https://api.github.com/repos/mafintosh/csv-parser/issues/59","id":266169938,"user":{"login":"baptistemanson","id":5444992,"avatar_url":"https://avatars.githubusercontent.com/u/5444992?v=3","gravatar_id":"","url":"https://api.github.com/users/baptistemanson","html_url":"https://github.com/baptistemanson","followers_url":"https://api.github.com/users/baptistemanson/followers","following_url":"https://api.github.com/users/baptistemanson/following{/other_user}","gists_url":"https://api.github.com/users/baptistemanson/gists{/gist_id}","starred_url":"https://api.github.com/users/baptistemanson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/baptistemanson/subscriptions","organizations_url":"https://api.github.com/users/baptistemanson/orgs","repos_url":"https://api.github.com/users/baptistemanson/repos","events_url":"https://api.github.com/users/baptistemanson/events{/privacy}","received_events_url":"https://api.github.com/users/baptistemanson/received_events","type":"User","site_admin":false},"created_at":"2016-12-10T01:49:25Z","updated_at":"2016-12-10T01:49:25Z","body":"Hi @emilbayes . FYI we processed about 1B rows without any issue today so the size of your file is not necessarily the reason. \r\nCan you reproduce the bug when you remove the 1280 first lines?\r\nYou can use \r\n```bash\r\ntail -n +1280 cities15000.csv > truncated.csv\r\n```\r\nto get a smaller file."}}
{"actor":{"display_login":"baptistemanson"}}