{"action":"created","issue":{"url":"https://api.github.com/repos/spion/promise-streams/issues/12","repository_url":"https://api.github.com/repos/spion/promise-streams","labels_url":"https://api.github.com/repos/spion/promise-streams/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/spion/promise-streams/issues/12/comments","events_url":"https://api.github.com/repos/spion/promise-streams/issues/12/events","html_url":"https://github.com/spion/promise-streams/issues/12","id":151918936,"number":12,"title":"How to continue processing on error","user":{"login":"zam6ak","id":838582,"avatar_url":"https://avatars.githubusercontent.com/u/838582?v=3","gravatar_id":"","url":"https://api.github.com/users/zam6ak","html_url":"https://github.com/zam6ak","followers_url":"https://api.github.com/users/zam6ak/followers","following_url":"https://api.github.com/users/zam6ak/following{/other_user}","gists_url":"https://api.github.com/users/zam6ak/gists{/gist_id}","starred_url":"https://api.github.com/users/zam6ak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zam6ak/subscriptions","organizations_url":"https://api.github.com/users/zam6ak/orgs","repos_url":"https://api.github.com/users/zam6ak/repos","events_url":"https://api.github.com/users/zam6ak/events{/privacy}","received_events_url":"https://api.github.com/users/zam6ak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-29T16:53:01Z","updated_at":"2016-05-05T02:55:30Z","closed_at":null,"body":"Hi\r\n\r\nI have a TypeScript program that processes CSV file and for each record a number of awaited operations are performed (DB calls, net lookups, etc, etc)\r\n\r\nThe questions I have are:\r\n\r\n* Do I need to worry and resolve backpressure issues or are those handled by promise-streams?\r\n* How do I continue processing in case a stream throws an error (e.g. in my case CSV parser throws)\r\n* Is there anything in the code flow (see below) regarding usage of promise-streams that I am not doing as intended?\r\n\r\n\r\n```typescript\r\nimport * as Promise from 'bluebird';\r\nimport * as fs from 'fs';\r\nimport * as csvParse from 'csv-parse';\r\nconst ps = require('promise-streams'); // no typings :(\r\n\r\nconst csvFile = __dirname + '\\\\..\\\\data\\\\sample.csv';\r\nconst csvParserOptions: csvParse.ICsvParseOpts = {\r\n  delimiter: ',',\r\n  rowDelimiter: '\\r\\n',\r\n  quote: '\"',\r\n  columns: true,\r\n  skip_empty_lines: true,\r\n  trim: true,\r\n  auto_parse: false\r\n};\r\n\r\n\r\nasync function processCsvFile(csvFile: string): Promise<void> {\r\n  let recordNum = 1;\r\n  const inStream: fs.ReadStream = fs.createReadStream(csvFile);\r\n  const csvParser: csvParse.CsvParser = csvParse(csvParserOptions);\r\n\r\n  return ps.wait(\r\n\r\n      inStream\r\n      .pipe(csvParser)\r\n      .pipe(ps.map({concurrent: 1}, (csvRecord: any) => {\r\n        recordNum++;\r\n        console.log(`1) Processing CSV record #: ${recordNum.toLocaleString()} , data: ${JSON.stringify(csvRecord, null, 0)}`);\r\n        return processCsvFileRecord(csvRecord);\r\n      }))\r\n\r\n  ).then(() => {\r\n    console.log(`ps.wait() Finished processing!`);\r\n  });\r\n\r\n}\r\n\r\nasync function processCsvFileRecord(csvRecord: any): Promise<void> {\r\n  console.log('2) About to do some other work #1...');\r\n  await dbOp1();\r\n  console.log('3) About to do some other work #2...');\r\n  await dbOp2();\r\n  return;\r\n}\r\n\r\n\r\n// just a simulation via delay...\r\nasync function dbOp1(): Promise<void> {\r\n  return  Promise.delay(1000).then(() => { console.log('2-1) running dbOp1 (1 sec)...'); });\r\n}\r\n\r\nasync function dbOp2(): Promise<void> {\r\n  return Promise.delay(2000).then(() => { console.log('3-1) running dbOp1 (2 secs)...'); });\r\n}\r\n\r\nasync function main(): Promise<void> {\r\n    console.log('Begin main()....');\r\n    await processCsvFile(csvFile);\r\n    console.log('The end!');\r\n}\r\n\r\n\r\nmain();\r\n```\r\n\r\n\r\n "},"comment":{"url":"https://api.github.com/repos/spion/promise-streams/issues/comments/217064038","html_url":"https://github.com/spion/promise-streams/issues/12#issuecomment-217064038","issue_url":"https://api.github.com/repos/spion/promise-streams/issues/12","id":217064038,"user":{"login":"spion","id":502412,"avatar_url":"https://avatars.githubusercontent.com/u/502412?v=3","gravatar_id":"","url":"https://api.github.com/users/spion","html_url":"https://github.com/spion","followers_url":"https://api.github.com/users/spion/followers","following_url":"https://api.github.com/users/spion/following{/other_user}","gists_url":"https://api.github.com/users/spion/gists{/gist_id}","starred_url":"https://api.github.com/users/spion/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spion/subscriptions","organizations_url":"https://api.github.com/users/spion/orgs","repos_url":"https://api.github.com/users/spion/repos","events_url":"https://api.github.com/users/spion/events{/privacy}","received_events_url":"https://api.github.com/users/spion/received_events","type":"User","site_admin":false},"created_at":"2016-05-05T02:55:30Z","updated_at":"2016-05-05T02:55:30Z","body":"I'm still working on error semantics, which will be fixed in promise-streams 2.0\r\n\r\nHere is my initial idea/proposal: \r\n\r\nhttps://gist.github.com/spion/ecdc92bc5de5b381da30#initial-proposed-error-semantics\r\n\r\n"}}