{"action":"created","issue":{"url":"https://api.github.com/repos/dominictarr/event-stream/issues/93","labels_url":"https://api.github.com/repos/dominictarr/event-stream/issues/93/labels{/name}","comments_url":"https://api.github.com/repos/dominictarr/event-stream/issues/93/comments","events_url":"https://api.github.com/repos/dominictarr/event-stream/issues/93/events","html_url":"https://github.com/dominictarr/event-stream/issues/93","id":126625668,"number":93,"title":"es.merge broke my data","user":{"login":"zwb-ict","id":10378547,"avatar_url":"https://avatars.githubusercontent.com/u/10378547?v=3","gravatar_id":"","url":"https://api.github.com/users/zwb-ict","html_url":"https://github.com/zwb-ict","followers_url":"https://api.github.com/users/zwb-ict/followers","following_url":"https://api.github.com/users/zwb-ict/following{/other_user}","gists_url":"https://api.github.com/users/zwb-ict/gists{/gist_id}","starred_url":"https://api.github.com/users/zwb-ict/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zwb-ict/subscriptions","organizations_url":"https://api.github.com/users/zwb-ict/orgs","repos_url":"https://api.github.com/users/zwb-ict/repos","events_url":"https://api.github.com/users/zwb-ict/events{/privacy}","received_events_url":"https://api.github.com/users/zwb-ict/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-01-14T10:35:12Z","updated_at":"2016-01-18T04:14:20Z","closed_at":null,"body":"I have 12 files to handle, so I want to create 12 ReadStreams and merge to one.\r\n\r\nIn each file, each line is a JSON string.\r\n\r\nMy code is like this: \r\n```\r\n  fileNames = fileNames.filter(Boolean);\r\n  es.merge(fileNames.map(function(item, index) {\r\n               return fs.createReadStream(item);\r\n             }))\r\n    .pipe(es.split())\r\n    .pipe(es.parse())\r\n```\r\n\r\nAfter run it, I got some JSON.parse error like this: \r\n> SyntaxError: Unexpected token \r\n\r\nand I found the error json looks like this:\r\n```\r\n{\"ts\":1452700832306,\"segments\":{\"sex\":1,\"age\":5,\"os\":1,\"deviceprice\":5},\"appkey\":\"560129b88bda20a5270f88da\",\"token\":\"jXL29pROAnEGng\",\"deviceid\":\"D4FB58CD-39{\"ts\":1452700810077,\"appkey\":\"560129b88bda20a5270f88da\",\"token\":\"jXL29pROAnEGng\",\"deviceid\":\"B8DA0F64C613E896B412788B9BDB06D4\"}\r\n```\r\n \r\nI grep **1452700832306** and **1452700810077**, and find they come from two different files:\r\n\r\nin file1, test1.log: \r\n```\r\n{\"ts\":1452700832306,\"segments\":{\"sex\":1,\"age\":5,\"os\":1,\"deviceprice\":5},\"appkey\":\"560129b88bda20a5270f88da\",\"token\":\"jXL29pROAnEGng\",\"deviceid\":\"D4FB58CD-39CF-4445-AD26-C85194C0D032\"}\r\n```\r\n\r\nin file2, test2.log:\r\n```\r\n{\"ts\":1452700810077,\"appkey\":\"560129b88bda20a5270f88da\",\"token\":\"jXL29pROAnEGng\",\"deviceid\":\"B8DA0F64C613E896B412788B9BDB06D4\"}\r\n```\r\n\r\nSo it looks like some of my data has been broken.\r\n\r\n**Node version**: v4.2.3\r\n**OS**: CentOS Linux release 7.0.1406 (Core)"},"comment":{"url":"https://api.github.com/repos/dominictarr/event-stream/issues/comments/172420443","html_url":"https://github.com/dominictarr/event-stream/issues/93#issuecomment-172420443","issue_url":"https://api.github.com/repos/dominictarr/event-stream/issues/93","id":172420443,"user":{"login":"dominictarr","id":259374,"avatar_url":"https://avatars.githubusercontent.com/u/259374?v=3","gravatar_id":"","url":"https://api.github.com/users/dominictarr","html_url":"https://github.com/dominictarr","followers_url":"https://api.github.com/users/dominictarr/followers","following_url":"https://api.github.com/users/dominictarr/following{/other_user}","gists_url":"https://api.github.com/users/dominictarr/gists{/gist_id}","starred_url":"https://api.github.com/users/dominictarr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dominictarr/subscriptions","organizations_url":"https://api.github.com/users/dominictarr/orgs","repos_url":"https://api.github.com/users/dominictarr/repos","events_url":"https://api.github.com/users/dominictarr/events{/privacy}","received_events_url":"https://api.github.com/users/dominictarr/received_events","type":"User","site_admin":false},"created_at":"2016-01-18T04:14:20Z","updated_at":"2016-01-18T04:14:20Z","body":"merge doesn't know where the lines in your flies are. when fs reads from the disk, it may break them up anywhere. you need the splits to be at the lines so that es.parse can handle them. \r\n\r\nthis should work:\r\n``` js\r\n  es.merge(fileNames.map(function(item, index) {\r\n               return fs.createReadStream(item)\r\n                  .pipe(es.split())\r\n                  .pipe(es.parse())\r\n             }))\r\n```\r\n"}}