title: Nodejs Parallel programming using threads
body: <p>I am using nodejs for a CPU intensive task ,which basicly generates large amount of data and stores it in a file. I am streaming the data to output files as it is generated for a single type of data. </p>
  
  <p><strong>Aim</strong> : I want to make the task of generating this data for multiple types of data in parallel (utilizing my multi-core cpu to its best).Without each of process having its own heap memory .Thus providing with larger process memory and increased speed of execution.</p>
  
  <p>I was planning to use node <a href="https://www.npmjs.com/package/fibers" rel="nofollow">fibers</a> which is also  used by <a href="https://meteorhacks.com/fibers-eventloop-and-meteor/" rel="nofollow">meteor js</a> for its own callback handling.But I am not sure if this will achieve what I want,as in one of the <a href="https://www.youtube.com/watch?v=AWJ8LIzQMHY" rel="nofollow">video on meteor fibers by <em>Chris Mather</em></a> mentions at the end that eventually everything is single threaded and node fibers somehow manges the same single threaded event loop  to provide its functionality.</p>
  
  <p>So,</p>
  
  <ul>
  <li><p>Does this mean that if I use node fibers I wont be running my task in
  parallel ,thus not utilizing my cpu cores ?</p></li>
  <li><p>Does node <a href="https://www.npmjs.com/package/webworker-threads" rel="nofollow">webworker-threads</a> will help me in achieving the
  functionality I desire.As is mentioned on modules home page which
  says that ,webworker threads will run on seperate/parallel cpu
  process ,thus providing multi-threading in real sense ??</p></li>
  <li><p>As ending question ,Does this mean that node.js is not advisable for
  such CPU intensive tasks ?</p></li>
  </ul>
  
  <p><em>note : I dont want to use asynchronous code structuring libs which are presented as threads,but infact just add syntatical sugar over same async code, as the tasks are largely CPU intensive .I have already used async capabilities to max .</em></p>
  
  <p><strong>// Update 1 (based on answer for clusters )</strong></p>
  
  <p>Sorry I forgot to mention this ,but problem with clusters I faced is : </p>
  
  <ul>
  <li><p>Complex to load balance the amount of work I have in a way which makes sure  a particular set of parallel tasks execute before certain other tasks.</p></li>
  <li><p>Not sure if clusters really do what I want ,referring to <a href="https://www.npmjs.com/package/webworker-threads#why-not-multiple-processes" rel="nofollow">these lines on webworker-threads</a> npm homepage </p></li>
  </ul>
  
  <blockquote>
    <p>The "can't block the event loop" problem is inherent to Node's evented model. No matter how many Node processes you have running as a Node-cluster, it won't solve its issues with CPU-bound tasks.</p>
  </blockquote>
  
  <p>..... any light on how ..would be helpfull.</p>