Take the 2-minute tour ×

I have large text files, which range between 30MB and 10GB. How can I count the number of lines in a file using Node.js?

I have these limitations:

  • The entire file does not need to be written to memory
  • A child process is not required to perform the task
share|improve this question
wc -l file ... –  zerkms Sep 17 '12 at 4:18
"using NodeJS" -- any real technical reason behind this requirement? –  zerkms Sep 17 '12 at 4:20
It's part of a script. –  hexacyanide Sep 17 '12 at 4:20
I'm sure that wc will be faster that any "native" nodejs solution –  zerkms Sep 17 '12 at 4:21
You could just count the lines-- stackoverflow.com/questions/6156501/… –  JoshRagem Sep 17 '12 at 4:41
show 2 more comments

2 Answers

up vote 7 down vote accepted

solution without using wc:

var i;
var count = 0;
  .on('data', function(chunk) {
    for (i=0; i < chunk.length; ++i)
      if (chunk[i] == 10) count++;
  .on('end', function() {

it's slower, but not that much you might expect - 0.6s for 140M+ file including node.js loading & startup time

>time node countlines.js video.mp4 

real    0m0.614s
user    0m0.489s
sys 0m0.132s

>time wc -l video.mp4 
619643 video.mp4
real    0m0.133s
user    0m0.108s
sys 0m0.024s

>wc -c video.mp4
144681406  video.mp4
share|improve this answer
Your benchmark isn't very convincing since you're running it on a file that is not structured into lines and as such is not representative of the sort of file the OP wants to process. The line if (chunk[i] == 10) count++; will be executed far more often during the analysis of a text file than during the analysis of a binary video file. –  ebohlman Sep 18 '12 at 7:14
I don't have 100mb text file :) And I don't expect any difference even in the case of similar 100mb text file but with 10x number of newlines - it's same linear search iterating every byte in each of Buffer chunks –  Andrey Sidorov Sep 18 '12 at 7:43
I replicated input script itself and concatenated it to a single text file, 1468750000 chars, 62500000 lines. WC time: 0m1.375s, node.js time: 0m6.254s. Same 4.5 times difference (wich could be better, but still good enough for JS vs C program) –  Andrey Sidorov Sep 18 '12 at 7:58
add comment

You could do this as the comments suggest using wc

var exec = require('child_process').exec;

exec('wc /path/to/file', function (error, results) {
share|improve this answer
add comment

Your Answer


