Technology Lab / Information Technology



Telling Nginx what to do

Node.js is many things; it has the capability to function as a Web server by itself, with a bit of configuration. However, we haven't put this much time and effort into building a robust Nginx set-up to not use it. Node.js isn't running at the moment, but when it does, it listens for incoming connections on TCP port 9001. We're going to configure Nginx to watch for HTTP requests to a path that doesn't really exist and to reverse-proxy those requests to Node.js on port 9001.

Reverse proxying—that is, retrieving something from one or more servers on behalf of a requesting client—is something Nginx is very, very good at. In fact, that's one of the predominant uses for Nginx—to sit in front of a pool of bigger, fatter application servers and act as a traffic cop between them and their users, quickly serving up static assets like images and text while the application servers generate the dynamic portions of a Web site.

We're not going to do anything so complex here, but we are going to take advantage of Nginx's presence on port 80. We can't run both Nginx and Node.js on port 80 at the same time, and so by proxying our Etherpad traffic through Nginx, we don't have to worry about telling users, "So, to use my cool Etherpad set-up, you'll need to add ':9001' at the end of the URL..." Nobody likes that.

Fortunately, we don't need to do much to Nginx to tell it what to do. We simply need to make up a location for Etherpad, and tell Nginx to take traffic bound for that location and pass it on to the waiting Node.js process, listening on port 9001.

Add this to your active Nginx virtual host configuration file, which if you've been playing along at home is /etc/nginx/sites-available/www:

location /etherpad {
     rewrite /etherpad/(.*) /$1 break;
     rewrite ^/etherpad$ /etherpad/ permanent;
     proxy_pass http://localhost:9001/;
     proxy_redirect / /etherpad/;
     proxy_set_header Host $host;
     proxy_buffering off;

When I originally wrote this procedure up for my blog, I did it such that it used its own hostname instead of a subdirectory, and also such that all traffic was forced to run over HTTPS. If you'd prefer to set your Etherpad instance up this way (or as a blend—HTTPS only from a subdirectory, for example), you can alter the configuration to look more like the one I'm using on my blog.

However, since not everyone is going to want to use HTTPS all the time, I've borrowed the Nginx configuration above from the Etherpad Github pages. It uses rewrites to fiddle with the actual text of the URL that Nginx passes to Node.js. This is important because our Etherpad traffic is all going to be sent to and from a URL with a directory name at the end of it: The Node.js process listening behind Nginx doesn't know anything about any /etherpad subdirectory—it's not a real subdirectory anyway, and certainly not one underneath the very real Etherpad directory we've created.

Using the rewrite directive lets Nginx take care of the confusion for us. Nginx will look for HTTP requests that match the regular expressions we entered above, and then alter the URLs on the fly before it proxies them to Node.js, and it will also translate back the replies it receives from Node.js. Neat!

Once that's in, reload Nginx to make the change active:

/etc/init.d/nginx reload

Adding in Redis support

Time to turn our attention to the database-side of things. Rather than gunking up the Etherpad code with all the different connectors necessary to work with lots of different database packages, Etherpad actually uses a database abstraction layer called UeberDB. This way, it doesn't particularly care what database you bring to the table: as long as UeberDB knows how to talk to it, we can use it.

It just so happens that there is an UeberDB connector for Redis, and we're going to install it inside the Etherpad directory that we cloned down way back near the beginning of this tutorial. Fortunately, installing it is super-easy; we can use the Node Package Manager to automatically pull it down and slot it into the correct place.

Back at the bash shell, change your directory so that you're at usr/share/nginx/etherpad-lite/src/node_modules/ueberDB, which is the location of the UeberDB package underneath the Etherpad directory, and install the Redis package:

cd usr/share/nginx/etherpad-lite/src/node_modules/ueberDB
npm install redis
Enlarge / Installing the Redis connector via Node Package Manager.
Lee Hutchinson

Configuring Etherpad

Getting closer! Now that we've given Etherpad the ability to store its stuff in our Redis database, we're going to actually instruct Etherpad to do so.

To do this, we need to edit Etherpad main configuration file, which is at /usr/share/nginx/etherpad-lite/settings.json. Looking inside the file, there are lots of settings that you probably want to change (like "title", to give your Etherpad instance a more descriptive name), but for now we're going to concern ourselves purely with the database ones.

Specifically, we're going to modify "dbType" and "dbSettings" from their default values to instead look like this:

"dbType" : "redis",
"dbSettings" : {
      "host" : "localhost",
      "port" : 6379,
  "database" : 0

This tells Etherpad that it should use its Redis database driver and that the Redis server is localhost and is listening on TCP port 6379. I spent some amount of time trying to get Etherpad to communicate via Unix sockets to Redis instead of via TCP—Unix sockets are faster and more efficient—but I couldn't make it work. I don't think the UeberDB database abstraction layer used by Etherpad included support for it. However, if someone does figure it out, please do drop me a line!

The last setting, "database" : 0, tells Etherpad to use Redis' db0. Redis can actually run multiple simultaneous independent key-value stores; the first one is db0 and is the one you'd most likely want to use first. If you end up setting up another Web application that uses Redis (like a second instance of Etherpad), you'd want to make sure that it's using a database other than 0 so that the applications don't eat each others' data!

Is it working? Let's take a peek!

We've got a couple of steps remaining, but it's time to check our progress and make sure Etherpad is actually, you know, working.

Kick off that Upstart job you created to get Node.js activated and Etherpad up and running:

start etherpad-lite

Then, open up your Web browser and navigate to If you've done everything right, you should see the Etherpad front page:

Enlarge / IT WORKS
Lee Hutchinson


But something’s missing...

If you'd like, you can stop now, because Etherpad is configured and everything's working. However, there's a little more we can do to add in some functionality that Etherpad is missing by default.

Right now, you can create a new "pad" to collaborate with simply by typing in a name. Try it out!

Enlarge / Creating a pad called "testpad" should yield a screen that looks like this. If this is what you're seeing, then Etherpad is functioning correctly!
Lee Hutchinson

But there's no easy way to see a quick list of all the existing pads on the server. This might be how you want it—for example, if your Etherpad instance is meant to be public, you might not want everyone who happens by to see that you're working on a pad titled "MY SECRET FINANCIAL INFO AND PASSWORDS" or something like that. However, I've found it beneficial to see exactly what pads are available on the server to edit.

To do this requires a bit of extra work, but the results are worth it.

Showing a list of pads

This is perhaps a bit hacky, but it works well enough. We're going to do a few different things to get a list of pads displayed on our Etherpad logon page. First, we need to figure out how to query our Redis database to actually produce the list. Then, we need to store that list in a text file. We need to be able to include that text file in our logon page. And, finally, we need to automate the Redis query so it's running over and over again, to keep our list of pads current.

Making the list

The Etherpad Github repository has instructions on how to use the command line to ask various databases to produce a list of pads; using the syntax provided, we can quickly extrapolate how to do the same with the redis-cli command line tool. Create a couple of pads in Etherpad so we have some database content to work with (simply type a name for your pad into the "New Pad" page and click "OK," then enter some content into the editing window). Then, from your root shell, run the following:


(If you're cutting and pasting the above code, you may need to manually remove the linebreak in it.)

This set of commands is an excellent example of the power of the Unix command line and serves as a beautiful demonstration of one of the main focuses of Linux and all Unix-like operating systems: they are ultimately made up of a collection of very powerful tools that can manipulate each others' textual output.

The commands are each separated by pipes; I suggest you try building the command line in sequence, pipe by pipe, to get a feel for what is happening. The pipes themselves tell bash to take the output of one command and use it for input into the next.

The first command, redis-cli keys 'pad:*', outputs a numbered list of all the Redis keys that contain the string "pad:". Grep is then used to strip out some of the cruft with a regular expression, leaving behind only the string "pad:" and whatever follows it to the first colon. Then, sed replaces "pad:" with an empty string, deleting it and leaving us with a bare list of pads. Next we run the list through sort, which orders the results, and uniq is called with the -c flag, which counts each unique entry and displays the total. These results are then run through sort again, this time with -rn, which sorts them in descending order by number, using the counts at the front. Lastly, the entire mess is piped through awk, which looks at the two columns we're left with, and if the first column is not "2" (which indicates that the pad exists but is empty), it displays the second column's text.

The end result is a list of non-empty pads, though you can modify the final awk statement to awk '{if ($1>"1") {print $2 }}' if you want a list of all pads, empty or not.

So, we have our list of pads:

Enlarge / Building our way through that last command's layers.
Lee Hutchinson

To get that list into a text file, you can simply append > output.txt onto the end of the command. Let's go ahead and do that, and output the file to a location that Nginx can serve it from. Use this path and file name:


So the full command should now look like this:

> /usr/share/nginx/html/padlist.txt

(Again, remove the linebreaks if you're copying and pasting.)

Expand full story

You must to comment.