Take the 2-minute tour ×

Node.js is a perfect match for our web project, but there are few computational tasks for which we would prefer Python. We also already have a Python code for them. We are highly concerned about speed, what is the most elegant way how to call a Python "worker" from node.js in an asynchronous non-blocking way?

share|improve this question
Hi, could you share with us what did you choose and how it worked out for you? There are libraries in Python which we all love to use while keeping the performance and non-blocking options. Thanks –  Maziyar Oct 1 '13 at 3:57

6 Answers 6

up vote 41 down vote accepted

For communication between node.js and Python server, I would use Unix sockets if both processes run on the same server and TCP/IP sockets otherwise. For marshaling protocol I would take JSON or protocol buffer. If threaded Python shows up to be a bottleneck, consider using Twisted Python, which provides the same event driven concurrency as do node.js.

If you feel adventurous, learn clojure (clojurescript, clojure-py) and you'll get the same language that runs and interoperates with existing code on Java, JavaScript (node.js included), CLR and Python. And you get superb marshalling protocol by simply using clojure data structures.

share|improve this answer
Thanks, it helped a lot :) –  Sushant Gupta Dec 25 '12 at 6:47
Do you know if something like this will work on Heroku, which has an ephemeral filesystem? –  cm2 Jul 6 '13 at 1:46
Hi Aleš Kotnik, I need to call soap(asmx, wcf) webservices from node.js via phyton. Could you provide some example code?? –  Udhaya Oct 18 '13 at 6:44

This sounds like a scenario where zeroMQ would be a good fit. It's a messaging framework that's similar to using TCP or Unix sockets, but it's much more robust (http://zguide.zeromq.org/py:all)

There's a library that uses zeroMQ to provide a RPC framework that works pretty well. It's called zeroRPC (http://zerorpc.dotcloud.com/). Here's the hello world.

Python "Hello x" server:

import zerorpc

class HelloRPC(object):
    '''pass the method a name, it replies "Hello name!"'''
    def hello(self, name):
        return "Hello, {0}!".format(name)

def main():
    s = zerorpc.Server(HelloRPC())

if __name__ == "__main__" : main()

And the node.js client:

var zerorpc = require("zerorpc");

var client = new zerorpc.Client();
//calls the method on the python object
client.invoke("hello", "World", function(error, reply, streaming) {
        console.log("ERROR: ", error);

Or vice-versa, node.js server:

var zerorpc = require("zerorpc");

var server = new zerorpc.Server({
    hello: function(name, reply) {
        reply(null, "Hello, " + name, false);


And the python client

import zerorpc, sys

c = zerorpc.Client()
name = sys.argv[1] if len(sys.argv) > 1 else "dude"
print c.hello(name)
share|improve this answer
Can zerorpc handle multiple states inthe event there are multiple client sessions? –  user1027169 Oct 31 '13 at 7:46

If you arrange to have your Python worker in a separate process (either long-running server-type process or a spawned child on demand), your communication with it will be asynchronous on the node.js side. UNIX/TCP sockets and stdin/out/err communication are inherently async in node.

share|improve this answer
Yes, of course, you're right, thank you. –  Cartesius00 May 27 '12 at 16:40

I'd consider also Apache Thrift http://thrift.apache.org/

It can bridge between several programming languages, is highly efficient and has support for async or sync calls. See full features here http://thrift.apache.org/docs/features/

The multi language can be useful for future plans, for example if you later want to do part of the computational task in C++ it's very easy to do add it to the mix using Thrift.

share|improve this answer

I'd recommend using some work queue using, for example, the excellent Gearman, which will provide you with a great way to dispatch background jobs, and asynchronously get their result once they're processed.

The advantage of this, used heavily at Digg (among many others) is that it provides a strong, scalable and robust way to make workers in any language to speak with clients in any language.

share|improve this answer

I've had a lot of success using thoonk.js along with thoonk.py. Thoonk leverages Redis (in-memory key-value store) to give you feed (think publish/subscribe), queue and job patterns for communication.

Why is this better than unix sockets or direct tcp sockets? Overall performance may be decreased a little, however Thoonk provides a really simple API that simplifies having to manually deal with a socket. Thoonk also helps make it really trivial to implement a distributed computing model that allows you to scale your python workers to increase performance, since you just spin up new instances of your python workers and connect them to the same redis server.

share|improve this answer

Your Answer


