rabbitmq + node.js = rabbit.js

For those who have been away from the internets, node.js is an evented JavaScript engine based on Google's V8. Because it is essentially one big, efficient event loop, it's a natural fit for programs that shuffle data backwards and forwards with little state in-between. And it's fun to program, an opinion apparently lots of people share, because there have been loads of libraries crop up around it.

Among the more impressive of these libraries is Socket.IO. One can combine Socket.IO with node.js's built-in web server to make a websocket server, with a socket abstraction for browsers that degrades to XHR tricks for when there's no websockets. (I would be happy to believe that node.js and Socket.IO were made for us by a benevolent and foresightful precursor race; but of course, they were made by hard-working clever people. Thank you, people!)

Once one has a socket abstraction in the browser, a whole world opens up. Specifically, for our purposes, a whole world of messaging. Since node.js has an AMQP client, we can easily hook it up with RabbitMQ; not only to bridge to other protocols and back-end systems, but also to provide messaging between browsers, and between application servers, and so on.

Following on from the work we've been doing with Martin Sustrik of ZeroMQ, I decided to make a very simple protocol for using on the browser sockets, reflecting the messaging patterns used in ZeroMQ (and thereby in RMQ-0MQ) -- pub/sub, request/reply, and push/pull (or pipeline). I wrote a node.js library that uses RabbitMQ to implement message patterns using its routing and buffering; the bridging then comes for free, since RabbitMQ has a bunch of protocol adapters and clients for various languages.

A brief explanation of the messaging patterns:

Publish/Subscribe is for the situation in which a published message should be delivered to multiple subscribers. In the general case, various kinds of routing can be used to filter the messages for each subscriber. This might be used to broadcast notifications from a backend system to users' browsers, for example.

Request/Reply is for RPC over messaging; requests are distributed round-robin among worker processes, and replies are routed back to the requesting socket. This might be used by browsers to query back-end services; or even for browsers to query each other.

Pipeline is for chaining together processes. Messages are pushed to worker processes in a round-robin, which themselves may push to another stage of processing. This might be used to co-ordinate a workflow among sets of users (or indeed individuals).

Having duly dispensed with ado, here is rabbit.js.

All it needs is a bare-bones RabbitMQ and node.js installed; and, the node-amqp and Socket.IO libraries. Instructions and the locations of these things are in the README. (Do note that you need my fork of node-amqp.)

It also includes a tiny message socket server; that is, a node.js server that accepts socket connections and speaks in length-prefixed messages. Since it's all going through RabbitMQ, you can talk to the browsers hooked up with Socket.IO via a socket. You can also use the in-process pipe server from code running in node.js itself.

All in all, I am surprised how much I could get done with only a handful of lines of code and some technologies that each hit a sweet spot -- node.js for fun network server programming, Socket.IO for magical browser sockets, and RabbitMQ for the no-tears messaging.

Tags: , , , ,

10 Responses to “rabbitmq + node.js = rabbit.js”

  1. Twitter Trackbacks for RabbitMQ » Blog Archive » rabbitmq + node.js = rabbit.js - Messaging that just works [] on Says:

    [...] RabbitMQ » Blog Archive » rabbitmq + node.js = rabbit.js - Messaging that just works – view page – cached RabbitMQ is a complete and highly reliable enterprise messaging system based on the emerging AMQP standard Tweets about this link [...]

  2. Rabbitmq + node.js = rabbit.js – Messaging that just works | Techarama Says:

    [...] View full post on Hacker News [...]

  3. Josh Mather » Bookmarks for October 29th through November 15th Says:

    [...] RabbitMQ » Blog Archive » rabbitmq + node.js = rabbit.js - Messaging that just works - [...]

  4. PHP Inter-process communication to monitor message queue - Programmers Goodies Says:

    [...] has a list of tutorials, libraries, and docs for PHP. Rabbit.js seems to be a good bet for Node.js. [...]

  5. Chris Adkin Says:

    I've attempted to run the example ( example/socketio.js ) provided with rabbit.js and I get:-

    throw e; // process.nextTick error, or 'error' event on first tick
    Error: Cannot find module 'sockets'
    at Function.resolveFilename (module.js:334:11)
    at Function.
    load (module.js:279:25)
    at Module.require (module.js:357:17)
    at require (module.js:368:17)
    at Object. (/home/chris/rabbit.js/example/socketio.js:8:13)
    at Module.compile (module.js:432:26)
    at Object..js (module.js:450:10)
    at Module.load (module.js:351:31)
    at Function.
    load (module.js:310:12)
    at Array. (module.js:470:10)

    It would appear that node.js is expecting sockets to be a module, however I cannot find any reference to a sockets node js module on github. Can someone please point me in the right direction so that I can get this working.

  6. scworldnetter Says:

    This looks really interesting, but when I went to try to work with node.js and the AMQP client, the client library have very cursory documentation at best, so really only the authors of the AMQP client can use the library effectively, for anyone else, it's pretty much hopeless. None of the code actually works out of the box, and requires major tweaking. Does anyone have a reference to a good blog or API documentation for this? Something similar to Javadoc? The one or two blogs I did find had code that was so buggy it was also pretty much useless.

    I would really appreciate a good reference for this.

  7. Michael Says:

    scworldnetter: Are you complaining about the AMQP library or about rabbit.js, or both? It is difficult to tell.

    For the AMQP client, what is wrong with the README?
    (By the way, rabbit.js will work with npm-installed node-amqp now).

    For rabbit.js, the examples run with node v0.4.x, but may not work with 0.5. Sorry about that.

  8. Jean-Baptiste DUPONT Says:

    I managed to get this example to work 01/dec/2012 - not straight forward though, because I bumbed into several smalls issues. Let's share my findings if this can help someone out there:
    The instruction below
    - rabbit.js$ npm install
    does not work move to the directory using cd like this
    cd rabbit.js/0.2.0/package/ and then issue the commands - this fix one of the issues.

    npm install amqp -g need be performed otherwise things dont go very far too you need the -g option

    Of course, you will also need to do a few more things to match your own configuration:
    populate in the server.js file your rabbitMQ IP and port (if not running on your local host) by editing the file with your favourite editor (I use nano).

    One thing I needed to figure out is that sub.connect('x'); means that the queue in rabbitMQ will have the name x

    Finally once you get the server up and running, from your browser, you key in something like localshost:8080 and then the page acting as the client will appear and you can run the example as stated.

    I'm not too sure what is really happening... I'm just giving it a go:

    • The client (the webpage) uses a SockJS and sends a message to the server (node server.js)
    • The server gets the message and put it in uppercase
    • Then the server sends the modified message to rabbitMQ to the queue named "uppercase"
    • The message is sent by rabbitMQ to a listener (in this example server.js listening for the queue "uppercase"
    • Then the server sends this back to the client (the page)
    • The page index.html publishes the message in the right hand side box

    Please correct me if I made it wrong.

    Overall great stuff achieved with not too much code. It gives a good example as to how you can demonstrate some of the functionalities rabbitMQ provdes. This is just what I needed on top of the examples provided in the tutorial in python and java.
    Many thanks for this interesting demo in javascript using node.js

    Hope my post will be useful to someone out there.
    Many thanks,

  9. Jean-Baptiste DUPONT Says:

    Just one more thing...
    If you send an empty chain or space the server doesn't like it much and crashes.
    A test is missing somewhere if the message is empty. Easy to fix though.
    Thanks a lot for this great example.

  10. Jean-Baptiste DUPONT Says:

    Here are some other fixes to get the examples to work:
    require('rabbit.js') need be replaced by require('../../index.js') when running examples you make from what is given - in fact I could not find a rabbit.js javascript file - rabbit.js is a folder instead.

    this bit is necessary as well

    espcially if you use a rabbitMQ server running on another machine just like me -

    var context = require('../../index').createContext('amqp://');

    I have fun playing with all this technology!

    Once again thanks,