Faruk Akgul

Cool Tools: ZeroMQ, Gevent, Websockets, node.js, web.py and Java

August 12, 2011 | View Comments

In this post, I'll share my experiences with the cool technologies of the Internet by creating a simple real time application.

  • ZeroMQ: Message/Queue socket library (I'm a contributor) with async I/O used by Mongrel2. Unlike RabbitMQ, it doesn't need a broker.
  • Gevent: Co-routine based Python network library based on libevent which provides high level sync API.
  • WebSockets: One of the cool features of HTML5 which provides communication with browsers via a TCP.
  • node.js: Evented I/O based on V8.
  • web.py: A Python web framework which we'll use in this post via gevent's high performance WSGI server based on libevent-HTTP.
  • Java: An island of Indonesia.

So, we have all the cool tools together and we can begin our little project. The scenario is;

  • We have a Producer that does some calculations.
  • Producer sends the results of the calculations to Server.
  • Server sends those information to WebSocket server.
  • We serve those information via a simple web.py application powered by gevent.

For the sake of simplicity, we don't do any fancy calculations instead we'll just send some random values to connected users. Our ZeroMQ Producer and Server will be in Java whereas the rest of the will be in Python (and of course a little bit Javascript).

Producer

Here's our simple ZeroMQ'd producer.

   import org.zeromq.ZMQ;
    import java.util.ArrayList;
    import java.util.Random;

    public class Producer {
        private static ArrayList<String> list = new ArrayList<String>();
        static {
            list.add("Milla Jovovich");
            list.add("David Duchovyn");
            list.add("Naomi Watts");
            list.add("Keanu Reeves");
        }

        static String choice() {
            int random = new Random().nextInt(list.size());
            return list.get(random);
        }

        public static void main(String[] args) throws InterruptedException {

            ZMQ.Context context = ZMQ.context(1);
            ZMQ.Socket pub = context.socket(ZMQ.REQ);
            pub.connect("tcp://localhost:6060");
            while (true) {
                String msg = choice();
                pub.send(msg.getBytes(), 0);
                String reply = new String(pub.recv(0));
                System.out.println("reply: " + reply);
                Thread.sleep(1000);
            }
        }
    }

What we do here is, we send random data from the list and send it server.

Server

The ZeroMQ'd server takes the data from Producer and sends it WebSocket. We also send back the data to Producer (no need to. I can't remember why I did it in the first place but let's hope I had a good reason).

   import org.zeromq.ZMQ;

    public class Server {

        public static void main(String[] args) throws InterruptedException {
            ZMQ.Context context = ZMQ.context(1);
            ZMQ.Socket sub = context.socket(ZMQ.REP);
            ZMQ.Socket pub = context.socket(ZMQ.PUB);

            sub.bind("tcp://*:6060");
            pub.bind("tcp://*:3030");
            while (true) {
                String msg = new String(sub.recv(0));
                System.out.println("received: " + msg);
                sub.send(msg.getBytes(), 0);
                pub.send(msg.getBytes(), 0);
                Thread.sleep(1000);
            }

        }
    }

Gevent WebSocket Server

Here, gevent waves its magic wand. We connect to ZeroMQ server and once we get some data from the ZeroMQ server, we send it to our web application via wsgi.websocket. This is a gevent WSGI server.

   import zmq
    import gevent
    from geventwebsocket.handler import WebSocketHandler
    from gevent.pywsgi import WSGIServer

    def app(environ, start_response):
        context = zmq.Context()
        ws = environ['wsgi.websocket']
        socket = context.socket(zmq.SUB)
        socket.setsockopt(zmq.SUBSCRIBE, '')
        socket.connect('tcp://localhost:3030')
        while True:
            msg = socket.recv()
            ws.send(msg)
            gevent.sleep(0.1)

    if __name__ == '__main__':
        print 'here we go...'
        server = WSGIServer(('', 4040), app,
        handler_class=WebSocketHandler)
        server.serve_forever()

And now, we can move on to our web application part, a simple web.py application.

web.py via gevent WSGI

Here's our simple gevent powered web.py application. Look out to monkey.patch_all() which turns thread-local storage of web.ctx into a greenlet-local storage.

   import web
    from gevent import monkey
    from gevent.pywsgi import WSGIServer

    monkey.patch_all()

    class Index(object):
        def GET(self):
            return """
                <div id="place"></div>
                <script>
                    var ws = new WebSocket('ws://localhost:4040');
                    var place = document.getElementById('place');
                    ws.onopen = function(e) {
                        document.write('connected');
                    };
                    ws.onmessage = function(e) {
                        place.innerHTML = e.data;
                    }
                </script>
            """

    urls = (r'^/', 'Index')
    app = web.application(urls, globals()).wsgifunc()

    if __name__ == '__main__':
        WSGIServer(('', 8080), app).serve_forever()

Here's a small problem; WebSocket isn't supported by all browsers. For example; Firefox disables it by default whereas Google Chrome supports it. In this case using socket.io, Faye (based on Bayeux protocol) for cross-browser support might be a better choice (or we could write a WebSocket server in node.js).

I must say that I'm quite getting started to warm up with these cool tools. I frequently make the same mistake which is I connect to wrong port and look at the blank screen while wondering "Why am I seeing a blank screen?" so my gentle advice would be double check the port you're connecting :)

Share:Tweet · reddit

blog comments powered by Disqus