Before and during the process of developing the concepts and architecture behind Orbited, I spent some time chatting with the folks in #cometd on irc.perl.org. I was a bit confused by the project for a while — I knew what problem domain they were attempting to solve, but I wasn’t sure how exactly they solved it. I was specifically confused on a number of points:

1. Why does Cometd have a different version of the server for every language?
2. How is the web-app supposed to send events to the browser?
3. How do I communicate with the Cometd server from Javascript?
4. How do I to run multiple Cometd nodes in a sensible way?

I’m going to give my best bet at answering each of these questions.

1. Why does Cometd have a different version of the server for every language?

   Ultimately there will be one or two Cometd implementations to rule them all. And probably they’ll be written in Java. In the meantime, smart hackers are using whatever language they know best to experiment. There are multiple implementations because no one really knows what will work in the end.

2. How is the web-app supposed to send events to the browser?

   The web application is supposed to pretend to be a browser that speaks Bayeux. So if the application wants to receive events, it will be using an iframe stream or long polling. If it wants to subscribe, unsubscribe, or publish, it will make json encoded Bayeux requests to the cometd server.

3. How do I communicate with the Cometd server from JavaScript?

   The obvious answer is to use Bayeux. But it’s really hard to implement and you aren’t expected to do so. The guys at Dojo Toolkit have already implemented Bayeux, and presumably there are more to follow. So the short answer is, use Dojo Toolkit.

4. How do I to run multiple Cometd nodes in a sensible way?

   I sort of have an answer, but I don’t fully understand it. I talked to some guys in #cometd on this issue, and the first response was “Good question… I don’t really know.” I did some research on the web and noticed this thread on a Cometd-user google group. The long and the short of it is “No one knows.” But on the other hand, Greg Wilkins told me that this is an easy problem to solve. You simply stick a Java Message Queue (JMS) behind your Cometd nodes and you’re done. I’m maybe a quarter willing to buy this, but in the face of this post by Greg, I have some doubts. I understand that Terracotta is completely unrelated to JMS, but it indicates that attempts at scaling aren’t going well. So the short answer is, design your app so that clusters of channels and users are isolated from each other such that each Cometd server supports a specific set of users/groups, and only those users and groups.

I hope my answers help someone who is interested in Cometd. Now I’m going to justify this post’s title. I have a number of problems with Cometd.

1. No easy way for my web app to send an event to a specific user.
2. You are locked into Dojo
3. There is no possible method to pre-process events, short of rewriting the Cometd server
4. There currently exists no security.
5. Lacks App-based configurability (or any sort of configuration)
6. Cometd is Inherently difficult to scale.

I’ll even substantiate my claims.

1. No easy way for my web app to send an event to a specific user.

   Any server whose purpose is comet-like communication has two points of interaction for the end developer: The web application and the browser. These are the most important parts because they dictate how a server can or cannot be used. Furthermore, the end-developer only ever sees these two parts of the server. If they are complicated, it doesn’t matter how beautiful the internal implementation is. APIs that are complicated or hard to implement are generally not well received and take longer to be adopted.

   So let’s say that I want to use Cometd with a currently unsupported language for my web application… Lisp. I have to implement a Bayeux client in Lisp. Now, I’ve spent half a year thinking about the problem space of Comet, read the Bayeux spec through multiple times, conversed with the authors of Bayeux, and even closely analyzed the source code of the twisted Bayeux implementation, as well as the Dojo client implementation. I am still lost. I will not be implementing Bayeux in Lisp, or any other language for that matter. Sure, eventually someone will after Cometd is popular, but what do I do in the meantime? And what if I don’t like the implementation that exists for a given language? Nothing, I’m stuck. It’s not as god awful as being stuck in some proprietary system, but it is still terrible in its own right.

2. Locked into Dojo

   The same logic applies for choice of Javascript toolkit. If you want to use Cometd, then you have to use Dojo Toolkit. Personally, I have little problem with that. But someone has a problem with it, otherwise everyone would already be using Dojo. And because it’s the Dojo guys who are pushing Bayeux, we aren’t likely to see them write an implementation for a competing framework.

3. There is no possible method to pre-process events, short of rewriting the Cometd server

   Lets take a moment to examine the flow of events through a Cometd server. Here is my understanding of the Cometd stack:

   

   Lets say that user A sends a chat msg to channel 1, which contains user A, B, C, and the webapp. The event is sent from User A’s browser to Cometd. Cometd takes that message and delivers it to user B, C and the webapp. But lets also say that user C is in Chile where they don’t speak English. We want our application to translate the message on the fly from English to Spanish. Unfortunately, the Web App isn’t the hub, its just a client like all the browsers. It can receive notification of the message, but it can’t alter the message before it gets to the other users.

4. There currently exists no security.

   There is no bar on who can join what channel, or even on who can publish to a given channel. In fact, check out the chat Cometd demo. Pay particular attention to line 51 and 52. This is the leave function, for leaving a chat room. Line 51 shows the browser unsubscribing from the chat channel, and line 52 shows the browser publishing a leave message to the channel. You don’t even have to be in a channel to publish to it. Is this by design, and if so what’s the advantage? I see this is a gaping security hole.

5. Lacks App-based configurability (or any sort of configuration)

   Okay, let’s entertain the thought for a moment that Cometd is really intended to allow users to publish to channels they aren’t subscribed to. But now we want to make certain rooms private. How do we do this? Well, right now you can’t. But assume that it became a configuration option. How would the web app tell Cometd which rooms to make private or public? There are a zillion other settings that need to be created and modified on the fly by the web app, but that doesn’t exist in Cometd. There are no hooks for the app. Your only bet is to reprogram the Cometd daemon itself, which is not very realistic. How do you kick a user out of a channel? How do you moderate a channel? How do you set a channel size limit, and change it as the app runs? I could keep asking questions, but the answer to all of them is: You cannot.

6. Cometd is Inherently difficult to scale.

   All of the previous problems are small beans though compared to my real problem: Cometd is inherently hard to scale. I mean, more so than Comet-style communication is inherently hard to scale. The reason is the publish subscribe mechanism that is the heart of Cometd. Every node in a Cometd cluster needs to contain information about every group. It needs to additionally know about every user and their membership in every group, or alternatively, relay every event to every server in the cluster. You’ll notice i used the word ‘every’ an awful lot just now, and that’s a problem. Every * every = n^2 of something, bandwidth or cpu, it doesn’t really matter. It’s difficult to scale an O(n^2) system. The cost of the second user is going to be a thousand times less than the cost of the thousandth user.

I honestly don’t understand why Cometd feels the need to tackle that problem. I’m not saying publish/subscribe isn’t ever necessary in a web app, rather that it’s a layer that doesn’t belong in a Comet server. Cometd shouldn’t be in the pubsub business… that’s a tough business. As if Comet wasn’t tough enough already.

In the end I think that Cometd is making amazing strides in advancing state of the art technology on the web, but it doesn’t mean we should be blind to alternatives.

This entry was posted on Thursday, April 19th, 2007 at 10:48 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.