Updated 2015-02-11 19:18:51 by ak

STOMP stands for the Simple (or Streaming) Text Oriented Messaging Protocol. STOMP provides an interoperable wire format so that STOMP clients can communicate with any STOMP message broker to provide easy and widespread messaging interoperability among many languages, platforms and brokers.

STOMP is loosely modeled after the HTTP protocol, where "messages" will typically start with an uppercase command (like GET in HTTP), followed by a number of headers (format is similar) and possibly followed by a body. Messages are ended by ^@, i.e. the zero character, aka ASCII 0, '\0'. STOMP advocates utf-8 on the wire and provides mechanisms that keep the connection alive, if necessary, thus making sure routers do not close connections. The complete specification is rather short and specifically targets scripting languages.

As I (EF) needed to understand STOMP in a better way, and despite there was already an implementation available at [1], I decided to start coding it from scratch again. An earlier implementation of the resulting library is in the half-bakery [2]. The library offers both a client and server implementation. The client implementation is almost complete, it is only missing support for transactions. The whole implementation has now moved to github. Please report issues and proposal for improvements through the facilities offered by github.

My client implementation differentiates itself from the other in the following ways:

  • It is coded in pure Tcl, without any object layer. It is coded to run on oldish implementations of the language, i.e. 8.4 and in JTcl.
  • It implements the heart-beating facilities as of the specification.
  • It handles automated reconnections, but also gentle disconnection, as of the specification.
  • It provides introspection facilities to be notified of the changes in connection state.
  • It supports receipts for messages.
  • Apart from a high-level interface to subscriptions, it also provides a lower-level interface to PUB/SUB mechanisms. This allows to provide full support for the different types of acknowledgements specified.
  • It provides an interface to low-level messaging on the wire.
  • Automatically sets the content-length of messages being sent.
  • Provides an API for accessing the content of messages, loosely modeled after the http library.
  • Provides some guards against packages that would be too long.
  • Can use an external command for socket creation, thus enabling TLS encryption of traffic if necessary.

The server supports the following features:

  • Also coded in pure Tcl, will even run under JTcl.
  • Implements heart-beating, similarly to the client.
  • Fans out incoming messages to all clients that have expressed interest via a subscription.
  • Basic security mechanisms to discard messages from non-connected clients or to discard messages that would be too long.
  • Supports virtual hosting.
  • Is able to check authorization against a "database", i.e. a list of username:password entries passed as an argument.

The current implementation has had little testing, initial tests against the Apache Apollo broker have shown proper behaviour. Please send any positive or negative feedback to me on the usual channels.

As part of the new project on github, three small utilities exist to exhibit the capabilities of the library:

  • A STOMP server implements a server, supporting user authentication and TLS encryption if necessary.
  • Stomper will send any non-empty line received from the standard input as a STOMP message to a topic which name is specified as part of the command-line options.
  • Printer does pretty much the opposite, i.e. it listens to a topic at a STOMP server and will print out the content of the messages that are sent to that topic.

The following script highlights some of the capabilities of the client library, there is example code to instantiate the server at the bottom of this page.
package require stomp::client

proc received { cid rid } {
    puts "Server has received message, id: $rid"
}
 
proc incoming { msg } {
    puts "Incoming message: [::stomp::message::getCommand $msg]"
}

# Connect to broker (following is default admin user for Apollo)
set s [::stomp::client::connect -user admin -password password]

# Subscribe to a queue at the server and arrange to receive
# messages for that queue in the procedure called "incoming"
# We wait before the subscription because connecting to a server
# implies a handshake mechanism. Instead, we could use the
# introspection mechanisms provided by the -liveness option to
# ::stomp::client::connect
after 1000 ::stomp::client::subscribe $s /queue/a \
    -handler incoming

# Send a message to the same queue, meaning we'll receive it
# in the "incoming" procedure. Also make sure we'll receive
# a receipt for the message in the procedure "received"
after 2000 ::stomp::client::send $s /queue/a -body "Hello world" -receipt received

# Disconnect after a while, you might want to force disconnection from
# the web UI at the server to see what happens in the mean time.
after 60000 ::stomp::client::disconnect $s

# Live for ever..
vwait forever

APN Very much interested in this, the server side as well. So happens I was looking for a simple message broker implementation in script and your post showed up. Serendipity!

EF You are welcome, and please try it as it's really in its early days... :) As long as you're not too impatient, I hope to be able to finish up some initial version of the server within the next few weeks. It needs some refactoring, moving out most of the actual code to a ::stomp::client namespace so I can start having common procedures in ::stomp and start with a ::stomp::server.

APN Any thought of putting it in a public repository ? The half-bakery is not the most convenient for keeping track of changes.

APN Tried it (took an hour to set up Apollo thanks to spaces in the path to the JRE). Looks pretty good. Stopped and restarted Apollo and life went on without a hitch. Didn't see how to get the data of received message so directly accessed the ::stomp::message::msg_* array. Also binary data did not seem to get through, probably I have to explicitly set the content-length header. Looks like a great start.

For anyone else interested in trying this out, here is a summary of steps to set up Apache Apollo on Windows:

  • Download and extract the Apache Apollo distribution
  • Download the Java server JRE (just the desktop JRE is not enough). There does not seem to be a 32-bit version of this. You probably have to download the entire JDK if you are on 32-bit Windows. The download is a compressed tar file though the extension is just gz. Rename it to .tar.gz so 7-zip will correctly handle it.
  • Extract to (for example) c:\src\jdk1.7.0_21
  • Then type the following commands in the command shell (assuming c:\src\stomp-test is the directory that will hold your Apollo config)
set JAVA_HOME=c:\src\jdk1.7.0_21 (Note no quotes around the path even if it has spaces else batch files will fail)
cd c:\src\stomp-test
C:\bin\x86\apache-apollo-1.6\bin\apollo create mybroker
mybroker\bin\apollo-broker.cmd run

  • A mybroker subdirectory is created
  • Ctrl-C to exit/stop
  • There is also an option to run as a service
  • To connect to the admin interface, connect to http://127.0.0.1:61680/

EF It'll probably end up in my efr-tools, or if suitable in the tcllib (dreaming?). There are a number of procedures to access the messages ::stomp::message::get*. This is loosely based on the API of the HTTP package where you have a set of procedures to access the most usual bits and pieces, but where you can access the array directly. There is some code to set the content-length automatically if you haven't set it, to be frank... there might be bugs there!

EF New version in the half-bakery. I have started to refactor the code, so most procedures now live in the ::stomp::client namespace instead, the main namespace being mostly empty. The new version behaves properly on binary messages, a bug that you APN had discovered. There is now a complete set of ::stomp::message::get* procedure to access the content of the messages, for example from a callback. I've updated the code example above to reflect the API change.

Can somebody help me so that some of the internal procedures from the main ::stomp namespace are directly available from, say the ::stomp::client namespace without having to prefix namespace parent:: to calls? You'll see an attempt at the beginning of the client.tcl implementation, but that does not work for GetOpt since that one relies on uplevel... I am looking for aliasing or similar...

APN How about using namespace path ? Something like
namespace eval ::stomp::client {
    namespace path [linsert [namespace path] 0 [namespace parent]]
}

EF Indeed, I was. Thank you very much! but that is construct which appeared in 8.5, meaning that it cannot be used when run under JTcl, which is one of my goals. Any other solution would be appreciated since the latest version performs some introspection in the linked procedures to access the variables from the callers. (ugly) APN Does JTcl support namespace import ? You could use that. Or perhaps even interp alias instead of proc wrappers. The problem with proc wrappers is that they introduce an additional frame level so it is difficult to write procs that use uplevel/upvar (the number of levels up depends on whether the proc is called from the same namespace or another namespace).

EF New version available in the half-bakery, now with some server-side code, able to handle both binary and textual messages. Creating the server is as simple as the following code. Updating the top of this page to describe the feature-set of the server.
package require stomp::server

set s [::stomp::server::new]
vwait forever

APN Played with latest version. Seems to work as advertised in basic usage.

EF New version, changed how packages depend between one another (see package require calls in the (updated) examples above) and adding a few features that were missing from the implementations:

  • Disconnections are now discovered sooner, i.e. often outside of the heart-beat mechanisms.
  • Added some security mechanisms to make sure messages that are too big can be discarded
  • Server handles disconnections properly
  • Clients that haven't properly connected to the server will be bumped away when trying to send any message that is not a CONNECT.