When we arrived at Comoyo for our summer internship, we were given a pretty exciting task: build a messaging web app, using any technologies at hand, but without using a backend. Well, not completely true – we already had a UNIX-socket backend that we were to hook into, so the problem soon became building a self-contained HTML app able to talk to it without any backend of its own.

The natural platform choice became WebSocket-heavy HTML5.

After implementing a WebSocket handler in our Netty backend we started thinking about our client-side technology stack. Apart from using WebSockets for server communication, we decided to use localStorage for client-side storage and cache, and Backbone.js for the actual front-end. As Backbone.js depends on Underscore.js, we’ll just as well use it throughout the application. To speed up development, we decided to go for CoffeeScript instead of pure JavaScript.

Other people have integrated Backbone with web sockets before us. However, they all seem to be using Node.js with Socket.io, so we thought we’d share our slightly different approach.

Heads up! This article focuses on integrating Backbone with an asynchronous web socket protocol, and assumes some level of comprehension of how Backbone works. Please check out the Backbone documentation and its examples for a quick introduction before checking back.

A pretty simple layered architecture emerged.

Our layered architecture

Before we take a stroll through the layers, let’s talk about the glue: the event dispatch.

The event dispatch

To let the components talk to each other in a nice and loose fashion, we needed some sort of event dispatch system. Luckily, it turns out Backbone supplies it for absolutely free through its Backbone.Events class:

window.dispatch = _.clone(Backbone.Events)

Using the event dispatch is easy: you subscribe to and trigger events with dispatch.on(eventName, callback) and dispatch.trigger(eventName, payload) respectively.

The communication layer

With event dispatch in place, we naturally started with the bottom layer: the communication layer.

A very simple Communicator class wound up looking like this:

 1 class Communicator
 2 
 3   # The messages used in the protocol look like:
 4   #   {"com.comoyo.CommandName": {"key": value}}
 5   # 
 6   # ...so we keep the namespace handy.
 7   commandNamespace: 'com.comoyo.'
 8   
 9   # Set up the web socket and listen for incoming messages
10   constructor: (@server) ->
11     @webSocket = new WebSocket(@server)
12     @webSocket.onmessage = @handleMessage
13     @webSocket.onopen = -> dispatch.trigger('WebSocketOpen')
14 
15   handleMessage: (message) =>
16   
17     # The message string is the message object's data property
18     strMessage = message.data
19   
20     # Now, parse that string
21     jsonMessage = JSON.parse(strMessage)
22   
23     # Grab the command name, ie. the first root key (using Underscore.js)
24     fullCommandName = _.keys(jsonMessage)[0]
25     commandName = _.last(fullCommandName.split('.'))
26   
27     # Trigger the command in event dispatch, passing the payload
28     dispatch.trigger(commandName, jsonMessage[fullCommandName])
29 
30   sendMessage: (commandName, messageData) =>
31   
32     # Full command name with namespace
33     fullCommandName = @commandNamespace + commandName
34   
35     # Build the message
36     jsonMessage = {}
37     jsonMessage[fullCommandName] = messageData
38   
39     # Serialize the object into a JSON string...
40     strMessage = JSON.stringify(jsonMessage)
41   
42     # ...and send it!
43     @webSocket.send(strMessage)

The Communicator simply encapsulates the web sockets, wrapping them in a couple of simple messaging methods. This should make it easy as a breeze to swap our beloved WebSockets with SSE or other long polling techniques, in case we want to provide working fallbacks in old browsers, for instance.

The protocol controllers

With our communicator in place, we can set up controllers handling the various parts of communication with the backend protocol. The controllers communicate through the Communicator: they listen to incoming messages through the event dispatch and send messages with Communicator.sendMessage calls.

A large part of the protocol we implemented relies upon sequences of messages. The typical pattern consists of the following steps:

  1. We subscribe to a resource
  2. We’re notified of a changed resource
  3. We request the changed resource
  4. We receive the resource

Let’s take a look at our login controller for an example of how to implement this sequential protocol.

A simplified version of out login process should clarify the pattern. It consists of only two simple sequential steps:

  1. Client registration
  2. Account login

To tie these together, we could build a controller looking like this:

 1 class LoginController
 2   
 3   # Let the initializer listen for relevant events,
 4   # delegating them to its respective event handlers.
 5   initialize: ->
 6     dispatch.on('WebSocketOpen',
 7                           @sendClientRegistrationCommand, this)
 8     dispatch.on('ClientRegistrationResponse', 
 9                           @handleClientRegistrationResponse, this)
10     dispatch.on('AccountLoginResponse', 
11                           @handleAccountLoginResponse, this)
12   
13   # The client registration command is a simple message 
14   # with some metadata to initiate the connection
15   sendClientRegistrationCommand: ->
16     payload =
17       clientInformation:
18         clientMedium: 'web'
19     communicator.sendMessage('ClientRegistrationCommand', payload)
20   
21   # Our client registration response handler should store
22   # required metadata and send an account login command
23   handleClientRegistrationResponse: (data) ->
24     # Store client registration data somewhere handy
25     
26     # Send account login command
27     payload =
28       userInformation:
29         username: "myrlund"
30         password: "ihazpassword"
31     
32     communicator.sendMessage('AccountLoginCommand', payload)
33   
34   # Handle the response to our AccountLoginCommand
35   handleAccountLoginResponse: (data) ->
36     # Store session keys and user data somewhere handy
37     
38     # Check response data to see if login was successful
39     if data.loggedIn
40       beHappy()
41     else
42       tryAgain()

Simply put, the various commands listen to responses and fire the next step as soon as the response is handled. It’s easy to implement, and the message sequences are easily understood from the listener declarations in the controller initializer.

The storage layer

We want a persistence layer capable of two things: storing data received from the backend controllers, and talking to the front-end part of our app. Since we’ve already looked at setting up the controller layer, let’s start with the former: storing data from the backend controllers.

Storing data

In case you’re not familiar with localStorage, fear not: you don’t need to be. It’s as simple a key-value store as they come.

localStorage.setItem('ourKey', 'someValue')
localStorage.getItem('ourKey') // => 'someValue'

Next, we run into a small problem in that localStorage doesn’t support storing objects. It is, however, pretty good at strings. A simple solution is to serialize our objects into the store. To make it so, we encapsulate the localStorage in an event-driven storage object:

class Store
  
  # We keep an in-memory cache to speed up reading of data
  data: {}
  
  # Set this.store to localStorage and load cache
  constructor: (@schemas) ->
    @store = localStorage
    @loadCache()
  
  # We'll call addItems from the backend controllers.
  # 
  # items: object, indexed on unique id.
  #   ex. {"1": {content: "Foo"}, "2": {content: "Bar"}}
  addItems: (schema, items) ->
    
    # Add or overwrite existing items
    _.extend(@data[schema], items)
    
    # Write cache to store
    @save()
  
  # Iterates over keys in cache, serializing
  save: ->
    for key in _.keys(@data)
      @store.setItem(key, JSON.stringify(@data[key]))
  
  # Populates cache with stored data
  loadCache: ->
    for schema in @schemas
      @data[schema] = @fetch(schema) || {}
  
  # Fetches object from store
  fetch: (schema) ->
    JSON.parse(@store.getItem(schema))

Talking to Backbone

Although Backbone is designed for AJAX REST APIs out of the box, it supports any kind of backend through an extremely simple synchronization interface. One simply sets Backbone.sync to a function that in some way can handle the basic CRUD operations – create, read, update and delete.

Let’s add a sync method to our store, along with some helper methods.

Note: We’re using a read-only API, so we don’t really handle writing to the store. It should, however, be easy enough to implement by triggering a change event resulting in an appropriate backend call.

class Store
  
  # ...
  
  # Attaches to Backbone.sync
  # 
  # method:  either "create", "read", "update" or "delete"
  # model:   the model instance or model class in question
  # options: carries callback functions
  sync: (method, model, options) =>
    resp = false
    schemaName = @getSchemaName(model)
    
    # Switch over the possible methods
    switch method
    
      when "create"
        # In our case, we never create models directly
        console.log "This shouldn't happen."
        
      when "read"
        # Read one or all models, depending on whether id is set
        resp = if model.id then
          @find(schema, model.id)
        else
          @findAll(schema)
        
        unless resp
          return options.error("Not found.")
        
      when "destroy"
        # Perform a fake destroy
        resp = true
    
    # Fire the appropriate callback
    if resp
      options.success(resp)
    else
      options.error("Unknown error.")
  
  # Simple getters for one or all models in a schema
  find: (schema, id) ->
    @data[schema] && @data[schema][id]
  findAll: (schema) ->
    _.values(@data[schema]) || []
  
  # Models either have a schema name attached to themselves
  # or through their collections
  getSchemaName: (model) ->
    if model.schemaName
      model.schemaName
    else if model.collection && model.collection.schemaName
      model.collection.schemaName

# Export and attach to Backbone.sync
this.store = new Store(['contacts'])
Backbone.sync = this.store.sync

Overriding the Backbone.sync allows Backbone to talk to our store, but web sockets are a two way street, and we still don’t have any way of telling our Backbone collections of incoming data.

So, to the actual talking to Backbone part… A simple way to allow collections to subscribe to changes to schemas is to trigger an event when adding items from the backend. Let’s add to our addItems method.

addItems: (schema, items) ->
    
    # Add or overwrite existing items
    _.extend(@data[schema], items)
    
    # Write cache to store
    @save()
+   
+   # Fire a notification passing the changed ids
+   payload = {}
+   payload[schema] = _.keys(items)
+   dispatch.trigger("store:change:#{schema}", payload)

Here is an example of a Backbone collection integrating with this event mechanism:

class ContactCollection extends Backbone.Collection
  
  model: Contact
  
  # We don't use URLs in our protocol, but 
  # Backbone requires that we set it...
  url: ''
  
  # We'll need to define a schema name for use
  # in the Backbone.sync method of our store
  schemaName: "contacts"
  
  initialize: ->
    # Bind to the store's appropriate change event
    dispatch.on("store:change:#{@schemaName}", 
                @updateContacts, this)
  
  # Called whenever new data is inserted into 
  # the data store.
  updateContacts: (data) ->
    
    # The contacts property of the passed data is
    # an array of ids of the changed contacts
    contactIds = data[@schemaName]
    
    for contactId in contactIds
      # Check if the contact exists
      if conversation = @get(contactId)
        # If it exists, simply _set_ its updated properties
        conversation.set(store.find(@schemaName, contactId))
      else
        # Elsewise, create it and add it
        contactData = store.find(@schemaName, contactId)
        @add(new Contact(contactData))

That’s it!

That should cover integrating Backbone with an arbitrary web socket protocol.

Note that this approach is especially well suited to our particular use case, and there are probably both easier and better ways to integrate with other protocols. However, our approach should be generic enough to be fitted to any sensible situation.

Issues

On our journey, we ran into some issues that you might do well to keep in mind if you’re trying to do something similar to us.

LocalStorage is completely unencrypted

Without a backend rendering our HTML, we don’t have any safe place to store user credentials. We’re also handling sensitive data, which shouldn’t be left plainly stored in any computer’s web cache.

We took some simple measures to secure our users’ data:

  1. When a user logs out, we clear the entire localStorage.
  2. In the login form, we present an option for whether the computer in use is a public computer. If so, persist as little as possible.

We’ve been looking into some recently matured client-side crypto libraries, and the possibilities of encrypting the store. However, there is no safe place to store the encryption key client-side, requiring us to get it from the server for every page reload, in turn requiring authentication. This stackoverflow thread pretty much sums it up.

Resources

  1. Introducing WebSockets by HTML5 rocks – a great intro to using WebSockets in HTML5
  2. backbone-localstorage.js by documentcloud – a simple adapter for using Backbone with localStorage
  3. Understanding Backbone by Kim Joar Bekkelund – a great tutorial on Backbone-ifying a typical jQuery app

Edit: Fixed small error in the Communicator code example.

comments powered by Disqus