uncategorized

Ok, This Won't Scale

I have been using socket.io to communicate between clients and the webserver for some time. I use it to count the number of visitors online, display system messages to the clients, etc. I piggyback socket.io on top of express. This has served me well but will not work as I move into a multi-server scenario:

  1. Online counts are stored in server memory.
  2. The messages will only be broadcast to the clients connected to an individual server.
  3. There can be issues when the client requests an upgrade to the websocket protocol unless you are using sticky sessions.

We can solve the first problem by persisting the online counts in redis or mongo but that would mean cleanup if a server crashed. We would really like to keep the count in memory because if the server crashes the online count should go to zero: If the server is offline, obviously the clients aren’t connected.

A new architecture which is scalable has the following requirements:

  1. Accurately reflect the number of players online.
  2. Not require changes to how/where messages are generated.

An architectue which fulfills the requirements is satisfied if we create a new application which acts only as a socket server. The application will not be load balanced but can be failed-over. The web servers then become clients of the new server and browser clients are now clients of the new server, not clients of the web server.

Socket Server

System messages are generated by a post request to a web server. The web server then prepares the message and emits a ShowMessage event to the socket server. The socket server then sends the message to all clients.

Message Flow

The only changes needed to the existing web server are:

  1. Remove the code that made it a socket server.
  2. Add the socket.io-client module and connect to the socket server.
  3. Instead of emitting events to the client emit them to the socket server.
  4. Change the client code to connect to the socket server instead of the web server.

The socket server and web servers are all docker containers running behind nginx. Because I don’t hard code context in my applications the web server application receives context via envars provided by Docker in the run command. The client’s html page obviously does not have access to the server’s environment variables so when the web server app is launched it replaces placeholders in the html page with the appropriate url of the socket server.

1
let fName=path.resolve(__dirname,"..","public/js/socketcounter.js");
	fs.readFileAsync(fName, "utf8")
		.then((data)=> {
			data = data.replace("__socketServerUrl__", app.get("nconf").get("SOCKETSERVERURL")).replace("__socketServerPort__", app.get("nconf").get("SOCKETSERVERPORT"));
			fs.writeFileAsync(fName, data, "utf8")
				.then((result)=> {
        ....

You can see this in action at aws.dailycryptogram.com but be advised this is a development version which may be unavailable at times and may disappear. Try opening the page in multiple tabs to verify the counts are changing. Also you should see a random quotaton on the bottom left of the page which changes every few minutes. The quotation is generated by an external system message request, but the actual quotation is fetched by the web server.

This scaling problem was simple to solve and greatly aided by the use of Docker. Would that all scaling issues were so easily solved.

Share