ApiBlaze: Websocket Backend

ApiBlaze is a tool to explore API specifications: Search for a keyword, filter for objects, properties, or endpoints, and immediately see descriptions and code examples. ApiBlaze helps you to answer a specific question about an API lightning fast. You can try it here: apiblaze.admantium.com.

In my previous articles, I covered why websockets are an important technique for ApiBlaze: They allow long-lasting, full duplex connections between two servers to continuously stream data. Since open API specification are quite big - the full Kubernetes Spec comes at 4MB of text - searching though this data, transforming it and sending it back to the browser is a rather compute intensive action. In ApiBlaze, these actions are done server-side, and then send back to the browser for immediate rendering.

This fulfills the following core requirement:

• TECH03 - Use WebSockets to Connect Frontend and Backend

This article details how to create a backend server build with the express framework and the socket.io library and how to connect it to a client-side frontend running in the browser. While the steps are explained in the context of ApiBlaze, they are generic and you can apply them for any frontend library like React, Vue or PlainJS apps.

Backend Server

The backend is based on express and socket.io (v2.3)[^1]. Let’s install the packages.

npm i -S express socket.io@2.3.0

Once the installation is done, we will create 4 files to separate the backend code into different areas of responsibilities.

• index.js: Implements the express server instance, opening a port at which the websocket connection can be accessed
• connect.js: Creates a websocket instance by receiving a node HttpServer instance, an object which is e.g. created with express
• handler.js: The handler defines all websocket events to which and determines how they are processed
• actions.js: Contains the concrete functions that will be called when a registered event is received, and they return their results to the handler which in turn returns them to the caller.

This simple layout helps you to keep the backend application code cleanly separated. Let’s now detail the contents of these files.

Webserver with WebSocket Endpoint

The express webserver is defined in index.js.

//*  index.js *//
const express = require('express')
const websocketConnection = require('./connect.js')

app = express()

const httpServer = app.listen(3000, () => {
  console.log(`BOOTING | api-blaze-backend v0.0.1`)
})

websocketConnection(httpServer)

In these few lines of code[^2], we create an express server instance to listen at port 3000 (line 7), and then pass this instance to the function websocketConnection (Line 11).

Connector

The connector defines how the websocket is configured. We create an instance called io (line 6), which receives the express server instance and an optional configuration object. Options are manifold, see the official documentation. Then, for the websocket instance, we define an event listener for the connection event (line 9). When this event happens, the handler will take control.

//*  connect.js *//
const websocket = require('socket.io')
const handleConnection = require('./handler.js')

function websocketConnection (httpServer) {
  const io = websocket(httpServer, {
    serveClient: false
  })
  io.on('connection', socket => handleConnection(socket))
}

module.exports = websocketConnection

Handler and Actions

In the handler.js file we define which messages the websocket processes and how to respond to them. Events are defined with the method io.on, which receives the name of the event, its arguments and a callback function that will be executed. In Line 6, we define how to handle the system:healthcheck message: We will log the received message, and then emit an answer with the message healthy. Similarly in Line 10, we define to handle the message app:api-search-action, which will execute the action function apiSearchAction.

//*  handler.js *//
const { apiSearchAction } = require('./actions')

const clients = {}

function handleConnection (socket) {
  console.log(`+ client ${socket.id} has connected`)
  clients[socket.id] = { connected: true }

  socket.on('system:healthcheck', msg => {
    console.log(msg)
    socket.emit('system:healthcheck', 'healthy')
  })

  socket.on('app:api-search-action', keyword => {
    console.log('app:api-search-action', keyword)
    socket.emit('app:api-search-action', apiSearchAction(keyword))
  })
}

module.exports = handleConnection

Actions are plain JavaScript functions. The apiSearchAction will load the API inventory, a static file containing the name, description and a backend-side file link to the API specification file. It will search for the keyword in this representation, by converting the keyword into a regexp, and then ranking all APIs by the number of matches of this keyword

Example for an action:

//*  action.js *//
const apiInventory = require('./spec/inventory.json')

function apiSearchAction (keyword) {
  const regex = new RegExp(keyword, 'ig')
  var res = []
  for (let [name, definition] of Object.entries(apiInventory)) {
    const occurences = JSON.stringify(definition).match(regex)
    const score = (occurences && occurences.length) || 0
    res.push({ name, score, definition })
  }
  return res.sort((a, b) => b.score - a.score)
}

Now we have explained the backend handling of a search. Let’s see how this connection is established and handled in the frontend.

Connecting the Frontend

The frontend offers two different options to install socket.io. You can either add a <script> tag manually, referring to a statically provided socket.io JavaScript file, or you can use a bundler such as Snowpack bundler which will automatically install the library.

To setup snowpack and the socket.io client library, execute this command:

npm i -s snowpack socket.io-client@2.3.0

Once completed, define a connect.js file which will create a websocket instance that connects to the backend server.

//*  connect.js (Frontend) *//
import io from 'socket.io-client'
export default io('ws://127.0.0.1:8081', { cookie: false })

Then, you can import the websocket instance in other files, for example in the index.js, and start with sending a health check message. Be careful: By importing socket.io-client, a number of objects in the global scope will be defined, like Server, Socket, Client - do not use similarly named objects in your application.

import websocket from './globals/connect.js'

function init () {
  websocket.emit('system:healthcheck', 'ok?')
  websocket.on('system:healthcheck', msg => {
    console.log(msg)
  })
}

init()

This websocket instance works similar to what we already saw in the backend. With websocket.emit, messages are send, and with websocket.on, handlers for incoming messages are defined.

You can use the instance that is exported in connect.jsin other classes as well. Here is an example for the SearchApiAction - it will emit the message app:api-search-action, and upon receiving an answer, it will pass the results to a callback function.

import websocket from '../globals/connect.js'

export default class SearchApiAction extends Action {
  action (cb) {
    websocket.emit('app:api-search-action', 'dummy')
    websocket.on('app:api-search-action', json => {
      cb(json)
    })
  }
}

Review: ApiBlaze Project Requirements

With the refactoring completed, we have the following status with ApiBlaze requirements:

• Searching for APIS
  • ✅ SEA01 - Search for APIs by Keyword
  • ✅ SEA02 - Show search results in a popup
  • ✅ SEA03 - Select a search results with arrow keys, enter and mouse click
• Framework
  • ✅ FRAME01 - Controller & Routing
  • ✅ FRAME02 – Stateful Pages & Components
  • ✅ FRAME03 - Actions
  • ✅ FRAME04 – Optimized Bundling
• Technologies
  • ✅ TECH01 - Use PlainJS & Custom Framework
  • ✅ TECH02 - Use SAAS for CSS
  • ✅ TECH03 - Use WebSockets to Connect Frontend and Backend

Conclusion

Using WebSockets to connect a backend with a frontend enables you to form long-lasting, full duplex connections for continuously streaming data. For the backend, the essential steps are: Import the socket.io library, create an node HttpServer instance, and use this instance to create a Socket.IO instance. Then, you define event listeners with the methods io.on and io.emit. The client needs to import the socket.io client library, create an instance that connects to the backend, and also use io.on and io.emit for defining and handling the messages that will be exchanged. Try to use WebSockets in one of your projects - they are powerful and easy to setup.

Footnotes

[^1]: At the time of writing, websocket.io v 3.0 was released, but I could not get it working and choose the older v2.3 instead. [^2]: In this example, the express configuration is rather plain, but you can add any other express module, e.g. for handling static files or for setting CORS values.

• Javascript
• Apiblaze