KiteProtocol primer

[usirin]

This is gonna be rather long, so bear with me to the end, and sorry in advance.

Let's start with our old friend math kite, with only one method defined on
it named square, which will take a number and it's gonna return square of that
number:

// math-server.js
const { KiteServer } = require('kite.js')

const math = new KiteServer({
  name: 'math',
  auth: false,
  api: {
    square: function(x, done) {
      done(null, x * x)
    },
  },
})

math.listen(7780)

The client for corresponding server will connect to it like this:

const { Kite } = require('kite.js')

const kite = new Kite({
  // assuming that they are running on the same host.
  url: 'http://0.0.0.0:7780',
  autoReconnect: false,
  autoConnect: false,
})

kite.on('open', () => {
  kite.tell('square', 5).then(res => console.log('result:', res))
  // => result: 25
})

kite.connect()

So what exactly is happening here?

Under the hood, Kite uses dnode-protocol to make calls to the each side
of the connection, but since dnode-protocol doesn't try to specify a
structure on how to send messages with extra/meta information (kite info,
authentication info, etc), we need to define our own structure.

It currently works like this:

Request from client:

{
  "arguments": [
    {
      "kite": {
        "id": "d43382a0-6b41-485c-8a44-48e5a851f302",
        "username": "anonymous",
        "environment": "browser-environment",
        "name": "math-remote",
        "version": "1.0.9",
        "region": "browser-region",
        "hostname": "browser-hostname"
      },
      "withArgs": 5,
      "responseCallback": "[Function]"
    }
  ],
  "callbacks": {
    "0": [
      "0",
      "responseCallback"
    ]
  },
  "links": [],
  "method": "square"
}

Response from server:

{
  "method": 0,
  "arguments": [
    {
      "error": null,
      "result": 25
    }
  ],
  "callbacks": {},
  "links": []
}

The way it works right now:

• We pack all the information in a temporary object(1).
• We specify this object as the only argument on the dnode-protocol request.
• We specifically name the callback as responseCallback in the temporary object.
• Once the responder(server) gets this request, we read this object(1)
  • It first checks if this is a special request by checking that the first
    argument on the request satisfies our rules.
  • If so, it first transforms this special object(1) to a valid dnode-protocol request, then since it's a normal dnode-protocol request right now, it lets it's proto to handle this request.
  • Therefore, the response will be another valid dnode-protocol message to be handled by the client's proto.

Problems with current implementation

1. Packed object has a withArgs property to define arguments even though the
   dnode-protocol defines an arguments object.
2. Packed object has a responseCallback as its callback resulting in always
   having callbacks: { '0': ['0', 'responseCallback'] } which means having to
   send a string as to define which callback is gonna be called each time. (More
   bytes over the wire)
3. withArgs is an array if there is more than one argument that needs to be
   passed, while it's a simple value without an array if there is only one
   argument needs passing (like our math kite's square method). This creates
   an unnecessary inconsistencies between requests, and therefor we need to
   handle this in a special way by checking if withArgs is an array or not.
4. The request is transformed into a regular dnode-protocol message before
   it reaches to the handler defined on the api, so even though we send the
   requester's kite info, the transformed object doesn't have this info.
   Meaning that the handlers cannot know who is making the request.
5. We are wrapping response in an object with a shape of {error, result},
   this requires first wrapping on the sender side, and de-wrapping on the
   responder side, resulting in unnecessary computation, and also more bytes
   over the wire.
6. Both requests and responses are being handled by the same method, namely
   handleIncomingMessage, which violates Single Responsibility Pattern
   resulting in more complex code, therefore more confusion.

Solution Proposal

• As far as I can see, the only reason we are using a specially packed object
  is to send the requester's info and authentication info.
• Instead of using a specially packed object with redundant properties, let's
  use dnode-protocol's regular arguments array, and let's send requester's
  info and authentication details as last argument.
• Instead of sending a responseCallback named property inside the object,
  let's make sure that the argument before last one is a function definition.
• Instead of sending an object with a shape of {error, result} as response
  message, let's again use the arguments array, and make sure it only has 2
  arguments, first is error and second is result.

The shape of request/response after proposed changes

Request from client:

{
  "arguments": [
    5,
    "[Function]",
    {
      "kite": {
        "id": "645f0802-51fc-4961-bdd0-55aed0245eae",
        "username": "anonymous",
        "environment": "browser-environment",
        "name": "browser-kite",
        "version": "1.0.9",
        "region": "browser-region",
        "hostname": "browser-hostname"
      }
    }
  ],
  "callbacks": {
    "0": [
      "1"
    ]
  },
  "links": [],
  "method": "square"
}

Response from server:

{
  "method": 0,
  "arguments": [
    null,
    25
  ],
  "links": [],
  "callbacks": {}
}

The code needs to be written will be the same, except now the handlers will
have information about the requester as their last argument:

// math-server.js
const { KiteServer } = require('kite.js')

const math = new KiteServer({
  name: 'math',
  auth: false,
  api: {
    square: function(x, done, { kite, authentication }) {
      console.log('kite info:', kite)
      // => kite info: {
      //   kite: {
      //     id: '645f0802-51fc-4961-bdd0-55aed0245eae',
      //     username: 'anonymous',
      //     environment: 'browser-environment',
      //     name: 'browser-kite',
      //     version: '1.0.9',
      //     region: 'browser-region',
      //     hostname: 'browser-hostname',
      //   },
      // }

      // send the response
      done(null, x * x)
    },
  },
})

This solves Problem 1, 2, 3, 4, 5 and the details of the implementation will
try to address Problem 6 as well, but since this problem is mostly related with
maintanence issues, it can be deferred to a later time.

Plan of action

• Refactor the current implementation without introducing new classes/types and
  make sure all the current tests are passing.
• Create a KiteRequest class which is responsible for creation and validation
  of request messages.
• Create a KiteResponse class which is responsible for creation and
  validation of response messages.
• Create a KiteProtocol class which is going to delegate to KiteRequest and
  KiteResponse classes for making sure that the connection and messages
  between 2 kites are correctly transferred, denies the requests/responses if
  something is wrong.

With these changes the connection between js version of kites will be
backwards compatible, but in order to be fully compatible and if we agree that
this is the way to go, golang implementation will need to adapt to these
changes as well.

Please let me know if there is something missing, or if I am assuming more than
I should, so that we can come up with a proper protocol definition.

/cc @gokmen @cihangir @rjeczalik @szkl @ppknap @sinan