Messages

WebSockets API is a connected API between a client (generally a browser) and MyScript Cloud or MyScript Server.

Opening the WebSocket

Open a websocket on this URL wss://cloud.myscript.com/api/v4.0/iink/document

Creating a new iink document

The client sends a createNewContentPackage message with an applicationKey.

Before using those APIs, you need to register on MyScript Cloud and have both an application and an hmac key.

Initialization with HMAC HandShake

If the application hmac security option is set (default mode), a HMAC Challenge must be answered before making any operation in the iink document.

Client to server message

{
  "type": "newContentPackage",
  "applicationKey":  "XXXX-XXXXXX-XXXXX-XXXX",
  "xDpi": 90,
  "yDpi": 90,
  "viewSizeHeight": 800,
  "viewSizeWidth": 1023
}

The server sends an ack message containing a hmacChallenge: The inkSessionId is used to reconnect to the doc in case of connection failure.

Server to client message

{
  "type":"ack",
  "hmacChallenge": "XXXX-XXXXXX-XXXXX-XXXX",
  "iinkSessionId": "YYYYYYYYYYYYYYYYYYYYY"
}

and a contentPackageDescription message:

{
  "type":"contentPackageDescription",
  "contentPartCount":0
}

The browser computes the hmac with this challenge and the private key.

Client to server message

{
  "type": "hmac",
  "hmac": "ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ"
}

Please find here a javascript example on how to compute the HMAC code.

Simple initialization (No HMAC HandShake)

Sequence if the application hmac security option is not set (refer to Administration interface),

Before using those API, you have to register on MyScript Cloud and have both an application key and an hmac key.

Client to server message

{
  "type": "newContentPackage",
  "applicationKey":  "XXXX-XXXXXX-XXXXX-XXXX",
  "xDpi": 90,
  "yDpi": 90,
  "viewSizeHeight": 800,
  "viewSizeWidth": 1023
}

The server will send an ack message:

Server to client message

{
  "type":"ack",
  "iinkSessionId": "YYYYYYYYYYYYYYYYYYYYY"
}

and a contentPackageDescription message:

Server to client message

{
  "type":"contentPackageDescription",
  "contentPartCount":0
}

This message is optional. Depending on the content part type, you can configure the recognition language, the margins or the solver behavior. Please refer to the configuration guide for an exhaustive list of configuration options.

Client to server message

{
  "type": "configuration",
    "math": {
        "solver": {
            "fractional-part-digits": 3,
            "decimal-separator": ".",
            "rounding-mode": "half up",
            "angle-unit": "deg"
        },
        "margin": {
            "top": 0,
            "left": 0,
            "right": 0
        }
    },
    "export": {
        "image-resolution": 300,
        "jiix": {
            "bounding-box": "false",
            "text": {
                "chars": "false",
                "words": "true"
            },
            "strokes": "true"
        }
    }
}

Set Theme

The theme is the default style of the content. The theme property is a CSS given as a String.

More insight about pen and theme style in this article

Client to server message

{
  "type":"setTheme",
  "theme":"ink {\ncolor: #B71C1C;\n-myscript-pen-width: 3;\n-myscript-pen-fill-style: none;\n-myscript-pen-fill-color: #FFFFFF00;\n}\n.math {\nfont-family: STIXGeneral;\n}\n.math-solved {\nfont-family: STIXGeneral;\ncolor: #A8A8A8FF;\n}\n.text {\nfont-family: ABeeZee;\nfont-size: 14;\n}\n"
}

Set Pen Style

Set the pen style

Client to server message

{
  "type":"setPenStyle",
  "style":"color: #ff7314;\n-myscript-pen-width: 2;"
  }

Set Pen Style Classes

Set the pen style classes

Client to server message

{
"type":"setPenStyleClasses",
"styleClasses":"greenThickPen"
}

Creating a new part

The client must declare a new part. It can declare mimeTypes which will be attached to the exported message. If mimeTypes are specified, the server will send an export message containing the recognition result after each contentChanged message.

Client to server message

{
  "type":"newContentPart",
  "contentType":"MATH",
  "mimeTypes":["application/x-latex","application/mathml+xml"]
}

Server to client message

{
  "type":"partChanged",
  "partIdx":0,
  "partId":"xjftuzof",
  "partCount":1
  }

{
  "type":"newPart",
  "idx":0,
  "id":"xjftuzof"
  }

Sending Strokes

Sending a stroke

pointerType can be “MOUSE”, “PEN” or “TOUCH”. Timestamps are expressed in milliseconds. p (optional) is the pressure.

Client to server message

{
  "type": "addStrokes",
  "strokes": [
  {
    "id": "pending-1",
    "pointerType" : "PEN",
    "x": [273, 278, 281],
    "y": [121, 128, 133],
    "t": [3185.7900000000004, 3213.8150000000005, 3222.5350000000003],
    "p": [1.4, 1.4, 1.4]
  }
  ]
}

Pending stroke management

At this point, the stroke is injected in the engine and its states is ‘pending’. The engine will decide if the stroke is a gesture or not. It will send SVG Patches containing indications on what SVG elements to add, remove or replace.

Server to client message

{
  "type": "svgPatch",
  "layer": "MODEL",
  "updates":[
    { "type": "REMOVE_ELEMENT", "id": "pendingStroke-1" }
  ]
}

SVG Patch

Server sends SVG Patch to update the browser view.

A patch is composed of updates which can be:

APPEND_CHILD
REMOVE_ELEMENT
REPLACE_ALL
REPLACE_ELEMENT
REMOVE_CHILD
APPEND_CHILD
INSERT_BEFORE
REMOVE_ATTRIBUTE
SET_ATTRIBUTE

The layer attribute in patch indicates the layer to apply the patch, either “CAPTURE” or “MODEL”.

Example:

Server to client message

{
  "type":"svgPatch",
  "updates":[
        {
          "type": "APPEND_CHILD",
          "parentId": "MODEL-viewTransform",
          "svg": "<g id=\"G7f33f00e1c80-MathArea\">\n  <g id=\"MODEL-ds000000000001000500ff\">\n    <path d=\"M 76.89,34.27l 0.10,-0.31l 1.60,2.07l 0.81,1.44l -1.07,-1.25l -1.44,-1.95z\"></path>\n  </g>\n</g>\n"
        }
      ],
  "layer":"MODEL"
}

Content Changed

Server sends a contentChanged message to the browser every time the content is modified.

Server to client message

{
  "type":"contentChanged",
  "partId": "xjftuzof",
  "canUndo":true,
  "canRedo":false,
  "empty": false,
  "undoStackIndex": 3,
  "possibleUndoCount": 2
}

Exported

Server sends exported message to the browser.

Server to client message

{
  "type":"exported",
  "partId": "xjftuzof",
  "exports":
  {
    "application/x-latex":"\\sqrt ",
    "application/mathml+xml":"<math xmlns='http://www.w3.org/1998/Math/MathML'>\n  <mo> &#x221A; <!-- square root --> </mo>\n</math>\n"
  }
}

If the mime type is binary type (for example image/jpeg or image/png), the value will be base64 encoded.

Client commands

pointerEvents

This message import external ink

Client to server message

{
  "type":"pointerEvents",
  "processGestures":"false",
  "events": [{
    "pointerType": "PEN",
    "pointerId": "1",
    "id": "AZDFDFD"
    "x": [273, 278, 281],
    "y": [121, 128, 133],
    "t": [3185.7900000000004, 3213.8150000000005, 3222.5350000000003],
    "p": [1.4, 1.4, 1.4]
  },{
    "pointerType": "PEN",
    "pointerId": "1",
    "id": "AAAZZA",
    "x": [173, 178, 181],
    "y": [221, 228, 233],
    "t": [6185.7900000000004, 6213.8150000000005,6222.5350000000003],
    "p": [1.4, 1.4, 1.4]
  }]
}

Change View Size

To ensure an accurate gesture detection, it is critical that the displayed content client-side and server-side are exactly the same. The ink will then be reflowed. Strokes or glyph elements will be replaced to fit at best on your screen; e.g. when you turn your tablet from landscape to portrait.

Client to server message

{
  "type":"changeViewSize",
  "width":1024,
  "height":768
}

Convert

To convert recognized strokes to glyphs

Client to server message

{
  "type":"convert",
  "partId": "XXXXX",
  "conversionState": "DIGITAL_EDIT"
}

Undo

Undo the last user action.

Client to server message

{
  "type":"undo"
}

Redo

Redo the last user action.

Client to server message

{
  "type":"redo"
}

Restore an iink session

Server stores document periodically. If your session is disconnected, you can restore it in a few minutes.

Client to server message

{
  "type":"restoreIInkSession",
  "applicationKey": "XXXX-XXXXXX-XXXXX-XXXX",
  "iinkSessionId": "YYYYYYYYYYYYYYYYYYYYY",
  "xDpi": 90,
  "yDpi": 90,
  "viewSizeHeight": 800,
  "viewSizeWidth": 1023
}

Import File

File import process has 2 steps: initialization and chunk sends. Splitting a file in chunks can be useful.

You can import JIIX or text (with a text part). Note that currently Math parts does not support import. You must begin the import with an importFile message once you have created a content part.

Set and keep importFileId which must be used in the next fileChunk messages. Supported mime types are application/vnd.myscript.jiix and text/plain

Client to server message

{
"type":"importFile",
"importFileId":"ZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZ",
"mimeType":"text/plain"
}

You can then send fileChunk messages.

Client to server message

{
{"type":"fileChunk",
"importFileId":"ZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZ",
"data":"plain text or base64 if binary",
"lastChunk":true
}

The server sends you an ack when a chunck is received.

Server to client message

{
  "type":"fileChunkAck",
  "importFileId":"ZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZ",
  "chunkIdx":0
}

After the last chunk, the server will incorporate the file and send SVG patches.

Export

This message allows to force an export client-side.

Client to server message

{
  "type":"export",
  "partId": "XXXXX",
  "mimeTypes": []
}

Possible values for mimeTypes are: application/mathofficeXML, text/plain, application/x-latex, application/vnd.myscript.jiix, application/vnd.myscript.iink, image/jpeg and image/png. The server will answer asynchronously with an “exported” message.

Ping-pong

The ping pong mechanism prevents the websocket from being disconnected with a timeout (client, server or network equipment in the middle).

Client to server message

{
  "type":"ping"
}

If server is alive, it will respond as soon as the ping message arrives with a pong message

Server to client message

{
  "type":"pong"
}

Common behaviour

On error, the server will send an error message and close the WebSocket.

{
  "type": "error",
  "message": "details about the error"
}