WebSockets API is a connected API between a client (generally a browser) and MyScript Cloud or MyScript Server. This page describes the main offscreen interactivity WebSockets messages
Open a websocket on this URL wss://cloud.myscript.com/api/v4.0/iink/offscreen with your applicationKey:
GET wss://cloud.myscript.com/api/v4.0/iink/document?applicationKey=XXXX-XXXXXX-XXXXX-XXXX
When the application hmac security option is set (default mode), an HMAC challenge must be sent to complete authentication.
Please find here a javascript example on how to compute the HMAC code.
Authentication sequence if the application hmac security option is not set (refer to Administration interface),
Once authenticated, you should use the following messages to set up your iink session.
To set up a new iink session, you should first send an initSession message
This message contains the scale to convert input coordinates unit into mm, such that (X coordinate unit * scaleX = mm) and (Y coordinate unit * scaleY = mm).
It also contains an optional configuration block: You can configure the recognition language.
Please refer to the configuration guide for an exhaustive list of configuration options.
{
"type":"initSession",
"scaleX":0.26458333333333334,
"scaleY":0.26458333333333334,
"configuration":
{
...
}
}The server sends you a sessionDescription containing the iinkSessionId.
{
"type":"sessionDescription",
"iinkSessionId":"fb9d7464-225a-452c-9109-052023c610b2",
"contentPartCount":0}The client must declare a new “Raw Content” part with a JIIX mimeTypes which will be used for the exported message:
The server will send an export message containing the recognition result after each contentChanged message.
{
"type":"newContentPart",
"contentType":"Raw Content",
"mimeTypes":["application/vnd.myscript.jiix"]
}{
"type":"partChanged",
"partIdx":0,
"partId":"rfvgkqum",
"partCount":1
}{
"type":"newPart",
"idx":0,
"id":"rfvgkqum"
}You are now ready to send strokes.
The server saves the document periodically. If your session is interrupted, you can recover it within a few minutes.
You must send a restoreSession message containing the iinkSessionId, scale values and configuration of your disconnected session.
{
"type":"restoreSession",
"iinkSessionId":"fb9d7464-225a-452c-9109-052023c610b2",
"scaleX":0.26458333333333334,
"scaleY":0.26458333333333334,
"configuration":{...}
}The server sends back the session description
{
"type":"sessionDescription",
"iinkSessionId":"fb9d7464-225a-452c-9109-052023c610b2",
"contentPartCount":1
}You must then ask to reopen your part, providing its id:
{
"type":"openContentPart",
"id":"rfvgkqum"
}{
"type":"partChanged",
"partIdx":0,
"partId":"rfvgkqum",
"partCount":1
}You are now ready to update your ink content.
{
"type":"addStrokes",
"processGestures":true,
"strokes":[
{
"id":"stroke-e9f50fb6-4cc7-41ce-a79c-67a8911fef44",
"pointerType":"mouse",
"p":[1, ... ,0.73],
"t":[1719912978137, ... ,1719912978821],
"x":[43,...,108],
"y":[194,...,295]
}
]
}At this point, the stroke is injected in the engine and its states is ‘pending’. The engine will decide if the stroke is a gesture in which case the server sends a “Gesture detected” or not in which case it will send a “Content changed”.
{
"type":"eraseStrokes",
"strokeIds":
["stroke-f88235b8-b46d-4bfe-95dd-5818fae7eeb4"]
}{
"type":"replaceStrokes",
"oldStrokeIds":
[
"stroke-d6a5f637-9be2-447a-b3f7-021cb115bed0"
],
"newStrokes":
[
{
"id":"stroke-61b12cfd-5d17-402b-ac8f-09dc142dbb89",
"pointerType":"mouse",
"p":[1,...,0.68],
"t":[1719923017231,...,1719923017683],
"x":[417,...,500],
"y":[253,...,395]
},
{
"id":"stroke-c99e4544-a4d6-4149-8e88-c3b53d643725",
"pointerType":"mouse",
"p":[0.77],
"t":[1719923019261],
"x":[983],
"y":[332]
}
]
}Applies a transform on strokes. Possible transforms are :
Example:
{
"type":"transform",
"transformationType":"TRANSLATE",
"strokeIds":["stroke-f0d251e4-3568-465b-9f70-c9875f3a9f38","stroke-f0d251e4-3568-465b-9f70-c9875f3a9f39"],
"tx":100,
"ty":0
}Server sends a gestureDetected message to the browser every time a gesture is detected over ink content when adding strokes.
{
"type":"gestureDetected",
"gestureType":"SCRATCH",
"gestureStrokeId":"stroke-66e664d1-b48e-4694-bc90-b309b9bfb874",
"strokeIds":
["stroke-d6a5f637-9be2-447a-b3f7-021cb115bed0"],
"strokeBeforeIds":[],
"strokeAfterIds":[],
"subStrokes":[
{"id":null,
"x":[503.00003,...,983.00006],
"y":[395.0,...,332.0],
"t":[1719923017691,...,1719923019261],
"p":[0.68,...,0.77],
"pointerType":null,
"pointerId":-1,
"fullStrokeId":"stroke-d6a5f637-9be2-447a-b3f7-021cb115bed0"
}
]
}You can then decide which action you want to perform, based on strokeIds information: for instance you may decide to erase corresponding strokes by using an eraseStroke message.
Server sends a contentChanged message to the browser every time the content is modified:
{
"type":"contentChanged",
"partId":"socftjfl",
"canUndo":null,
"canRedo":null,
"empty":null,
"undoStackIndex":0,
"possibleUndoCount":0
}Client can then decide which action to proceed: add strokes, get recognition result, etc.
Before retrieving the recognition result, you must make sure that the recognition engine is idle by sending a “waitForIdle” message to the server and receiving its “idle” message.
{
"type":"waitForIdle"
}{
"type":"idle"
}Once recognition engine is idle, you can get recognition result by sending an export message to the server.
{
"type":"export",
"partId":"rfvgkqum",
"mimeTypes":["application/vnd.myscript.jiix"]
}{
"type":"exported",
"partId":"rfvgkqum",
"exports":
{"application/vnd.myscript.jiix":"{\n \"type\": \"Raw Content\",\n \"bounding-box\": {\n \"x\": 25.7229156,\n \"y\": 54.0333328,\n \"width\": 64.1770782,\n \"height\": 42.7458344\n },\n \"elements\": [ {\n \"id\": \"raw-content/11\",\n \"type\": \"Text\",\n \"bounding-box\": {\n \"x\": 25.7229156,\n \"y\": 54.0333328,\n \"width\": 64.1770782,\n \"height\": 42.7458344\n },\n \"label\": \"he\",\n \"words\": [ {\n \"label\": \"he\",\n \"candidates\": [ \"he\", \"hel\", \"hee\", \"hal\", \"hl\" ],\n \"first-char\": 0,\n \"last-char\": 1,\n \"bounding-box\": {\n \"x\": 25.7229156,\n \"y\": 54.0333328,\n \"width\": 64.1770782,\n \"height\": 42.7458344\n },\n \"items\": [ {\n \"type\": \"stroke\",\n \"id\": \"stroke-f88235b8-b46d-4bfe-95dd-5818fae7eeb4\",\n \"full-id\": \"stroke-f88235b8-b46d-4bfe-95dd-5818fae7eeb4\"\n } ]\n } ],\n \"chars\": [ {\n \"label\": \"h\",\n \"candidates\": [ \"h\", \"H\", \"b\", \"n\", \"k\" ],\n \"word\": 0,\n \"grid\": [ {\n \"x\": 25.1212311,\n \"y\": 52.4228401\n }, {\n \"x\": 45.3454437,\n \"y\": 52.4228401\n }, {\n \"x\": 45.3454437,\n \"y\": 116.49025\n }, {\n \"x\": 25.1212311,\n \"y\": 116.49025\n } ],\n \"bounding-box\": {\n \"x\": 25.7229156,\n \"y\": 54.0333328,\n \"width\": 21.3145828,\n \"height\": 38.5124969\n },\n \"items\": [ {\n \"type\": \"stroke\",\n \"id\": \"00000000010064000000\",\n \"full-id\": \"stroke-f88235b8-b46d-4bfe-95dd-5818fae7eeb4\"\n } ]\n }, {\n \"label\": \"e\",\n \"candidates\": [ \"l\", \"e\" ],\n \"word\": 0,\n \"grid\": [ {\n \"x\": 45.3454437,\n \"y\": 52.4228401\n }, {\n \"x\": 88.1287155,\n \"y\": 52.4228401\n }, {\n \"x\": 88.1287155,\n \"y\": 116.49025\n }, {\n \"x\": 45.3454437,\n \"y\": 116.49025\n } ],\n \"bounding-box\": {\n \"x\": 45.0374985,\n \"y\": 64.6166611,\n \"width\": 44.8624954,\n \"height\": 32.1625061\n },\n \"items\": [ {\n \"type\": \"stroke\",\n \"id\": \"000000006500eb00ff00\",\n \"full-id\": \"stroke-f88235b8-b46d-4bfe-95dd-5818fae7eeb4\"\n } ]\n } ]\n } ],\n \"id\": \"MainBlock\",\n \"version\": \"3\"\n}"
}
}Contextless gesture message allows you to have strokes recognized without any ink context:
{
"type": "contextlessGesture",
"scaleX":0.26458333333333334,
"scaleY":0.26458333333333334,
"stroke":
{
"id":"stroke-f0d251e4-3568-465b-9f70-c9875f3a9f38",
"pointerType":"mouse",
"p": [1,0.75,0.68,0.74],
"t":[1721834093742,1721834093800,1721834093814,1721834093819],
"x":[375,370,369,366],
"y":[255,259,262,266]
}
}Once this message has been processed by the server, it sends a contextlessGesture message with corresponding gesture type and strokeID:
{
"type":"contextlessGesture",
"strokeId":"stroke-f0d251e4-3568-465b-9f70-c9875f3a9f38",
"gestureType":"scratch"
}Undo / redo messages allow you to undo and redo previous action granting you same recognition result. The changes array contain the list of actions to undo/redo with their strokes, for instance “addStrokes”, “eraseStrokes”, …
{
"type":"undo",
"changes":
[
{
"type":"addStrokes",
"processGesture":false,
"strokes":
[
{
"id":"stroke-7abceacd-2b3d-4bcf-a0f7-2d9932966357",
"x":[340,349,370,381,395,402,407,407,404,395,390,382,379,378],
"y":[347,340,323,311,293,279,259,250,240,249,259,286,302,330],
"t":[1722240815034,1722240815110,1722240815139,1722240815150,1722240815181,1722240815199,1722240815217,1722240815247,1722240815274,1722240815373,1722240815386,1722240815416,1722240815431,1722240815464],
"p":[1,0.66,0.48,0.6,0.52,0.6,0.55,0.81,0.68,0.64,0.67,0.47,0.6,0.47]
}
]
}
]
}{
{
"type":"redo",
"changes":
[
{
"type":"replaceStrokes",
"oldStrokeIds":["stroke-c84f9dab-2ef3-414e-a36e-ea0de77260bf"],
"newStrokes":
[
{
"id":"stroke-7abceacd-2b3d-4bcf-a0f7-2d9932966357",
"x":[340,349,370,381,395,402,407,407,404,395,390,382,379,378],
"y":[347,340,323,311,293,279,259,250,240,249,259,286,302,330],
"t":[1722240815034,1722240815110,1722240815139,1722240815150,1722240815181,1722240815199,1722240815217,1722240815247,1722240815274,1722240815373,1722240815386,1722240815416,1722240815431,1722240815464],
"p":[1,0.66,0.48,0.6,0.52,0.6,0.55,0.81,0.68,0.64,0.67,0.47,0.6,0.47]
}
]
}
]
}
}The ping pong mechanism prevents the websocket from being disconnected with a timeout (client, server or network equipment in the middle).
{
"type":"ping"
}If server is alive, it will respond as soon as the ping message arrives with a pong message
{
"type":"pong"
}On error, the server will send an error message and close the WebSocket.
{
"type": "error",
"message": "details about the error"
}