MyScript iink SDK Web REST Recognizer API

Call to <host_of_rest_endpoint>/api/v4.0/iink/recognize/ allows you to use our recognizer engine to recognize digital content from your already collected strokes.

Depending on its Content Type configuration, the iink REST recognizer API performs a specific following types of recognition.

Overview of the request

The body of the request contains several parts to adjust the recognition (in bold are the mandatory parts):

  • Recognition type
  • Scale of the writing area
  • Configuration (default values will be used if no configuration is provided)
  • Ink strokes

Overview of the request body:

{
    "scaleX": 0.625,
    "scaleY": 0.625,
    "contentType": "Text",
    "configuration": {
        "lang": "en_US",
        ...
     },
    "strokes": []
}

You will find complete request bodies in the Swagger-ui or in the examples listed below.

Setting the scale of the input surface

Although you might not display the strokes, you need to provide proper scale values. These correspond to the scale of your input surface, be it a digital screen or a sheet of paper, to convert input coordinates unit into mm, such that (X coordinate unit * scaleX = mm) and (Y coordinate unit * scaleY = mm).

The scaleX and scaleY values must be strictly positive.

Ink strokes

There is no margin with recognize API.

Stroke

An input stroke is represented by arrays:

  • The x array contains the x coordinates of the points of the stroke
  • The y array contains the y coordinates of the points of the stroke
  • The optional t array contains the timestamps in milliseconds of the points of the stroke. There is no need for them to represent the exact epoch time of the point as the recognition engine will treat each of them in comparison with the first one.
  • The optional p array describe the normalized pressure for each stroke point. Pressure values must respect 0 < value < 1.

You can assign an optional id attribute to the stroke. You can provide an optional pointerId attribute. The pointer id identifies the device that was used to write the stroke (the id of the stylus or pencil, the finger that is moving in case of two-finger touch, ...).

Ink strokes are grouped in strokes.

Example of strokes

Here is an example of the iink input for a series of three strokes.

"strokes": [
    {
        "x":[167,164,162,160,158,157,156,155,154,153],
        "y":[124,129,133,137,143,146,149,152,155,159],
        "t":[0,141,159,175,191,208,225,241,275,308]
    },
    {
        "x":[212,210,209,208,208,207,204,203,202,201,201,201,201],
        "y":[123,131,135,139,142,145,150,153,157,160,163,166,169],
        "t":[1238,1292,1309,1326,1342,1360,1393,1409,1427,1443,1460,1476,1493]
    },
    {
        "x":[148,151,155,158,163,168,171,176,180,184,187,190],
        "y":[170,169,168,167,167,167,167,167,168,170,171,174],
        "t":[4897,5043,5078,5094,5128,5159,5177,5210,5227,5260,5293,5328],
    }
]

Language choice

The language for the text recognition in Text and Raw Content is configured in the configuration.lang attribute. The language code is build that way:

  • ISO 639-1 two-letter code of the language.
  • The character "_".
  • ISO 3166-1 two-letter code of the country.

For example, use en_US for American English.

When using a Raw Content contentType, it is possible to use the mul_Latn value to recognize multiple Latin languages text.

Supported languages list for iink SDK 4.5 is available here.

Recognition type

The recognition type is configured using the contentType attribute. It can have the value: Text, Math, Raw Content, Shape or Gesture.

Text recognition

The text recognizer of the iink SDK enables recognition of multiline cursive text in the selected language. If you ask for application/vnd.myscript.jiix output, you can have information about how the text was recognized and what were the alternative recognition candidates. You can also ask for raw text output by using the text/plain mime type.

Here is an example of Text recognition.

The text recognizer supports the recognition of vertical Japanese by configuring the language to "ja_JP" and using the "vertical" text configuration name :

    "configuration":{
        "lang":"ja_JP",
        "text":{
            "configuration": {
                "name": "vertical"
            }
        }
    }

When using vertical Japanese recognition, the recognition result format is limited text/plain mime type only.

Math recognition

The math recognizer of the iink SDK enables recognition of math equations. The recognition output is available in LaTeX format with the application/x-latex mime type.

Here is an example of Math recognition.

Raw Content recognition

The raw content recognizer of the iink SDK enables analyzing unstructured inks to separate parts that correspond to structured text and/or shapes and/or math from parts that do not.

The raw-content.classification.types controls the list of content types that can be output by the ink strokes classifier. If content types are removed from this list, the corresponding strokes are output as part of the most likely alternative choice among the remaining content types.

The raw-content.recognition.types defines the type of recognition that will be performed on this content and the result that will be available in the JIIX export.

The classification and recognition outputs are available in jiix output format with the application/vnd.myscript.jiix mime type. Please refer to the configuration guide for an exhaustive list of configuration options for the Raw Content recognition.

The Raw Content recognizer supports the recognition of multiple Latin languages text by configuring the language to "mul_Latn":

{
    "configuration":
    {
        "lang":"mul_Latn",
    },
    "contentType":"Raw Content",
    ...
}

Here is an example of Raw Content recognition.

Shape recognition

The shape recognizer enables the recognition of shapes only: no matter which ink content is sent to the Shape recognizer, it returns the shape(s) that it recognizes, with alternative recognized shape(s), if any.

By default, the shape beautification is enabled: the beautification process consists in aligning shapes with each other, connecting shapes that almost touch each other and scaling shapes having close sizes. You can disabled the shape beautification with the shape.beautification.enable.

The recognition output is available in jiix output format with the application/vnd.myscript.jiix mime type.

Here is an example of Shape recognition.

Gesture recognition

The gesture recognizer is dedicated to the recognition of gestures without any text context. The recognition output is available in jiix output format with the application/vnd.myscript.jiix mime type.

Here is an example of Gesture recognition.