MyScript iink SDK Web REST Batch API

Call to <host_of_rest_endpoint>/api/v4.0/iink/batch/ allows you to use our legacy recognition engine to recognize digital content from your already collected strokes.

Depending on its Content Type configuration, the iink REST batch API performs Text, Math, Diagram or Raw Content of recognition.

Overview of the request

The body of the request contains several parts to adjust the recognition (in bold are the mandatory parts):

• Size of the writing area
• Configuration (default values will be used if no configuration is provided)
• Recognition type
• Inks
• Styling
• Conversion indication

Overview of the request body:

{
    "xDPI": 90,
    "yDPI": 90,
    "width": 800,
    "height": 1200,
    "contentType": "Text",
    "conversionState":"DIGITAL_EDIT",
    "theme": ".text { font-family: Open Sans; font-size: 10;}",
    "configuration": { },
    "strokeGroups": [ {
        "penStyle":"color: #00FF00;",
        "strokes": []
        }
    ]
}

You will find complete request bodies in the Swagger-ui or in the examples listed below.

Recognition type

The recognition type is configured using the contentType attribute. It can have the value Text, Diagram, Math or Raw Content.

Ink sending

Setting the size of the writing area

The size of the writing area is configured using the non mandatory attributes:

• width
• height

width and height if omitted will be automatically calculated using the strokes and the potential margin configuration.

Setting the resolution of the input surface

Although you might not display the strokes, you need to provide proper dpi values. These correspond to the “resolution” of your input surface, be it a digital screen or a sheet of paper. They will provide the recognizer with a sense of “writing scale” and help it favor the right hypotheses.

• xDPI the number of pixels per inch of the input surface in the X dimension.
• yDPI the number of pixels per inch of the input surface in the Y dimension

xDPI and yDPI have a default value of 96. You might have to tune these values to improve recognition results, particularly if your horizontal and vertical DPI values are different.

Stroke

An input stroke is represented by arrays:

• The x array contains the x coordinates of the points of the stroke
• The y array contains the y coordinates of the points of the stroke
• The optional t array contains the timestamps in milliseconds of the points of the stroke. There is no need for them to represent the exact epoch time of the point as the recognition engine will treat each of them in comparison with the first one.
• The optional p array describe the normalized pressure for each stroke point. Pressure values must respect 0 < value < 1.

You can assign an optional id attribute to the stroke. This feature is for future uses.

You can provide an optional pointerType attribute to the stroke. Allowed values are "PEN", "TOUCH" and "ERASER". The default value is "PEN".

You can provide an optional pointerId attribute. The pointer id identifies the device that was used to write the stroke (the id of the stylus or pencil, the finger that is moving in case of two-finger touch, ...).

Stroke groups

Lists of strokes are grouped in strokeGroups. A stroke group allows to associate a pen style to a series of strokes. Pen styles can be set in two ways:

• by setting valid CSS in penStyle
• by setting CSS classes in penStyleClasses. The listed CSS classes have to be defined in the general theme attribute.

Styling

Styling enables personalisation of your writings. A general CSS style for the whole writing area can be set in the theme attribute. Plus, for each series of strokes you can indicate a different style for the pen by setting in the corresponding strokeGroup:

• direct CSS in the penStyle attribute
• references to CSS classes of the theme in the penStyleClasses attribute.

More on CSS styling can be found in the Styling reference page.

Example of strokeGroups

Here is an example of the iink input for a series of three strokes, the last one being not of the same color as the two first ones.

"strokeGroups": [
        {
            "strokes": [
                {
                    "x":[167,164,162,160,158,157,156,155,154,153],
                    "y":[124,129,133,137,143,146,149,152,155,159],
                    "t":[0,141,159,175,191,208,225,241,275,308]
                },
                {
                    "x":[212,210,209,208,208,207,204,203,202,201,201,201,201],
                    "y":[123,131,135,139,142,145,150,153,157,160,163,166,169],
                    "t":[1238,1292,1309,1326,1342,1360,1393,1409,1427,1443,1460,1476,1493]
                }
            ]
       },
       {
           "penStyle": "color: #00FF00;",
            "strokes": [
                {
                    "x":[148,151,155,158,163,168,171,176,180,184,187,190],
                    "y":[170,169,168,167,167,167,167,167,168,170,171,174],
                    "t":[4897,5043,5078,5094,5128,5159,5177,5210,5227,5260,5293,5328],
                }
            ]
       }
    ]

Language choice

The language for the text recognition in Text, Diagram and Raw Content is configured in the configuration.lang attribute. The language code is build that way:

• ISO 639-1 two-letter code of the language.
• The character "_".
• ISO 3166-1 two-letter code of the country.

For example, use en_US for American English.

Supported interactive and non interactive languages list for iink SDK 4.5 is available here.

Text recognition

The text recognition of the iink SDK enables recognition of multiline cursive text in the selected language. If you ask for application/vnd.myscript.jiix output, you can have information about how the text was recognized and what were the alternative recognition candidates. In addition to the application/vnd.myscript.jiix output format, you can also ask for the following output:

You can customize the text recognition using custom resources that you priorly built. You can also set your own custom lexicon for each call.

Please refer to the configuration guide for an exhaustive list of configuration options for the Text recognition.

Here is an example of Text recognition.

Diagram recognition

The diagram recognition of the iink SDK handles a series of shapes, text and connection features. You can find more about Diagram structure in the description of the jiix format. In addition to application/vnd.myscript.jiix output format, you can also ask for the following output:

Regarding the Microsoft Office clipboard format, saved as a file at a location you define. It is up to you to place it into the clipboard using the Art::GVML ClipFormat key. Note also that Microsoft Office does not support pasting content from the system clipboard on all platforms.

You can customize the text recognition in the Diagram using custom resources that you priory built. You can also set your own custom lexicon for each call.

Please refer to the configuration guide for an exhaustive list of configuration options for the Diagram recognition.

Here is an example of Diagram recognition.

Math recognition

The math recognition of the iink SDK enables recognition of math equations. Here is a list of math symbols that can be recognized and rules to combine them. In addition to application/vnd.myscript.jiix output format, you can also ask for the following output:

Equations sent to the iink SDK can also be solved. If the solver is enabled, the equation will be solved automatically for the application/vnd.myscript.jiix output (the answer node will be marked as generated) and upon conversion for the other output formats.

You can restrict the number of math symbols that can be recognized using a custom grammar. There is two way to provide a custom grammar:

• You can provide the text of the grammar in the configuration (configuration.math.customGrammarContent). This gives you the opportunity to construct it dynamically.
• You can upload a grammar resource using the admin UI and provide the id in the configuration (configuration.math.customGrammarId). We incite you to choose this solution if you have a predefined set of grammars.

Please refer to the configuration guide for an exhaustive list of configuration options for Math recognition.

Here are some examples of Math recognition.

Raw Content recognition

The raw content recognition of the iink SDK enables analyzing unstructured inks to separate contents that correspond to structured text and/or shapes from contents that do not.

The raw-content.classification.types controls the list of content types that can be output by the ink strokes classifier. If content types are removed from this list, the corresponding strokes are output as part of the most likely alternative choice among the remaining content types. Possible values are text, shape and drawing

Those contents can be retrieved in the application/vnd.myscript.jiix.

The raw-content.recognition.types defines the type of recognition that will be performed on this content and the result that will be available in the JIIX export. Possible values are text and shape.

In addition to the application/vnd.myscript.jiix output format, you can also ask for the following output:

You can customize the text recognition using custom resources that you priorly built. You can also set your own custom lexicon for each call.

Please refer to the configuration guide for an exhaustive list of configuration options for the Raw Content recognition.

Here is an example of Raw Content recognition.

Conversion

Conversion is the act of replacing ink content with a typeset equivalent. It is different from the recognition process it relies on. You can ask for conversion of the output using the facultative attribute conversionState. In iink 4.5 it can only have the value DIGITAL_EDIT. The effects of conversion in iink SDK REST recognition are:

• For Diagram content type, texts and shapes are replaced by a typeset equivalent. The recognition server uses the elements of the theme to determine what font to use. You can see an example of the effect of conversion in the Diagram example.
• For Math content type, conversion solves a solvable equation in all output format if the math solver is enabled. You can see an example of the effect of conversion in the Math example.
• Conversion removes all the strokes of the jiix output, even if they are asked in the configuration. In that case the strokes will be present but the array will be empty.