Glossary

Name  Definition
Application key Key provided on registration that identifies you as a valid user.
Candidate Refers to a recognition candidate: The recognizer keeps in memory all the alternatives it has considered during the recognition process (e.g. “cat”, “cal”, “eat”). These alternatives are called candidates.
Digital ink Hand-drawn strokes that are captured digitally via a touch screen, stylus device or other digital writing implement. The strokes are an array of points, their ink coordinates (XY) and possibly their corresponding timestamp (T).
HMAC key This key is a way is used to crypt an application key and to check both the integrity and the authentication of a request.
Recognition Process of turning digital ink into machine-readable elements such as digital text or vector shapes.
Recognizer There is one recognizer per protocol (REST or WebSocket) and per recognition type (Text, Math, Shape, Analyzer or Music). The recognizer can only receive input components: In most cases, these are strokes, but can also be strings, characters, music clefs, etc.
Resource Internal information used within the MyScript recognizer. This can be a dictionary or lexicon, a language resource that helps it convert a certain language, etc.
Result This is the result of the MyScript engine’s recognition process. The end result is the final choice for the whole ink sample. For text, however, you can obtain more of the hierarchical tree that constitutes the result by asking to see x candidates at various levels.
WebSocket This protocol is a connected handwriting recognition mode, allowing you to create a unique scrambled connection, where Text or Math requests are continuously sent and received.

Recognition overview

Digital ink as MyScript input

MyScript performs on-line recognition and works mainly on digital ink as input*. Digital ink can be defined as a series of strokes. A stroke is defined as the trajectory of the pen from the moment it is touching the surface until it is lifted again. Digital ink thus refers to a dynamic process that takes into account where strokes start, where strokes end, and in which order they were written.

These trajectories are captured mathematically, in an ink signal composed of a sequence of 2D points (XY) ordered by time (T). This signal is captured by some writing capture device (digital pen, tablet, touchscreen, etc.). Whatever the handwriting surface, digital ink is always plotted according to a matrix with X and Y axes and a point of origin (the XY position equals 0,0).

You need to have in place the system for capturing raw ink data. This data needs to be sent in the JSON format request that you create from scratch (see JSON examples).

* Digital ink is not the only input supported by the MyScript engine: It can also recognize strings, characters, music clefs, etc. To see which input is supported, refer to the add methods in the API reference guide.

Text recognition results

MyScript is based on artificial intelligence and so works mainly on probability. It gives the most probable result it finds, in accordance with the criteria it has to work with.

For Text recognition, the result is delivered as a hierarchical tree.

The result is considered on 3 levels: character, word and text (the whole ink sample). At each level, the recognizer can keep several candidates, or likely alternatives that it also found during its deduction. You can get several recognition candidates, at any level, if you ask for them in the initial request. For example, this could allow your application to provide alternative recognition choices. You can ask in your request to see the different levels of results and ask for a certain number of candidates for each level.

A score is provided, from 0 to 1. The resemblance score reflects the level of confidence in the recognizer’s raw output. The normalized score reflects how confident the recognizer is, taking into account all the resources it has used to recognize (dictionaries, lexicons, etc.). It is probably the latter score that you should refer to.

For Shape and Math, results are more straightforward. For Shape, it is a shape type (e.g. a circle), a set of shape “primitives” (the shape’s composing elements) and knowledge on how the shape appears spatially (so it can be reproduced). For Math, it can be a LaTeX or MathML string, which you will have to render yourself if you wish to display it as an image.

Limitations and errors

MyScript can recognize different types of input, but except for Analyzer (Shape + Text), these types cannot be combined in a single request to the same URL. Each request is for a particular type of service.

Examples of input that would NOT be recognized correctly.

In the above examples, text, shapes and math expressions are combined in the same request, which is not currently possible. Your application needs to have some way of ensuring that different elements are sent in distinct requests.

Text limitations

The Text recognition service:

  • recognizes also digits, common punctuation and symbols, in the usual style for the given language.
  • can only recognize ink samples in the order in which they are sent. Any “reassembly” of parts of a document has to be done in pre-processing.
  • can recognize many common items such as many proper names, phone numbers, email addresses, simple Web addresses, with varying degrees of accuracy.
  • is case sensitive during recognition. It does not “adjust” the case, so for example if you write in all capitals, the result is highly likely to be all in capitals too.
  • recognizes text in one language. You cannot mix for example Russian and Spanish in a single request. However, most languages allow to recognize common “International” English terms. The East Asian languages support some English words appearing mixed in with native Chinese/Korean/Japanese ideograms.
  • cannot recognize super- or sub-script.

Math limitations

The Math recognition service:

  • can recognize math symbols as described on the Math recognition page, although we recommend clearly separating the different phases of the Math request,
  • can recognize equations that are not true equations, or that are improbable,
  • can recognize equation elements with almost limitless levels of recursiveness (for example, square roots with subscript),
  • can only recognize single equations - but not systems of equations,
  • cannot recognize complex arithmetic layouts like long divisions,
  • cannot recognize other text or characters or shapes (assuming that its input is always an equation),
  • cannot support corrections or ink incidents.

There is one specific case that is not supported:

Unsupported case Description Supported alternative
This is not currently supported. The second elements are too close to the first and will not give a correct result (the segmentation of the elements is likely to be incorrect). This type of equation can be recognized but care must be taken to provide adequate spacing.

Shape limitations

The Shape recognition service:

  • can recognize simple diagrams, organization drawings, freehand drawings, etc.,
  • cannot recognize text items mixed with shapes,
  • cannot recognize complex shapes such as diagrams for electronics or engineering, maps, architectural plans and so on.
Connectors will be created on the beautified shapes to make them as geometrically attractive as possible. If lines and shapes are very close, it will tend to make them touch to create an illusion of connection.

Analyzer limitations

The Analyzer recognition service can recognize only 2 types of input (Text and Shape) and:

  • complies with the Text and Shape limitations described above.
  • can recognize text mixed with simple diagrams, organization drawings, freehand drawings, etc.
  • cannot recognize complex shapes such as diagrams for electronics or engineering, maps, architectural plans and so on.
  • cannot recognize Math items.
Connectors will often be created on the beautified shapes to make them as geometrically attractive as possible. If lines and shapes are very close, it will tend to make them touch to create an illusion of connection.

Potential errors

Besides standard HTTP errors, some API-specific exceptions may occur.

Exception Description
TextApiParsingException ShapeApiParsingException MathApiParsingException AnalyzerApiParsingException IllegalInputParameterException There is incorrect input in the JSON request.
NotImplementedMethodException You have used bad syntax or used parameters etc that do not exist.
BadIpAddressApplicationKeyException BadRefererApplicationKeyException The filtering value is unknown or incorrect.
InvalidApplicationKeyException You have used an application key that does not exist or is incorrect.
RevokedApplicationKeyException Your application key is no longer valid or does not have appropriate permissions.
ApplicationDeactivatedException This application is not currently active. Visit the account console to fix this if necessary.
ContentTypeResourceNotFoundException A content type resource (used to restrict recognition to a specific type of linguistic input, such as city names) has been requested but is not present on the server. You may have misspelled it or asked for a resource that doesn’t exist in that particular language.
SKResourceNotFoundException A subset knowledge resource (used to restrict recognition to a specific set of characters) has been requested but is not present on the server. You may have misspelled it or asked for a resource that doesn’t exist in that particular language.
TextLanguageException You have asked for a language that is not supported by MyScript or the language code is incorrectly entered.
RecognitionApiTimedOutException The processing of your request lead to a timeout in the recognition engine.
IOException RuntimeException There is an internal server error.

JSON Examples

These examples show only the source JSON code. You will need to add both your application and HMAC keys and carry out URL encoding before sending the request.
Recognition type Description Example
Text Here is the visual representation of the ink sample used in the Text recognition example. The POST request made to supply the input and the result can be seen here.
Shape Here is the visual representation of the drawn shapes used in the shape recognition example. The POST request made to supply the input and the result, can be seen here.
Analyzer Here is the visual representation of the text and drawn shapes used in the analyzer recognition example. The POST request made to supply the input and the result, can be seen here.
Math Here is the visual representation of the ink sample used in the math recognition example (infinity symbol). The POST request made to supply the input and the result, in both LaTeX and MathML format, can be seen here.