Off-screen usage

This page explains how you can use iink SDK without using iink SDK rendering.

MyScript iink SDK was primarily designed as a graphical-oriented, interactive solution that lets end users easily input and edit their handwriting. With the Editor, interactivity is rendering driven, relying on iink SDK rendering.

Sometimes, however, it makes sense to integrate iink SDK off-screen, without any user interface or with a user interface based on your own rendering, not on iink SDK one.

Non interactive use cases

It may be required to add handwriting recognition capabilities to an existing non-interactive application in batch mode. It might also be useful to process input in incremental mode without displaying the recognition result, for instance to implement an ink search indexing service.

Setup

Off-screen setup is very similar to the graphical use case:

1. Create and configure an engine.
2. Create a renderer from the engine and pass it a null render target.
3. Create a package and a content part of the appropriate type to work on.
4. Create an editor, set its view size and attach a font metrics provider (for most platforms, you can reuse the one coming with the UI Reference Implementation).
5. Attach the part to the editor.

Common pitfalls

Example

The editor can now be used to process pointer events and export the results.

Offscreen Interactivity use case

Up to iink SDK 2.0, it was necessary to use the iink SDK rendering in order to benefit from iink SDK interactive features. In 2.1, while keeping our classical Editor and its related objects and interfaces, we introduced the Offscreen Interactivity feature based on an OffscreenEditor.

This new feature lets the integrator programmatically drive the content model update by keeping the incremental recognition principle and using gesture notifications. The application can rely on its own rendering to manage the captured strokes and display its model. Recognition results are available on demand with dedicated APIs.

Offscreen Interactivity main objects

For the moment, this feature is only available for “Raw Content” parts. It relies on these new objects and interfaces:

• OffscreenEditor is a service allowing offscreen interactivity. It is the entry point to edit an associated part and receive change notifications.
• ItemIdHelper allows working with item IDs (i.e. strokes). It is associated with an OffscreenEditor.
• IOffscreenEditorListener the listener interface for receiving OffscreenEditor events.
• IOffscreenGestureHandler The listener interface for handling gesture events.

Offscreen Interactivity undo/redo

• HistoryManager gives you access to undo/redo stack manipulation, just like the legacy Editor and content of changesets. The goal of the HistoryManager is to allow you to use undo/redo to preserve text recognition, rather than redoing strokes, which can change stroke order and recognition output.
• To help you synchronize your undo stack with ours, we also give you access to changesets, so you can know what our next undo/redo will do. A changeset contains N ChangesetOperation which each corresponds to an operation (ADD, ERASE or TRANSFORM) on a list of item IDs.
• For performance reasons, a HistoryManager is associated with an OffscreenEditor on creation, only if offscreen-editor.history-manager.enable is set to true (by default this parameter is false). It can then be accessed with OffscreenEditor getHistoryManager() method.

Offscreen Interactivity API and samples