Off-screen usage

This page explains how you can use Interactive Ink SDK without a graphical interface.

Use case

Interactive Ink SDK was primarily designed as a graphical-oriented, interactive solution that lets end users easily input and edit their handwriting.

Sometimes, however, it makes sense to integrate iink SDK off-screen, without any user interface. It may be required to add handwriting recognition capabilities to an existing non-interactive application or to process input “in the background”, for instance to implement an ink search indexing service.

Common off-screen use cases often consist in batch processing content, i.e. processing a series of pointer events and exporting a result.

Setup

Off-screen setup is very similar to the graphical use case:

  1. Create and configure an engine.
  2. Create a renderer from the engine and pass it a null render target.
  3. Create a package and a content part of the appropriate type to work on.
  4. Create an editor, set its view size and attach a font metrics provider (for most platforms, you can reuse the one coming with the UI Reference Implementation).
  5. Attach the part to the editor.

Common pitfalls

  • dpi values - Although you do not display the strokes, you need to provide proper dpi values to the renderer. These correspond to the “resolution” of your input surface, be it a digital screen or a sheet of paper. They will provide the recognizer with a sense of “writing scale” and help it favor the right hypotheses.
  • Text guides - As explained in the editing part, text guides improve the recognition accuracy, provided that handwriting uses them as baselines. If handwriting does not match the guides, however, the quality of the recognition can be negatively impacted. In most off-screen scenarios, it is thus recommended to turn them off.
  • Editor view size - The editor requires you to provide a size before setting the part. This size shall be consistent with your writing surface and the dpi values you provided to the renderer. It will be used by some export options.
  • Font metrics provider - Although you may not want to explicitly convert, you need to attach a font metrics provider to the editor before setting the part. This is because font metrics are useful beyond mere display, like for content layout management or some export options.
  • Clear the editor between sessions - A common mistake in batch mode is to forget cleaning the part between imports, which causes new ink to be stacked on top of the one from the previous session. In most cases, you thus have to clear the part before processing new input. Creating a new part for each new session and removing the previous one will free the memory from a potentially unwanted undo/redo history.

  • Gesture processing - When calling pointerEvents() on the editor, make sure to set the processGesture parameter to false to avoid side-effects and lengthy recognition time.
  • Export after recognition is complete - Make sure to call waitForIdle() on the editor to wait for the recognition to complete before exporting.

Example

// Configure the engine to disable guides (recommended)
engine.getConfiguration().setBoolean("text.guides.enable", false);

// Create a renderer with a null render target
float dpiX = ...;
float dpiY = ...;
Renderer renderer = engine.createRenderer(dpiX, dpiY, null);

// Create the editor
Editor editor = engine.createEditor(renderer);

// The editor requires a font metrics provider and a view size *before* calling setPart()
DisplayMetrics displayMetrics = getResources().getDisplayMetrics();
Map<String, Typeface> typefaceMap = new HashMap<>();
editor.setFontMetricsProvider(new FontMetricsProvider(displayMetrics, typefaceMap));
editor.setViewSize(640, 480);

// Create a temporary package and part for the editor to work with
ContentPackage package = null;
try {
  package = engine.createPackage("text.iink");
} catch (IOException e) {
  e.printStackTrace();
}
ContentPart part = package.createPart("Text");
editor.setPart(part);

The editor can now be used to process pointer events and export the results.

We use cookies to ensure that we give you the best experience on our website Read the privacy policy