Editing

This page focuses on the role of the Editor object, the central point to interact with content.

Setting a part

An Editor works on a ContentPart. To setup the link between them, call the editor’s setPart() method and pass it the part you opened or created:

// Create a new package
ContentPackage contentPackage = engine.createPackage(packageNameFile);

// Create a new part
ContentPart contentPart = contentPackage.createPart("Text");

// Accessing editor
Editor editor = editorView.getEditor();

// Associate editor with the new part
editor.setPart(contentPart);
You have to make sure that you previously called editor.setViewSize() and attached a font metrics provider to the editor before setting the part. If you use the reference implementation, calling editorView.getEditor() will do it automatically for you.

Input capture

Pen, finger and tool management

Following Interactive Ink interaction patterns, iink API considers that pen events are dedicated to write or edit content (text, math or shape content, edit gestures, etc.) and touch events to manipulate content (select, drag & drop, scroll, etc.).

If your user has an active stylus, it is strongly recommended to adopt the Interactive Ink principle of using the pen to write and the finger to manipulate content.

As you are in charge of propagating the events to the editor - in case your users are writing with a finger or a capacitive stylus or if your application is built around a modal set of tools - you can choose what iink SDK shall consider as pen or touch events.

The type of input is defined by the PointerType enum.

You may use it to select tools with specific behaviors as well (for example the eraser).

Incremental input

Interactive Ink SDK typically processes user input in real time. You thus have to tell how pointers (pen, finger) are interacting with the capture surface (typically a screen or a graphical tablet).

This can be done by calling the following methods of the Editor object:

  • pointerDown() - When the pointer first touches the surface.
  • pointerMove() - When the pointer moves while staying in contact with the surface.
  • pointerUp() - When the pointer is lifted from the surface.

Each of these methods requires you to provide:

  • x and y - The coordinates of the pointer on the surface
  • t - The timestamp of the pointer event
  • f - The pressure information associated to the event (normalized between 0 and 1)

  • pointerType - The type of pointer (pen, finger or a predefined tool like the eraser: see the PointerType enum)

  • pointerId - An identifier for this pointer.

Example:

editor.pointerDown(0.0f, 0.0f, new Date().getTime(), .7f, PointerType.PEN, 1);
editor.pointerMove(1.1f, 2.0f, new Date().getTime(), .6f, PointerType.PEN, 1);
editor.pointerUp(2.0f, 4.0f, new Date().getTime(), .5f, PointerType.PEN, 1);
You can call pointerCancel() to have the editor drop and ignore an ongoing event sequence.

Remarks:

  • The timestamp is typically the time in ms since Jan 1st, 1970. You can set it to -1 to let iink SDK generate one for you based on the current time of the system.
  • Interactive Ink SDK does not use the pressure information. It is stored in the model and can be retrieved at export or when implementing your own inking. If you don’t have or need this information, you can set it to 0.
  • If you only have one pointer simultaneously active, you can pass a pointer id of -1.

In the most simple case, you can write something like:

final long NO_TIMESTAMP = -1;
final float NO_PRESSURE = 0.0f;
final int NO_POINTER_ID = -1;

editor.pointerDown(0.0f, 0.0f, NO_TIMESTAMP, NO_PRESSURE, PointerType.PEN, NO_POINTER_ID);
editor.pointerDown(1.1f, 2.0f, NO_TIMESTAMP, NO_PRESSURE, PointerType.PEN, NO_POINTER_ID);
editor.pointerDown(2.0f, 4.0f, NO_TIMESTAMP, NO_PRESSURE, PointerType.PEN, NO_POINTER_ID);

Series of events

In some cases, you may want to send a set of strokes to the engine in a single pass, for instance if you import ink from outside of the iink model or if you want to extract a math equation from a digital ink drawing.

Interactive Ink SDK provides a method to input in a single pass a series of pointer events, pointerEvents(), that take in parameter an array of PointerEvent objects.

Here is an example:

// Build the pointer events array
PointerEvent[] events = new PointerEvent[]{
  // Stroke 1
  new PointerEvent(PointerEventType.DOWN, 184, 124),
  new PointerEvent(PointerEventType.MOVE, 184, 125),
  new PointerEvent(PointerEventType.MOVE, 184, 128),
  new PointerEvent(PointerEventType.MOVE, 184, 133),
  new PointerEvent(PointerEventType.MOVE, 184, 152),
  new PointerEvent(PointerEventType.MOVE, 184, 158),
  new PointerEvent(PointerEventType.MOVE, 184, 163),
  new PointerEvent(PointerEventType.MOVE, 183, 167),
  new PointerEvent(PointerEventType.MOVE, 183, 174),
  new PointerEvent(PointerEventType.MOVE, 183, 183),
  new PointerEvent(PointerEventType.UP, 183, 184),
  // Stroke 2
  new PointerEvent(PointerEventType.DOWN, 150, 126),
  new PointerEvent(PointerEventType.MOVE, 151, 126),
  new PointerEvent(PointerEventType.MOVE, 152, 126),
  new PointerEvent(PointerEventType.MOVE, 158, 126),
  new PointerEvent(PointerEventType.MOVE, 166, 126),
  new PointerEvent(PointerEventType.MOVE, 184, 126),
  new PointerEvent(PointerEventType.MOVE, 190, 128),
  new PointerEvent(PointerEventType.MOVE, 196, 128),
  new PointerEvent(PointerEventType.MOVE, 200, 128),
  new PointerEvent(PointerEventType.MOVE, 207, 128),
  new PointerEvent(PointerEventType.MOVE, 208, 128),
  new PointerEvent(PointerEventType.UP, 209, 128),
};
// Feed the editor
editor.pointerEvents(events, false);
When calling pointerEvents(), it is possible to set the optional second parameter processGestures to false to explicitly prevent gesture detection for this input.

Edit and decoration gestures

Interactive Ink SDK supports all the standard gestures defined as part of Interactive Ink.

There is nothing particular to do to benefit from gestures. The SDK will take care of detecting and applying the effect of the gestures from the provided input without any plumbing needed.

Decorations can be styled and will be taken into account when generating some of the export formats (ex: text underlining will be ignored for a simple text export, will turn bold in the case of a docx export and be semantically tagged in a jiix export).

Other edit operations

The following operations can be directly made on the content of a part via an Editor object:

  • Undo
  • Redo
  • Clear

Most operations, however, are to be done on content blocks.

If you need to integrate iink SDK undo/redo with your own undo/redo stack, refer to this page (advanced)

Recognition feedback

The UI Reference Implementation comes with a “smart guide” component that lets you provide real-time text recognition feedback to your end users and allows them to select alternative interpretations from the engine.

Refer to the page describing how to work with text recognition candidates for more information.

Monitoring changes in the model

There are cases where it makes sense to be notified of what occurs in the model. For instance, you may want to update the states of the undo/redo buttons of your interface or only permit an export when there is something to export.

You can call addListener() and removeListener() to respectively attach and remove an object of your choice that implements the IEditorListener interface: the contentChanged() method will tell you when any change occurs within the model.

IEditorListener also provides an onError() method that you can implement to be notified when anything goes wrong. It is strongly recommended to implement it, as it allows detecting some frequently seen issues such as recognition assets or configurations not found by the engine.

In addition, the Editor class provides other useful methods/properties, such as:

  • isIdle(), that will return true if any ongoing processing by the engine is over
  • waitForIdle(), that will block your thread until the engine is idle. This may be useful for instance in the case you want some synchronous recognition.
isIdle() will always return false in a contentChanged() notification.
To avoid deadlocks, do not call waitForIdle() from inside an IEditorListener notification.

Block management

Content blocks

A content block is a semantic subdivision of the content, and may contain data and/or other blocks. It has a unique id, a defined type (“Text”, “Math”, “Diagram”, “Drawing” or “Container”) and a bounding box.

For example:

  • A “Math” part will only contain a single block, hosting the math content itself.
  • A “Text Document” part will be more complex, as it can host text paragraphs, math equations, diagrams and drawings, arranged in a complex layout, sometimes one after the other, sometimes alongside one another. This is where “Container” blocks can be used to semantically group sub-blocks together.

The following illustration shows how these different blocks relate together inside their parent parts:

Package “Text” part “Math” part “Text Document” part 0 1 2 get root block “Text” “Math” “Container”blocks “Math” “Text” get part “Diagram” Blockhierarchy Serialization

When diagram.enable-sub-blocks is set to true in the configuration, “Diagram” blocks contain sub blocks of type “Text”, “Node”, “Edge” or “Polyedge” describing the content of the diagram.

The different blocks form a hierarchy, which root can be obtained by calling the getRootBlock() method on the parent part. Each block, in turn, has a getChildren() method that will return its own children, if any.

It is important to note that a block hierarchy is only valid at a given point in time. For instance, in the case of a Text Document, inserting new blocks, removing a text paragraph using a gesture, etc. are some examples of events that may invalidate the block hierarchy you previously retrieved.

You can check if a block is still valid by calling its isValid() method. Alternatively, watching contentChanged() events using an IEditorListener on your Editor object will provide hints that your blocks may have become invalid (the list of ids of impacted blocks is provided in parameter).

Block addition

Any part you create will contain a root block.

You may however use the addBlock() method of the editor to add a new block at a given location in compatible parts, as a way to import content (only “Text Document” parts support this feature as of now).

A dedicated addImage() method allows you to insert an image inside a Text Document part.

Operations with blocks

Some operations are possible with blocks, at a finer granularity than at the sole part level:

  • hitBlock() lets you know the top-most block at a given location, if there is any. It makes it possible for example to know which block is tapped or pressed by a user.
  • removeBlock() lets you remove a non-root block.
  • convert() lets you convert the ink inside a given block.
  • export_() lets you export the content of a specific block, including its children.
  • copy() lets you copy a block into the internal clipboard. You can then paste it at a given location using paste(), much like you would add a new block. This only works on a “Text Document” part.
  • You can monitor the events affecting of a given block, as information about the impacted blocks is provided to you by the contentChanged() method of the IEditorListener interface.

Back to the example

You previously created an Editor object. As you rely on the reference rendering implementation provided by MyScript, ink input will be transparently managed by the EditorView. All you have to do is to load the part into the editor at the end of the openContent() method you previously wrote:

public void openContent()
{
  // Load the part in the editor
  editor.setPart(contentPart);
}

If you launch the app, you can now write in it and see actual ink being rendered!

Next, you will plug the undo, redo and clear buttons.

This can be conveniently done by calling the appropriate Editor methods in reaction to user actions:

public void undo()
{
  editor.undo();
}

public void redo()
{
  editor.undo();
}

public void clear()
{
  editor.clear();
}

As a “Math” part consists of a single root block, you will be able to convert it very easily. This will be the topic of the next step of this guide!

We use cookies to ensure that we give you the best experience on our website Read the privacy policy