Editing

This page focuses on the role of the IINKEditor object, the central point to interact with content.

Setting a part

An IINKEditor works on an IINKContentPart. To setup the link between them, call the editor’s part property and pass it the part you opened or created:

// Create a new package
IINKContentPackage *package = [self.engine createPackage:@"newPackage.iink" error:nil];

// Create a new part
IINKContentPart *part = [package createPart:@"Text" error:nil];

// Create an editor
IINKEditor *editor = [self.engine createEditor:renderer];

// Associate editor with the new part
editor.part = part;
You have to make sure that you previously called setViewSize: on the editor and attached to it a font metrics provider before setting the part. If you use the reference implementation, the EditorViewController will do it automatically for you.

Input capture

Pen, finger and tool management

Following Interactive Ink interaction patterns, iink API considers that pen events are dedicated to write or edit content (text, math or shape content, edit gestures, etc.) and touch events to manipulate content (select, drag & drop, scroll, etc.).

If your user has an active stylus, it is strongly recommended to adopt the Interactive Ink principle of using the pen to write and the finger to manipulate content.

As you are in charge of propagating the events to the editor - in case your users are writing with a finger or a capacitive stylus or if your application is built around a modal set of tools - you can choose what iink SDK shall consider as pen or touch events.

The type of input is defined by the IINKPointerType enum.

You may use it to select tools with specific behaviors as well (for example the eraser).

Guides

Text Document and Text parts have guides set by default. Guides provide useful hints for end users to know where to write and at what size. They also improve the recognition accuracy, provided that handwriting uses them as baselines.

You can enable or disable the guides of a Text part via the text.guides.enable key of the engine or editor configuration. The vertical spacing between guides can be tuned via the text styling options.

If you know that your input will not match the guides, for instance with ink coming from an unstructured context such as a sheet of paper, you must disable them to ensure a good recognition.

Incremental input

Interactive Ink SDK typically processes user input in real time. You thus have to tell how pointers (pen, finger) are interacting with the capture surface (typically a screen or a graphical tablet).

This can be done by calling the following methods of the IINKEditor object:

  • pointerDown:at:force:type:pointerId:error: - When the pointer first touches the surface.
  • pointerMove:at:force:type:pointerId:error: - When the pointer moves while staying in contact with the surface.
  • pointerUp:at:force:type:pointerId:error: - When the pointer is lifted from the surface.

Each of these methods requires you to provide:

  • x and y - The coordinates of the pointer on the surface
  • t - The timestamp of the pointer event
  • f - The pressure information associated to the event (normalized between 0 and 1)

  • type - The type of pointer (pen, finger or a predefined tool like the eraser: see the IINKPointerType enum)

  • pointerId - An identifier for this pointer.

Example:

[self.editor pointerDown:CGPointMake(0.0f, 0.0f) at:(int64_t)([[NSDate date] timeIntervalSince1970] * 1000) force:.7f type:IINKPointerTypePen pointerId:1 error:nil];
[self.editor pointerMove:CGPointMake(1.2f, 2.0f) at:(int64_t)([[NSDate date] timeIntervalSince1970] * 1000) force:.6f type:IINKPointerTypePen pointerId:1 error:nil];
[self.editor pointerUp:CGPointMake(2.0f, 4.0f) at:(int64_t)([[NSDate date] timeIntervalSince1970] * 1000) force:.5f type:IINKPointerTypePen pointerId:1 error:nil];
You can call pointerCancel:error: to have the editor drop and ignore an ongoing event sequence.

Remarks:

  • The timestamp is typically the time in ms since Jan 1st, 1970. You can set it to -1 to let iink SDK generate one for you based on the current time of the system.
  • Interactive Ink SDK does not use the pressure information. It is stored in the model and can be retrieved at export or when implementing your own inking. If you don’t have or need this information, you can set it to 0.
  • If you only have one pointer simultaneously active, you can pass a pointer id of -1.

In the most simple case, you can write something like:

[self.editor pointerDown:CGPointMake(0.0f, 0.0f) at:(int64_t)-1 force:0.0f type:IINKPointerTypePen pointerId:-1 error:nil];
[self.editor pointerMove:CGPointMake(1.2f, 2.0f) at:(int64_t)-1 force:0.0f type:IINKPointerTypePen pointerId:-1 error:nil];
[self.editor pointerUp:CGPointMake(2.0f, 4.0f) at:(int64_t)-1 force:0.0f type:IINKPointerTypePen pointerId:-1 error:nil];

Series of events

In some cases, you may want to send a set of strokes to the engine in a single pass, for instance if you import ink from outside of the iink model that you want to process as a single batch.

Interactive Ink SDK provides a method to input in a single pass a series of pointer events, pointerEvents:, that take in parameter an array of IINKPointerEvent objects.

Here is an example:

// Build the pointer events array
NSInteger n = 23;
IINKPointerEvent* events = malloc(sizeof(IINKPointerEvent) * n);
IINKPointerEvent *p = events;

IINKPointerType pointerType = IINKPointerTypePen;

// Stroke 1
*p++ = IINKPointerEventMakeDown(CGPointMake(184, 124), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 125), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 133), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 152), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 158), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 163), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(183, 167), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(183, 174), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(183, 183), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeUp(CGPointMake(183, 184), -1, 0, pointerType, 0);

// Stroke 2
*p++ = IINKPointerEventMakeDown(CGPointMake(150, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(151, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(152, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(158, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(166, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(184, 126), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(190, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(196, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(200, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(207, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeMove(CGPointMake(208, 128), -1, 0, pointerType, 0);
*p++ = IINKPointerEventMakeUp(CGPointMake(209, 128), -1, 0, pointerType, 0);

// Feed the editor
[self.editor pointerEvents:events count:n doProcessGestures:NO error:nil];

free(events);
When calling pointerEvents:count:doProcessGestures:error: to process a large quantity of strokes, you should set doProcessGestures to NO to explicitly prevent gesture detection and allow better performances.

Edit and decoration gestures

Interactive Ink SDK supports all the standard gestures defined as part of Interactive Ink.

There is nothing particular to do to benefit from gestures. The SDK will take care of detecting and applying the effect of the gestures from the provided input without any plumbing needed.

Decorations can be styled and will be taken into account when generating some of the export formats (ex: text underlining will be ignored for a simple text export, will turn bold in the case of a docx export and be semantically tagged in a jiix export).

Other edit operations

The following operations can be directly made on the content of a part via an Editor object:

  • Undo
  • Redo
  • Clear

Most operations, however, are to be done on content blocks.

If you need to integrate iink SDK undo/redo with your own undo/redo stack, refer to this page (advanced)

Recognition feedback

The UI Reference Implementation comes with a “smart guide” component that lets you provide real-time text recognition feedback to your end users and allows them to select alternative interpretations from the engine.

Refer to the page describing how to work with text recognition candidates for more information.

Monitoring changes in the model

There are cases where it makes sense to be notified of what occurs in the model. For instance, you may want to update the states of the undo/redo buttons of your interface or only permit an export when there is something to export.

You can attach a delegate conforming to the IINKEditorDelegate protocol to an IINKEditor object’: the contentChanged:blockIds: method will tell you when any change occurs within the model.

IINKEditorDelegate also provides an onError: method that you can implement to be notified when anything goes wrong. It is strongly recommended to implement it, as it allows detecting some frequently seen issues such as recognition assets or configurations not found by the engine.

In addition, the IINKEditor class provides other useful methods/properties, such as:

  • idle - Returns true/YES if any ongoing processing by the engine is over
  • waitForIdle - Blocks your thread until the engine is idle. It allows waiting for recognition to be complete before exporting, converting or manipulating the content.
isIdle will always be false/NO when accessed from inside a contentChanged: notification.
To avoid deadlocks, do not call waitForIdle from inside an IINKEditorDelegate notification.

Block management

Content blocks

A content block is a semantic subdivision of the content, and may contain data and/or other blocks. It has a unique id, a defined type (“Text”, “Math”, “Diagram”, “Drawing” or “Container”) and a bounding box.

For example:

  • A “Math” part will only contain a single block, hosting the math content itself.
  • A “Text Document” part will be more complex, as it can host text paragraphs, math equations, diagrams and drawings, arranged in a complex layout, sometimes one after the other, sometimes alongside one another. This is where “Container” blocks can be used to semantically group sub-blocks together.

The following illustration shows how these different blocks relate together inside their parent parts:

Package “Text” part “Math” part “Text Document” part 0 1 2 get root block “Text” “Math” “Container”blocks “Math” “Text” get part “Diagram” Blockhierarchy Serialization

When diagram.enable-sub-blocks is set to true in the configuration, “Diagram” blocks contain sub blocks of type “Text”, “Node”, “Edge” or “Polyedge” describing the content of the diagram.

The different blocks form a hierarchy, which root can be obtained by calling the rootBlock property on the parent part. Each block, in turn, has a children property that will return its own children, if any.

It is important to note that a block hierarchy is only valid at a given point in time. For instance, in the case of a Text Document, inserting new blocks, removing a text paragraph using a gesture, etc. are some examples of events that may invalidate the block hierarchy you previously retrieved.

You can check if a block is still valid by testing its valid property. Alternatively, watching contentChanged:blockIds: events using an IINKEditorDelegate on your IINKEditor object will provide hints that your blocks may have become invalid (the list of ids of impacted blocks is provided in parameter).

Block addition

Any part you create will contain a root block.

You may however use the addBlock:type:error: method of the editor to add a new block at a given location in compatible parts, as a way to import content (only “Text Document” parts support this feature as of now).

A dedicated addImage:file:mimetype:error: method allows you to insert an image inside a Text Document part.

Operations with blocks

Some operations are possible with blocks, at a finer granularity than at the sole part level:

  • hitBlock: lets you know the top-most block at a given location, if there is any. It makes it possible for example to know which block is tapped or pressed by a user.
  • removeBlock:error: lets you remove a non-root block.
  • convert:targetState:error: lets you convert the ink inside a given block.
  • export_:mimeType:error: lets you export the content of a specific block, including its children.
  • copy:error: lets you copy a block into the internal clipboard. You can then paste it at a given location using paste:error:, much like you would add a new block. This only works on a “Text Document” part.
  • You can monitor the events affecting of a given block, as information about the impacted blocks is provided to you by the contentChanged:blockIds: method of the IINKEditorDelegate protocol.

Editor-level configuration

While iink SDK can be globally configured at engine level, it is possible to override this configuration at editor level. This is particularly useful in form-like use cases, where you need to manipulate fields with different configurations.

You can access the configuration of a specific editor via the configuration property and set the values of the keys that should override the global configuration, in a cascade-like manner. Values of the keys you do not explicitly set at editor-level still follow engine-level configuration.

For example:

IINKConfiguration *globalConfig = engine.configuration;
IINKConfiguration *editorConfig = editor.configuration;

// Global configuration values apply ...
NSString *globalUnit = [globalConfig getStringForKey:@"math.solver.angle-unit" error:nil]; // -> "deg"
NSString *editorUnit = [editorConfig getStringForKey:@"math.solver.angle-unit" error:nil]; // -> "deg"
[globalConfig setString:@"rad" forKey:@"math.solver.angle-unit" error:nil];
globalUnit = [globalConfig getStringForKey:@"math.solver.angle-unit" error:nil];           // -> "rad"
editorUnit = [editorConfig getStringForKey:@"math.solver.angle-unit", error:nil];          // -> "rad"

// ... except if overridden at editor level
[editorConfig setNumber:4 forKey:@"math.solver.fractional-part-digits" error:nil]
[globalConfig setNumber:2 forKey:@"math.solver.fractional-part-digits" error:nil]
double editorDigits = [editorConfig getNumberForKey:@"math.solver.fractional-part-digits" error:nil]; // -> 4
double globalDigits = [globalConfig getNumberForKey:@"math.solver.fractional-part-digits" error:nil]; // -> 2
Language-related settings like text.configuration.bundle and text.configuration.name behave in a particular way, in that they are only considered the very first time a given part is set to an editor and cannot be changed afterwards.

Back to the example

You previously created an IINKEditor object. As you rely on the reference rendering implementation provided by MyScript, ink input will be transparently managed by the EditorViewController. All you have to do is to load the part into the editor at the end of the openContent: method you previously wrote:

- (void)openContent
{
  // Select the package and part to use
  ...
  // Associate the part with the editor
  self.editor.part = part;
}

If you launch the app, you can now write in it and see actual ink being rendered!

Next, you will plug the undo, redo and clear buttons.

This can be conveniently done by calling the appropriate IINKEditor methods in reaction to user actions:

- (IBAction)undo:(id)sender
{
  [self.editor undo];
}

- (IBAction)redo:(id)sender
{
  [self.editorViewController.editor redo];
}

- (IBAction)clear:(id)sender
{
  [self.editor clear];
}

As a “Math” part consists of a single root block, you will be able to convert it very easily. This will be the topic of the next step of this guide!

We use cookies to ensure that we give you the best experience on our website Read the privacy policy