OffScreen interactivity

Now you are aware of the most important APIs of iinkTS let's dive into OffScreen Interactivity specificities.

Integration fully based on iinkTS

GetStarted

You can see below an example fully based on iinkTS: It uses iinkTS ink capture, rendering and model. To use this integration option, all you need is an iinkTS Editor configured with offscreen enabled (See also GetStarted).

This Editor instantiates an OIBehaviors object that defines and instantiates all the objects used by the offscreen interactivity.

Menu customization

We provide a set of ready-to-use menus:

• A styling menu, which allows you to choose the content color, thickness, etc. For more details on styling parameters, see next section.
• An intention menu, which allows you to choose the intention of the event on the content, such as pen when writing or drawing some content, eraser when aiming to erase some content.
• An action menu, allowing you to choose the action to be applied to the content, such as delete, convert, ...
• A context menu that allows you to display the list of possible actions on the current selection.

All you have to do is configure whether you want to show or hide these menus:

For more details, you can also refer to the configuration reference page

Styling

You can modify the following styling properties:

• For all items (ink, text, shapes) you can set the width, color, opacity and fill style. In addition for text, you can set the fontSize and choose the fontWeight (normal or bold). This can be done with the styling menu, or directly in your code:

editor.behaviors.updateSymbolsStyle([stroke1.id], {  width: 42, color: "#808080" , fill: "#ffff00" , opacity: 50 })
editor.behaviors.updateTextFontStyle([text.id],{ fontSize: 20, fontWeight: "bold" })

• You can also adjust the canvas grid styling with the rendering.guides.* properties. For example:

        const options = {
          configuration: {
            offscreen: true,
            ...
            rendering: {
              guides: {
                type: "grid",
                gap: 120
              }
            }
          },
        };

Integration based on another canvas rendering

You can also use your own framework for ink capture, rendering and model and use the OIRecognizer to send strokes updates to our recognition Engine and receive all server updates.

We have developed a React app example based on a tldraw canvas to illustrate such a custom integration:

This example is based on the following main objects:

• A Recognizer extending OIRecognizer, which handles all communication with the recognition server.
• A Synchronizer which is responsible for intercepting canvas events and translating them to the Recognizer. It also receives gesture notifications from the Recognizer and invokes the gesture to perform associated actions.
• a Gesture which is responsible for applying the actions associated with gestures.
• a Converter which is responsible for converting handwritten content into a typed equivalent.

Please refer to this callflow for a more detailed description of these objects and how they interact with iinkTS and tldraw: