Introduction

Tangible interactions are interactions that establish a physical connection between the real-world of a user and the virtual world of a screen. For example, placing an object directly on a screen to pull up information about a specific topic. Tangible objects can have very different shapes, enabling their use across fields such as museums, restaurants, trade shows, and any other business where the use of objects can bring novelty, fun, and utility to user interaction.

NOTE: To work with tangible objects, Intuiface requires the use of Windows-based PCs and requires the hardware/middleware detecting those objects to communicate using the TUIO protocol. The only exception is if you use hardware from Interactive Scape, see the section about TUIO 2.0 at the end of this article. 

How tangible object recognition works

Intuiface does not detect or recognize tangible objects, that is the job of hardware/middleware dedicated to tangible object detection and recognition. Rather, Intuiface enables you to react to the appearance, movement, or removal of these objects. The process is as follows:

• The hardware/middleware detects one or more tangible objects and identifies them.
• Information about the detected objects is sent to a Windows PC via the TUIO protocol.
• Intuiface's TUIO Interface Asset captures the TUIO-encoded information
• Intuiface raises the appropriate trigger associated with the TUIO Interface Asset - object detected / object moved / object lost - with information about each detected object (unique identifier, X & Y location, orientation).
• As with any other trigger, you can use these TUIO triggers to call any one or more actions in your Intuiface experience.
• You can also display a collection of currently detected objects using a data template.

Third-party solutions for tangible object detection

Eyefactive

eyefactive: Intuiface partner providing tangible object recognition. Setting up the MORE Engine (Multi Object Recognition) with Intuiface is plug&play without any need for a configuration or calibration. The slim and sleek tags/markers can be attached to products or integrated into any kind of object/sculpture (conductive as well as non-conductive materials). Contact eyefactive for more information and free demos.

 

Promultis

Promultis: Intuiface partner who manufactures displays and middleware that, together, can be used to detect tangible objects. Contact Promultis for more information.

 

Interactive Scape

Interactive Scape: Intuiface partner who manufactures their own display/middleware combination to support tangible object recognition. Scape X objects are supported; use of Scape displays is required. Contact Interactive Scape for more information.

Displax

Displax: Object recognition is available out-of-the-box with their touch foils; the only separate purchase is for the tags. Below is a short video showing their object recognition tech in action. Contact Displax for information.

 

About the TUIO protocol

TUIO (Tangible User Interface Objects) is an open framework defining a common protocol and API for use with multi-touch surfaces that optionally support tangible object recognition. The TUIO protocol enables the transmission of an abstract description of both touch events and tangible object states. You can read more information about this specification on the official TUIO website.

Intuiface already supports touch events transmitted by the TUIO protocol as tuio/2Dcur events. To add support for tuio/2Dobj TUIO object events, you have to add the TUIO Interface Asset to your experience and use its properties and triggers to control your experience.

If you don't have a TUIO enabled device when building your experiences with Composer, you can use one of the TUIO Simulators listed towards the bottom of this TUIO website page. We recommend using the Java-based standard TUIO Simulator application: download / ReadMe on GitHub.

TUIO Interface Asset

NOTE: This Interface Asset is still a work in progress. It is provided as beta software in the sample below.

Properties

• TUIO Port: 3333 by default on most TUIO-enabled devices, but can be changed if needed. 
• Experience Width: Specify your experience width (default: 1920) to automatically convert the TUIO coordinates (0 to 1) to usable values in your bindings. 
• Experience Height: Specify your experience height (default: 1080) to automatically convert the TUIO coordinates (0 to 1) to usable values in your bindings.  
• Use predefined IDs: 
  • if unchecked, any object recognized will appear in the TUIO Objects data feed. 
  • if checked, only uses the objects listed in the next section. 
• Predefined IDs: if "Use predefined IDs" is checked, these are the only IDs that should be tracked in this experience. IDs should be entered as a comma-separated list. For example: 1,2,3
• X Offset: if "Use predefined IDs" is checked, this is the offset applied between each object when they are moved out of the scene (= not detected).
• Y Offset: if "Use predefined IDs" is checked, this is the offset applied to move the object out of the scene when not used (= not detected). 

Triggers

Video

Sample

You can download a test sample containing the TUIO Interface Asset by clicking this link.

 

TUIO 2.0 Protocol over Web Sockets

The majority of manufacturers use the original TUIO 1.1 protocol over UDP. Some manufacturers who want to go beyong what that protocol offers decided to implement the TUIO 2.0 protocol and deliver messages over Web Sockets instead of UDP. 

This is the case of Interactive Scape with their latest generation of tables and objects. 

A new version of the TUIO Interface Asset was created in TypeScript to handle this protocol and is available for our Player for all platforms but Windows.

It handles the same objects-related events as described above in this article and exposes the detected objects as a data source. 

You can download a sample that contains this Interface asset by clicking this link. 

Check this article to see how to reuse this Interface Asset in your own Experience.