The Creation of the Monolith Multi-Touch Surface Computer

The Technical Research Of Vision Based Multi Touch Solutions

Infrared light spectrum

The monolith design will utilise infra red light for the tracking of the user’s fingers across the surface of the screen.

Infra red light is a frequency range within the electromagnetic spectrum, existing just below the human spectrum of visible light. This spectrum separation is very important to the monolith design,

By using a simple infra red pass filter on a camera lens we can filter out all visible light, enabling the use of a video camera to track the infra red blobs caused by reflection or frustration of the infra red light when finger tips or another object are placed onto the screen. The IR spectrum used is completely separate and filtered from the light visible to the human eye so it is not detectable by the user and there is no contamination from the visible light projected onto the same surface.

Johnny Lee and his Wii remote research

Johnny Lee takes cutting edge technology concepts and delivers them to the masses using standard home hardware [9]. Lee’s main aim is to design software/hardware concepts that can be reproduced easily by the general public utilising public or similar open source software licences. The body of his research in this area of interest revolves around the Wii remote and its built in infra red camera. Lee states he uses this device for many reasons such as: it’s relatively cheap price for the consumer, availability and the fact that it is already designed to do the onboard processing and deliver the information to a computer over Bluetooth as a HID (human interfacing device) [9]. This is useful for the development of cheap and easy to use software as most of the development is in utilizing the X,Y,Z coordinates already converted by the Wii remote.   

Wii Remote - Interactive Whiteboard

One of the programs Lee created and was one of the inspirations for the monolith design was the interactive whiteboard program using the Wii remote. This program was designed as a system which can track up to four purpose built infra red switchable pens across a projected computer screen by following the infra red dots created by the pens. 

This system only projects a normal visual spectrum of light from a standard projector and uses a standard un-modified Wii remote so requires minimal equipment and calibration. This enables the user to set up and use the system quickly and easily with minimal related expertise.

The first multi touch prototype of the monolith was created using this method, but projected from the underside of a glass table with a light diffuser on the touch surface so the projected image could be seen by the user opposed to projecting above the user interface area.

 The infrared pen, due to the long wave characteristics of infrared, penetrated the diffuser and can be tracked by the Wii remote easily.

The system can be easily set up to a fully functional multi touch display within just a couple of hours.

The benefit of Lee’s system is that it produces an output as a mouse driver so is useable in most everyday applications but also outputs as TUIO messages so is compliant to the standard input of open source multi-touch software being developed.

Rear projected infrared light.

The rear projected infra red light method or diffused illumination (DI) as used in the Microsoft surface™ [2] is a series of infrared illuminator arrays projected and diffused onto the surface of the table. When something is placed onto the surface, it reflects the infrared light back towards a camera situated at the base of the table, and the reflected infrared light is then tracked and used as input signals to interface with the computer [2]. The major drawback to this system is that the resolution of the touch events is significantly lower than that of Jefferson Hans work. Microsoft employ a series of up to five cameras stitched together tracking the surface to gain high enough finger touch resolution.

Frustrated total internal reflection (FTIR)

Frustrated total internal reflection is a phenomenon describing the behaviour of light projected directly into a clear dense material such as Perspex or glass [1][10]. It describes light travelling through a material with a lower index of refraction (glass to air) causing total internal refraction of light within the material, trapping the light with extremely low levels of loss [1][7][10]. This phenomenon is commonly used in fibre optics for instance, but is extremely useful for acquiring touch image information at extremely high resolutions. [1][10]

The system uses a series of infra red LEDs along the edge of the acrylic causing the infra red light to be trapped within the sheet.

When the surface is touched the index of refraction is disrupted and the light escapes downwards from the point of contact, giving the computer vision hardware and software a high resolution image to track the touch location, this then shows up as an infrared blob at the touch point on the screen.

There is no limitation to the amount of simultaneous touches that can be physically triggered on the screen using this method[10], but the tracking software used may have a limit of objects it itself can simultaneously track. This is computer hardware and software design/code complexity issue and theoretically the process itself should have no limit, if the right processing power was available and the right tracking software developed.

Which Infra red wavelength?

Infra red bulbs come in a variety of different wave lengths generally 840nm (nano-meters), 860nm and 940nm.  Each of these different wave lengths have different advantages for different methods of tracking, for instance the shorter wave length of 840nm is more easily picked up by most domestic cameras, whereas the longer wave lengths can sometimes be out of the range of the same cameras range.

The advantage of a higher IR wave length is that longer waves can travel through solid objects more easily than shorter ones [11]. This is an advantage when using a second material under the main touch screen, such as a diffuser for the visual light as a longer IR wavelength has more chance of penetrating the surface and not being affected by the diffuser.

Infrared camera

Most domestic video cameras have the capability to detect and record infra red light although some need to be modified by removing a protective IR filter on the inside of the lens. This only exists on a few cameras but is easily modified. CCD cameras are usually regarded as the better type opposed to CMOS for their ability to see more of the IR scale.

The camera then needs to be filtered using an IR pass filter, filtering all light other than infrared. These kinds of filters can be bought from any standard photography store but DIY methods are extremely easy to achieve. One of the most common methods of DIY infrared pass filtering is to use exposed film stock from the end of a reel or a piece of the recordable surface of a floppy disk [10].  Exposed film mildly filters visual light so you can easily layer up the film to gain the desired strength, whereas floppy usually is harsher and only requires one layer [10].

Multi Touch driver Software

The information from the touch screen needs to be translated into a useful form for the computer to use as positional information.  There is many open source software solutions available online. Two of the most successful packages we tested on the prototype were:

• Tbeta/CCV(newer  version)  by the NUI community

• Touch lib by the NUI community

Tbeta/CCV and Touchlib as with all of the other tracking software takes the video stream from the camera and applies a series of filters to the stream, applying a series of high pass filters and background removal filters, leaving only the brightest surface touch information.