A History of Multi Touch
History of multi touch interfaces.
Multi touch human interfacing has existed in the experimental computing community since 1982 when the University of Toronto developed the first multi touch display based on finger pressure technology. By 1983 Bell labs published a comprehensive discussion of touch screen based interfaces leading on to the creation of bells labs own touch screen interface, which could manipulate images with more than one hand. The University of Toronto stopped working on hardware, moving onto software interfaces with the expectation of access to the Bell labs work.
By 1991 Pierre Wellner published a paper on his multi touch “digital desk” a device that not only supported multi finger interfacing but introduced one of the first instances of touch gestures with his pinching motions.
Windows surface™ multi touch surface computing
The concept for the Microsoft surface™ was pioneered by Steven Bathiche of Microsoft Hardware and Andy Wilson of Microsoft Research.[2] After presenting the idea to Microsoft’s chairman Bill Gates the idea was to be developed - by creating a team to work on the first prototype called the T1. This first prototype was based on a standard IKEA table with a hole cut in the centre and a piece of architect vellum used as a projector light diffuser. Eighty five prototypes were made in the development of the surface leading up to its release, with the final design completed in 2005[2].
The year after the initial concept (2002) the film director Stephen Spielberg was in consultation with Microsoft for the film “Minority Report” where he conceived a very similar futuristic device for the film based on gesture interfacing [8].
2008 was the year when Microsoft released the surface to its first retail customers the first being on April 17, 2008 at AT&T[3]
followed shortly by Harrah’s Entertainment at Rio iBar[4] and MSNBC used the surface on November 4th 2008 to map the US elections live on air.
The Microsoft surface uses the rear projected infrared (DI) method projecting diffused infrared light onto the surface of the computer.
This infra red light is then reflected back by any object that comes into contact with the surface and is imaged by a series of high resolution cameras that tracks the objects movement and a form of barcode can also be tracked to tell the computer what the object is. This information is then used to manipulate the user interface. The idea is to make the interface “transparent” to the user so they intuitively know how to interact with the computer.
Open source alternatives to multi touch surface computing
After the release of the Microsoft surface™ the open source community went into overdrive to create copycat designs, applications and driving software. The two most widely used and functional are Touchlib and Tbeta/CCV, both are open source software developments for tracking surface reflections or shadows on surface/touch screen interfaces. The two software products are in use in the majority of the multi touch screen interfaces created by the open source community. Both software output the tracked finger touches in x,y coordinates as TUIO, a form of OSC messages. These messages can be used as an input for programs which present themselves to windows as a standard HID (human interface device). This can enable single touch can be used in windows xp/vista replacing the mouse or multi touch in windows vista/7 (multiple mouse).
TUIO can also be used and translated by a java based system called FLOSC to convert and input OSC into Flash 9 upward as XML packet data. Other programming languages have libraries to do similar of use the raw TUIO data.
Research by the media research laboratory - New York University
One of the most famous and reproducible, scalable, high resolution methods of multi touch finger tracking is the work done at New York University by Jefferson Han Using the frustrated total internal reflection method (FTIR).
He wrote a paper entitled “low-cost multi-touch sensing Through Frustrated Total Internal Reflection” detailing how to create a multi touch surface based on his design. A further paper “Multi-touch interaction wall” was written showing an example of the methods scalability.
Jefferson Han (alongside Microsoft) is arguably one of the major pioneers of multi-touch surface computing in this form, not because of his hardware design, as it is revisited technology. But for his work done promoting the surface computing concept and his suggestions of where it might lead. His talk at the TED conference in 2006 where he demonstrated his multi touch surface (FTIR). This has been avidly watched on the internet by the growing DIY multi touch surface community. Arguably due to Jefferson Han, FTIR has become the most popular multi-touch finger tracking solution. FTIR seems to me to be the best of the solutions for high resolution finger tracking but has limitations in its design as nothing above the surface can be tracked or “seen” by the computer, so this method is limited to just finger tracking.