Multi-touch Technology and Software

What's special about multi-touch technology?

Multi-touch computer interaction is based on the idea of using your fingers to manipulate objects on a computer screen. It's called 'multi-touch' because unlike traditional touch screens, where you use a single finger as if it were a mouse pointer, you can use multiple fingers at the same time. This addition of more touches makes a big difference though. With multi-touch support you can start treating objects on screen more as if they were physical objects, for example by squeezing with two fingers to scale a document or an image. Supporting multiple touches also means you can share a display with multiple people, and each person can use multiple fingers too.

How does it work?

Well, the main idea is that somehow you have to communicate the position of each finger touching the screen to the software those fingers should be manipulating. Their are many techniques for getting the position of the fingers on screen, but in the Demand Evolution screens we are using a form of computer vision. With this technique an infra-red (IR) camera is watching behind the screen, and when a finger reflects some IR light by touching the IR illuminated screen it shows up on the camera as a small burst of light. The software driver, such as the NUI Touchlib, then processes this video to determine the exact X,Y coordinates of each touch. (Another typical type of touch sensor, for example as used on the iPhone, utilizes a transparent, grid-like circuit that measures changes in capacitance as you touch it. These are currently very expensive and limited to small sizes though.) These X,Y points are sent to the software just like a mouse would send its current location, and then it's up to each application to make use of these points in a meaningful way.