by Klony Lieberman

Lumio Inc., 5 Nahum Hefzadi st. Beit Ofer, Jerusalem, Israel

A novel, edge emitting plastic optical fiber element has been developed to efficiently generate thin, planar light fields adjacent to a touch surface. This fiber element, referred to as an Active-Barrier^TM is placed at the edge of the touch surface along three sides, illuminating a thin layer just above the surface. High speed, high resolution linear optical sensors are placed along the fourth side of the touch surface to sense interactions with the light field. Both the illumination fibers and the sensors are embedded in a thin frame, a few millimeters high, which can be bonded directly to a display glass or table top. The optical efficiency of the Active-Barrier^TM fiber enables the sensing frame to be scaled up to cover very large interaction regions. Multi-touch operation is also supported by incorporation of redundant sensors

and/or detection zone separation.

1. Introduction

Optical imaging is the method of choice for touch activating medium and large scale surfaces. This is primarily a result of the scalability of optical techniques when compared to physical overlays whose cost grows exponentially as the size of the interaction area increases as a consequence of, among other things, yield issues. Additional benefits include the fact that as a completely digital solution optical imaging is free from the drift and aging that plague resistive and capacitive technologies so systems never needs to be recalibrated. A fast sensor response time minimizes latency and a high frame rate coupled with high optical resolution provides the performance required for high end sensing applications such as graphic stylus input and jitter free cursor control.

Optical sensing techniques can be divided into two categories which depending on the geometry of the detection. In-plane sensing techniques use illumination and sensing components that lie, as the name suggests, within the touch sensing plane - i.e. just above the sensing surface. Out-of-plane sensing techniques on the other hand rely on two dimensional imaging sensors that view the sensing plane from above or below the interaction surface. Out-of-plane techniques have the advantage that they inherently support multi-touch sensing since each location on the image sensor corresponds directly to a unique location on the interaction surface but are limited in their usefulness due to the geometrical viewing constraints requiring large volumes to be freed up on one side of the surface to obtain an unobstructed view of the entire sensing region. The 2D sensors are also more expensive for a given resolution and typically do not support high frame rates. In-plane sensing on the other hand can be embedded in a thin frame that surrounds the interaction region but requires more sophisticated algorithmic and hardware support in order to track more than one event at a time. The linear sensors used for in-plane sensing can also provide much higher resolutions and frame rates. Both techniques scale economically to very large sizes.

In this paper we describe the underlying technology that enables Lumio's in-plane optical touch frame and the techniques that are employed to support multi-touch interaction.

2. Triangulation bases sensing

The basic construction of Lumio's sensing frame comprises two high resolution linear optical imaging modules located along one edge of the sensing region and an optically emitting active barrier located around the other edges as shown below in Figure 1. All of the components can be mounted either on a self contained frame that surrounds the sensing region or directly onto the glass plate that defines the front surface of a display, providing high resolution optical sensing with no wear and no degradation of display quality. The barrier floods the region in front of the interaction surface with a thin layer of light that is directed towards the sensors from all directions providing a uniform reference signal. When an object is inserted into this plane the imaging sensors will record a decrease in the light from a particular direction and the location is determined by straightforward triangulation. Since the sensing relies on blocking of the reference light emitted by the barrier it will respond to any object under just about any conceivable illumination.

Figure 1 Schematic illustration of the optical touch screen plate showing the location of the sensor modules, active-barrier^TM fiber and infra-red illumination light emitting diodes.

3. Optical barrier construction

The active barrier is comprised of a customized edge-emitting plastic optical fiber. The fiber is designed with a patent pending structure and has a non-circular cross section that is optimized to extract light along the length of the fiber from one edge only and focus it into a parallel beam. Infra-red light from LEDs is coupled into the fiber from either end to illuminate the entire length of the fiber which then emits a uniform, thin planar beam of light that completely fills the sensing region. The optical efficiency of the active barrier technique, in which the LED emission is almost entirely re-directed into the sensing plane, enables the frames to be scaled up to very large dimensions without requiring multiple discrete illuminators placed around the edge.

Figure 2. 19 inch glass plate with 2mm active-barrier^TM fiber mounted the edges. The plastic fiber can be bonded directly to the glass plate for a very low profile optical sensing solution.

It is the position of this illumination along with the sensing modules with respect to the screen that determines the actual location of the sensing plane. For small sensing regions fibers as thin as 1mm may be used. For large screens, white board and tabletop applications larger fibers with 2 or 3 mm diameters would typically be used. In either case a finger must be brought essentially into contact with the screen in order to activate the touch sensor just as with ordinary touch screen technologies

4. Optical sensing modules

The optical sensing modules are based on high resolution linear CMOS images sensors. A typical sensing implementation would use two 2048 pixel sensors at adjacent corners of the screen to provide over 4 million discrete sensing points. Sub-pixel interpolation can then increase the actual resolution further. While the actual resolution will vary over the sensing field this is sufficient to provide sub millimeter resolution over the largest display systems.

A typical sensing module consists of a linear sensor die, focusing lens, IR filter and aperture stop packaged into a 3-4mm high package as shown in Figure 3 below.

Figure 3. Schematic illustration of complete linear sensor imaging module assembly.

A sensing module designed for corner operation in a rectangular sensing region requires a 90 degree field of view in order to cover the entire region as shown schematically below. Careful lens design and proper placement of the aperture stop are important in order to obtain a working depth of field from less than 10 to more than 1000 mm while providing sufficient signal for a rapid response.

Figure 4. Sensor module optical schematic showing the linear sensor die, imaging lens and field of view.

5. Touch screen operation

When a finger or other object touches the glass surface it blocks the light from the barrier that would otherwise reach the sensor from that direction. This results in a change in the signal on the CMOS sensor at a particular location. A sophisticated algorithm is employed to process the data, filter the signal and subtract background interference. This processing enables robust operation under any ambient illumination. A representative image of the sensor signal from two sensor modules is shown superimposed in Figure 5 below. The location of each peak along the linear sensor corresponds to the direction of the object and the absolute location of the object is then calculated by the control board via straightforward triangulation and output with standard touch screen protocols.

Figure 5. Output of the left sensor module (red) and right sensor module (green) for an object activating the touch screen.

6. Large touch screen assembly

An example of a large, 47 inch LCD screen onto which an optical touch frame has been attached is shown below in Figure 9. Three barrier segments, each with small IR LEDs coupled into both ends are placed on three sides of the screen and illuminate the entire field. Two small image sensor modules are mounted in the corners of the fourth side to complete the sensing system. The entire opto-mechanical assembly protrudes about 4 millimeters above the surface of the screen and is small enough to allow retrofitting into existing display systems. A an even larger glass plate with a 67 inch diagonal measurement onto which a touch frame has been mounted is shown in Figure 10.

Figure 6. Photograph of a disassembled 47" LCD screen showing the placement of the sensor modules and barrier illuminators directly on the LCD frame.

Figure 7. Photograph of a 67" glass plate with barrier illuminator and sensor modules