Gesture sensing and recognition is a growing market, driven by technology advancements and expanding applications, especially within the Internet of Things (IoT) movement. Markets where gesture recognition is becoming particularly prominent include portable and wearable devices in the gaming, healthcare, automation, automotive and consumer electronics sectors.
Figure 1: Touchless gesture control has arrived in the automotive infotainment market, with the latest iDrive launch from BMW. Advances in sensors and control circuitry are making the same technology accessible to designers of wearable devices.
In its simplest form, a gesture sensing circuit might include IR LEDs, IR photodiodes, ambient light sensors, and proximity sensors. However, to help designers reduce their time to market, more highly integrated devices, subsystems and modules are emerging, incorporating multiple optical sensors and control circuitry.
This article will consider one such recently launched solution, showing the ams TMG4903 optical sensing module. The associated evaluation board, TMG4903 EVM-ND helps designers incorporate gesture recognition into their next-generation products swiftly and efficiently, thereby enabling them to put their product on the market faster and with a lower bill of material (BOM) cost compared to building functionality from discrete devices.
For engineers and hobbyists wanting to explore gesture recognition at a simpler level, the SparkFun SEN12787 gesture sensor breakout board, based on the Avago APDS9960 provides a low-cost start point.
Market growth
According to market research company, Global Industry Analysts1, the worldwide market for gesture recognition is forecast to exceed US $12.7 billion by 2020. Demand is fuelled by the need for simple, intuitive interfaces to control ever increasing functionality in handheld and wearable devices, such as smartphones, health monitors and fitness gadgets. Dramatic improvements in sensor technology for portable and wearable devices are also driving the search for new human machine interfaces (HMI). Further growth will come from innovative new developments in areas including 3D vision, eye tracking and machine control.
Interpreted by mathematical algorithms, gesture recognition and sensing enable more creative, natural, and intuitive methods to communicate with electronic equipment today. Gesture sensing in wearable devices, for example, eliminates the need to design in buttons. The technology can be exploited to allow people to express meaningful commands using full body movements, hand motions and/or facial expressions.
Components such as sensors and control circuitry for wearables need to be tough, small and use very little power in order to conserve battery life. Recent improvements in accuracy, speed and simplicity of design have raised quality levels, making wearable devices an increasingly viable solution for a wider range of medical applications. First generation consumer wearable devices, such as smart watches and fitness monitors, were successful for their novelty value, but these are now benefiting from more accurate and reliable sensors. Thus gesture recognition is seen as the next generation product differentiator.
Design challenge
European-headquartered ams is a global leader in the development of smart sensors such as Infrared LEDs, combined with integrated control circuitry. The company has produced a useful and explanatory Technical Article aimed at designers tackling the wearables marketREF2. The article explains that, typically, a gesture is recognized by analyzing the IR light reflected back to the photodiode when the user’s hand passes over the LEDs.
The design can be challenging however. Normally a wearable device will be exposed to ambient light (such as sunlight), which might include some IR luminance. In addition, the sensor’s aperture on the face of the device may be obscured by contaminants such as sweat, dust or dirt. Very sensitive photodiodes are required to distinguish reliably between IR light attributable to ambient light (which is background noise in this application) and IR light reflected from the LEDs. As a result, advanced analog semiconductor technology is also required.
Typically, in operation, a gesture recognition system regularly wakes up to scan for gestures. In order to minimize battery use, power down and sleep modes as well as low power wake-up procedures are needed, and optimally, without necessitating the operation of the IR LEDs.
With appropriate hardware in place, the next step is to generate gesture control application software to interpret the raw measurements of IR light. The software needs to calculate the velocity and direction of movement of the user’s hand, calculate the distance of the hand from the device, distinguish between ambient light and light reflected from the IR LEDs, and then interpret the movements in order to recognize distinct gestures. According to ams, the quality of the system, that is its ability to recognize gestures quickly and predictably, is dependent as much on the application software as on the sensor hardware.
Manufacturers of wearable devices are increasingly looking for ready-integrated sensor systems rather than discrete sensor and processing components. Such an optimized system can save the device manufacturer design time and effort as well as delivering the high performance and reliability demanded by consumers in the next generation of wearable electronics devices. Key performance benefits of an integrated approach include: optical crosstalk that is more easily modeled and controlled (predictable); power variability that can be more reliably calibrated; an optimized combination of IR LED, light sensor and driver; and tighter physical tolerances of the module.
Module vs discrete
Taking, as an example, the TMG4903 from ams, this highly integrated optical sensing module incorporates an IR LED and advanced LED driver, ambient light sensor and a color diode array. UV/IR blocking filters and parallel ADCs produce simultaneous 16-bit results, while the IR beam pattern generator comes with RAM and specialized control logic. At its heart is a gesture and proximity engine that enables true 3D gesture recognition, enhancing simple north-south, west-east 2D gesture sensing. Thus, the module delivers multiple advanced optical sensing functions, including universal remote control, barcode emulation, and RGB color sensing, as well as proximity and 3D gesture detection.
Meeting the demands for small size and low power, the module is contained in a multi-module package measuring just 2 x 5 x 1 mm, the voltage supply requirements are 1.7 to 2 V, and it consumes typically 150 µA (200 µA max) in its active state, 30 µA (60 µA max) when idle, and just 0.4 µA (5 µA) in sleep mode.
Figure 2: Block diagram of the TMG4903 gesture and proximity sensing engine from ams, showing the integration of the various functional modules.
For 3D gesture recognition and proximity sensing, the module features automatic adjustment of the IR LED’s timing and power output, thereby minimizing noise and power consumption while optimizing sensitivity and dynamic range. Importantly, the module is capable of rejecting ambient light noise while automatic calibration eliminates electrical noise and optical crosstalk. The integration of an LED driver ensures that the engine is dedicated to gesture recognition, freeing up the software of the host device and reducing processing requirements.
Getting started
To get started, the TMG4903 EVM evaluation module is comprised of a main controller board featuring the gesture recognition device and a PIC microcontroller, with industry standard USB2.0 interface and cable. The simple, intuitive software allows the designer to set up the logging and control functions of the key features, such as the ambient light sensor, proximity sensor and allied gesture recognition. A comprehensive user guide steps designers through the process to set up parameters such as length of pulses in a gesture cycle, number of pulses in a cycle and frequency of gesture cycles.
In operation, raw gesture data is captured by the four directional photodiodes and presented on a 3D grid together with Z data taken from the proximity sensor. In addition to detecting and identifying moving gestures, ‘long’ gestures can also be recognized, simulating a button press, for example.
Figure 3: ams evaluation module for the TMB4903 gesture and proximity engine.
Unleash the inner inventor
For designers and hobbyists looking for a low-cost introduction to gesture sensing and recognition, the SparkFun SEN12787 RGB and Gesture Sensor breakout board provides the wherewithal to investigate ambient light and color measuring, proximity detection and touchless gesture sensing. It is based on the Avago APDS9960, an integrated module incorporating filters and diodes as well as an I2C interface.
The device itself measures 3.94 x 2.36 x 1.35 mm and incorporates an IR LED and factory calibrated LED driver. Gesture and proximity detection is based on four directional photodiodes, plus data from the ambient light sensor. Operational features of the engine include automatic activation, ambient light subtraction and crosstalk cancellation. Power consumption and noise are minimized with adjustable IR LED timing. The recommended power supply voltage range is from 2.4 to 3.6 V. At 3 V, supply current figures range from 1 µA (sleep state), 38 µA (wait state), through 200 µA (active) to 790 µA (gesture and proximity pulsing).
The development board has broken out a number of pins for convenience, including: VL (optional power to IR LED), GND, VCC, SDA (I2C data), SCL (I2C clock) and INT (interrupt). The module can be used to demonstrate gesture recognition for controlling a microcontroller-based device as well as a computer or robot.
Summary
Touchless gesture sensing and recognition is a growing market, and solutions are evolving fast, with new devices launched on a regular basis. It is likely to become the next ‘must-have’ HMI feature for very small battery-operated portable and wearable devices, allowing them to be reduced in size and run at lower power, while providing easy operation.
Design can be challenging, however. This article has considered how two topical solutions can give engineers a head start: a highly integrated, high performance module from ams, and a low-cost starter board from SparkFun.
References:
- Market Report, Global Industry Analysts, Gesture Recognition: A Global Strategic Business Report
- Technical Article: ams: Why wearable electronics devices call for a new generation of highly integrated, smart sensor solutions
