Project End: March 2007

Diss. ETH No. 18075

-

FingerMouse worn on body

User input devices are essential to any interactive computer systems. For Wearable Computers, there is a need for special input devices, that are less obtrusive and thus do not hinder the user from performing another task.

In this project, we implement and evaluate the use of integrated computer vision modules (which we call “FingerMouse”) as input devices, enabling the user to simply use his free hand to interface with the Wearable Computer. Worn on the user's body, the FingerMouse measures in real time the position (and movement) of the hand, and can acquire and transmit a segmented image of the hand’s shape as a 1-bit bitmap (new version), also acting as a smart camera.

process flow

Objectives

The goal of the Fingermouse Project is to implement an integrated Wearable Vision Hardware/Software Platform and to analyze issues arising in that context.

During this task, we especially focus on

- the increase of robustness in wearable environments

- the decrease of need in computational resources

- higher integration

- analysis of performance/ power consumption trade offs

Achievements

The latest prototype uses a specifically designed ASIC, computing images at 20GOp/s.

The ASIC performs real time stereo block matching to create a depth map. In a second step, it combines depth with color information to segment the hand from the background and to track it. The system outputs the hand coordinates as well as a 1-bit image of the hand segmented from the background.

ASIC-Fingermouse (only cameras mounted)

[key specs]

system:

overall power consumption: less than 200mW PCB size: 43mm X 18mm cameras: Omnivision OV7649 low voltage CMOS VGA imager

processing rate: 5 Mpixels / sec

ASIC:

technology: umcL250, 250 nm power consumption: 78mW @80 MHz / 2.5V core / 3.3V IO

Publications

2002

Fingermouse: A Wearable Hand Tracking System, P. de la Hamette, P. Lukowicz, G. Tröster, T. Svoboda, UbiComp 2002: Adjunct Proceedings of the 4th International Conference on Ubiquitous Computing, September 2002, pages 15-16. [link]

Poster: Fingermouse: A Wearable Hand Tracking System, UbiComp 2002, [link]

2004

Laser Triangulation as a means of robust Visual Input for Wearable Computers, P. de la Hamette, M. von Waldkirch, G. Tröster, ISWC 2004: Proceedings of the Doctoral Colloquium, pages 18-20, Arlington VA, Oct. 31-Nov. 3, 2004. [link]

Poster: Hand Input with Laser Triangulation, ISWC 2004 , [link]

2006

FingerMouse – A Button Size Visual Hand Tracking and Segmentation Device, P. de la Hamette, G. Tröster, ARCS 2006: Architecture of Computing Systems - ARCS 2006: 19th International Conference, Frankfurt/Main, Germany, March 13-16, 2006. Proceedings, pages 31-41. [link]

Poster: FingerMouse – A Button Size Visual Hand Tracking and Segmentation Device, ARCS 2006, [link]

FingerMouse – Architecture of an ASIC-based Mobile Stereovision Smart Camera, P. de la Hamette, G. Tröster, ISWC 2006 Proceedings of the 10th IEEE International Symposium on Wearable Computers, Montreux, Switzerland, 11.-14. October 2006 [link]

Poster: FingerMouse – Architecture of an ASIC-based Mobile Stereovision Smart Camera, P. de la Hamette, G. Tröster, ISWC 2006

2007

Architecture and applications of the FingerMouse: a smart stereo camera for wearable computing HCI, P. de la Hamette, G. Tröster, Journal of Personal and Ubiquitous Computing Journal, Springer London, ISSN: 1617-4909 (Print) 1617-4917 (Online) [link]