EyeScreen: A Vision-Based Desktop Interaction System

Abstract

EyeScreen provides a natural HCI interface with vision-based hand tracking and gesture recognition techniques. Multi-view video images captured from two cameras facing a computer screen are used to track and recognize finger and hand motions. Finger tracking is achieved by skin color detection and particle filtering, and is greatly enhanced by the proposed screen background subtraction method that removes the screen images in advance. Finger click on the screen can also be detected from multi-view information. Gesture recognition based on binocular vision is presented to improve the recognition rate. The experimental results show that EyeScreen is able to perform natural and robust interaction in desktop environment.