EyeScreen: A Vision-Based Desktop Interaction System

  • Yihua Xu
  • Jingjun Lv
  • Shanqing Li
  • Yunde Jia
Schlagworte: Vision-based interaction system, Human computer interaction, Finger tracking, Gesture recognition, DDC: 004 (Data processing, computer science, computer systems)


EyeScreen provides a natural HCI interface with vision-based hand tracking and gesture recognition techniques. Multi-view video images captured from two cameras facing a computer screen are used to track and recognize finger and hand motions. Finger tracking is achieved by skin color detection and particle filtering, and is greatly enhanced by the proposed screen background subtraction method that removes the screen images in advance. Finger click on the screen can also be detected from multi-view information. Gesture recognition based on binocular vision is presented to improve the recognition rate. The experimental results show that EyeScreen is able to perform natural and robust interaction in desktop environment.
The 5th International Conference on Computer Vision Systems