Assisting persons with dementia during handwashing using a partially observable Markov decision process.

  • Jesse Hoey
  • Axel von Bertoldi
  • Pascal Poupart
  • Alex Mihailidis
Schlagworte: assistive technology, human-computer interaction, video analysis, decision making, DDC: 004 (Data processing, computer science, computer systems)


This paper presents a real-time system to assist a person with dementia wash their hands. Assistance is given in the form of verbal and/or visual prompts, or through the enlistment of a human caregiver's help. The system uses only video inputs, and combines a Bayesian sequential estimation framework for tracking hands and towel, with a decision theoretic framework for computing policies of action -- specifically a partially observable Markov decision process (POMDP). A key element of the system is the ability to estimate and adapt to user states, such as awareness, responsiveness and overall dementia level. We demonstrate the system in a set of simulation experiments, and we show examples of real-time interactions with actors.
The 5th International Conference on Computer Vision Systems