This project has been shortlisted for the DPH 2025 Innovation Prize – Best Data Driven Innovation
Team: Satoko Ohmatsu and Kanako Nakamura (Digireha Inc.)
Outline: Conventional assessment methods in rehabilitation, special education, and disability care are predominantly manual and subjective. These approaches lack standardization, are time-consuming, and hinder the effective sharing of results among interdisciplinary teams. As the demand for individualized and continuous monitoring grows, there is an urgent need for a scalable, objective, and accessible evaluation framework.
To address this gap, we are developing a universal “Assessment DX” platform – a gamified, sensor-integrated dashboard designed to evaluate multiple functional domains across the lifespan, from children to older adults. The system harnesses multimodal sensor inputs, including PC camera-based skeletal tracking (MediaPipe), eye tracking, and inertial measurement units (IMUs), captured during structured gameplay. These data streams are automatically analyzed to generate standardized indicators aligned with conventional evaluation tools (e.g., Box and Block Test), enabling reproducible and efficient assessments without the need for specialized equipment or training.
In a pilot study involving 20 typically developing preschool children, two features extracted from a PC camera-based game – the number of successful trials and the frequency of velocity peaks in fingertip motion – were found to significantly predict motor performance on the Box and Block Test, demonstrating high test–retest reliability (ICC > 0.6). These findings support the viability of gamified, sensor-based evaluation as a digital proxy for traditional assessments.
Crucially, the platform extends beyond measurement to drive actionable behavioral change, supporting five core functions:
- Delivering clear, interpretable feedback on functional changes to caregivers and professionals.
- Recommending individualized support strategies tailored to users’ sensory and cognitive profiles.
- Guiding subsequent training and goal-setting across motor, cognitive, and ADL domains.
- Clarifying the relationship between functional capacities and everyday performance.
- Offering practical, family-friendly suggestions for home-based activities and adaptations.
The dashboard’s core features include:
- Goal-setting support, visualizing trends and progress over time.
- Behavioral insight modules, offering communication and engagement strategies based on user traits.
- Automated reporting, with intuitive charts and brief narrative summaries to enhance stakeholder understanding.
- Role-specific interfaces for families and institutions, promoting seamless collaboration and sustained engagement.
By combining gamified interaction, automated sensing, and data-driven analytics, the Assessment DX platform introduces a novel infrastructure for functional evaluation – one that is inclusive, efficient, and standardized. It reduces assessment burden while enhancing data richness, interpretability, and cross-setting applicability. This approach is especially valuable in environments with limited resources, and offers a transformative bridge between clinical precision and real-world usability.
This project was conducted as part of the SBIR Promotion Program supported by the New Energy and Industrial Technology Development Organization (NEDO), Japan.
