Browsing by Author "Bernardino, Alexandre"
Now showing 1 - 8 of 8
Results Per Page
Sort Options
- An exploratory study on techniques for quantitative assessment of stroke rehabilitation exercisesPublication . Lee, Min Hun; Siewiorek, Daniel P.; Smailagic, Asim; Bernardino, Alexandre; Bermúdez i Badia, SergiTechnology-assisted systems to monitor and assess rehabilitation exercises have an opportunity of enhancing rehabilitation practices by automatically collecting patient’s quantitative performance data. However, even if a complex algorithm (e.g. Neural Network) is applied, it is still challenging to develop such a system due to pa tients with various physical conditions. The system with a complex algorithm is limited to be a black-box system that cannot provide explanations on its predictions. To address these challenges, this paper presents a hybrid model that integrates a machine learn ing (ML) model with a rule-based (RB) model as an explainable artificial intelligence (AI) technique for quantitative assessment of stroke rehabilitation exercises. For evaluation, we collected thera pist’s knowledge on assessment as 15 rules from interviews with therapists and the dataset of three upper-limb stroke rehabilitation exercises from 15 post-stroke and 11 healthy subjects using a Kinect sensor. Experimental results show that a hybrid model can achieve comparable performance with a ML model using Neural Network, but also provide explanations on a model prediction with a RB model. The results indicate the potential of a hybrid model as an explainable AI technique to support the interpretation of a model and fine-tune a model with user-specific rules for personalization.
- Augmented Human Assistance (AHA)Publication . Bermúdez i Badia, Sergi; Odekerken-Schröder, Gaby; Mahr, Dominik; Čaić, Martina; Lee, Min Hun; Siewiorek, Dan; Smailagic, Assim; Gamboa, Hugo; Belo, David; Carnide, Maria Filomena Araújo da Costa Cruz; Baptista, Maria de Fátima Marcelina; Simão, Hugo; Avelino, João; Sousa, Honorato; Paulino, Teresa; Gonçalves, Afonso; Cardona, John Muñoz; Cameirão, Mónica S.; Bernardino, Alexandre; Gouveia, Élvio RúbioAging and sedentarism are two main challenges for social and health systems in modern societies. To face these challenges a new generation of ICT based solutions is being developed to promote active aging, prevent sedentarism and find new tools to support the large populations of patients that suffer chronic conditions as result of aging. Such solutions have the potential to transform healthcare by optimizing resource allocation, reducing costs, improving diagno ses and enabling novel therapies, thus increasing quality of life. The primary goal of the “AHA: Augmented Human Assistance” project is to de velop novel assistive technologies to promote exercise among the elderly and patients of motor disabilities. For exercise programs to be effective, it is essential that users and patients comply with the prescribed schedule and perform the ex ercises following established protocols. Until now this has been achieved by hu man monitoring in rehabilitation and therapy session, where the clinicians or therapists permanently accompany users or patient. In many cases, exercises are prescribed for home performance, in which case it is not possible to validate their execution. In this context, the AHA project is an integrative and cross-discipli nary approach of 4 Portuguese universities, the CMU, and 2 Portuguese industry partners, that combines innovation and fundamental research in the areas of hu man-computer interaction, robotics, serious games and physiological computing (see partner list in Appendix A). In the project, we capitalize on recent innova tions and aim at enriching the capabilities and range of application of assistive devices via the combination of (1) assistive robotics; (2) technologies that use well-understood motivational techniques to induce people to do their exercises in the first place, and to do them correctly and completely; (3) tailored and relevant guidance in regard to health care and social support and activities; and (4) tech nologies to self-monitoring and sharing of progress with health-care provider enabling clinicians to fine-tune the exercise regimen to suit the participant’s ac tual progress. We highlight the development of a set of exergames (serious games controlled by the movement of the user’s body limbs) specifically designed for the needs of the target population according to best practices in sports and human kinetics sciences. The games can be adapted to the limitations of the users (e.g. to play in a sitting position) so a large fraction of the population can benefit from them. The games can be executed with biofeedback provided from wearable sensors, to pro duce more controlled exercise benefits. The games can be played in multi-user settings, either in cooperative or competitive mode, to promote the social rela tions among players. The games contain regional motives to trigger memories from the past and other gamification techniques that keep the users involved in the exercise program. The games are projected in the environment through aug mented reality techniques that create a more immersive and engaging experience than conventional displays. Virtual coach techniques are able to monitor the cor rectness of the exercise and provide immediate guidance to the user, as well as providing reports for therapists. A socially assistive robot can play the role of the coach and provide an additional socio-cognitive dimension to the experience to complement the role of the therapist. A web service that records the users’ per formances and allows the authorized therapists to access and configure the exer cise program provides a valuable management tool for caregivers and clinical staff. It can also provide a social network for players, increasing adherence to the therapies. We have performed several end-user studies that validate the proposed ap proaches. Together, or in isolation, these solutions provide users, caregivers, health professionals and institutions, valuable tools for health promotion, disease monitoring and prevention.
- Co-design and evaluation of an intelligent decision support system for stroke rehabilitation assessmentPublication . Lee, Min Hun; Siewiorek, Daniel P.; Smailagic, Asim; Bernardino, Alexandre; Bermúdez i Badia, SergiClinical decision support systems have the potential to improve work flows of experts in practice (e.g. therapist’s evidence-based rehabilitation assessment). However, the adoption of these systems is challenging, and the gains of these systems have not fully demonstrated yet. In this paper, we identified the needs of therapists to assess patient’s functional abilities (e.g. alternative perspectives with quantitative information on patient’s exercise motions). As a result, we co-designed and developed an intelligent decision support system that automatically identifies salient features of assessment using reinforcement learning to assess the quality of motion and generate patient-specific analysis. We evaluated this system with seven therapists using the dataset from 15 patients performing three exercises. The results show that therapists have higher usage intent on our system than a traditional system without patient-specific analysis (𝑝 < 0.05). While presenting richer information (𝑝 < 0.10), our system significantly reduces therapists’ effort on assessment (𝑝 < 0.10) and improves their agreement on assessment from 0.66 to 0.71 F1-scores (𝑝 < 0.01). This work discusses the importance of human centered design and development of a machine learning-based decision support system that presents contextually relevant information and salient explanations on its prediction for better adoption in practice.
- A dataset for the automatic assessment of functional senior fitness tests using kinect and physiological sensorsPublication . Bernardino, Alexandre; Vismara, Christian; Bermúdez i Badia, Sergi; Gouveia, Élvio; Baptista, Fátima; Carnide, Filomena; Oom, Simão; Gamboa, HugoThis work presents a dataset of functional fitness tests acquired with Kinect v2 and physiological sensors. The dataset contains both young and senior subjects executing a number of fitness tests meeting scientific standards of reliability and validity. The main objective is the ability to assess lower body strength, endurance, gait speed, agility and balance from the data obtained from commercially accessible devices. The dataset can be used to develop algorithms to automate the assessment of fitness levels in low-cost computer based systems for use at home, gymnasiums or care centers.
- Eye gaze patterns after stroke: correlates of a VR action execution and observation taskPublication . Alves, Júlio; Vourvopoulos, Athanasios; Bernardino, Alexandre; Bermúdez i Badia, SergiThe concept of a partially shared neural circuitry between action observation and action execution in healthy participants has been demonstrated through a number of studies. However, little research has been done in this regard utilizing eye movement metrics in rehabilitation contexts.In this study we approach action observation and action execution by combining a virtual environment and eye tracking technology. Participants consisted of stroke survivors, and were required to perform a simple reachand-grab and place-and-release task with both their paretic and non-paretic arm. Results showed congruency in gaze metrics between action execution and action observation, for distribution and duration of gaze events. Furthermore, in action observation, longer smooth pursuit segments were detected when observing the representation of the paretic arm, thus providing evidence that the affected circuitry may be activated during observation of the simulated action. These results can lead to novel rehabilitation methods using virtual reality technology.
- A human-ai collaborative approach for clinical decision making on rehabilitation assessmentPublication . Lee, Min Hun; Siewiorek, Daniel P. P.; Smailagic, Asim; Bernardino, Alexandre; Bermúdez i Badia, SergiAdvances in artificial intelligence (AI) have made it increasingly applicable to supplement expert’s decision-making in the form of a decision support system on various tasks. For instance, an AI-based system can provide therapists quantitative analysis on patient’s status to improve practices of rehabilitation assessment. However, there is limited knowledge on the potential of these systems. In this paper, we present the development and evaluation of an interactive AI-based system that supports collaborative decision making with therapists for rehabilitation assessment. This system automatically identifies salient features of assessment to generate patient-specific analysis for therapists, and tunes with their feedback. In two evalu ations with therapists, we found that our system supports thera pists significantly higher agreement on assessment (0.71 average F1-score) than a traditional system without analysis (0.66 average F1-score, p < 0.05). After tuning with therapist’s feedback, our sys tem significantly improves its performance from 0.8377 to 0.9116 average F1-scores (p < 0.01). This work discusses the potential of a human-AI collaborative system to support more accurate decision making while learning from each other’s strengths
- Interactive hybrid approach to combine machine and human intelligence for personalized rehabilitation assessmentPublication . Lee, Min Hun; Siewiorek, Daniel P.; Smailagic, Asim; Bernardino, Alexandre; Bermúdez i Badia, SergiAutomated assessment of rehabilitation exercises using machine learning has a potential to improve current rehabilitation practices. However, it is challenging to completely replicate therapist’s deci sion making on the assessment of patients with various physical conditions. This paper describes an interactive machine learning approach that iteratively integrates a data-driven model with ex pert’s knowledge to assess the quality of rehabilitation exercises. Among a large set of kinematic features of the exercise motions, our approach identifies the most salient features for assessment using reinforcement learning and generates a user-specific analysis to elicit feature relevance from a therapist for personalized rehabilita tion assessment. While accommodating therapist’s feedback on fea ture relevance, our approach can tune a generic assessment model into a personalized model. Specifically, our approach improves performance to predict assessment from 0.8279 to 0.9116 average F1-scores of three upper-limb rehabilitation exercises (𝑝 < 0.01). Our work demonstrates that machine learning models with feature selection can generate kinematic feature-based analysis as expla nations on predictions of a model to elicit expert’s knowledge of assessment, and how machine learning models can augment with expert’s knowledge for personalized rehabilitation assessment.
- Towards personalized interaction and corrective feedback of a socially assistive robot for post-stroke rehabilitation therapyPublication . Lee, Min Hun; Siewiorek, Daniel P.; Smailagic, Asim; Bernardino, Alexandre; Bermúdez i Badia, SergiA robotic exercise coaching system requires the capability of automatically assessing a patient’s exercise to in teract with a patient and generate corrective feedback. However, even if patients have various physical conditions, most prior work on robotic exercise coaching systems has utilized generic, pre-defined feedback. This paper presents an interactive approach that combines machine learning and rule-based models to automatically assess a patient’s rehabilitation exercise and tunes with patient’s data to generate personalized corrective feedback. To generate feedback when an erroneous motion occurs, our approach applies an ensemble voting method that leverages predictions from multiple frames for frame-level assessment. According to the evaluation with the dataset of three stroke rehabilitation exercises from 15 post-stroke subjects, our interactive approach with an ensemble voting method supports more accurate frame level assessment (p < 0.01), but also can be tuned with held-out user’s unaffected motions to significantly improve the perfor mance of assessment from 0.7447 to 0.8235 average F1-scores over all exercises (p < 0.01). This paper discusses the value of an interactive approach with an ensemble voting method for personalized interaction of a robotic exercise coaching system.