Recognizing human functional patterns in IADLs through hand detection and artificial intelligence

Background: Cognitive impairment is one of the characteristics shown by patients with Alzheimer's dementia (Montenegro et al., 2020; Kelley & Petersen, 2007). This affects the executive functions of most of these patients from the early stage (Guarino et al., 2019) and consequently, have difficulty in carrying out their daily activities (Dubbelman et al., 2020). These activities require precise coordination of the person's movements and interaction with their environment, so instrumental activities of daily living (IADLs) are predictors of such impairment (Barberger-Gateau & Dartigues, 1993), as several research studies have shown(Brovelli et al., 2017). Therefore, assessing the performance of IADLs by analyzing human biomechanical markers such as hand movements is of great interest. This, together with the application of artificial intelligence techniques, in particular deep learning, will allow the development of technological tools for the creation of an automated model to support the early diagnosis of Alzheimer's disease. Method: This work was inspired by the development of Alejandro Acosta et al, where human biomechanical markers were analyzed throughout the performance of IADL activities to recognize the human functional pattern (Acosta-Franco et al., 2021). To improve the accuracy of the baseline model, we propose a hand detection algorithm based on a probabilistic approach considering skin color as a descriptor. To test the proposed algorithm, we used the dataset of egocentric videos containing the performance of IADL activities, which is organized into four classes, based on the prehensile patterns of the hands: force and precision, and on the kinematics of the instruments: displacement and manipulation. Result: With the improvement made to the base model we obtained an accuracy higher than 74% in recognizing movement patterns of displacement and manipulation of objects, and a good prediction of the depth of the anatomical planes. Conclusion: Our improved model obtained a good percentage of recognition accuracy for the instrument's kinematics, which can help in the development of technological tools for the creation of an automated model to support the diagnosis of early Alzheimer's disease.