PREDICTING BEHAVIOR CHANGE IN STUDENTS WITH SPECIAL EDUCATION NEEDS USING MULTIMODAL LEARNING ANALYTICS

Abstract

The availability of educational data in novel ways and formats brings new opportunities to
students with special education needs (SEN), whose behavior and learning are highly sensitive to
their body conditions and surrounding environments. Multimodal learning analytics (MMLA)
captures learner and learning environment data in various modalities and analyses them to explain
the underlying educational insights. In this work, we applied MMLA to predict SEN students’
behavior change upon their participation in applied behavior analysis (ABA) therapies, where ABA
therapy is an intervention in special education that aims at treating behavioral problems and
fostering positive behavior changes. Here we show that by inputting multimodal educational data,
our machine learning models and deep neural network can predict SEN students’ behavior change
with optimum performance of 98% accuracy and 97% precision. We also demonstrate how
environmental, psychological, and motion sensor data can significantly improve the statistical
performance of predictive models with only traditional educational data. Our work has been applied
to the Integrated Intelligent Intervention Learning (3I Learning) System, enhancing intensive ABA
therapies for over 500 SEN students in Hong Kong and Singapore since 2020.