Multimodal Feature-Assisted Continuous Driver Behavior Analysis and Solving for Edge-Enabled Internet of Connected Vehicles Using Deep Learning

Omar Aboulola, Mashael Khayyat, Basma Al-Harbi, Mohammed Saleh Ali Muthanna, Ammar Muthanna, Heba Fasihuddin, Majid H. Alsulami

The emerging technology of internet of connected vehicles (IoCV) introduced many new solutions for accident prevention and traffic safety by monitoring the behavior of drivers. In addition, monitoring drivers’ behavior to reduce accidents has attracted considerable attention from industry and academic researchers in recent years. However, there are still many issues that have not been addressed due to the lack of feature extraction. To this end, in this paper, we propose the multimodal driver analysis internet of connected vehicles (MODAL-IoCV) approach for analyzing drivers’ behavior using a deep learning method. This approach includes three consecutive phases. In the first phase, the hidden Markov model (HMM) is proposed to predict vehicle motion and lane changes. In the second phase, SqueezeNet is proposed to perform feature extraction from these classes. Lastly, in the final phase, tri-agent-based soft actor critic (TA-SAC) is proposed for recommendation and route planning, in which each driver is precisely handled by an edge node for personalized assistance. Finally, detailed experimental results prove that our proposed MODAL-IoCV method can achieve high performance in terms of latency, accuracy, false alarm rate, and motion prediction error compared to existing works.