Hermes Javier Mora, Tomás B. Echaveguren, Esteban J. Pino
Abstract
Driver somnolence remains a major challenge for road safety, not only for its detection, but especially for forecasting when drowsiness will impair driving performance. To address this matter, various physiological signals and facial images are employed to identify signs of sleepiness. However, predicting the driver’s drowsiness condition within a few minutes earlier is more complex than classifying their current status. This study introduces a novel forecasting method based on BiLSTM (Bidirectional Long-Short-Term Memory) to predict when a driver will reach a predefined drowsiness threshold within a seven-minute window. A set of non-intrusive sensors, including force-sensing resistors (FSR) and vehicle measurements (Telemetry data), alongside physiological data (EEG, ECG, EMG), is employed to detect and forecast the upcoming drowsy events. Moreover, a combination of drowsiness detectors based on regression models and a ResNet architecture was implemented to evaluate the performance of these models. This multimodal database was collected from 30 volunteer drivers in a controlled virtual driving environment using a driving simulator in three different scenarios. The results of this study allow evaluation of whether the performance of the BiLSTM model is enhanced when compared to non-intrusive sensor data. In comparison to existing classification-based approaches, the proposed BiLSTM forecasting model demonstrated superior predictive outcomes, reducing classification error rates and improving accuracy in forecasting drowsiness events. This improvement highlights the advantage of integrating regression-based detection with time-series forecasting, thereby enhancing the reliability of driver monitoring systems. Furthermore, the best regression model achieved a test accuracy of 0.964, while the best-performing forecasting model scored 0.86 on the same metric. Notably, the entirely non-intrusive FSR alternative achieves a promising detection accuracy of 0.905. These findings demonstrate the feasibility of using time-series data, non-intrusive sensors, and a forecasting technique to predict upcoming drowsiness events, enabling a practical alternative for continuously monitoring the drowsiness status of drivers.
