Journal Article

Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection

Liu, Weihuang, Pan, Jiahui, Qian, Jinhao, Jiao, Xintao, Yao, Zengwei

2019

Overview

Road traffic accidents caused by fatigue driving are common causes of human casualties. In this paper, we present a driver fatigue detection algorithm using two-stream network models with multi-facial features. The algorithm consists of four parts: (1) Positioning mouth and eye with multi-task cascaded convolutional neural networks (MTCNNs). (2) Extracting the static features from a partial facial image. (3) Extracting the dynamic features from a partial facial optical flow. (4) Combining both static and dynamic features using a two-stream neural network to make the classification. The main contribution of this paper is the combination of a two-stream network and multi-facial features for driver fatigue detection. Two-stream networks can combine static and dynamic image information, while partial facial images as network inputs can focus on fatigue-related information, which brings better performance. Moreover, we applied gamma correction to enhance image contrast, which can help our method achieve better results, noted by an increased accuracy of 2% in night environments. Finally, an accuracy of 97.06% was achieved on the National Tsing Hua University Driver Drowsiness Detection (NTHU-DDD) dataset.

• Publisher: MDPI AG
• Subject: Accident prevention / Accuracy / Algorithms / Artificial neural networks / Casualties / Classification / Computer vision / Driver fatigue / Drivers / fatigue detection / Feature extraction / feature fusion / gamma correction / Image contrast / Image enhancement / Internet / Localization / Methods / Mouth / multi-task cascaded convolutional networks / Neural networks / optical flow / Optical flow (image analysis) / Researchers / Traffic accidents / Traffic accidents & safety