The future of individual mobility belongs to autonomous vehicles. It is particularly important that not only the driver receives information about the vehicle and the surroundings but also that the vehicle receives relevant information about the driver and the driver state. This improves the interrelations between the driver, the vehicle and the surroundings with the objective of preventing accidents caused by fatigue, a lack of vigilance and attention and of optimally supporting the driver according to his current performance prerequisites.

To enable the most comprehensive estimation of the driver state possible, the FZI Research Center for Information Technology, cooperation partner of IAV GmbH, developed a camera-based system. This system uses a camera to capture and track the driver’s face inside the vehicle and evaluates it with regard to his heart rate, blink rate, head pose and emotions. The special feature of the approach is that a standard webcam without additional technology or special lighting attached to the driver’s body is used, which allows a cost-effective application in the vehicle. The heart rate is optically recorded by analyzing changes of color values in the driver’s face depending on the pulsing of the blood.

Relevant facial areas are analyzed with innovative image and signal processing algorithms. The eye blinking detection, facial expression recognition and head pose estimation are based on the analysis of the features of a developed model of a driver’s face by means of adequately trained classifiers.

The measured parameters can now be merged and function as input for a precise determination of the driver state with regard to the level of fatigue, stress, attentiveness or excitement. Such data provides information to optimize existing driver assistance systems, such as fatigue detection, that are so far mainly based on the analysis of driving behavior. The topic is especially interesting for future driving assistance systems, such as distraction warning or a medical emergency brake assistant. On this basis, solutions for the adaptation of driver assistance systems and HMI elements both depending on the driver state or solutions for taking back control over the vehicle in a controlled manner in partly automated driving can also be developed.

At the CES fair stand, we determine your personal driver state and present the functionalities of camera-based systems as well as the measured vital parameters live.