How cars are getting safer with interior cameras and AI

Patrick Laufer is specialized in artificial intelligence (AI) and develops systems that increase safety in cars for all occupants. Common restraint systems – such as seat belts and airbags, which secure vehicle occupants in their seats in the event of an accident and protect them from injury in the event of a collision – are designed for a man who is 1.75 m tall and weighs 78 kg. Therefore, they only provide optimal protection for people who meet this standard.
To fix this, the engineer has developed a method that uses an interior camera, originally installed in the vehicle for drowsiness and distraction detection of the person driving, to identify the gender and estimate the height and weight of all vehicle occupants. This will allow future restraint systems to be personalized and optimize safety for each person in the car. “I always call it the democratization of restraint systems,” Laufer says.

He works with a so-called neural network (CNN) that processes images and looks for patterns in the pixels to infer certain characteristics, such as gender. The system currently employs two strategies: First, it uses a segmentation method to generate person silhouettes, which additionally include the visible points of articulation of the upper body and the prominent facial points of the persons, and feeds them into a CNN. Second, the face images are directly fed into the CNN. The face is particularly well suited for estimating in-vehicle occupant features because, unlike the upper body, it is mostly uncovered.

In order to evaluate the models for scenarios inside the vehicle, Laufer and his team applied them to test subjects sitting in the car and captured by a vehicle interior camera. This allows them to evaluate the performance of the various deep-learning models inside the vehicle.

The video shows a prototype of how an interior camera segments vehicle occupants in practice and generates skeletal models in the form of dots, thus estimating occupant characteristics for each individual subject. The system recognizes three people and their position in the car. If a dangerous situation occurs, the vehicle would detect if a person is in a non-upright position (out-of-position) and independently decide whether to deploy an airbag and if so, which one.
It is particularly important that the forces are perfectly adjusted. This works best if the weight and seat height of the vehicle occupants are known. The method Laufer has developed should be able to provide this in the future. The result is an accurate picture of each person in the car, which can be used to optimize and significantly increase safety.

“Gender definitely plays an important role in vehicle safety. Women differ from men particularly in their lower bone density and muscle mass, Laufer explains. That is a disadvantage in the event of an accident, he says. In addition, women often have longer torsos and shorter legs, which results in different leverage ratios in an accident, he adds. “Our algorithm currently has a 98% probability of correctly guessing a person’s gender by face.”

At the InCabin fair in Phoenix (March 15-17, 2023), Patrick Laufer will speak and present part of his research, “Body height an weight estimation of vehicle occupants.”
The Autosens trade fair is an annual international conference focused on technologies and applications related to automotive sensing. Topics featured at the conference include sensor development for driver assistance systems, autonomous driving and vehicle interior monitoring. It also provides a platform for industry experts to present and discuss their latest research findings and developments.

For further information please contact Patrick Laufer on LinkedIn.