Vehicle In-Cabin Monitoring

Inexpensive components and the trend to automated driving mean that cameras are being increasingly used in passenger compartments. They enhance passenger comfort and safety, generating highly coveted data for numerous disciplines. IAV has implemented initial pilot projects and supports its customers along the entire development chain.

Person or parcel? Current systems for seat occupancy detection cannot necessarily make the right distinction in borderline cases. A simple pressure sensor cannot tell whether the weight on the seat comes from a person or an inanimate object. Cameras do not have the same problems: they easily see the difference so that the seat occupancy detection function doesn’t tell the parcel to fasten its seat belt.

And that’s not all. In future, image detection can make the vehicle aware when the driver starts to show signs of fatigue. It can also register the size and seat position of the occupants for more individual adjustment of the restraint systems. After an accident, pictures could be transmitted directly to the emergency center to make the emergency services better prepared when they arrive at the scene.

The example indicates the contribution that cameras in passenger compartment can make to vehicle safety. In the past, their use has been limited to large vehicles, but this is now changing. “The components cost far less”, reports Dr. Andrea Paggel, head of the Sensors and Actuators department at IAV. “Another driving factor is the trend to automated driving. The next step is level 3, where the camera lets the vehicle know whether the driver is still alert and can resume control again at any time.” Furthermore, in future camera data could let passenger restraint systems adapt accordingly in response to a crash situation, moving them into a position that reduces the risk of injury.

This by no means exhausts the possibilities offered by cameras inside the passenger compartment. Many comfort functions can be greatly improved or made possible in the first place thanks to the “electronic eyes”, including biometric authentication in the vehicle. “The camera inside the vehicle detects whether the person sitting on the driver’s seat is actually authorized to enter the vehicle”, explains Danny Thiel, project manager in the Vehicle Electronics division at IAV. “Positive confirmation is needed before the vehicle will start up.” The authentication process can also be linked to individual settings such as personal chassis adjustments or the seat position.

IAV has been working with its American partner FaceTec Inc. to develop a camera solution that operates with mobile devices and stationary cameras inside the vehicle, regardless of the specific platform. Biometric authentication is based on a 3-D face scan that is not deceived by changeable personal attributes such as wearing spectacles, beards or make-up. The occupant detection function also works reliably in differing light conditions. It is used among others in IAV’s proprietary IAV Coquille development (see article on page 28) which was presented at the CES 2019 in Las Vegas. At the exhibition booth, visitors could also see how the system works based on a stationary camera.

Besides biometric authentication, cameras in the vehicle interior also offer other comfort functions. “It would be possible, for example, to adapt the interior lighting to the driver’s mood, adjust the infotainment display to the occupants’ line of vision, regulate the air-conditioning to the current condition of the people in the car or even measure vital signs such as heart rate”, says Robert Büthorn, development engineer in the Automated Driving Functions section at IAV. When it comes to detecting the driver’s state and line of vision, IAV is currently working at proprietary solutions that function reliably even under adverse conditions.

Many of the key components involved in smart interior sensor systems, including artificial intelligence and image detection, are already used in projects at IAV. “We therefore offer our customers support throughout the entire development process, from the initial concept via the specification through to validation”, says Stefan Olders, Technical Consultant for Vehicle Safety at IAV. “Typical questions look at positioning the camera, choosing the best algorithms or assessing potential suppliers.” Knowledge about suitable test data and validation methods is also a key aspect in the development process. Furthermore, it is important to decide which cameras are to be used, how many are needed and whether infrared lighting is necessary.

The use of interior cameras also brings new challenges for the development engineers. For example, up to now three kinds of dummies were needed to depict various body sizes of vehicle occupants during crash tests. In this day and age of personalized restraint systems, more precise differentiation is needed. In future, this will also include road trials where many different test persons will be involved. And future high-speed cameras in autonomous vehicles will produce high data rates with all the associated challenges in terms of transport and processing. But the effort will be worthwhile: cameras inside the passenger compartment promise a hitherto unknown level of comfort and safety.