Child or beer crate? AI recognizes who is sitting in the back seat.

Where is Blackberry Ivy used?

Simply put: on a control unit in the vehicle. There, Blackberry Ivy uses the sensors that are installed in the car – this could be the seat occupancy sensor, the steering angle sensor or the camera in the vehicle – and builds synthetic sensors from them. The local sensors form the basis for synthetic sensors. These enable us to provide additional functions.

One additional functionality, for example, would be … ?

Let’s assume there is a beer crate in the back seat of my car. An ordinary weight sensor might misidentify it as a child, as both have similar weights, and falsely display a seat belt sign. Blackberry Ivy, on the other hand, uses camera data to determine what is actually placed on the seat and can tell the difference between a beer crate and a child.

What is a synthetic sensor?

A synthetic sensor is a mathematical model that helps machines to extract information from data. For example, a machine cannot initially do anything with a camera image. But by using a synthetic sensor based on facial recognition, it can identify a face in the image and deduce that it is a human. In particular, it can recognize whether the face in the image belongs to a child.

How does the translation from this mathematical equation to recognition work?

The translation of the mathematical equation for recognition works similarly to our brain. It uses individual neurons to decide whether we recognize a face. This neural connection can be mathematically replicated and taught to the system so that it is able to recognize a face.

Neurons and mathematics, that sounds like artificial intelligence. Can you explain what that means?

Yes, artificial intelligence is actually an interconnection of neurons. It used to be called a neural network, but now it’s part of artificial intelligence. Basically, it is simply an interconnection of different neurons that have been trained beforehand. This is done so that probabilities can be used to determine whether there is a crate of beer or a child in the back seat.

In which situations is this knowledge useful?

This knowledge is useful in various situations. For example, the vehicle can be programmed to lock the door when it detects a child in order to prevent the child from opening the door – a function that is unnecessary with a crate of beer. In addition, ENCAP regulations will require such detection systems from 2025. Vehicle manufacturers need this technology to achieve the best ratings for their vehicles.

Blackberry Ivy can recognize the mood of drivers. How does it work?

Blackberry Ivy can recognize and react to the mood of drivers by combining different data sources. Let’s say a driver is stressed because of heavy traffic. A camera installed in the car identifies her as the driver and additional data, such as weather information, is retrieved from the cloud. To improve her mood, Blackberry Ivy could offer a soothing playlist based on her music preferences. This function is trained by using images of the driver and her preferred music for modelling.

And how do the camera and artificial intelligence recognize the mood? By face or posture?

Mood recognition by the camera and artificial intelligence is based on the analysis of facial features, similar to how humans do it. The AI observes the mouth region to identify emotions: Raised corners of the mouth indicate happiness, while a straight, pressed-together mouth can indicate tension. The AI also takes the eye region into account to detect any surprise or fright. All these characteristics can be recorded and evaluated by the AI.

How well is the mood recognition already working? Does artificial intelligence know my taste in music over time?

The ability of AI to recognize moods has already reached advanced stages. As an adaptive system, AI can better understand and predict a person’s behaviour in certain situations over time.

Does this mean that if the driver skips a song from the playlist because she doesn’t like it, Blackberry Ivy will no longer suggest this song in future?

Yes, that’s exactly how it works. The artificial intelligence learns from the interactions and adapts its actions accordingly.

What other applications is Blackberry Ivy suitable for?

Blackberry Ivy can be used in many different ways. It can classify vehicle occupants and thus optimize safety systems such as airbags. In electric vehicles, it supports the assessment of the battery status and provides speed recommendations. It also helps with predictive maintenance by monitoring tire and brake wear, taking into account driving behaviour and tire type. For example, the AI can make a suggestion as to which tires are suitable for the driver’s driving profile.

How do Blackberry Ivy and occupant classification work together?

Blackberry Ivy is a software platform that facilitates the integration of artificial intelligence into vehicle control units. It enables the operation of neural networks that are developed for functions such as occupant classification. Blackberry Ivy is therefore a key tool for using artificial intelligence efficiently in vehicles.

What other data, apart from weather data, could Blackberry Ivy use?

In addition to weather data, which is relevant for both mood and climate control, Blackberry Ivy could use traffic information to improve navigation. General online information could help to optimize various tasks. Data from smartwatches could monitor the health of drivers and trigger an alarm in an emergency. So there is a multitude of possible data sources.

The biggest benefit of Blackberry Ivy is…?

The biggest benefit of Blackberry Ivy is its ease of use and hardware independence. Crucially, unlike previous practices where control units were developed individually for each vehicle, Blackberry Ivy is universally applicable. Algorithms that have been developed once for a specific vehicle can be transferred to any other vehicle, which saves considerable resources.

Contact: 

ruediger.christfried.kewitz@iav.de

LinkedIn