Voice Assistant Platform “Made in Germany”

While driving, keep your hands on the steering wheel and at the same time operate the navigation device or send a message: Speech recognition assistants make it possible and the demand for speech recognition is growing. Current market studies indicate that within the next four years, 25 percent more devices with speech recognition functions will be used annually in both the private and commercial sectors.

Data secure, open, modular and scalable

A major problem regarding speech assistants: data security. Some OEMs have integrated speech assistance systems into their infotainment solutions – microphones are constantly active for keyword recognition. But the user cannot control where dialogs are processed and stored. Sensitive data from research and development must also be protected. However, the current market leaders do not work according to European standards. A platform “Made in Germany”, on the other hand, would guarantee compliance with the requirements of the general data protection regulation of the European Union.
This is exactly where the Speaker Project of the Fraunhofer Institutes for Integrated Circuits IIS and for Intelligent Analysis and Information Systems IAIS and its partners comes in. IAV is participating with two application scenarios: In the B2B segment, a speech assistant is being set up to support vehicle development. As a B2C case, IAV is developing a cross-device, personalized speech assistant. In this way, IAV is helping to develop solutions that offer state-of-the-art technology and data sovereignty and can be adapted to workflows, specialist terminology and the needs of different sectors. The innovative speech assistance platform will provide services, data and technologies via combinable interfaces and will therefore be modular and scalable. The speaker modules will be multilingual, retrainable and not only run cloud-based, but also offline and on-premises (on locally operated servers).

«Our most important contribution to the overall Speaker project is to bring the platform to the automotive sector with specific use cases»

Maurizio Guida — IAV project manager

IAV is the only consortium partner from the automotive sector with the necessary expertise in vehicle architecture and development and is able to integrate speech assistants with a wide range of functions in a robust and reliable manner. “Our most important contribution to the overall Speaker project is to bring the platform to the automotive sector with specific use cases”, explains IAV project manager Maurizio Guida. “One of the ways we do this is by incorporating both data protection specifications and technical requirements from our industry into the development of the platform.”

B2C – cross-device communication with multiple users

New applications of speech assistants can make driving safer. IAV aims to further improve user-friendliness by enabling personalized dialogs to be continued virtually seamlessly across multiple terminals. At present, it is not possible to switch from a smartphone to the vehicle infotainment system, for example. Moreover, personal profiles are not tied to the voice but to login data.

Spracherkennung Auto

IAV is developing multi-user scenarios so that several speakers can interact with the assistant. On the Speaker platform, it will be possible to distinguish between several users by means of speaker identification and thus conduct more complex dialogs with different people present. With access points – i.e. microphone arrays with the interface to the speaker platform – different users will be able to use the assistant in a personalized way. This is made possible by IAV’s expertise in the fields of speech dialog development, UX, HMI development, vehicle integration, acoustic protection and the creation of relevant algorithms.

IAV has already developed a smart speech recognition system that allows software to react differently to different people in the vehicle. This is made possible by individual rights assignment and zone recognition that identifies whether a voice is coming from the driver’s seat or the back seat. The use case was presented at the CES 2020 technology fair in Las Vegas.

More efficiency in vehicle development tests

In vehicle development, too, an intelligent speech assistant can help to record data, control complex test sequences and record errors that occur during testing as well as voice notes. Whereas two people were previously required for test drives and their documentation, in the future the driver will be able to carry out and document the tests on his own. Usability is a decisive factor in ensuring that a speech assistant is actually used later on. In order to scale this up as much as possible, components such as Automatic Speech Recognition (ASR), Natural Language Understanding (NLU) and Natural Language Generation (NLG) must interact robustly and users must already be involved in the development process of the assistant.

IAV is developing a dialog for the Speaker platform that is specifically tailored to this application scenario. The vocabulary is adapted specifically for this purpose so that technical terms, product names and abbreviations are also recognized by the speech assistant. IAV also contributes knowledge of vehicle electronics and connectivity as well as extensive experience in vehicle interior acoustics. This interaction is essential for recording speech signals while driving in such a way that the signal quality is suitable for further machine processing.

So far, non-European companies in particular have dominated the market for voice-controlled dialog assistants – although research in Germany has achieved groundbreaking results in this area. The Speaker project will integrate all of these on the new platform – and help to usher in a new, more secure data age.


The article was published in automotion 02/2020, the automotive engineering magazine of IAV. Here you can order the automotion free of charge.

Stay up to date

Subscribe to the newsletter