See what the future holds – HMI concepts in AR and VR

What a vehicle will look like from the inside can be quickly checked these days with the help of 3D views. But how do you make a human machine interface (HMI) tangible long before the vehicle even enters pre-series development? IAV has created a development environment that uses virtual reality (VR) and augmented reality (AR) to configure user interfaces quickly and easily.

Digitalization is opening up more and more new possibilities in the area of human-machine interfaces, such as operation via sensitive surfaces or augmenting the environment via a head-up display (HUD). This creates new design scope for the design of a vehicle HMI. The trend is clear: from simple operation to a holistic experience. And for car buyers, an intuitive HMI is increasingly becoming a purchasing criterion. Manufacturers are therefore faced with the challenge of bringing new HMI display/control elements to market as quickly as possible. To do this, they need to make them tangible as early as possible – while they are still in the prototype phase – so that the HMI can be tested on the end customer if possible. However, vehicle manufacturers not only need fast solutions, but also cost-effective ones: Instead of having to make physical prototypes of these elements – which is time-consuming and expensive – other options are needed for evaluation. IAV’s user experience specialists led by Yves Tamborini, Team Manager UX Research & Concepts, have developed a virtual environment for this purpose: AR/VR scenarios bring the HMI to life virtually – even before pre-series production.

Virtually everything is possible

Yves Tamborini explains: “We work with different test methods. In the area of VR, for example, we are rebuilding the entire interior of a vehicle using 3D software so that we can simulate displays there in different places – both on the vehicle displays and in the arrangement of other controls.” Test subjects then wear VR goggles and a data glove to spatially experience and test the virtual interior.

In the area of AR, the team is still in the experimental phase, but initial benefits are already emerging: Unlike VR, work can be done in a real environment – for example, in an existing vehicle or a seat box. AR scenarios can also be experienced by the test subject through a pair of glasses: “The test person sees their real environment, but with the help of the AR glasses, we can define certain components to be replaced or overlaid with 3D elements via AR,” Tamborini explains. For example, a head-up display can be superimposed over the entire windshield. And other features such as data gloves even enable haptic feedback. Impulses in the fingertips give the test person an impression of switches, controllers and other elements.

The HMI of the future

An important tool in handling the virtual HMI concept is the so-called IAV Fast & Easy AR/VR prototyping approach. “The HMI in the virtual vehicle is not static, but can be adapted in real time,” explains Tamborini. A tablet can be used to quickly and easily assemble a wide variety of HMI elements using drag-and-drop or to load HMI configurations, which then appear on the virtual displays in the AR/VR scene in the vehicle.

Another plus point of the IAV Fast & Easy AR/VR prototyping approach: Designs and themes can be quickly exchanged or changed – this shortens development time. “So you no longer need to be an AR/VR expert to create HMI variants,” says Tamborini. Such virtual test environments thus create flexibility as well as cost efficiency and also already map technical restrictions of the components. If physical display/operating elements are available at a later stage of development (e.g. displays, HUD, etc.), the HMI can also be displayed and operated there using the approach chosen by IAV.

Making the design and functions of the HMI tangible and, above all, evaluable at an early stage helps manufacturers in many ways. Here, too, IAV relies on virtual methods: Gaze duration and trajectories, type and number of interactions, errors, etc. can also be analyzed in virtual scenes. But not everything can be tested virtually: The National Highway Traffic Safety Administration (NHTSA) has defined stringent requirements for vehicles that are to be registered in the USA. “Manufacturers can have planned HMI systems evaluated in advance by our experts and tested using test subjects,” says Tamborini.

yves tamborini

«This gives manufacturers certainty at a very early stage that all functions comply with the regulations and that nothing stands in the way of approval by the NHTSA.»

Yves Tamborini — Team Manager UX Research & Concepts AT IAV

HMI evaluations in the virtual world coupled with tests on real systems therefore create the best conditions for an intuitive and compliant user experience in the future vehicle – and show users today what the future holds.

The article was published in automotion 01/2021, the automotive engineering magazine of IAV. Here you can order the automotion free of charge.

Stay up to date

Subscribe to the newsletter