Quality Control from Development to the User

Opening the door with the smartphone, preconditioning the passenger compartment or checking the battery charge status from a distance: these are just a few examples of smart online services in the vehicle. Warranting smooth interaction of all components is increasingly a challenge, given the growing complexity of the numerous services and of the vehicle itself. IAV has developed a comprehensive, end-to-end test and monitoring service for automated monitoring and longterm analysis of online service behavior from the user’s perspective.

Skedsmokorset, a small village 25 kilometers northwest of Oslo. The temperature is -14°C. It’s great when the vehicle’s charge timer and auxiliary air-conditioning system work handin- hand. As a result, the electric vehicle is both warm and fully charged. Preconditioning the passenger compartment with energy from the power grid helps to maximize the cruising range in the winter months, thanks to vehicle online services for remote advance control of functions such as the vehicle’s airconditioning. ”Always on“ has arrived in our vehicles. Recent years have seen a great increase in the number of online services for vehicles, with further rapid growth expected in future.

A growing number of apps must remain compatible with the existing vehicles. More and more different service providers are being integrated in offers that should remain available in the vehicle. The data is becoming more dynamic, which means more individual and context-related. Users expect their vehicles to offer the same seamless flow of information that they are used to with their smartphones. Among others, these expectations include using the phone to control increasing numbers of vehicle functions, as well as collecting and receiving information from ever more different sources. This demands a complex backend infrastructure.

Servers must check identities and rights, and also collect, provide and send information. Time is a critical factor: if you want to open the door with your phone, you don’t want to be standing around next to your locked car for several minutes. If you want to replace the ignition key with the more universal smartphone, you don’t want to be restricted to one device among many.

In order to meet these challenges, IAV has specialized in all-encompassing validation of all systems. ”We offer end-to-end monitoring that safeguards automated monitoring and long-term analysis of the run-time behavior of the online services from the user’s perspective. Quality control of individual components such as backend, telematic units or cell phone networks would only cover partial aspects“, explains Björn Steffen, Senior Vice President for Connectivity & Analystics at IAV. Even if no faults are reported by the individual components in isolation, it is still possible for processes and workflows between the systems to break down so that the elements do not work together properly. In the end, this results in poor performance, low service quality and finally, dissatisfied users. ”IAV’s end-to-end monitoring simulates typical user behavior. It analyzes objective data about the availability and response times of individual subsystems for more precise troubleshooting“, says Steffen.

IAV’s monitoring software operates the applications on the GUI of user devices in just the same way as a genuine user. Possible user devices can include smartphones or tablets, a website on the internet or the vehicle’s infotainment system. In the case of remote-controlled, auxiliary air-conditioning, for example, it is possible to test how long it takes until the air-conditioning is activated in the vehicle after entering the start command in an app. The monitoring software registers the time it takes and also any possible unintentional states. An automatic check of all interdependencies between the process components then indicates the causal fault in the overall system.

The test or monitoring process is time-controlled and begins with automatic installation of the app, for example on a virtual or real smartphone. After logging in, the appropriate vehicle is selected. The ”auxiliary air-conditioning“ service is then selected and started. The control command is not just triggered by a real device but also passes through the real cell phone connection and the real backend to the vehicle communication unit. This is set up on a test rack together with other real-world communication-relevant vehicle components. The system simulates all those function-relevant real parts that are not actually installed in the test setup. During the complete test run, all data generated along the complete chain are recorded and stored in accessible fashion.

When the command is given to start the auxiliary air-conditioning, the vehicle side of the process reacts accordingly and triggers the workflow in the vehicle simulation. A message goes to the backend, confirming that the service started successfully. The backend forwards the message to the app, thus concluding the use process outlined by way of example.

The test run checks among others whether the app is able to trigger auxiliary air-conditioning in the vehicle, and whether the start command arrives correctly and is processed properly in the vehicle. Indirectly, the system also checks the possibility of using an app to log into the backend and exchange data with the vehicle. All measuring points can be time-stamped to monitor the performance of individual components and the overall system over the course of time.

Thanks to its fully automatic operation, IAV’s test automation can be used to set up end-toend monitoring for one of the different stages typically used, such as development/validation or productive/live.

This kind of monitoring and the associated alerting is vital for the live stages to allow constant operation and performance monitoring from the customer’s perspective and for prompt reaction to any faults. The communication log is used for retrospective analysis of operational anomalies. The same questions can also be answered for test and validation stages, which are often not as easily available as the live stages. It can be difficult to classify errors observed during service trials. Automated end-to-end monitoring makes it clear which stages were running with the necessary stability or which services were not available at a given point in time. It also makes it possible to narrow down the error fields for troubleshooting.

In future, production vehicles already used by the end users will need modifications and extensions in the form of updates or new services. When activating a new backend software, for example, how is it possible to ensure that all vehicle variants sold in previous years using this backend for communication will still work properly after a software update? The automated end-to-end tests can be used here for swift, partly virtualized and reproducible testing of such backend deployments for existing series.

Last but not least, end-to-end test automation can also be used for classic testing. The automated recording and analysis of all traces of communication, vehicle connectivity and control units result in swifter error documentation, and error tickets can even be produced automatically. The system is also interesting for comparing different technologies and observing their performance. This can refer to backend technologies and to technology for vehicle control units. IAV’s system is so flexible that it can operate and connect frontends and test racks all over the world so that local conditions can be taken into account (such as the cell phone network).