No More Boredom on the Backseat

Things are looking up for side windows: IAV’s “Side Window Entertainment” development is turning them into an augmented reality display so that in future they offer more than just a nice view. Possibilities include providing passengers with information about places of interest en route or the chance to play games with passengers in other vehicles.

Two cameras make it all possible. An outside camera films what is going past the vehicle, while a 3-D camera inside the passenger compartment monitors the line of vision of the vehicle occupants. The vehicle’s GPS data and automatic object detection lets the system know exactly what the passenger is looking at. A transparent OLED display on the side window marks the places of interest with small touch points that follow the objects during the journey. When the passenger presses a touch point, he sees information about the selected object obtained by the system from the internet (from Wikipedia, for example).

But Side Window Entertainment could offer far more in future. It should soon be able to mark restaurants or cinemas where tables can be booked or cinema tickets ordered simply by pressing the touch point. Another possibility is an “X-ray view” that lets the passenger see inside a building by superimposing additional information on the real picture. “This could show, for example, where the elevators are in Berlin’s TV tower”, reports Daniel Danz, head of the Navigation and Hybrid Multi-Media Functions department at IAV. “Or the passenger can find out where the individual stores are located in a shopping mall.” Here again, the requisite data are obtained from public sources on the internet. The IAV developers even want Side Window Entertainment to keep passengers happy when their vehicle is held up in traffic congestion. When two vehicles standing next to each other are connected, the passengers can play tic-tac-toe on the windows.

Following the presentation at the Congress for Automotive Electronics 2018 in Ludwigsburg, Danz and his colleagues also showcased the system at the CES 2019 in Las Vegas. A reconstructed passenger compartment gave visitors an opportunity to look “outside” through a transparent display where a second monitor showed a video clip of a journey through Berlin. As in real operation, the touch point followed all movements of the head. At the touch of a finger, it then revealed information about the Reichstag parliament building, the central station or the TV tower. “It felt just like being in a car”, says Danz. “And so it was no great surprise that our demonstrator went down really well with the visitors at the CES 2019”.

Side Window Entertainment could go into production in just two to three years, giving OEMs a new USP. “Future vehicles will also be distinguished by the on-board entertainment they offer. Windows fitted with transparent OLED displays will have an important role to play”, says Danz. As well as providing useful information about the surroundings, they can also be used for Office applications or learning apps. In addition, in autonomous vehicles the windshield can also serve as a movie screen during the journey.

The new entertainment possibilities are not limited to cars. Indeed, they could add a further dimension to the journey experience offered by sightseeing buses, trains, ships or aircraft. Moreover, in future it should be possible for Side Window Entertainment to be used in entire mobility chains where travelers can continue to use films or games in subsequent buses, trains and car-sharing vehicles.