Interior analysis based on cameras is nowadays most likely to be associated with driver attention and drowsiness monitoring, but less so with its application in automated vehicles. However, interior analysis systems are essential in the vehicles of the future.
More and more automated shuttles and robo-taxi fleets are starting (test) operations. In addition to large Tier-1 suppliers and OEMs like ZF with Beep and VW with MOIA, big tech companies like Google with Waymo and Amazon with Zoox are also in the race to automate mobility. In addition to interesting developments in the value chain and innovative business models, exciting details are also hidden in the technical aspects of these vehicles. While the hype around automated driving mostly focuses attention on the automated driving functions, in this article we want to look at an often forgotten but important part of the complex puzzle: the interior analysis.
A world without bus drivers
The word “bus driver” does not do justice to the many tasks this person performs. Besides the main task of getting passengers to their destination with safe driving, a bus driver has to take care of many other things to ensure safety and comfort, such as recognising medical emergencies, intervening when passengers are endangered or the cabin is damaged, assisting with boarding and exiting, and much more. But what happens when this important role is no longer performed by a human? This is where interior analysis comes into play. Such systems consist primarily of one or more cameras and potentially other complementary sensors that are integrated into the interior. The sensors are connected to a processor on which AI software creates an analysis of the interior status. This status is used to realise functions in three essential domains: safety, comfort/support and fleet efficiency.
Safety
The reliable provision of safety functions is one of the most important tasks of the interior analysis. Before starting a journey, it must be ensured that the vehicle is ready to depart. Each manufacturer of automated shuttles interprets this “cabin-safe check” slightly differently. Generally, it is checked whether the maximum number of passengers has been reached and whether all passengers have fastened their seatbelts or are ready for departure. During the journey, the interior analysis warns in case of detection of prohibited objects (e.g. weapons), vandalism or medical emergencies. For the somewhat more distant future, adaptive passive safety systems are envisioned which, in the event of a crash, trigger specifically according to the situation in the passenger compartment.
Comfort/Support
Interior analysis provides a wealth of information about passenger characteristics and behaviour that can be used to realise innovative comfort and support features. For example, multi-modal interaction concepts can be implemented that combine voice controls with body language to provide a particularly intuitive and innovative driving experience. Controlling and interacting with displays and other elements of the cabin via eye contact and gestures are just two examples of future interaction possibilities. In addition, objects such as wheelchairs, walking assistance and strollers are recognised to provide the passengers with best support when getting on and off the shuttle.
Efficiency
Most fleets plan to employ so-called operators who can access camera streams like in a security centre and also establish a voice connection with the shuttles. To ensure the most profitable operation of the automated fleet, however, those operators should only rarely have to intervene. This means that the vehicle should solve safety-relevant functions as independently as possible. Furthermore, the interior analysis must also recognise when maintenance-relevant factors change. For example, the system should report dirt or damage to the interior. In difficult situations, unsolvable by the vehicle itself, the help of the remote operator is automatically requested.
Conclusion
Without interior analysis, the safe and smooth operation of automated fleets would be impossible. Therefore, in the medium term, such systems will be found in all automated vehicles, implementing vital as well as comfort-related functions. emotion3D is working with a wide range of Tier-1 suppliers and OEMs on solutions for conventional as well as automated vehicles. Interior analysis may not attract as much attention as automated driving functions, but it is necessary to ensure the safety of us all.
This article was written by emotion3D and was first published in a Technology Report by ACstyria in December 2022.