Nowadays, almost every new car has some form of Advanced Driver Assistance Systems (ADAS) on board, greatly enhancing both safety and driving experience. While all of those systems are very well aware of their surroundings, many of them are blind to a crucial variable in the equation: the driver. Let’s take a look at why the driver plays such a crucial role for safety even in higher autonomy levels and why it is important to know their state at every moment.
SAE Automated Driving Levels – Based on SAE J3016
Level0 & Level1
With Level0 and Level1 cars (see description of Levels below) – we all wouldn’t call them autonomous vehicles – In-Cabin Monitoring Systems (ICMS) have the potential to save hundreds of thousands of lives each year simply by warning drivers in case they get distracted, drowsy or perform any other dangerous action. We know that around 94% of accidents happen due to human error (NHTSA) which results in around 1.3 million deaths every single year (WHO). We also know that many of those accidents can be avoided with appropriate warning systems in place, which is why many governments and automotive institutions are creating new regulations making driver monitoring systems mandatory in new vehicles. For more details, see our comprehensive collection of all relevant regulations around this topic, our Regulatory Radar.
Level2
While Level0 and Level1 vehicles help you stay safe on the road but don’t remove much of the hassle, driving a Level2 vehicle actually feels much more like a game changer. When automatic steering and braking/accelerating work together, they remove a lot of cognitive load – the driver becomes a supervisor of the system (mostly) not having to do anything more than checking if what the car does is in the realm of what he or she considers as safe. Over time, if the system performs well – which we all hope – the driver will gain trust, possibly resulting in thoughts like “It can’t harm to check my phone for a sec”, or to the extreme “Let’s watch a movie – the system drives anyway”. The levity that comes with blindly trusting the system can bear a high cost. There have been several crashes, some fatal, with vehicles in Level2 mode while the driver was performing actions other than supervising the system.
However clearly it is stated that the driver’s attention is required at all times during Level2 mode, it is understandable that people are blinded by the perceived capabilities of their car. There have even been attempts to manipulate hands-on-wheel detection systems in order to knowingly cheat a Level2 into a Level3 vehicle – just google “tesla orange video” and be surprised by people’s recklessness. OEMs cannot rely on drivers’ good judgement when someone’s life is on the line. Even if most people are sensible, there will always be some that are not or some that simply don’t know better.
Luckily, the industry becomes more and more aware of this problem, looking for solutions which cannot be cheated so easily. Camera-based approaches out there are focusing on the eye gaze direction of the driver. While this is clearly an improvement to simple hands-on-wheel detection, it’s not the ultimate solution because nobody knows if the driver’s hands are in a ready-state to take over control in case of emergency. Neither the combination of gaze tracking and hands-on-wheel detection is the answer to all problems – after all, driving experience should be improved, not worsened.
We believe that comprehensive vision-based systems are a crucial part of the final answer. Vision-based technology is the only one capable of understanding complex actions and situations in the vehicle interior. The already used eye and head tracking are only fractions of a comprehensive approach. It needs hand and body tracking, object detection (e.g. smartphones), action recognition and more for the system to fully understand what is happening and what actions should be taken in any given situation. We, emotion3D, prepared our software to take care of those tasks.
Even on a regulatory level, first proposals for a multi-factor approach have been made, as for example in a new proposal for an UN regulation on lane keeping systems. Those suggestions are highly important for a uniform standard, however, more work has to be done on how to optimize safety while creating an outstanding driving experience experience.
emotion3D’s Occupant Monitoring software provides relevant information about the driver and occupants
Level3
The big difference between Level2 and Level3: Level3 vehicles are self-aware. They know that there is a situation coming up that they cannot handle. Level2 vehicles don’t have much clue about their limitations. Hence, Level3 is a big leap forward: it allows the driver to be disengaged from the driving and supervising tasks until the vehicle tells them to get their eyes back on the road and their hands back on the steering wheel.
ICMS like in Level2 will still be needed, however, they might not be active continuously – only before the handover from system control to driver control (and during driver control to monitor distraction and drowsiness).
Although the technology might already be there, regulatory and liability issues have so far prevented true Level3 vehicles to drive on our roads. Many experts say that we might be stuck with Level2 for quite a while, or as Richard Matsu from Autoliv said at a WardsAuto conference: “I’m a fan of pushing Level2 to be Level2.99.” Until regulators haven’t found a solution for the liability issues, carmakers will naturally be inclined to place the driver at the top of the control-hierarchy.
Level4 & Level5
In the era of Level4 and Level5 there won’t be drivers anymore, only passengers. However, this does not mean that ICMS are becoming obsolete. As passengers will be able to move more freely inside the vehicle, it is crucial to know where they are, how they are positioned and what they do. In case of an inevitable impact, the vehicle’s safety systems have to optimally respond to the given situation inside the vehicle. They have to deploy airbags accordingly and might catch/fix objects inside the cabin before they become bullet-like. This is a huge challenge that will keep engineers busy for many years to come. In case of a public autonomous shuttle, ICMS will have to automatically detect weapons, fights or medical emergencies.
As you can imagine, we are not at all worried that our market vanishes with a rising level of autonomy – quite the opposite is about to happen.
Future vehicles will enable passengers to use cabins much more flexibly – In-Cabin Monitoring is necessary to ensure safety and best user experience
What’s next?
We believe that Level2, or “Level2.99”, will be around for a while before Level3 or Level4 (except super-slow autonomous shuttles) vehicles appear on the broad market. It is still quite a mystery when Level5 will broadly become reality – at the moment it seems that most OEMs adjusted their outlook to a more realistic, further away point in time. While it is crucial to have many companies working on Level5 technology, as an industry, we must accept that we cannot jump from the start to the finish line but rather take incremental steps to build truly safe systems, beginning in the lower levels. It might not attract as much media and hype as Level5 does but it is needed to keep all of us safe. And keeping all of us safe should be the industry’s primary goal.
Along with autonomous driving features, In-Cabin Monitoring Systems will increasingly get into vehicles. First, simple Driver Monitoring for assistance. Then, more advanced features enabling innovative user experience and the control of passive safety systems (e.g. airbags). In-Cabin Monitoring will be an integral part of the future in which driving will be safer and more enjoyable for all of us.
Feedback, ideas or other topics you’d like to discuss with us? Contact us!