Are Suppliers Ready for the Imminent Merger of Driver and In-Cabin Monitoring Systems?

Jungo, an automotive software divestiture from Cisco Systems, recently announced a collaboration with Analog Devices, a semiconductor manufacturing company, to develop a unified driver and in-cabin monitoring solution that merges Analog Devices’ Time-of-Flight (ToF) technology with Jungo’s CoDriver software. CoDriver is a 2D infrared camera-based driver monitoring software solution that uses deep learning, Machine Learning (ML), and computer vision algorithms to provide a real-time picture of the driver’s condition. The full-cabin version uses standard infrared cameras to provide the number of people in the cabin and their age, ID, gender, and position. The ToF technology will add facial and gesture recognition to Jungo’s solution and enable Augmented Reality (AR) capabilities beyond gesture control. Today, Driver Monitoring Systems (DMS) and in-cabin or Occupant Monitoring Solutions (OMS) are often commercialized separately. With the collaboration, Jungo will be able to provide a single solution that complies with the European Union’s DMS requirements and the Euro NCAP recommendation of child presence detection that will both come into force in 2022.

It is now well accepted that camera-based DMS is the most appropriate way to directly track driver drowsiness and distraction and perform safe, vehicle-initiated handover in semi-autonomous cars. The Euro NCAP proposes the use of DMS starting in 2020, and the system will become mandatory in the European Union for M1, M2, M3, N1, N2, and N3 new vehicle categories starting in 2022. Also, in February 2020, the NTSB concluded that steering wheel torque is an ineffective measure for driver engagement in SAE Level 2 vehicles, recommending the use of DMS. DMS includes at least one camera pointed at the driver’s face that provides a real-time evaluation of the driver’s state. The system sends alerts and warnings when it detects that the driver is tired or not paying attention to the road ahead. In semi-autonomous cars, the system notifies the driver when a vehicle-initiated handover is required. If the driver fails to engage after the notifications, the system may be able to slow down the car speed, activate the brakes, or even perform an emergency stop maneuver.

OMS is widely used in commercial vehicles to monitor events such as an unauthorized passenger, cabin clutter, or the driver reaching around the cabin. The solution is now getting popular among passenger vehicles after over 70% of automakers in the United States voluntarily agreed to make child detection a standard feature by 2025. The feature has been mandatory in Italy since 2019, and the Euro NCAP will reward Original Equipment Manufacturers (OEMs) that offer child presence detection after 2022. OMS can also provide cabin comfort and convenience via facial or gesture recognition, such as the operation of the infotainment system via gesture or automatic set-up of preset preferences. Moreover, OMS will be critical for fully autonomous ride-sharing vehicles because it can enable payments and identify objects left behind, violent conduct, or events that require immediate action such as food spillage and health emergencies. Initially, the OMS demand will mainly be driven by the rear-seat child detection requirements, which will drive hardware opportunity.

Considering drivers will eventually become occupants in SAE Leve 4 and 5 autonomous vehicles, DMS and OMS will eventually merge in the medium term. The tradeoffs of a DMS and OMS merger are the positioning and type of sensors; DMS’s driver-facing camera cannot detect the second row of the vehicle, whereas OMS’s sensors cannot capture the driver’s eye movements. At the example of Jungo and Analog Devices, vendors must be aware of the market trend and prepare for a seamless merge to remain competitive in the medium term.

Despite being aware of the need to provide a single driver and occupant monitoring solution, most vendors lack real-world deployment, having vague roadmaps based solely on theoretical applications. The use of ToF camera and in-vehicle radar along with a driver-facing camera is a possible solution to merge OMS and DMS. However, it would require carmakers to install additional hardware in different locations in the car. Another possible solution would be the use of Artificial Intelligence (AI) and ML to track and learn as many traits of driver distraction from as many different angles as possible so the cameras can be positioned in a higher range of places as opposed to in front of the driver. However, they would still lack the precision of a driver-facing camera.

Moreover, once DMS and child detection functionalities become commoditized, OEMs will seek to provide comfort and convenience functionalities, such as AR, to deliver added value in luxury vehicles. Thus, OMS and DMS suppliers should offer solutions that can be further upgraded to provide additional features.