The Partnership of Renesas and Syntiant Points to An Edge AI Future of Multimodal Learning

In July 2021, Renesas and Syntiant launched an edge Artificial Intelligence (AI) computational platform that supports Machine Learning (ML) based computer vision and voice recognition in edge devices. The solution combines an RZ/V Series microprocessor unit (MPU) from Renesas with the Neural Decision Processor (NDP) 120 from Syntiant. Renesas MPU is responsible for computer vision inference workloads, while Syntiant’s chipset supports voice-related inference workloads.

Renesas mentioned that this partnership is essential for targeting edge AI use cases outside of automotive. As a strong player in automotive, Renesas is keen to expand its presence in the non-automotive space. By leveraging the market presence of an established edge AI player like Syntiant, Renesas aims to combine always-on, low-power voice and speech recognition with classification, object detection, and image segmentation in non-automotive applications, such as security systems, personal devices, industrial and manufacturing, transportation and logistics.

When edge AI was first introduced, the edge ML models were mainly designed to analyze a single source of data type, whether image, audio, or time-series data from sensors, such as pressure, temperature, vibration, and motion. However, as more and more companies realized the importance of running AI workloads in edge devices, they are no longer satisfied with simple ML applications that rely on a single input. Instead, they are looking to draw more insights by combining the analysis from various inputs to generate a contextual understanding of an alert or event.

In 2019, ABI Research predicted the potential rise of multimodal learning in our report on multimodal learning (AN-4955). The report highlighted the primary objective of multimodal learning, namely, to consolidate the learning process from heterogeneous data streamed from various sensors and other data inputs into a single model, either for prediction or inference. Through a hybrid mixture of rules-based and deep learning-based multimodal learning models, companies can combine signals from different modalities to generate more robust inference or improve on unimodal ones because modalities can carry complementary information about each other.

As such, the partnership between Renesas and Syntiant allows both companies to provide a comprehensive solution for developers who wish to leverage the computer vision capabilities provided by Renesas MPU and always-on audio capabilities from Syntiant, who specializes in voice AI.

In recent years, main Microcontroller (MCU) players, namely NXP and STMicroelectronics, have been actively offering solutions that target a broad spectrum of edge AI applications through hardware and software enhancement. NXP has introduced a dedicated Neural Processing Unit (NPU) to support dedicated AI workloads and eIQ ML Development Environment for ML developers with limited ML expertise. STMicroelectronics’ STM32 microcontroller portfolio features a group of 32-bit MCUs with the STM32 X-Cube-AI extension pack, which can run ML inference. In May 2021, STMicroelectronics acquired Cartesiam, a startup specializing in AI development tools enabling machine-learning and inferencing on Arm-based microcontrollers.

Instead of acquiring a startup or developing a dedicated chipset for audio and language processing, Renesas decided to partner with Syntiant, a key player in ultra-low power AI applications. Both companies aim to provide developers with a comprehensive set of hardware to deploy their ML-based computer vision and audio processing applications through this partnership. As for software solutions, Renesas mentioned that it would be combining its software toolkit with Syntiant’s ML development platform. Since software user experience is crucial for ML developers, creating the optimal software toolkit should be the key focus of both parties moving forward. Another possible alternative is to partner with third-party ML developer software platform vendors, such as Edge Impulse, SensiML, and ONE Tech, who is already leveraging Renesas’ MCU for edge AI applications.

Renesas and Syntiant joining force will undoubtedly push the competition in the edge AI chipset to another level. Nonetheless, edge AI is only at its nascent stage. The multimodal AI solution developed by both companies will create edge AI systems that are more intelligent and provide more contextual analysis and understanding for the end users. This will lead to more excellent user experience and benefits, furthering expanding the potential of edge AI.