Renesas and Syntiant develop AI for IoT edge use

  • July 28, 2021
  • Steve Rogerson

Renesas and California-based deep-learning chip firm Syntiant have jointly developed a voice-controlled multimodal AI product that enables low-power contactless operation for image processing in vision AI-based IoT and edge systems.

Applications are in self-checkout machines, security cameras, video conference systems and smart appliances such as robotic cleaning devices.

The product combines Japanese company Renesas’ RZ/V vision AI microprocessor and the low-power multimodal, multi-feature Syntiant NDP120 neural decision processor to deliver voice and image processing capabilities.

The joint offering features always-on functionality with quick voice-triggered activation from standby mode to perform object recognition, facial recognition and vision-based tasks that are critical in security cameras and other systems. For example, while user-defined voice cues drive activation and system operation, vision AI recognition tracks operator behaviour and controls operation or issues a warning when suspicious actions are detected.

The multimodal architecture makes it easier to create contactless user experiences for vision AI-based systems. Using a dedicated, power-efficient chip for voice recognition reduces standby power consumption while speeding up system development because it is possible to develop software independently of the vision AI functionality.

“We anticipate that demand for multimodal systems that use multiple streams of input information – both image and voice – will increase moving forward as a way to improve both ease of use and safety,” said Hiroto Nitta, senior vice president at Renesas. “Through the collaboration between Renesas, a leader in low-power image AI technology, and Syntiant, a leader in voice AI technology, we will accelerate the adoption of low-power, ultra-small smart voice AI technology in embedded systems and deliver new combinations to customers globally.”

The RZ/V for vision AI incorporates Renesas’ DRP-AI (dynamically reconfigurable processor AI) accelerator and combines precision AI inference with power efficiency. This eliminates the need for heat dispersion measures such as heat sinks or cooling fans, which reduces costs and makes it possible to integrate vision AI into a wide range of embedded applications.

The Syntiant NDP120 chip incorporates AI capabilities that can be used to implement many high-precision, hands-free voice functions, including speaker recognition, keyword detection, multiple wake words and local command recognition. Packaged with the Syntiant Core 2 neural network inference engine, the NDP120 can also run multiple applications simultaneously while reducing power consumption to 1mW battery power.

The voice-controlled multimodal AI uses multiple mutually compatible devices from the broader Renesas portfolio to provide an elevated prototyping platform for faster time to market and reduced risk.

“Voice-based user interfaces will make it possible for customers to deliver new user experiences that bring the next generation of innovative ideas from concept to reality,said Syntiant CEO Kurt Busch. “We’ve already shipped more than 15 million of our deep learning NDPs globally to enable always-on voice in a wide variety of consumer and industrial IoT applications. Our collaboration with Renesas delivers a powerful, low-power voice and image solution that is certain to accelerate traction among a global customer base in a variety of devices and use cases.”

The reference design for the multimodal AI is available now, including circuit diagrams and BoM lists.

Founded in 2017 and headquartered in Irvine, California, Syntiant moves AI and machine learning from the cloud to edge devices. Its chips merge deep learning with semiconductor design to produce low-power deep neural network processors for always-on applications in battery-powered devices, such as smartphones, smart speakers, earbuds, hearing aids and laptops.