Himax Technologies and IoT edge specialist Seeed Studio have developed an ultralow power real-time monitoring IoT device. The SenseCAP Watcher provides vision, sound sensors and speaker that can detect, understand, and respond to complex scenarios.

SenseCAP Watcher integrates ultralow power edge AI inferencing together with cloud-based large language models (LLMs). This hybrid approach enables high-performance processing while minimising energy consumption, paving the way for smarter, more efficient IoT solutions.

The SenseCAP Watcher is designed to address growing demands of IoT for real-time data processing. By processing data locally through WiseEye on-device AI, SenseCAP Watcher is designed to provide real-time accurate responses to user commands and event detections.

The device can detect objects such as people, pets, or items using WiseEye technology and utilise cloud-based LLMs to capture precise user-specified events, behaviours and the status of these objects. Once the SenseCAP detects specified events, it can interact with the user through voice responses, ensuring that users are kept informed of any specific changes or events in their environment.

A real-world application of this could be in a department store, where, when a customer picks up a product, the SenseCAP responds with “Good choice!” and immediately notify the salesperson with a message saying, “Someone is interested in the latest product! Please check the shelf ASAP.” By combining real-time monitoring with intelligent interaction, the SenseCAP Watcher serves as an essential tool for IoT applications that require immediate action, such as security monitoring, automation, and critical decision-making.

The SenseCAP Watcher combines hardware integration, on-device AI and cloud-based LLMs, and no-code model training and deployment. The device is based on the Seeed’s ESP32S3 platform and integrated with Himax’s WiseEye2 HX6538 AI processor, featuring Arm Cortex-M55 CPU and Ethos-U55 NPU.

It combines on-device AI and LLM. Incorporating Himax’s WiseEye2 HX6538 AI processor, the on-device AI runs Yolo v8n object detection models, enabling real-time image and vector data processing. It can detect and classify over 80 object categories, including people, vehicles, license plates, animals, household items, and parcels. When combined with cloud-based LLMs, it provides scenario analysis while minimising LLM API calls and reducing operational costs

It provides no-code model training and deployment through Seeed’s SenseCraft AI platform. Users can deploy pre-trained models from the SenseCraft AI Model Zoo. Employing Himax’s SDK, the SenseCraft AI toolchain provides machine learning frameworks and AI algorithms for image processing “At Seeed Studio, we’re passionate about making advanced technology accessible to everyone. We’re thrilled to continue our partnership with Himax by integrating their cutting-edge WiseEye2 into our AI agent SenseCAP Watcher. This collaboration brings powerful LLMs to the physical world, enabling data to be processed locally on-device for quick, responsive actions. The exceptional AI inference performance of WiseEye2 is crucial to advancing intelligence across a wide range of IoT applications, including smart buildings and homes,” said Joey Jiang, GM of Industry and Application Group at Seeed Studio.

“Himax is delighted to partner with Seeed Studio again to showcase the power of our WiseEye2 AI processor in the SenseCAP Watcher. This ultralow power edge AI solution redefines IoT innovation by enabling real-time, precise detection and analysis directly on the device, while seamlessly integrating cloud-based LLM intelligence when needed. By tackling challenges such as latency, bandwidth, and power efficiency, we are empowering smarter, more responsive IoT applications that cater to the dynamic and diverse demands of IoT users across various applications,” said Mark Chen, Vice President of Smart Sensing Business at Himax.