EMQ rolls out shadow service for IoT data caching

  • November 23, 2022
  • Steve Rogerson

California-based EMQ has introduced a shadow service for its EMQX Cloud fully managed MQTT messaging platform; the service provides out-of-the-box platform-side data caching, helping accelerate the development process for many IoT use cases.

Previously, developers needed to transfer IoT data to third-party services through the data integration component of EMQX Cloud before they could conduct further data processing, analysis and application development. The shadow service avoids reliance on third-party services, allowing developers to centralise device data caching, modification and viewing directly within EMQX Cloud. They can quickly create object models, device shadows and other resources related to data reporting and distribution that reduce build times, latency and transmission costs.

Quickly connecting IoT devices and platform applications to support rapid implementation and market validation is becoming key to shaping core competitiveness and achieving business innovation. However, in the case of message interactions between IoT devices and applications, it is very common that the device side network is unstable, low-power devices are dormant, and mobile applications do not consume data regularly. To ensure reliable interactions, caching or persisting data, such as the latest reported messages from devices or configuration parameters issued by applications, in the MQTT message access layer can be important.

The shadow service solves these problems, helping companies speed up development while enhancing the reliability of data processing. After activating the service, users can configure shadow models, access data through a standard API, and monitor usage without requiring third-party integrations.

The service was informed by real customer needs across the automotive, manufacturing, finance and other smart industries deployed in EMQX environments. Many IoT applications can benefit from the shadow service, such as the interactions between smart home devices, low-power smart meter data collection and configuration, and telematics message pushing.

With the data caching capability, users can develop many applications without configuring external storage and network connectivity. It’s especially suitable for tracking device status.

Application requests to get device status are complicated by several conditions:

  • Devices may go online and offline frequently because of network instability, so they cannot respond to application requests normally.
  • Devices may need to respond to requests from multiple applications simultaneously, straining their limited processing capacity.
  • Devices may transmit information even when there is no data consumer.
  • Devices may transmit the same information in response to every request, even though different applications need to read different parts of the device information.

With the use of device shadowing, the device state change only needs to be synchronised to the device shadow once. Regardless of whether the application is online, the number of requests and whether the device is online, the current state of the device can be obtained from the device shadow cache, enabling the decoupling of the application and the device.

Applications can send commands to devices, but when a device is offline or the device goes online and offline frequently because of an unstable network, the commands will fail to be sent. Using the device shadow mechanism, the commands issued by the application can be stored in the device shadow with a timestamp. When the device goes online again, it can obtain the commands from the device shadow and determine whether to execute them according to the timestamp.

Furthermore, the service provides both MQTT and Rest API interfaces to add, delete and check cached documents for easy invocation by MQTT devices and application services. The MQTT interface provided by the service is compliant with the MQTT standard, and the client can invoke the shadow service without the need for a customised SDK and without platform binding.

With the managed MQTT messaging service provided by EMQX Cloud combined with the shadow service, users can integrate MQTT device access and message caching, accelerating the speed of IoT application development. The flexible message caching data structure in the service can also help later business expansion, providing a guarantee for continuous business development.

At present, the shadow service provides a seven-day free trial with up to 1Gbyte of data. Users can log in to EMQX Cloud and provision the service through the value-added service module on the top main menu or the shadow service module on the left menu of the deployment screen.