At last month’s Retail Tech exhibition in Tokyo, Hitachi unveiled a cashier-less store called Co-Uriba where users can shop, select products and pay for them using only facial recognition, without a wallet or smartphone payment function.

Users register their face photos and credit-card information in advance, and verify their identity by projecting their face on a screen installed in the store. After authentication, users’ payment is automatically performed when they leave the store with a product from one of the store’s display stands in their hands.

The system completes the purchase procedure so users can select products and leave the store quickly and easily without having to use their wallet or smartphone to make a payment.

The core of this service is the 3D lidar time-of-flight (ToF) sensor from Hitachi-LG Data Storage (HLDS) that can accurately detect 3D information about an object within a distance of about ten metres in real time. The sensor detects the movement of shoppers in the store, instantly identifies which products have been sold, and accurately determines whether a customer returns a product once held in their hand or when multiple customers select a product at the same time.

Multiple sensors are used in the configuration of such a cashier-less shop. HLDS has internally developed a stitching technology that maps 3D data so they can operate as one sensor by connecting the 3D data of multiple sensors, thus providing more accurate tracking.

ToF sensors also provide information that can be used for marketing in smart stores, including checking the number of customers in many stores, tracking movements and analysing dwell time. The information is visualised in various ways through HL-DP, a cloud-based, enterprise-grade IoT operation platform also unveiled by HLDS.

It is expressed simply as a number, but it can be viewed through a heat map, real-time imaging or a time series graph. These data are used as customer analysis data with various types of data – gender, age, sales and so on – along with dwell time and hand touch (tracking the customer’s reach) information based on movement information.

Based on the LwM2M IoT device management standard, the HL-DP platform provides not only the display of data collected from sensors, but also various feature such as remote sensor device monitoring and control.

Through HL-DP, users can operate the device with controls without visiting the site by using the fine adjustment feature to change the angle of the sensor, and can save maintenance costs through fault monitoring, self-diagnosis and firmware over-the-air updates.

Another strength of the platform is it has interworking API features that enable management by the integration of multiple sensors. In addition to the ToF sensors used for a smart store, it can be managed by connecting to temperature, dust and humidity sensors. In particular, when interworking with face recognition-based devices that are scheduled to be released by HLDS, a synergy effect is expected as types of customer information can be secured and connected.

HLDS sensors for smart retail stores are already being commercialised through collaboration with retailers. They are being provided to large convenience stores in Korea and Japan to implement cashier-less stores, and smart retail technology is being applied and piloted in various types of retail stores. These sensors are in the spotlight because of their accuracy and relatively low investment and maintenance costs compared with existing technology for cashier-less stores.

HLDS is a joint venture between Hitachi and LG Electronics of Korea and specialises in optical drives, such as DVD and Blu-Ray players for PCs. HLDS has developed 2D and 3D sensors using high-precision optical technology in these optical drives and, recently, in cashier-less stores.

Recently introduced convergence products such as Safe Pass Plus, which converges digital signage and face recognition access control devices, are also attracting attention.