Automotive AI firm Cerence has launched a product that allows drivers and passengers to interact with points of interest outside the car. Cerence Look is likened to having the knowledge of a human copilot in the automotive assistant.

Cerence Look is being deployed first by Mercedes-Benz in its new Travel Knowledge feature in the new generation MBUX.

Designed to provide a natural way to interact with the environment outside the car, Cerence Look uses environment reconstruction, existing hardware, and car sensor data to deliver information about points of interest along a driver’s route. Where drivers previously had to manually search for information about stores, restaurants and cafes, hotels, landmarks, etc., they can now simply look at a building and ask the automotive assistant for information, creating a faster and more productive way of gaining needed knowledge while on the road.

A driver can look at a coffee shop and ask, “What is the name of that restaurant?” or “What is that café on my left?” Or, the driver can look at a historic landmark like the Burg Teck castle in Germany, and get information by asking, “Tell me about this castle.”

“In a typical car setting, it can be challenging for drivers to gather information about the world around them without compromising their safety by pulling out their smartphones,” said Stefan Ortmanns, EVP & General Manager, Core Products, Cerence. “Cerence Look solves for this, providing intuitive, multimodal interaction that, when combined with voice recognition, makes it easier and more natural than ever to engage with the automotive assistant and gain critical knowledge about the outside world.”