Amazon is using generative AI to improve the home-automation features on Alexa and control devices with Echo Hub and Map View.

Consumers have connected more than 400 million smart home devices to Alexa, and use Alexa hundreds of millions of times each week to control those devices. The tech giant now hopes that advancements in generative AI can create homes that are more intuitive, intelligent and useful.

One problem with Alexa is users still have to remember specific phrases and device names, make multiple requests, or sometimes repeat themselves. Amazon has developed a large language model (LLM) to make the experience more intuitive and conversational.

The Alexa LLM can manage a nearly endless amount of variation, and it can understand how specific homes are set up to trigger the right APIs and actions. Understanding these nuances is critical for making Alexa more intuitive. For example, if the user says “Alexa, I’m cold”, Alexa will turn up the temperature. Or, “Alexa, it’s too bright in here”, and Alexa will dim the lights.

Alexa should also be better at understanding context. So, when more devices are added to the home, the user will be able to say “Alexa, turn on the new light in the living room”, and Alexa will infer what is meant and act accordingly.

The LLM should also make Alexa more conversational with a higher level of smart home intelligence. Multiple requests can be combined into one by saying “Alexa, close all the blinds, turn off all the lights and start the vacuum”. Or setup an Alexa Routine entirely by voice, without any manual programming in the Alexa app.

This can be done by saying things such as “Alexa, every morning at 8am, turn off my sound machine, open the blinds, turn on my bedroom lights, and make me some coffee”.

Amazon has already added Matter support to over 100 million Echo and Eero devices. It is also making its tools and APIs publicly available, and continues to invent ways for device makers to build with Alexa. It has announced two LLM-powered options that will make building for Alexa easier.

Device makers can use Dynamic Controller to tell Alexa the unique things their device can do. So, when the user says “Alexa, make it look spooky in here”, Alexa will infer that it should set a temporary lighting scene. Also added is Action Controller, which lets device makers define the simple actions their devices support so users can be more conversational when they ask for something.

For example, saying “Alexa, the floor is dirty” means Alexa will infer the action is to vacuum. Amazon is working with GE Cync, Philips, GE Appliances, iRobot, Roborock and Xiaomi on this. More on developer tools can be found at: developer.amazon.com/en-US/blogs/alexa/alexa-skills-kit/2023/09/alexa-llm-fall-devices-services-sep-2023.

Forty per cent of smart home actions are initiated by Alexa, without users saying anything. Asking Alexa to control the lights is one of the most common smart home requests. Soon, with a compatible Echo or motion and ambient light sensor, Alexa will be able to detect the brightness level and activity in a room, and intelligently decide to turn the lights on or off. Users will no longer need to walk into a dark room and search for a light switch, or even ask Alexa to do it for them.

Amazon is also adding more Routines to help automate the day: Featured Routines for Ring will bring pre-populated lists of Routines to the Ring app and the Ring devices in the home. This includes actions such as turning off lights when the Ring Alarm is set to Away Mode, turning on porch lights when someone rings the doorbell at night, or having Alexa announce which door opened.

More morning Routines include character alarms, daily affirmations and stories for the kids’ wake-up time.

Millions of users have more than 20 devices connected to Alexa, and Amazon is looking to make it easier to control those devices. One way the industry has tried to solve smart home control is with custom touchscreen panels, but they’re expensive, become outdated and typically require professional installers. To fix that, Amazon has invented Echo Hub (www.amazon.com/dp/B0BCR7M9KX), an Alexa-enabled smart home control panel that makes connecting and managing smart devices easier and more intuitive.

Users can talk to Alexa or tap its 20cm customisable touchscreen to group, manage and view smart home devices. They can also start a Routine, arm the Ring security system, and view multiple camera snapshots or live views at once. When it’s not in use, it becomes an ambient display with family photos or information such as the weather and time.

Echo Hub supports smart home protocols such as Zigbee, Matter, Thread, Bluetooth Low Energy and Amazon Sidewalk. It can connect to the internet via wifi or Ethernet with a compatible power-over-Ethernet (PoE) converter.

With Map View (amazon.com/mapview), users can build a digital representation of the home’s floor plan and pin connected devices to each room. Users choose which rooms are added to the map, select which devices show up, and can unpin individual devices at any time. Through the map, users can turn off all the lights downstairs with a single tap, adjust the temperature, view live feeds from the cameras, and check that the front door is locked, all in one place.

Map View will be available in the USA later this year through the Alexa app on compatible lidar-equipped iOS devices, and come to Echo Hub early next year.