At this month’s Farnborough International Airshow in the UK, Microsoft launched Project AirSim, an end-to-end simulation platform for training drones.

Called Project AirSim, the platform runs on Microsoft Azure to build, train and test autonomous aircraft safely through high-fidelity simulation.

In realistic environments, AI models can run through millions of flights in seconds, learning how to react to countless variables much like they would in the physical world: How would the vehicle fly in rain, sleet or snow? How would strong winds or high temperatures affect battery life? Can the drone’s camera see a turbine’s arms on an overcast day just as well as a clear one?

Project AirSim uses the power of Azure to generate massive amounts of data for training AI models on exactly which actions to take at each phase of flight, from takeoff to cruising to landing. It will also offer libraries of simulated 3D environments representing diverse urban and rural landscapes as well as a suite of pretrained AI models to help accelerate autonomy in aerial infrastructure inspection, last-kilometre delivery and urban air mobility.

Project AirSim is available today in limited preview.

“It arrives as advances in AI, computing and sensor technology are beginning to transform how we move people and goods,” said Gurdeep Pall, Microsoft corporate vice president. “And this isn’t just happening in remote areas home to wind farms; with urban density on the rise, gridlocked roads and highways simply can’t cut it as the quickest way to get from point A to point B. Instead, businesses will look to the skies and autonomous aircraft.”

He said autonomous systems would transform many industries and enable many aerial scenarios, from the last-kilometre delivery of goods in congested cities to the inspection of downed power lines from 1500km away.

“But first we must safely train these systems in a realistic, virtualised world,” said Pall. “Project AirSim is a critical tool that lets us bridge the world of bits and the world of atoms, and it shows the power of the industrial metaverse – the virtual worlds where businesses will build, test and hone solutions and then bring them into the real world.”

High-fidelity simulation was at the heart of AirSim, an earlier open-source project from Microsoft Research that is being retired but inspired this launch. AirSim was a popular research tool, but it required deep expertise in coding and machine learning.

Now, Microsoft has transformed that open-source tool into an end-to-end platform that helps advanced aerial mobility (AAM) users more easily test and train AI-powered aircraft in simulated 3D environments.

“Everyone talks about AI, but very few companies are capable of building it at scale,” said Balinder Malhi, engineering lead for Project AirSim. “We created Project AirSim with the key capabilities we believe will help democratise and accelerate aerial autonomy, namely the ability to accurately simulate the real world, capture and process massive amounts of data and encode autonomy without the need for deep expertise in AI.”

With Project AirSim, developers will be able to access pretrained AI building blocks, including models for detecting and avoiding obstacles and executing precision landings. These out-of-the-box capabilities eliminate the need for deep machine-learning expertise, helping expand the universe of people who can start training autonomous aircraft.

Airtonomy, which participated in an early access programme for Project AirSim, used it to help launch remote inspections of critical infrastructure quickly and safely, without the time, expense and risk of sending a crew to remote locations, and without deep technical backgrounds.

“We create autonomous capture routines for the frontline worker, people who don’t use drones and robots on a regular basis but need them to act like any other tool within their service vehicle,” said Josh Riedy, CEO of Airtonomy. “With Airtonomy, not only does the drone inspect the asset automatically, the captured data are automatically contextualised at the moment of capture. These features can be extended to any asset in any industry, enabling novel and automated end-to-end workflows. It’s amazing to see those responsible for our nation’s infrastructure use these tools literally with a push of a button and have a digital representation at their fingertips for things like outages, disaster response or route maintenance. Project AirSim is transforming how robotics and AI can be used in an applied fashion.”

Using data from Bing Maps and other providers, Project AirSim users will also be able to create millions of detailed 3D environments and access a library of specific locations, such as New York or London, or generic spaces, such as an airport.

Microsoft is also working with industry partners to extend accurate simulation to weather, physics and the sensors an autonomous machine uses to see the world. A collaboration with Ansys leverages its high-fidelity physics-based sensor simulations to enable users with rich ground truth information for autonomous vehicles. Microsoft and MathWorks are working together so users can bring their own physics models to the AirSim platform using Simulink.

As simulated flights occur, huge volumes of data are generated. Developers capture all those data and use them to train AI models through various machine-learning methods.

Gathering these data is impossible to do in the real world, where you can’t afford to make millions of mistakes, said Matt Holvey, director of intelligent systems at Bell, which also participated in Project AirSim’s early access programme. “Often, you can’t afford to make one,” he said.

Given that, Bell is turning to Project AirSim to hone the ability of its drones to land autonomously. It’s a tough problem. What if the landing pad is covered in snow, or leaves or standing water? Will the aircraft be able to recognise it? What if the rotor blades kick up dust, obscuring the vehicle’s vision? AirSim lets Bell train its AI model on thousands of what-if scenarios in a matter of minutes, helping it practice and perfect a critical manoeuvre before attempting it in the real world.

The emerging world of aerial mobility will launch a diverse cast of vehicles into the skies, from hobbyist drones to sophisticated eVTOLs (electric vertical take-off and landing) aircraft carrying passengers. And the potential use cases are almost limitless: inspecting powerlines and ports, ferrying packages and people in crowded cities, operating deep inside cramped mines or high above farmlands.

But technology alone won’t usher in the world of autonomous flight. The industry must also chart a pathway through the world’s airspace monitoring systems and regulatory environments. The Project AirSim team is engaged with standards bodies, civil aviation and regulatory agencies in shaping the necessary standards and means of compliance to accelerate this industry.

Microsoft also plans to work with global civil aviation regulators on how Project AirSim might help with the certification of safe autonomous systems, Pall said, potentially creating scenarios inside AirSim that an autonomous vehicle must successfully navigate. In one case there is blinding rain, in one case deep winds, in one case it loses GPS connectivity. If the vehicle can still get from point A to point B every time, Pall said, that could be an important step towards certification.

And the industry is moving closer towards scalable commercial operation of autonomous systems. Bell recently used Project AirSim to prepare for Nasa’s Systems Integration & Operationalisation (SIO) extension project, which aims to accelerate the use of autonomous aircraft in the national airspace system. The company’s APT autonomous pod transport aircraft flew through a corridor in the Dallas-Fort Worth area, successfully demonstrating the craft’s ability to maintain contact with ground-based radar monitoring systems.

“AirSim allowed us to get a true understanding of what to expect before we flew in the real world,” Holvey said. “It’s going to be one of the tools that will accelerate the timeline for scaling aerial mobility. If we have to test and validate everything by hand, or in a physical lab, or on a flying aircraft, we’re talking about decades, and it’s going to cost billions. But Project AirSim pulls that forward through high-fidelity simulation.”

Ashish Kapoor, the creator of the of the original AirSim in Microsoft Research, helped the simulation engine evolve from a research tool that existed largely in code into a more robust platform that any business can use without a deep technical background. An aviator himself, Kapoor can’t wait to see what that means for the world of flight.

“A ton of data gets generated when an aircraft flies through space in Project AirSim,” said Kapoor, now general manager of Microsoft’s autonomous systems research group. “Our ability to capture those data and translate them into autonomy is going to significantly change the landscape of aviation. And because of that we are going to see many more vehicles in the sky, helping to monitor farms, inspect critical infrastructure, and transport goods and people to the remotest of places.”