On July 13, Boston’s Merlin Labs announced that it would be working with the US Air Force to add autonomy to the C-130J Super Hercules cargo transport plane. Merlin’s technology is a kind of advanced auto-copilot, designed to take over the responsibilities of one crew member in flight while being supervised by a human pilot. If the technology delivers as promised, it will allow planes that normally fly with two human pilots to operate with just one, and could even allow single-seater planes to fly fully autonomously.
The same day that Merlin announced its partnership with the Air Force, it also announced a second round of $105 million in funding, which combined with a first round means the company has $130 million of runway to develop its technologies. (In Air Force terms, $130 million is enough money to buy one F-35A and have some change left over.)
This funding, says Merlin Labs CEO Matthew George, will help the company continue to develop “the world’s most capable, safest and flexible pilot, that will eventually enable very large aircraft to fly with reduced crew and small aircraft to fly totally uncrewed.”
When George says “pilot,” he means it, drawing a distinction between traditional autopilots and Merlin’s product, the way that having cruise control in a car is different from having a driver. That means not just maintaining a course in flight, but also performing takeoffs and landings, responding to challenges like turbulence and storms as they come up, and even talking with air traffic control.
While such a pilot may conjure images of a humanoid robot sitting in a captain’s chair (or, to a certain generation, the goofy inflatable contraption from 1980 comedy Airplane!) Merlin’s pilot is software, plugged into the plane. This software will receive the same kinds of sensor information a pilot does, though it will get it directly as data instead of having to read it off instrument displays.
[Related: This company is retrofitting airplanes to fly on missions with no pilots]
The Merlin Pilot is “a box that goes into the aircraft,” says George. “Depending on the aircraft, we have a bunch of different interfaces that allow the Merlin pilot and the Merlin intelligence to be able to go and control the aircraft surfaces.”
It may not have a literal hand on the yoke controlling roll, pitch, and yaw, but it is intended to steer the plane all the same by skipping the physical interface and going directly to the electrical controls. Or, on older aircraft that are not fly-by-wire, installing the Merlin pilot means adding servos and actuators so that the system can work with the plane.
As this technology is still in development, George says the company is hiring human factors scientists to figure out the best relation between the Merlin pilot-in-a-box and a human overseer. But even at this stage, the device is set up so that a human pilot can monitor what the Merlin pilot is doing, in much the same way a flight instructor would keep an eye on a student co-pilot.
“We haven’t announced how we’re gonna do that yet,” says George. “You can assume a tablet type device, where the human pilot is able to monitor the system to understand what the system is doing. If the human pilot doesn’t like what the system is doing, they’ll be able to take over for the aircraft.”
In addition to operating the plane mechanically, for Merlin’s pilot to function like a human pilot, it will need to take and follow commands from air traffic control. These commands are verbal, carried out around the world on a fairly uniform basis, but always with the expectation that a human pilot is talking to a human air traffic controller.
“Philosophically, we believe that air traffic control needs to be able to interact with an autonomous aircraft,” says George, adding that this includes not just uncrewed aircraft but also people-carrying planes flown by a crew that includes human and autonomous pilots. “The system present-day is designed to be talked to just like a human pilot and will respond just like a human pilot, albeit with a slightly funny voice.”
Merlin is working with the Federal Aviation Administration (FAA) in the United States and the New Zealand Civil Aviation Authority (CAA) as the first regulators for certifying its pilot, and is still figuring out if the autonomous system needs to identify itself as such to air traffic controllers when talking to them.
The Merlin pilot was trained on voice data from a wide variety of air traffic controllers around the world. This training was done through a machine learning process, with the intention of the AI being able to respond on its own, rather than just following a set script of fixed rules for speaking. This makes it more similar to Alexa or Siri than to a bot that simply reads a pre-recorded script.
Before certification comes something called certification basis, which gives the regulator a framework for later certification.“This is the first time ever that a regulator has issued a certification basis to a system that has a machine-learned element,” says George. “Everything in avionics prior to us has been rules based and deterministic.”
For air traffic controllers, and governments more broadly, to trust AI-enabled autonomous pilots means the rules have to accommodate the choices and actions of the AI. When it comes to air traffic control, those choices made by AI will be in spoken words, and will come with the added safety feature of a human pilot on board who is able to clear up any discrepancies.
Merlin plans to deploy its co-pilot on cargo routes in New Zealand first on Cessna Caravans, where relatively empty skies and rough terrain make it ideal for testing aerial supply lines. This will be a civilian air route, but the company intends to take lessons from these routes to flying US Air Force C-130 planes on cargo runs.
“In a world where pilots are becoming more scarce, we can enable pilots to be able to go perform other missions where human brains are even more needed,” says George. “The Air Force, I think, has picked the C-130J as the first testbed for this because it’s the most ubiquitous transport aircraft out there. It is a really good platform to start to think about autonomy in the cockpit in a very real and practical way.”
Merlin’s first task will be adapting the digital pilot to fly the plane, but once it has that figured out, the possibilities of a plug-in autonomous pilot for the Air Force are many. In an immediate sense, the Air Force could reassign human pilots to more sensitive missions while the autonomous pilot works with human crew on routine flights. In short, a cockpit of two human pilots is now just one.
“You can imagine, especially in the early days of the war in Ukraine, we two had areas of Ukraine that were cut off. If you were going to resupply those areas, having the fewest possible human crew members aboard to be able to go perform those vital humanitarian missions, to be able to resupply villages, cities, other places—that’s critical.” says George. “It gives the Air Force a really flexible tool in a way that they can use in a variety of different ways while still having that human in the cockpit and still having that human supervision of the autonomous system.”
As exciting as the possibility of autonomous co-pilots on wartime resupply missions are, George emphasizes that the tech at present is delivering an interesting future. Cargo running in the sky above New Zealand with an AI that can talk to air traffic control is not just a step to future technology—it is a feat in and of itself. With this week’s announcement, Merlin has more funding to make it possible. It remains to be seen how much future those investors are buying with $130 million.
Correction on July 15, 2022: This article has been updated to clarify that the cargo routes in New Zealand will utilize Cessna Caravans, not Beechcraft King Airs.
