Seattle residents can start keeping their eyes out for matte black Toyota Highlanders, clad with a wealth of sensors on their roofs, plying the streets of their drizzly city. These retrofitted SUVs belong to Zoox, a self-driving car company owned by Amazon that wants to one day operate autonomous taxi services—like Uber, but with no driver behind the wheel.
Self-driving cars are nothing new, of course, although the vision that Zoox has is especially slick. The company is working on an autonomous vehicle that, in general, belongs to a category of vehicles that tend to jokingly be compared to rolling toasters. In Zoox’s case, their future taxi will seat four, and can drive bi-directionally, since it has no front or back the way a regular car does. It also boasts four-wheel steering, and lacks any spot for a driver. It has no official name just yet. The company likes to compare it to a carriage, because the four passengers will sit facing each other.
In Seattle, the company is planning to open an office next year and is also bringing a double-digit fleet of Toyota Highlanders (not the futuristic carriage-type vehicles) to the streets. In fact, the company has discreetly already made some exploratory moves on the streets of the city that’s home to their parent company, Amazon.
“We actually first sent a Highlander to Seattle in late 2019,” says Jesse Levinson, the company’s co-founder and CTO. With these SUVs, the company can use the sensors on them—devices like lidar units and cameras—to create highly-detailed maps of the urban environment, a key prerequisite for a self-driving car. “We were able to build a map of downtown Seattle very quickly, and then on our next trip we were able to drive fully autonomously in downtown Seattle,” Levinson says. “Daytime, night time, and in the rain, and it worked really, really well.”
“We did that very quietly,” he adds.
Searching for soggy skies
Now, the company isn’t being quiet about it, announcing today that they’re officially expanding to Seattle. They’ve already been testing their Highlander vehicles in other locations: two spots in California, and in Las Vegas.
Levinson notes that Seattle will bring some challenges for these Toyota Highlanders, which boast level-three autonomy and have safety drivers behind the wheel. One of those challenges will be the city’s famous drizzle. “We’re looking for some rain, to test our sensors out,” Levinson says. “We’ve really designed our vehicle to handle inclement weather, but we don’t get a ton of it in San Francisco, and we get even less of it in Las Vegas, so having frequent rain will be really useful for that purpose.”
He also notes that just being in a new location will help train the company’s software. “Getting more diversity of data is good for machine learning algorithms,” he says.
[Related: What will it take for humans to trust self-driving cars?]
At a point yet to be announced, Zoox hopes to open a robo-taxi service for customers who would hail one of their new carriage-like vehicles, something like one of their rivals, Waymo, is operating right now in the Phoenix area with driverless Chrysler Pacificas. The next-gen Zoox taxis sport lidar sensors on their corners, as well as other sensors, like cameras, radar units, and an IMU, or inertial measurement unit. Plus, the vehicle’s roof will feature a sunroof and hundreds of LED lights to set a vibe for future passengers. (Another company, Cruise, is working on a similar vehicle, also in the toaster-on-wheels category, called Origin.)
Meanwhile, the Toyota Highlanders are the company’s more pragmatic stepping-stone vehicles, which they’ll use to conduct mapping and autonomous driving in Seattle. The company doesn’t intend to use the Toyotas as part of the future ride-hailing service, but Levinson says the sensor set-up on the Highlanders is similar enough to the next-gen vehicle that the data they gather with the SUVs is still useful. “They have the same sensors, and they’re in almost identical geometric configurations,” he says.
Modern-day cartography
For a self-driving car to know where it is in the urban streetscape, it needs a detailed three-dimensional map to reference as well as onboard sensors to perceive what’s around it. The map, of course, is much different from what a human driver navigates with when they fire up a navigation app.
“The maps that we use in self-driving cars are entirely different, and they’re much higher detailed, much higher fidelity,” says Taylor Arnicar, a staff technical program manager at Zoox. “And also things that humans might think of as being important have no real bearing to the self-driving car itself—the self-driving car doesn’t care what the name of the street is, actually.” (Instead, the car is using numbers to understand what street it’s on.)
But before the vehicles can use those maps, the company has to build them, which they do using one of those Highlanders. “We drive those vehicles around in a new area that we want to be exploring in the future,” Taylor says. That allows them to connect the information they need to make the maps.
[Related: Kitty Hawk’s electric airplane will fly you around—with no pilot]
Interestingly, the process works better if the terrain is varied, as opposed to including spans of bridges or tunnels that contain very repetitive features and thus could be confusing. “These mapping systems are strongest when they are operating in areas with lots of unique 3D structures in the world,” he says. Human eyes could have trouble determining their location on a desolate stretch of road in a featureless desert, or a long span of bridge that looks just like the part before it, and the same is true of a robot car.
The ‘reality phase’
Certainly, Zoox is not the only player in the autonomous vehicles space. Other notable competitors include Aurora, Argo AI, Cruise, Motional, and of course Waymo, which has already made a driverless taxi service a reality in the Phoenix, Arizona area. Waymo is also conducting some rides in the San Francisco area, on a smaller scale and using electric Jaguars.
Cars driven by computers are certainly still imperfect, and tend to hit snags that might not throw off a human driver. (Then again, they never get tired or distracted.) In San Francisco, some Waymo vehicles have been showing up mysteriously on a dead-end street. In Phoenix, Waymo has reportedly struggled with issues like left-hand turns (which are hard for self-driving cars and regular cars, too) and even puddles.
[Related: How Waymo is teaching self-driving cars to deal with the chaos of parking lots]
The industry goal isn’t solely to build robo-taxis: other sectors involve autonomous trucking, self-driving shuttles, and the also the kind of driver assistance features that are designed for regular cars, like GM’s Super Cruise.
Currently, the self-driving car industry has reached a type of “reality phase,” says Raj Rajkumar, who directs the Metro21: Smart Cities Institute at Carnegie Mellon University. He noticed a “massive hype cycle” that peaked around 2018, and then industry “doldrums” in 2019.
“I think it’s beginning to bounce back,” he says. “Right now, the hype is a lot more muted.”