Autonomous vehicles have difficulty navigating bad weather. Heavy rain, snow, and fog can do a lot to scramble an AV’s perception systems, which largely rely on cameras, radar, and lidar to “see” the world around it. Wet roads can create reflections that confuse cameras. Fog can screw up sensor data. And ice can form on the lidar, inhibiting the sensor’s ability to send out laser points to gauge distance from other objects on the road.
Waymo, the Alphabet company that aims to deploy fleets of robotaxis all over the country, has been quietly investing in weather research since its early days as a “moonshot” project out of Google’s X division. And while its autonomous vehicles still struggled to navigate certain inclement conditions, the company has gotten a lot better at detecting and predicting the weather in the cities in which it operates — even creating a first-of-its-kind fog map for San Francisco.
The map is a product of millions of data points collected by Waymo’s fleet of autonomous vehicles as they traverse San Francisco’s fog-drenched streets. Combined with special weather-detecting vehicles equipped with visibility sensors, Waymo is able to create a new meteorological “metric” that it then feeds to its autonomous “Waymo Driver” to aid its decision-making.
GIF: Waymo
Waymo has been working to improve its fog navigating skills.
“We’re describing our vehicles as mobile weather stations,” said Daniel Rothenberg, a trained meteorologist and a member of the company’s weather team, in an interview with The Verge. “And that’s exactly how they’re functioning.”
This level of on-the-ground accuracy will become more important as Waymo gets closer to deploying fully driverless vehicles as part of its commercial robotaxi service in California. The company is operating driverless rides in Phoenix and the surrounding towns and is close to offering “rider only” trips in San Francisco after receiving approval from the California DMV.
Waymo’s early testing was concentrated in Arizona, with its sunny, dry conditions and relatively flat environment. But in the past few years, the company has expanded its testing to include more treacherous conditions, including snowy Novi, Michigan, rainy Kirkland, Washington, foggy San Francisco, and humid Florida. And, of course, Arizona wasn’t without its edge cases (see: giant dust clouds called “haboobs”).
Early on, Waymo said it saw value in gathering more granular weather data to help inform its autonomous testing. Traditionally, weather stations are considered the best source for real-time weather information. But more are located at airports to support aviation safety and climate monitoring applications and can be imprecise in measuring local conditions.
Map: Waymo
Waymo’s street-level detection helps supplement data from weather stations.
Using this map, Waymo’s fleet can track the progression of coastal fogs flowing from the Pacific Ocean and as they burn off when the sun rises. It can detect drizzle and light rains that lead to wet roads in situations that are invisible to the National Weather Service’s local Doppler weather radar. “These weather observation capabilities allow us to localize where the weather conditions are beginning to deteriorate or improve,” the company says in a blog post published today. “We’re using these now to enable our ride-hailing services in San Francisco and Phoenix, and we’ll create similar weather maps for additional cities as we scale.”
“We’re understanding in real time the actual weather conditions that are impacting our vehicles in a really hyperlocal context,” Rothenberg said. “And that’s just something that hasn’t been done before.”
Fog has been a “huge focus” for the company, said Robert Chen, product lead for Waymo’s weather team. That makes sense, considering the financial and symbolic importance the San Francisco Bay Area represents to the company.
Map: Waymo
Waymo’s first of its kind fog map.
“We’re we’re actually able to do light fog conditions pretty well,” Chen said. “And then also some degree of rain.” By integrating more hyperlocal, real-time data into its prediction and decision-making capabilities, Waymo anticipates that its vehicles will be even more successful in navigating through soupy conditions in the near future.
Ultimately, the goal is to build a vehicle that can reliably and safely drive through all types of bad weather in a bid to outperform human drivers, many of whom are overconfident in their own ability to see through especially thick fog.
“I really can’t overstate how important and impactful this has been to have this ability to understand the actual weather conditions,” Rothenberg said.