The technology was breathlessly hyped with flashy events inviting news cameras into the autonomous vehicles (AVs) for test rides. Everyday Pittsburghers could take driverless Ubers on certain routes in the city. In some Uber AVs, workers would play the theme music from Star Wars or Indiana Jones to get passengers hyped up.
But now, the big players in the AV game, like Uber and Ford’s Argo AI, are trying to tamp down those expectations.
Uber confirmed to Pittsburgh City Paper they “realize that this technology will take years to commercialize” and even if it does happen, Uber envisions “a hybrid network where Uber trips are shared across traditional driver partners and self-driving vehicles.” According to Uber, the company has been consistent on this since starting their AV operations, and “there's too much to perfect to enable a 100 percent autonomous future, and we really do need the skilled expertise of drivers to handle the more challenging scenarios that are not ideal use cases for AVs.”
Argo AI, Pittsburgh’s other large AV outfit, is a bit more optimistic, but cautiously so. Argo CEO Bryan Salesky wrote in an October 2017 Medium post that AV systems remain as complex as ever. “Those who think fully self-driving vehicles will be ubiquitous on city streets months from now or even in a few years are not well connected to the state of the art or committed to the safe deployment of the technology,” wrote Salesky. On Jan. 20, Ford’s president of mobility Marcy Klevorn said in Automotive News the for the company's commercial vehicles, Ford would be focusing its efforts on autonomous tools to help drivers pay attention to the wheel, not have the car chauffeur them around.
This new reality has led some to look critically at the future of these companies. A local engineer and former Uber employee are sounding the alarm on “public shadow driving,” in which workers sit behind the wheel of AVs, but computers do the driving. They believe this method of testing is putting lives at risk and should only be a tiny part of advancing the technology. A professor at Duke University backs up some of these claims and has urged the government to increase regulations on AVs.
But Pittsburgh’s tech companies believe AVs still have an important place in the future of mobility, and defend driving them on public roads, citing increased safety measures they have made recently. Employees believe the future for AVs is still bright, but realize it is a lot farther off than originally marketed.
When a pedestrian was struck and killed by an Uber AV in Tempe, Ariz. last spring, the industry took a hard look at how it should operate in public spaces. The public shadow driver in the Uber was allegedly watching a video right before the crash, and a National Transportation Safety Board (NTSB) investigation showed the car’s technology observed the pedestrian, but that the AVs automatic braking was disabled “to reduce the potential for erratic vehicle behavior.”
The Pittsburgh Post-Gazette also reported that a few days before the Tempe crash, an Uber AV in Pittsburgh swerved onto the sidewalk and continued driving without the technology correcting the behavior, according to a whistleblower.
After the crash in Tempe, Uber halted testing for months and was eventually cleared by PennDOT to test again in Pennsylvania in December. Over that time, Uber made significant changes to its testing to operate in safer scenarios. Testing now starts with software simulation. Once the simulations are completed, AVs try those scenarios on private test tracks. Only after that are Uber AVs put onto public roads, with a driver behind the wheel ready to take over.
“Prior to resuming self-driving operations, we felt it was critically important to share our holistic approach to safety,” says Uber spokesperson Sarah Abboud.
Uber is currently only testing AVs in autonomous mode on a loop of public streets near their office in the Strip District. They tested about 70 different scenarios on test tracks before that. Beyond that, Uber employees are manually driving AVs in select streets in Pittsburgh, as well as in San Francisco and Toronto, to gather data. Uber AVs will not be picking up passengers for the time being.
Argo AI is following a very similar process and is also only currently testing on public roads in the Strip District. Argo spokesperson Alan Hall says “testing will likely expand over time.” Hall says safety is paramount for Argo AI, and safety measures, like when drivers manually takeover AVs, are a plus for the company.
“Manual takeovers are viewed as a positive,” says Hall. “Argo does not lose any value from a manual takeover because they are able to review the sensor data and run simulations to understand what the self-driving system would have done.”
Michael DeKort, an engineer who lives in the North Hills, recently penned an article in Autonomous Vehicle Engineering magazine. The story outlines DeKort’s belief that AV companies need to adapt a more rigorous simulation process, one that mirrors that of the aerospace industry, a combination of manual movements and complex software.
Right now, most AV companies primarily use only software to simulate the scenarios for AVs, and DeKort says that is woefully inadequate.
“They don’t know what simulation can do,” says DeKort, who used to work at aerospace giant Lockheed-Martin. “They thought simulation wasn't good enough. There is a reason there has never been a space accident involving software.”
DeKort believes simulation should be the vast majority of testing for AVs, then a small percentage should be conducted on a test tract, and even smaller percentage (1 percent or less) should be testing on public roads, or with public shadow driving.
A former Tempe Uber engineer who spoke to CP on the condition of anonymity says pre-Tempe crash, simulation was not a big part Uber’s AV testing.
“NTSB said you need to meet these 39 different scenarios, but there are millions of different scenarios that AVs can encounter,” says the former employee. “Simulation was not a word that I even heard when I was at Uber.”
A spokesperson at Aurora, another Pittsburgh AV company, says its autonomous technology is put through thousands of software simulations daily. DeKort says that number is more in line with what is necessary to equal the rigor applied to the aerospace industry, but he says working with or mirroring aerospace testing should be implemented.
Representatives at Uber, Argo AI, or Aurora could not confirm if they were working the aerospace industry or utilizing their simulation techniques.
DeKort has started a company to consult and eventually help implement these complex simulators in AV companies. DeKort believes AV companies need to utilize full-motion simulators if they are going to achieve the millions of difference scenarios they need in simulation, before they hit the test tracks or public roads.
Without that simulation, DeKort says that AVs could be putting Pittsburghers in danger.
“If you want to make these things better than a human, you need to drive something like 500 billion miles, maybe more,” says DeKort. “[Artificial intelligence] does not learn fast.”
Professor Missy Cummings of Duke University backs up this assertion. Cummings researches human-autonomous system collaboration and has spoken to U.S. Congress, warning them that she believes AVs are not relatively close to operating safely on public streets.
Cummings told congressional staffers in April 2018 that AV technology had suffered from a lack of testing on its computer systems, leaving them easily confused at times. A research paper she wrote in March 2018 suggested that AV companies might have to return to the drawing board in order to ensure AVs are safe on the road.
“Significant work is needed to develop more transparent algorithms, visualizations, and interaction modalities such that people with advanced degrees in machine learning can understand the limits of such approaches,” she wrote of AVs.
The Pennsylvania Department of Transportation, which permits AV companies for testing on public roads in the state, says safety is a shared responsibility between the state, AV companies, and others involved. PennDOT spokesperson Jan Huzvar says AV companies must adhere to strict guidelines of the state’s Operational Design Domain.
For example, Uber says that it only completes public shadow-driving testing during daylight hours and in agreeable weather conditions. PennDOT says it has followed the recommendations of industry and academic experts in issuing regulations.
AV companies say they have made the significant changes to put safety as their top priority. The former Tempe Uber employee says this new focus on safety and data gathering is a encouraging.
And a current Uber employee at the offices in San Francisco who spoke to CP on the condition of anonymity says that while that new focus on safety is necessary, it means that advancing AVs will be much slower and methodical.
He says a more likely estimation for AV implementation is in the ten to 15-year range, which he believes is a good thing.
“The trajectory that was originally painted was completely unrealistic,” says the current employee. “I think Uber has learned a lot of lessons on how to make this safer. It brings real benefits.”
He believes the future is far off, but still possible. And in contrast to most tech companies who see regulation as burdensome, he thinks state and federal legislators should help guide AV companies, ensuring their future is beneficial to the public.
PennDOT agrees.
“It is our hope moving forward to see substantive legislation passed related to automated and connected vehicles at both the state and federal level,” says Huzvar. “We stand ready to assist our federal and state partners in helping to craft a path forward.”
