Some 16 months after Ford initially announced its intention to invest $1 billion in a startup with only a handful of people on its staff, remarkably little real information has been revealed about Argo AI. Based in Pittsburgh, home of Carnegie Mellon University and some of the most advanced robotics research in the world, Argo is tasked with creating a production-ready automated driving system for Ford.
Two weeks ago, Argo moved into shiny new offices located on the bank of the Allegheny River, next door to where the company started in late 2016. The engineering team currently occupies the top floor of a five-story building. They eventually expect to fill half of the structure as the company grows. As of this month, Argo has 338 employees divided among Pittsburgh; Dearborn, Michigan; and Miami, along with an office in Cranbury, New Jersey, where the lidar specialists from Princeton Lightwave, a recent Ford acquisition, are based.
Eleven years ago, Carnegie Mellon University revamped its team competing in the DARPA Urban Challenge Program for autonomous vehicles. The new team featured Chris Urmson leading the hardware group and Bryan Salesky, now the chief executive officer at Argo AI, leading software development.
Their Chevrolet Tahoe, nicknamed Boss, went on to win the challenge, and Urmson and Salesky have continued their efforts to commercialize automated driving. After stints working on automated mining trucks for Caterpillar and the Google/Waymo self-driving project, Urmson and Salesky each started their own companies that are working with automakers to develop production-ready systems. Urmson and his co-founder at Aurora Innovation, Sterling Anderson, are working with Volkswagen, Hyundai, and Chinese EV startup Byton. Salesky and CMU/Uber alum Peter Rander founded Argo AI.
In many respects, the story of Argo mirrors that of Cruise Automation. San Francisco–based Cruise was acquired by GM 11 months before Ford invested in Argo. As with Argo, Cruise has been tasked with developing the automated driving system for GM vehicles. The product development teams at the respective OEMs are handling the vehicle development and integration at their home bases while the startups work primarily at their remote locations. Both are nominally independent companies despite the ownership stakes of the automakers.
Salesky emphasized the tight integration between the Argo team and their counterparts at Ford. The engineers “hold a daily standup meeting with the team in Dearborn,” Salesky said.
Computing Power Supercharges Development
As an alumnus of the DARPA Grand Challenge program, Salesky is part of a relatively small cohort of people with experience developing automated vehicles over a long period.
“The technology continues to evolve at such a rapid pace that I don’t really pretend to know what has been done elsewhere,” said Salesky. “The approach that we’re taking now is different than how we approached it three years ago, which is different than how we approached it six years ago.”
What, if anything, has changed about the approach to developing a vehicle that drives itself?
“The biggest difference from now and just a few years ago is the evolution of computing technology. The amount of CPU and GPU horsepower available to us lets us approach problems in ways that we never approached before. We can run algorithms that used to be completely intractable from a runtime perspective.”
A key aspect of that is the kind of machine learning and deep neural networks that are now common in this field. It used to be that those sorts of algorithms could only run on massive computing clusters. Today, Nvidia is rolling out ever more powerful graphics-processing units optimized for automated driving that can crunch through those machine-learning models with ease. While racks of servers filled the back of the DARPA Chevy Tahoe, the Ford Fusion hybrids that Argo is using as test beds today easily contain far more capability in their trunks, and production-ready systems will soon be arriving that are even smaller, if still power hungry.
While Salesky may not have insight into how every other company in this field is tackling the virtual-driver problem, it is believed that at least some are using massively deep machine-learning systems from end to end, making the entire stack into a black box. You can see what inputs go in and what control signals come out, but little else.
Like most of the other big companies in this field, Salesky, Radner, and Argo vice president of robotics Brett Browning, along with their benefactors and partners at Ford, are committed to making sure their system is safe and reliable. To that end, the software is architected with what Salesky and Browning call a modular-decomposition approach. The entire software stack is broken into manageable chunks that can be individually tested and then assembled.
“We’re not religious about machine learning or any other approach,” added Salesky. “We have a full toolbox so we can use the right tool for every job.”
That means machine learning where it makes the most sense, or a classic rules-based approach where that yields a better result. When issues are found, even where neural nets have been used, this enables tracing them much closer to the source.
Another notable feature of Argo’s software stack is the prediction engine. Automated driving systems generally include a perception layer that is designed to pick out all of the objects in the environment around the car, a path planner that determines where it should go, and the actuator that sends commands to the hardware. Argo’s prediction engine sits between the perception and the path planner and tries to determine where the detected objects are going and what they are likely to do.
To some degree, all AV systems do this, because it helps to decide if a car should stop to avoid a pedestrian about to cross the road or proceed because the pedestrian is going to stop and wait. This is actually one of the more challenging elements of the whole system because it can have a big impact on overall smoothness and refinement, something the developers of the Cruise system were still struggling with last November.
Entertainment Experience Meets Self-Driving Tech
Argo has hired people with a variety of different backgrounds to contribute to its development effort, including Peter Carr and several of his colleagues from the Walt Disney Company. You might wonder what someone from Disney can contribute to automated-driving software: Carr and his team were specialists in programming cameras to look where the action is going to be in sports coverage, something Disney does extensively as the owner of ESPN.
The knowledge brought by Carr includes the use of game theory in the prediction engine to better understand how other road users will respond to inputs around them, including other users, vehicles, and traffic signals.
This situational awareness generated from the perception and prediction engines all flows into the path-planning module that determines where the car should go. Of course, a car needs to understand where it is before it can determine where to go. A key component of that is simultaneous localization and mapping, another technique pioneered by the Carnegie Mellon team during the first DARPA Grand Challenge. Before a car starts driving autonomously, the fleet goes out and repeatedly records the entire operational area in 3D using cameras, lidar, and radar.
This data is fed into tools built by Argo that allow everything to be labeled and annotated. This helps the car understand where it is in space down to 0.4-inch accuracy as well as where it can go, where it can’t, and how fast. One of the interesting details of the map is the location of traffic signals. With this information, the perception system can know exactly where in the frame to look for signals and signs, which speeds up the scanning process and improves performance.
The Argo mapping team is continuously improving its internal tool set to reduce the time required to build the maps before the cars can start testing in a new area. The team is currently in the process of mapping out the area around Michigan Avenue between the Ford product development campus in Dearborn and Detroit’s Corktown neighborhood, home of Michigan Central Station, which Ford just purchased.
Based on all of the inputs, the path planner strives to produce what Salesky and the team refer to as naturalistic driving for the environment where the car is operating. That means replicating behaviors consistent with local customs, following the rules, and—most important—providing a safe, predictable, and comfortable journey for passengers. Comfort is actually an important component, because it builds trust in the system so that people will use it, and because it minimizes the risk of motion sickness. Predictability is also important in a world where automated vehicles still have to coexist with human-driven vehicles.
Test Fleet Mapping Miami
Two blocks away from Argo’s office is a garage, known internally as the depot. This is where the vehicles are stored and maintained and where the operators report for their shifts. Operations manager Brandon Duncan begins and ends each shift with a briefing about what is going on that day. Each of the three locations operates two shifts daily that overlap, and that midday briefing includes all three teams together over a video conference setup in the garage.
In addition to Pittsburgh, Argo has similar depots in Dearborn and in Miami. In February 2018, Ford announced its intention to expand its test program to the South Florida city, where it would also include partners Lyft, Domino’s, and Postmates, as it strives to develop a viable business model for mobility and transportation as a service. Since then, Argo has been recruiting and training vehicle operators from the area and building out the high-definition maps.
Currently, Argo’s test fleet is operating in a section of Miami, racking up miles and collecting data that is fed back to the engineers in Pennsylvania, Michigan, and California where Ford Smart Mobility is based. In addition to the development of the core automation system, lessons from the work in Miami will help in the development of the logistics platform that, Ford hopes, will allow it to keep its vehicles generating revenue nearly around the clock. That will be crucial to both making a profit and funding ongoing development of functional and security updates for the life of each vehicle.
Argo and Ford aren’t on quite the aggressive timeline of some other companies, such as Waymo and General Motors, that want to launch commercial services late this year and into early 2019. In some respects, they aren’t quite as far along in development as some companies might appear to be. On the other hand, with a planned 2021 launch, they have time to get their system done right while avoiding dangerous shortcuts, and they certainly seem to be on the right track.
This article originally appeared in Car and Driver