NVIDIA is the inventor of the GPU, the computer device that creates interactive graphics on laptops, workstations, mobile devices, notebooks, and personal computers. It is Tesla’s supplier of choice for the components that power its 17″ touchscreen in the Model S. NVIDIA is a global company with interests in many emerging computer technologies, one of which is autonomous driving systems.
A team of NVIDIA engineers working out of a former Bell Labs office in New Jersey decided to use deep learning to teach an automobile how to drive. As it happens, Bell Labs is where the earliest research into deep learning — sometimes called artificial intelligence or AI — took place.
The project was called DAVE2, in honor of DARPA Autonomous Vehicle (DAVE) program. Using a convolutional neural network (CNN), it was able to learn the entire process needed to steer an automobile. The team wanted to bypass the usual way self driving systems work. The team says detecting lane markings, guardrails, and other cars requires coding for a nearly infinite number of “if, then, else” statements.
Instead, they used an NVIDIA DevBox and Torch 7 (a machine learning library) for training and an NVIDIA DRIVE PX self driving car computer to do all the processing. The team trained a CNN with time-stamped video from a front-facing camera in the car synced with the steering wheel angle applied by the human driver.
They collected the majority of the road data in New Jersey, including two lane roads with and without lane markings, residential streets with parked cars, tunnels, and even unpaved pathways. More data was collected in clear, cloudy, foggy, snowy and rainy weather, both day and night. With the data collected, the team trained a CNN to steer the same way a human driver would.
Once the neural network proved effective in a driving simulator, the team uploaded it to a DRIVE PX computer and took it out for a real road test in an actual car. The vehicle drove along paved and unpaved roads with and without lane markings, and handled a wide range of weather conditions. As more training data was gathered, performance continually improved. The car even flawlessly cruised the Garden State Parkway.
The engineering team never explicitly trained the CNN to detect the outline of the road. Instead, using the steering wheel angles a human driver would use and using the road as a guide, it began to understand the rules of engagement between vehicle and road. Alert readers will notice that the system relies heavily on input from cameras, something that Tesla has recently decided not to do. Instead, Tesla says it will rely primarily on radar images because cameras cannot penetrate fog, rain, snow, and smog adequately.
The field of autonomous driving is developing right before our eyes. In ten years, self driving systems will use standardized components, but for now researchers are experimenting with various technologies to discover which ones work best. It’s an exciting time to be an automotive engineer.
You can read the entire NVIDIA research paper, End To End Learning For Self Driving Cars for yourself or watch the video to learn more about how artificial intelligence is teaching cars how to drive themselves.