Researchers have discovered a new way to train autonomous vehicles. This uses an approach very similar to the way children learn to ride a bike. The car is given a safe environment to practice in, after which the vehicle learns independently how to stay within its lane based on corrections by a human instructor.
The approach has been developed by Wayve, a company founded by a team from Cambridge University's Engineering Department. Members for the team have worked for Cambridge & Oxford University, BCG, Ubisoft, Goldman Sachs, Microsoft Research, Skydio, NASA and Facebook AI Research, among others. Wayve claims to have developed a unique end-to-end machine learning approach that allows autonomous vehicles to learn to drive in unfamiliar places more efficiently than competing technology.
Machine learning
Autonomous vehicles use machine learning algorithms, which are trained using extensive data sets. This is a time-consuming process and requires a high-quality dataset. "Contemporary self-driving cars are equipped with a large number of sensors and are told how to drive based on a long list of rules manually created through slow development cycles," Wayve explains in a blog post.
Wayve is taking a different approach, stating that it goes back to basics. The team teaches an autonomous vehicle to stay within its lane through a trial and error process, which is very similar to how people learn to ride a bike. "Can you remember how you learned to ride a bike as a child? Enthusiastic and slightly nervous, you probably sat on a bike for the first time and pedalled while an adult was bent over you, ready to catch you if you lost your balance. After a few shaky attempts, you may have managed to keep your balance for a few metres. A few hours later, you were tearing across gravel and grass through the park," Wayve said.
Turning observations into actions
The team points out that in this, children are not given an extensive set of instructions, do not need a detailed 3D map of the park and do not have to wear sensors on their heads. An adult simply provides the child with a safe environment in which they can learn to convert observations into actions and ride a bike based on them.
Wayve largely replicated this process for autonomous vehicles. This is possible thanks to the use of a machine learning algorithm called a deep reinforcement learning algorithm. The researchers point out that this algorithm is freely available and has not been developed for this specific task. The algorithm is presented with data collected by a single camera, based on which the algorithm can determine its position within a driving lane. Based on this data, the vehicle tries to stay within its lane. If the vehicle threatens to cross the road markings, a human instructor corrects the vehicle and the task is performed again.
Training a self-driving car in 20 minutes
Based on these corrections, the algorithm can improve its operation, requiring fewer and fewer corrections and allowing the vehicle to stay within its lane with increasing accuracy. As a result, Wayve's team managed to teach a vehicle how to stay within its lane in 15 to 20 minutes during a test. The video below shows the training process.
The researchers expect that their development could significantly improve the quality of self-driving vehicles. "Contemporary self-driving cars are stuck at a performance level that is good, but not good enough. We are now providing evidence for the first feasible framework to rapidly improve driving algorithms from mediocre to drivable. The ability to learn how to solve tasks via a smart trial and error process have turned humans into highly versatile machines capable of evolving and surviving. We learn through a combination of imitation and to a significant extent trial and error for tasks ranging from cycling to cooking," Wayve explains.
In addition, the team points out that self-driving cars are usually equipped with a large number of sensors and, in principle, an unlimited budget can be spent on such equipment. However, the study shows that these sophisticated sensors are not necessary for simple driving tasks and can also be performed with a monocular camera image combined with information about the vehicle speed and steering angle.
DeepMind
That deep reinforcement learning has promising potential also previously demonstrated DeepMind, part of Google's parent company Alphabet. DeepMind used this method to train artificial intelligence (AI) to play Go, Chess and a number of computer games, among others. In almost all cases, the AI managed to beat its human opponents. The researchers point out that DeepMind's algorithms take millions of attempts to solve a task. The researchers therefore call it remarkable that in less than 20 attempts, they managed to teach a self-driving vehicle to consistently stay within its lane.
All of Wayve's findings are described in the research paper 'Learning To Drive in a Day'.
Author: Wouter Hoeffnagel
Source: Wayve
Source: Research paper 'Learning To Drive in a Day'
