Self-Driving Cars Learning from Self-Driving Cars


By Dr. Lance B. Eliot, the AI Insider for AI Trends

When my daughter started to drive, I did my fatherly duty and sat in the passenger seat while providing gentle tips and suggestions on how to drive a car (ranging from putting the car into gear to strategies of freeway traversal). For anyone that has ever aided a novice driver, you know how terrifying it can be. There you are, urging to keep their foot on the brake, and the next thing you know they are stomping on the accelerator and the car is leaping forward. You watch as they struggle to make those corners or stop fully at stop signs. After each such session, I’d get out of the car with weak knees, but at least I knew that I was doing my part to help her become a full-fledged driver and do so with some insight about the nature of the driving task.  I’ll add that today, a few years later, she’s a wonderful driver, and as a passenger I sit back and enjoy the ride (though, perhaps a bit fast for my taste).

Next, when my son then came along and started to drive, I was better prepared and felt that I could do an even better job at coaching and guiding about how to drive a car. There is a twist though to this that I had not anticipated. Turns out that he had been learning from my daughter, and so he was already somewhat up the learning curve on driving a car. My daughter, being my eldest, did not have an older sibling to help teach her, but my son, realizing that his older sister could help, had enlisted her assistance. Who knew this would happen? I suppose that I should have guessed it would, since this same pattern had occurred for other things in life, whereby she would for example take a history class and then later on he’d take the same history class, and they’d compare notes and such.

In the realm of human learning, these are examples of an “expert” teaching a “novice” (that’s me teaching my children), along with an example of a “novice” teaching a fellow “novice” (that’s my daughter teaching my son). Now, you can argue about whether my daughter was still a novice when she was helping to teach my son, since she by then had a few years of driving experience under her belt. Let’s perhaps say she was a “semi-expert” driver, somewhat above a novice but still not quite what one might consider an expert driver. I suppose you could even contest whether I am an “expert” driver, which, I use that moniker simply due to having been a driver for many years and so somewhat think I am an expert at driving. Of course, if you compare me to say an Indie race car driver then I’m no longer an expert per se on a relative basis, and would rightfully agree that the race car driver is more of an expert than me.

Both of my children could have tried to learn on their own. I’d say they both did in the following sense – they each had watched my driving and had tried to glean what they could about driving by the manner in which I drive a car. This is a kind of self-learning, whereby there isn’t explicit instruction provided to the learner. Instead, the learner uses observation to try and learn the nature of a task. In some respects, your children will do this without being overtly aware, since they while being passengers in a car are soaking up the driving experience anyway. They subconsciously likely notice how fast you drive, how hard your stops are, and otherwise can by osmosis gain some semblance of the driving task. But, they can also be explicitly aware, such as the times that my son or daughter would notice that I suddenly swerved the car to avoid debris in the roadway, and they would then ask what had happened, why I had responded the way I did, etc. This was part of the subtle but indeed focused self-learning about this task.

What does this have to do with self-driving cars?

The answer is straightforward, namely that self-driving cars can and ought to be learning not only by being programmed but also by learning from other self-driving cars. I’ll let that soak in for a moment. It’s a key area of research and practice that we’re doing at the Cybernetic Self-Driving Car Institute.

There are robots in the manufacturing realm that are programmed to do a particular task. Let’s say that we want a robot arm to grasp a cup made of glass, and then we want the arm to swing over to a second position and place the cup safely onto a pedestal. Programmers can write instructions in specific robotic control programming languages that will step-by-step indicate what the robot arm is supposed to do. First step, swing the arm over to the cup. Second step, move the robot hand to the cup. Third step, grasp the cup. Fourth step, while grasping the cup then swing to the pedestal. Fifth step, while positioned over the pedestal then let go of the cup. Something like that would be the series of instructions.

We are pushing the envelope further in robotics by saying that maybe we don’t need to always be there to program a robot, and instead setup the robot to be able to learn a task by itself. In this manner, the robot can be readily “re-programmed” to do lots of tasks and it does not take a human programmer to be able to reprogram the robot.

One notion is to establish overall parameters and then let the robot via trial-and-error try to ascertain how to do a task.  This is sometimes referred to as a “robot learning in the raw” (the use of the word “raw” suggests that this type of learning is going to be often slow and ponderous, since it is based on a trial-and-error approach). For example, a robotic arm that was “learning” to grasp a cup had made over 800,000 attempts, and along the way it “realized” that grasping a soft object versus a hard object required differing amounts of grasping strength and pressure. The robot had not been programmed directly to softly grasp a soft object, and strongly grasp a strong object, but instead it had essentially deduced this by thousands of attempts at grasping various objects.  This robot had used a relatively large-scale convolutional neural network (CNN), and the artificial neural network had found these patterns over the course of the trial-and-error series.

Please note that I am wanting to be careful in my use of the word “learn” when it comes to robotics. The nature and type of learning that we are having robots do is not necessarily the same as humans do learning, and in fact we are still not fully sure how the human brain indeed learns things. So, I ask that you keep in mind that I am trying to avoid anthropomorphizing robots and assume that you are aware that I am aware that they aren’t doing robotic learning as we humans do learning per se.

Let’s suppose we are able to have a robot arm that successfully learns how to grasp an object. Could we have that robot then “teach” another robot to do the same thing? Or, alternatively, could we have one robot that “learns” from another robot about how to do that same thing? Well, whether you think this is a good idea for society and mankind, which some are worried it sounds like a doomsday scenario, it is certainly something of keen interest to see if it can be done. This is the latest focus for many in the robotics field.

We are doing the same with self-driving cars. Rather than having to program a self-driving car to be able to do the driving task, let’s have a self-driving car teach another self-driving car about how to drive. Or, alternatively, have a self-driving car learn from another self-driving car. The self-driving car that is teaching another self-driving car would presumably be doing so with the awareness that it is doing teaching. This is akin to my having taught my daughter and son how to drive. I was sitting in the passenger seat and knew that I was teaching, and they knew that I was teaching them. In contrast, the circumstance of them watching me drive, and subtly picking up clues about the driving task, that’s a case of them learning about driving but without necessarily my even knowing that I was “teaching” them – I was merely driving.

Here are the ways in which self-driving cars can learn:

a) Self-driving car learns from the collective wisdom of other self-driving cars

b) Self-driving car learns by programmatic insertion of knowledge from other self-driving cars

c) Self-driving car learns by passive observational learning of other self-driving cars

d) Self-driving car learns by active shared learning with other self-driving cars

We’re making use of each of these methods at our Lab.

I’ll address here each of these approaches briefly. Note that there is nothing mutually exclusive regarding these approaches, and as such you should keep in mind that a self-driving car could make use of one or more of these, and do so at the same time or at different points in time. This is not a one shoe size fits all aspect.

a) Self-driving car learns from the collective wisdom of other self-driving cars

This is well exemplified by the existing Tesla approach of learning for self-driving cars. A centralized collective is being used whereby the Tesla cars on-the-road are reporting into the collective global system with various aspects about their respective driving experiences. The collective can then share that “wisdom” down into the self-driving cars. This is done by the over-the-air updates that the Tesla cars are ready to receive.

For a collective approach, you should keep in mind that not everything that is being individually learned is necessarily of value or even appropriate for other self-driving cars to learn from. Suppose one car happens to learn that to navigate a bumpy road off highway 40 is done by going slowly for the first 90 yards and then doing an acute turn to the left. It is hard to imagine that this rather idiosyncratic aspect is going to provide any particular value to all the other cars in the collective.  On the other hand, suppose that a car has devised a better way to parallel park a car, in this case the collective could gain great value if the learned task is something that can be applied on a widespread basis.

b) Self-driving car learns by programmatic insertion of knowledge from other self-driving cars

For many self-driving cars, we are using artificial neural networks for purposes of machine learning and deep learning. If you’ve got a large-scale neural network that underlies say the freeway driving prowess of a certain model of self-driving car, the question arises whether you could take that neural network and plug it into another self-driving car so that it suddenly would have that same knowledge about how to drive on freeways.

Right now, there are numerous models for neural networks and thus it is difficult to share any piecemeal aspects of one neural network with another self-driving car. It is like one self-driving car that is based on VHS and the other based on Beta (historical note, these were famous standards factions during the early days of video tapes).

Furthermore, we don’t yet have any across the board standards of what a neural network consists of, and so trying to extract a portion of one and insert into another is problematic. It would be akin to trying to do brain surgery on a human and take part of the brain of one person and surgically place it into another person’s brain. We aren’t at a juncture where the human brain is well enough understood to do something like this. For artificial neural networks, in a somewhat crude analogous manner, we also cannot yet just carve out one chunk and place it into another artificial neural network such that we could transplant the neural knowledge from one robotic type system to another.

c) Self-driving car learns by passive observational learning of other self-driving cars

We’ve been having one self-driving car watch other self-driving cars, and try to glean from passive observations any tricks or techniques for driving a car. This is similar to what I was saying earlier about my children watching me drive a car. In the case of a self-driving car, the learner self-driving car does not necessarily let the other self-driving car even know that it is being observed. Instead, the self-driving car that is learning is just watching and trying to figure out whatever it can figure out on its own.

This makes sense too in that once we have lots of self-driving cars on the roadways, we might as well have them learning by observing each other.  I’ll also point out that the self-driving car can be learning by watching human driven cars too. There is nothing keeping the learner self-driving car from observing both human driven cars and also other self-driving cars. You could suggest that it is “better” for a self-driving car to learn from another self-driving car, since they are both of the same ilk. I get that idea, but at the same time the techniques being used by humans to drive a car offer plentiful insights too.

One important aspect to keep in mind about the passive observational learning mode is that the self-driving car learning in this fashion might learn the wrong thing. Back to my children and their watching me, suppose that each time that I got on the freeway and it was bumper-to-bumper traffic that I drove right up to the edge of being on the bumper of the car ahead of me. This is not a very safe way to drive. But, my children might not realize the dangerous aspect of such driving, and thus inadvertently learn how to drive in a rather unsafe manner. Any self-driving car that is learning by passive observation needs to have some added means to gauge whether the learning itself is worthwhile or not.

d) Self-driving car learns by active shared learning with other self-driving cars

In this mode, a self-driving car is engaged in a connection with another self-driving car, and the self-driving cars share what they know. This is a peer-to-peer form of learning. One self-driving car might be the teacher and be pretty much doing a one-way of sharing knowledge about driving over to the “novice” self-driving car. Or, they might both have aspects worthy of sharing with each other. Suppose one self-driving car is adept at driving in the city, while the other self-driving car is adept at driving on the open highways. The city driving self-driving car could share what it knows, and so the other self-driving car now has awareness about city driving, and likewise the open highway driving self-driving car shares what it knows with the city driving self-driving car.

They can share in various ways. One aspect of sharing is their respective collected sensory data. The city driving self-driving car has lots of data about city landscapes and so it can quickly pump over to the other self-driving car aspects about what city roads, signs, buildings, etc. are like. They can also potentially share their neural networks (though keep in mind my caveats earlier mentioned herein), and other AI code.

For all of the above approaches, there are aspects about learning that need to be kept at the forefront of doing any kind of self-driving car to self-driving car learning.

Here’s some key points:

1) Must be capable of learning

Most of the self-driving cars of today are being built with the idea that human programmers are essentially teaching the self-driving cars what they need to know. We need to make sure that self-driving cars have a learning capacity built into them. Without which, they will be somewhat stagnant and wholly dependent upon being human programmed.

2) Must be willing to learn

Besides having the capacity to learn, a self-driving car must be “willing” to learn in the sense that it is able to open itself to learning, when so appropriate. If the self-driving car is busy trying to drive and it is being tasked with some tough driving circumstance, that’s likely not the best time to turn-on the learning mode.

3) Must be available to learn

If a self-driving car is trying to convey to another self-driving car that is has some handy driving tip to share, the other self-driving car must have the capacity to learn and must be willing to learn and must be available to learn. For example, suppose the self-driving car is parked at the side of the road and waiting to be commanded by a human to undertake a driving task. This might be a good time for the self-driving car to learn something, perhaps doing so as other self-driving cars whisk past. Or, maybe while parked next to another self-driving car. I suppose you can imagine this as though it is two chauffeurs waiting for their driving gigs, each sharing some tips about driving.

4) Must be able to judge what is learned

There is a real danger that anything learned might be wrong or might be only usable in certain circumstances. A self-driving car cannot allow itself to blindly learn something. It needs to have a means to gauge whether or not what is learned has value and when to make use of it. This is especially important not only by inadvertently learning something that is amiss, but also as a means of security. If a self-driving car is willing to blindly learn anything, a nefarious person could have their self-driving car communicate to another self-driving car that it should go drive off a cliff. The self-driving car that is open to learning must be able to judge the appropriateness of what it is learning.

5) Must be able to apply learnings properly

Besides learning something from other self-driving cars, a self-driving car needs to know how to apply it correctly. Suppose a self-driving car that can accelerate from 0 to 60 miles per hour in 2 seconds tries to teach another self-driving car that it can escape from dangerous situations by rapid acceleration. But suppose that the other self-driving car is sluggish and it goes from 0 to 60 miles per hour in 8 seconds. The notion that rapid acceleration is a driving technique needs to be applied accordingly to the learning self-driving car.  It might not be able to use that technique or might need to adapt it according to its own ways and capabilities.

This whole topic of self-driving cars learning from other self-driving cars is an exciting means of pushing forward on getting self-driving cars to become true self-driving cars (i.e. reaching the vaunted Level 5). Perhaps a Level 4 self-driving car can get to become a Level 5 self-driving car by learning from some other self-driving car that is already at Level 5. This could tremendously speed-up getting self-driving cars up the learning curve. Does this also portend that we might have self-driving cars that become sentient beings and overrun humans? I don’t think so. For those that believe we are on the verge of a Frankenstein by making use of learning within self-driving cars, I just don’t see that this is a likely outcome. Of course, I realize some will say that I am blind to this aspect and that I am leading us toward the emergence of Skynet (per the movie “Terminator”). If so, all I can say is oops.