The Head Nod Problem in AI for Self-Driving Cars


By Dr. Lance B. Eliot, the AI Insider for AI Trends and contributes regularly.

I drove up to a four-way stop in my neighborhood the other day, and arrived just as another car happened to arrive (catty-corner of me). There we were, two cars and their human drivers, trying to ascertain which of us would proceed next. You’ve undoubtedly been in this same kind of stand-off. Should you go ahead, or should you wait for the other driver to proceed? You might inch forward, hoping to suggest you want to go first. Meanwhile, the other car inches forward too. Realizing that you both were starting to roll forward, you both abruptly come to a complete stop again. How long will this last? We might stay this way for a minute, five minutes, or maybe five years.

Finally, by a head nod, I signaled subtly to the other driver that he could make passage safely through the intersection and I would wait accordingly. The other driver hit the gas and went along his merry way. Accident averted. Another world crisis solved. By the rules-of-the-road in my state, he admittedly had right-of-way since the rule is that if two cars arrive at the four-way at the same moment in time, the one to the right gets to proceed. I think we were both unsure of whether we had exactly both arrived at the same instant of time. Neither of us wanted to make a leap in judgement that could produce a collision. I was not in a hurry and so was willing to just wait as long as needed to have the other driver go ahead.

This brings us to the topic of self-driving cars. You might be aware that some of the early trials of self-driving cars on-the-road were somewhat comical due to the four-way stop kind of circumstance. A self-driving car came up to a similar stop, and waited for the other car to proceed. In some cases, the other car, being driven by a human, opted to wait and see what the self-driving car would do (rightfully so, since one should be justifiably suspicious of these experimental self-driving cars). Thus, both cars just sat there for an inordinate amount of time, each waiting for the other to proceed. There were also cases of a self-driving car coming up to a stop and then detected that the other car was going ahead by essentially refusing to stop completely, so the self-driving car waited, even if it had right-of-way by the rules. Of course, as happens often at peak traffic periods, another human driven car came up to the four-way intersection, stopped just ever so briefly (a so-called “rolling stop” in cop parlance), and once again the self-driving car continued to remain stationary. Car after car, driven by humans, proceeded to do the same, and the self-driving car sat motionless since it refused to try and play chicken with these human driven cars.

Though perhaps comical, it brings up an important factor about self-driving cars and AI. Those early developers of those self-driving cars complained that humans are “bad” drivers and that it is the fault of those humans that the self-driving car got in a pickle. They further lamented that those darned humans should get off-the-road, and once we all have self-driving cars, and there are no human driven cars, there won’t be any need to worry about this issue. The self-driving cars will presumably stick to the letter of the law and so there won’t be any ambiguity about what is supposed to happen. To these remarks, I have but one word: dreamers!  There are an estimated 250 million cars in the United States alone, and I can assure you they are not going to magically be set aside for self-driving cars overnight. It will be years and years before those cars are replaced with self-driving cars and/or augmented to become self-driving cars (which is unlikely for various technical reasons).

In essence, any self-respecting self-driving car is going to have to learn how to deal with human driven cars. That’s a fact. If self-driving cars aren’t able to contend with human driven cars, you might as well then put your forecast for self-driving cars to become so far in the future that we can’t even see that date from here. There is going to be a mixture on-the-road of self-driven cars and human driven cars, at least for the foreseeable future (maybe someday this won’t be the case, but realistically it will be an instrumental and inevitable interim step toward a possible all-and-only self-driving car future). Sure, some cities are going to perhaps consider having some roads designated for human drivers and other roads for self-driving cars, in an effort to separate the two, but this “solution” is not especially tenable and exceedingly costly, so as to be impracticable generally.

It is a core principle that self-driving cars will need to be able to predict and deal with human driving idiosyncrasies. Furthermore, a robust self-driving car should also be able to deal directly with humans. By dealing directly, I am referring to the head nod. In my story above about coming to the four-way stop, the other driver looked at me, I looked at the other driver. We made eye contact, from afar. He saw me nod my head. He interpreted this to mean I was relinquishing the roadway to him. Humans do this all the time. It can literally be a head nod, or it could be a waving of the hand, or a mouthing of words, or even just a staring look of the eyes (you know, that piercing look that says go ahead even if I think you are a jerk, laden with emotion). Some of the software and hardware developers for self-driving cars are entirely missing the mark about this aspect, and they are not taking into account the human driver-to-driver communication that happens continually while driving our cars.

How can this head nod problem be dealt with? Sensors on the self-driving cars include sonar, radar, and other capabilities, including for some there are cameras on-board too. Via sensor data fusion, the AI of self-driving car tries to examine what is going on and make decisions about how to drive the car. The cameras are usually looking for things like a cow standing in the road, or a child’s ball that rolled into the street. These cameras also need to be able to visually detect other drivers, which is a vital element that needs to be included in any truly realistic self-driving cars. I know that some self-driving car developers will wince at this notion, since they are already so busy with trying to program so many other self-driving car technical matters.  Plus, as mentioned earlier, some of those developers are still using the simplistic belief that self-driving cars will take over the roads and therefore there aren’t going to be human drivers to deal with anyway.

I assert that the head nod problem is real, it is going to exist, it must be dealt with, and self-driving car developers should be coping with it. We already know that some of the underlying capabilities exist, for example consider how Facebook has popularly been able to do facial recognition to find your friends hidden in your Bahamas pictures that you posted. Indeed, there is a hot trend now of doing sentiment analysis of faces. You walk into a store to look at the latest shirts, and a camera captures your facial image, does a sentiment analysis, and maybe the retailer discovers that most people make an ugly face when they look at the shirts (probably time to redesign those Hawaiian prints). We can build upon this kind of facial and human expression recognition, and begin to tackle the head nod problem head-on. It is an undeniably thorny problem because it involves the subtleties of human movement and interaction, the moving of the head or arms, the waving of hands, etc. I am confident it can be figured out.

That being said, one question I often  get asked involves how does the self-driving car respond?  Unless we have android human-looking robots driving the car, there is no place for the human driver to vent their frustrations and not a ready means for the self-driving car to wave back or do a similar head nod. Imagine for a moment if we were having human-looking robots that drove self-driving cars, then it would be a human-to-robot style of communication. Logically, it might lead to some incredible road rage that might ensue — you can just imagine a human driver and a robot driver, they get out of their cars to settle a roadway dispute, and go to fists at the side of the road, angry over who did what while driving their cars.  Fortunately, we aren’t heading in that direction with our self-driving cars.

The reality for self-driving cars is that humans will be willing to make faces and gestures in the direction of the self-driving, regardless whether there are robotic eyes and hands waving back at them. A camera can detect these human responses by human drivers. Too, the self-driving cars will likely be outfitted with some exterior signaling to communicate toward the human driver. Keep in mind that all of this head nodding and discussion about the role of human drivers and self-driving cars is still nascent, and for many of the self-driving cars manufacturers and developers it is barely on their radar. They are trying to get the basics going first. I predict that those that are the more forward thinking are or will soon be working on the head nod problem.  Nod your head if you agree.