Making AI Sense of Road Signs


By Dr. Lance B. Eliot is the AI Insider for AI Trends and contributes regularly.

Road signs. Can’t live with them, can’t live without them. A few years ago, I was tricked (I claim) by a road sign that was being used by a motorcycle cop to write numerous traffic tickets. Here’s my story. In downtown Los Angeles, there is a popular one-way street that reaches another popular one-way street that is perpendicular to it. Everybody comes up to the corner to make a right turn from one of those streets onto the other. When there is a red light, of course traffic normally comes to a full stop prior to trying to make the right turn, and proceeds only when the pedestrian walkway is empty and otherwise it is legal to proceed. This particular right turn is well known and frequented by especially the evening heading-home-from-work downtown drivers.

So, having made that same turn hundreds of times, I drive up to it one evening around 6 p.m., I dutifully came to a full stop since the traffic light was red, I made sure there was no one in the pedestrian walkway, and then I made the turn. Voila, a motorcycle cop parked just a few yards up that street waves for me to pull over. My mind raced as I tried to figure out what I had done wrong. Nothing that I could imagine. I roll down my window and the officer asks me if I had seen the street sign posted at the corner. I indicated that I’ve seen the signs there many times, including that for example you can’t park along that street during the morning and evening commutes. He says to me, not those signs, the new one. Huh? Well, they had just put up a temporary sign that said that for the next two weeks there is no right-turn on red allowed between the hours of 5:30 p.m. to 7:30 p.m. on weekday evenings.

I grumbled to myself as he wrote the ticket, and he simultaneously pulled over a dozen other cars. This was going to be a bonanza for the Los Angeles city coffers. I drove around the block because I wanted to see this temporary sign. It was mounted onto a sign post that already had five other signs on it. Only someone that was directly looking at the post and that was intent on reading all five signs would have realized that the temporary one even existed. Friends thought I should go to court and fight the ticket, but I decided that there was no point in trying to fight city hall on this one.

Why all this rambling about a road sign? Because road signs are going to be crucial to the success of self-driving cars. I realize that such a statement seems odd on the face of things. How could old-fashion road signs make any difference to state-of-the-art self-driving cars. It’s simple. Self-driving cars need to obey the traffic laws and so they need to know what the signs say in order to comply with them. For those of you that instantly retort that in this electronic age that it is ridiculous to rely upon road signs, I can only say that until we are able to replace road signs with electronic versions, they are still the ruling kings of the road. The cost to replace conventional road signs with electronics ones is pretty high and so don’t hold your breath that we’ll suddenly see an overhaul of road signs to become digitally based.

You might also be thinking that certainly all road signs must already be captured someplace electronically, such as via Google’s mapping of the roads. You’d be surprised. There are indeed lots and lots of pictures and video that have roads signs somewhere embedded in them, but there is no nationwide and nor global database that has the road signs information electronically available for access. It is still the “I’ll know it when I see it” circumstance, namely as you drive down a road you’ll see a road sign and have to figure out what it says.

Plus, even if we did a snapshot in time, like let’s say we scanned Google road images captured a year ago, think about my example above of when I got nabbed. It was a road sign put up for just two weeks. These kinds of signs are popping up all the time. Now, I am not saying that in the entire realm of road signs that the new ones and the temporary ones are a big chunk, but it is just that even if you had the road signs in pictures and video, you’d still be looking at just one snapshot in time.

Speaking of which, you might be tempted to believe that you could just use a massive neural network to look at the pictures and videos of roads and readily find the road signs. This is not as easy as it sounds. Road signs are notorious for being obscured by tree branches. They are often covered in graffiti. They are bent by kids that like to hang on the signs. They are twisted away from the road and you can only catch a glimpse at it based on your angle of approach. They are sometimes hanging loose and the writing is weather worn. If you’ve ever seen the standard road signs shown in a book or driving guide, you know from real-world driving that they are rarely so pristine and readable.

Therefore, even if we had images of that street sign down there on Mulberry street, the odds of easily doing an automated scan to determine what it means is problematic. That’s the reason why AI can help out. Just as AI and machine learning can review thousands upon thousands of images of cats, and then generalize to what a cat looks like, similarly we can do the same with street signs. A startup called Mapillary is trying to do a crowdsourced approach to collecting photos from around the world of streets, which could potentially be used to have machine learning learn about street signs. They claim to have already collected 116,140,578 photos covering 2,462,653,892 meters of area. They allow a limited amount of free access to the data set and then charge for more access.

Here’s ultimately what we might have evolve about street signs and self-driving cars. There will be some enterprising entrepreneurs that put together AI-based systems that learn how to read street signs. Those making the self-driving cars will either tap into that aspect by licensing from those firms, or maybe buy such a firm, or maybe opt to do the same thing themselves. Either way, a self-driving car while driving will need to use its sensors, primarily cameras, in order to detect a road sign by its overall shape and position, and then flow the image over to its internal AI-specialist component that can figure out what the road sign says. Then, the road sign information will be relayed to the overarching self-driving car AI, which will take into account whatever the road sign purports to indicate.

The self-driving car might alter its plan of action based on the contents of the sign. For example, suppose the sign says the speed limit is 35 miles per hour. The self-driving car maybe was coming down a hill where it had just been going 45 miles per hour, and now has to reduce speed to accommodate the 35 mph. In other cases, the road sign might not invoke any reaction by the self-driving car. Where I live, there is a tsunami sign posted on the coast road that I use to get to work. I am guessing that the self-driving car would not need to take into account that the car itself is in a tsunami zone, though if the self-driving car can also turn into a boat then I had better buy one in case I encounter a tsunami.

In the future, the multitude of self-driving cars on the road might end-up solving the street signs recognition problem by sharing with other self-driving cars what they know about the street signs. I am not sure whether this is going to happen per se, since it might be that one maker of a self-driving car might want to claim that their self-driving car is better than other competing self-driving cars because it does a better job of recognizing street signs, and so they presumably would not want to share what their AI system knows about street signs.

You might laugh at the notion that when you go to buy a self-driving car that it might be distinctive by an ability to read street signs, but I wouldn’t be so quick to think it silly. Keep in mind that if we actually get to a situation where all cars are self-driving cars, somehow each maker has to find a means to make you want to buy their self-driving car over some other brand. It probably won’t be sold on the features we think of for today’s cars, and so a high-quality highly reliable street sign detector might just be one key factor that sways you toward the one brand that has it. Can’t you see the ads for this, broadcast during the Super Bowl, showcasing an “inferior” self-driving car that zooms past a sign that says “Sinkhole ahead” while the advertiser’s self-driving car comes to a halt to avoid the deathtrap. I ask you, which self-driving car would you want to buy?

This content is original to AI Trends.