Driverless cars are inching closer than ever to becoming permanent fixtures on the road. But if, or when, cars become completely autonomous, there will still be a human element to them, especially with regard to those people who shun the robots behind the wheel, preferring to do the driving themselves. Many of these people will be the ones who cut corners while driving, never stopping fully at a stop sign, or changing lanes without signaling. Leading the way in self-driving car development, Google has found that it’s not the cars that are their biggest obstacle but rather the aforementioned corner-cutting drivers.

The problem with autonomous cars is that they are programmed to follow all rules to the "T." They won’t cut corners the way human drivers do. That means they’ll stop fully at crosswalks, ease gently to a stop at a red light, and wait for all pedestrians to move through a crosswalk before accelerating. This caused one of the Google test cars to run into a conundrum in 2009: It couldn’t find its way through a four-way stop, because human drivers wouldn’t stop fully at the stop sign. So, it just stayed there waiting for a chance to move forward.

“The real problem is that the car is too safe,” Donald Norman, director of the Design Lab at the University of California, San Diego, who studies autonomous vehicles, told The New York Times. “They have to learn to be aggressive in the right amount, and the right amount depends on the culture.”

The Google cars are still facing the hard truth that humans drive a little differently than robots. The cars are incredibly cautious drivers, but only because humans are much more reckless when they drive. “It’s always going to follow the rules, I mean, almost to a point where human drivers who get in the car and are like ‘Why is the car doing that?’” Tom Supple, a Google safety driver, told The New York Times during a recent test drive.

However, the caution isn’t without merit. Since 2009, the Google cars have only been in 16 accidents, all of them nothing more than a fender bender. Conversely, from January to June of this year alone, 19,000 people have died in car crashes. According to the company, however, humans were at fault for all 16 accidents. In one instance, a pedestrian was crossing a crosswalk, so the Google car slowed to stop, but a Google employee applied the brakes as well, thus causing a car to smash them from behind. This elicits the question: If a human wasn’t there, could the car have stopped closer to the crosswalk and avoided the accident? Google doesn’t have the answer.

Humans and machines coexist more than ever in this world, but it’s an imperfect mix. While autonomous cars might make our roads safer — and cut greenhouse gas emissions — they still have kinks that need ironing out. Developers need to find the perfect mix between aggression and safety. Humans, conversely, have always found a way to create accidents, and probably always will.

So, what do we do? Hand the keys over to driverless cars and see what happens? Continue to slowly implement better safety technology in our cars until one day accidents are a thing of the past? There’s no one definitive answer, but as time goes by and driverless cars get better at driving, we might have to turn over the keys and see what happens.