While Google racks up miles in their self driving car it is likely that they are collecting data for two big reasons. One is to train the car to better recognize its environment and operate intelligently within it; machine learning requires massive amounts of data. The second is that they should be able to quickly verify if a change they have made in software changes how the car behaves in scenarios it has already seen. After each change, any attempts to improve the car’s behavior, the software can be exposed to the same inputs it received while driving all of its cars past 1,000,000 miles. In this way Google can verify that it performs no worse than previous versions.
In software development this is called regression testing. Rather than making a change, then jumping in the car to see whether it works, you simply rerun the tests. In addtion, if you find a scenario where the car performed poorly, say it ran a stop sign, while actually driving the car, that scenario can be annotated with the desired behavior, “When the car sees a stop sign, it should stop before the white line.” Engineers would then modify the car until they got the desired result, passing the test for that scenario and feel confident that they haven’t made other behaviors in the car worse by solving for this one.
Here is what is interesting to me though. How can the public be sure that self driving cars are safe? I think that this sort of testing should be done in the open and be run by an independent organization. This is how the public could be sure that the latest version of Company X’s self driving car doesn't have a death wish. An independent test would expose the car’s software to a huge number of scenarios through either simulated sensory inputs or a public collection of sensor recordings, and then validate that the car responds appropriately. Essentially this allows the public to test drive a car in a compressed time span. A car could be exposed to decades worth of driving in a fraction of the time.
By making testing a public endeavor, positive and negative examples could be shared between manufacturers. If Google's car successfully avoids a woman chasing a duck with a broom, we can ensure than Mercedes performs as well. I can’t begin to imagine what the self driving car market will look like, but guaranteeing safety should be a collaborative, not a competitive effort.