Getting it right by looking at what went wrong

Any time we work to make improvements - whether in our professional or personal lives – it is natural to look at what is going right and try to build on what appears to be a winning formula. But we must always be cautious of focusing too much on success and disregarding the important lessons to be learned from failure.

I am reminded of the story of a mathematician, Abraham Wald, who was working as part of a top-secret statistical research group in New York during the Second World War. The U.S. Air Force came to Wald to ask him to figure out how to place armour strategically on their airplanes to provide the best protection from being shot down while at the same time using the least amount of armour possible so that the planes remained fast and efficient.

Examining planes that had returned from battle, Air Force engineers saw that bullet holes tended to cluster along the main body of the airplane. Based on this observation, they determined that they should increase protective armour along the body of the plane because that’s where they were getting hit most often.

Wald took a different view. He knew little about airplanes and even less about war, but he knew about one critical statistical concept that applied here: survivor bias. In order to figure out how to armour the planes better, Wald said, they needed to examine the planes that were shot down, not those that returned. 

While that was impossible, he made one further simple but brilliant deduction. He theorized that the shot-down planes would have bullet holes where the surviving planes did not. He told the engineers to increase the armour where there were the fewest bullet holes on the surviving planes—near the engines—and not to bother with extra armour on the fuselage. Wald’s advice was successfully used by the U.S. Air Force in the next two wars, in Korea and Vietnam.

Wald’s thinking offers an important lesson for us here at CCO. As we work to improve health systems, we must remember that it is just as important to study things that don’t work so well as it is to study our successes.    

Questions or comments about this blog post? Contact us at publicaffairs@cancercare.on.ca