Dorian’s Development of Overstress Probe Testing
By Richard D. Shainin
In 1961, Dorian was engaged by Grumman to help with their bid for the Apollo program’s lunar module. The Apollo spaceship consisted of three modules: the command module, the service module and the lunar module. The command module would carry the astronauts to the moon and back. The service module was located below the command module and contained life-support resources such as oxygen tanks. The lunar module sat above the command module and was designed for the trip from the command module to the surface of the moon and back.
Once the ship achieved lunar orbit, two astronauts would move from the command module to the lunar module for the descent to the lunar surface. The lunar module had to perform a series of functions:
- Control descent to the lunar surface
- Land safely
- Ascend to Lunar orbit
- Rendezvous with the Command Module
- Dock with the Command Module
- Provide life support
If any of these functions failed there was a high chance that the astronauts wouldn’t be able to return safely to Earth.
The contracts for the Command Module and the Service Module had already been awarded to a large aerospace company, North American Rockwell. Everyone expected that they would also win the bid for the Lunar Module. Grumman was a manufacturer of Naval aircraft but was not a player in the space program. They saw the Lunar Module as their opportunity to expand into aerospace.
Dorian attended a NASA bidders conference with Grumman engineers. NASA insisted that the Lunar Module had to be 99.99% reliable. That would mean 9,999 successful missions out of 10,000. That is a very high reliability level. When Dorian asked how NASA had arrived at that standard, they said they had contacted the AAA (American Automobile Association) and asked the probability of a fatality if you drove from New York City to Los Angeles and back. Perhaps a better perspective was the fear that the astronauts would land safely on the moon, but would not be able to return to the command module.
In assessing the reliability of a design, there are two key elements. The reliability prediction and the confidence level for that prediction. In standard Weibull analysis it is common to use the median best fit line which would equate to a 50% confidence level in the prediction. In other words, you could predict a 99.99% reliability with a 50% confidence. That would mean that half the time, the product would meet or exceed the 99.99% reliability and half the time it would fall short.
Dorian developed an innovative approach to reliability testing with a modified use of Weibull that allowed Grumman to commit to 99.99% reliability with 90% confidence. That was superior to the competing bids and was a key factor in Grumman’s winning bid.
Dorian’s revolutionary approach was called Overstress Probe Testing. It contained the following elements:
- Samples were tested to failure.
- Tests were conducted under multiple combined environments.
- The environments were modeled under the best understanding of the expected conditions.
- When failures occurred, the modified Weibull predicted if the failure would occur within the expected operating conditions.
- Potentially unsafe failure modes led to improved designs.
- Safe failures, those beyond the expected operating conditions, meant the design was reliable.
- Testing started at the component level and advanced to assembly, sub-system and system testing which allowed flexibility in making design improvements.
Resulting improvements in the design of the Lunar Module led to 100% success in every mission. When the Apollo 13 mission experienced an explosion in the service module that damaged the command module life support systems, the astronauts were moved into the Lunar Module which acted as a life raft so that they could orbit the moon and then return safely to earth.
Subsequent to the successful development of the Lunar Module, Overstress Probe Testing has been applied to the reliable introduction of electronic displays for gas station fuel delivery pumps; the development of Detroit Diesel’s most reliable engine; the design of chemical tank cars for freight trains, a hybrid bus powertrain system, and numerous other products.
It remains a key element in Shainin’s Resilient Engineering toolbox for mitigating risk in the design and development of new products and processes.