How One-Factor-at-a-Time Experimentation Can Lead to Greater Improvements Than Orthogonal Arrays

This paper attempts to explain the empirically demonstrated phenomena that, under some conditions, one-at-a-time experiments outperform orthogonal arrays (on average) in parameter design of engineering systems. Five case studies are presented, each based on data from previously published full factorial experiments on actual engineering systems. Computer simulations of adaptive one-at-a-time plans and orthogonal arrays were carried out with varying degrees of pseudo-random error added to the data. The average outcomes are plotted for both approaches to optimization. For each of the five case studies, the main effects and interactions of the experimental factors are presented and analyzed to explain the observed simulation results. It is shown that, for some types of engineering systems, “one-at-a-time” designs consistently exploit interactions despite the fact that these designs lack the resolution to estimate interactions. It is also confirmed that orthogonal arrays are adversely affected by confounding of main effects and interactions.