Abstract
The vast majority of applied optimization falls into the category of first order optimization. This paper attempts to make the case for increased use of second order optimization techniques. Some of the most serious criticisms against second order methods are discussed and are shown to have lost some of their validity in recent years. In addition, some positive advantages of second order methods are also presented. These advantages include computational efficiency, compatibility with new advances in hardware and spill-over benefits in areas such as minimum sensitivity design. A simple second order constrained optimization algorithm is developed and several examples are solved using this method. A comparison is made with first order methods in terms of the number of function evaluations. The results show that the second order method performs much better than the first order methods in this regard. The paper also suggests some directions for future research in second order optimization.
