Algorithms

The optimiser algorithms are provided by the free/open source library NLopt developed and maintained by Steven G. Johnson who is professor of Applied Mathematics and Physics at MIT.

The library contains over 20 optimiser algorithms, however, we have currently only implemented "Derivative-free" optimisation algorithms.That is algorithms that do not require the calculation of the partial derivatives (or sensitivities) of each objective and constraint with respect to every parameter. The calculation of each derivative requires one additional simulation run for each parameter and so is in general expensive. However, algorithms that do require derivatives to be calculated, usually converge much more quickly.

In this topic:

List of Available Algorithms

Here is a complete list of currently available algorithms. Use the value in the Code column as the value of the alg parameter for the .OPTIMISER statement or single-analysis optimiser statement.

Algorithm Code Details
Nelder-Mead NELDER_MEAD Unconstrained only. Recommended algorithm for unconstrained problems. Nelder-Mead is also known as Downhill Simplex and the paper that described this method was first published in 1966. Many researchers dismiss this algorithm as it is known to be capable of converging on a non-optimal point. While this is undoubtedly true, our experience is that it works well for many circuit simulation problems and is tolerant of the noise and discontinuities that can be present in that application
COBYLA COBYLA Supports constraints. Recommended algorithm for constrained problems. COBYLA (Constrained Optimization BY Linear Approximations) was developed by the late Michael Powell who was a professor of Mathematics at Cambridge university
Nelder-Mead with augmented Langrangian NELDER_MEAD_AL Supports constraints. Nelder-Mead coupled with Augmented Lagrangian, see notes below
BOBYQA BOBYQA Unconstrained only. Bound Optimization BY Quadratic Approximation. Developed by Michael Powell. This algorithm attempts to model the objective function as a quadratic which tends not to work that well in circuit simulation applications
BOBYQA with augmented Langrangian BOBYQA_AL Supports constraints. BOBYQA coupled with Augmented Lagrangian, see notes below
Subplex SBPLX Unconstrained only. This is a variant of Nelder Mead and claims to be superior. However this is not what we have found in circuit simulation applications
Subplex with augmented Langrangian SBPLX_AL Supports constraints. SBPLX coupled with Augmented Lagrangian, see notes below
PRAXIS PRAXIS Unconstrained only. Principal axis method developed by Richard Brent
PRAXIS with augmented Langrangian PRAXIS_AL Supports constraints. PRAXIS coupled with Augmented Lagrangian, see notes below

Augmented Lagrangian

The augmented Lagrangian algorithm is a derivation of Lagrangian multipliers and is used to convert a constrained problem into an unconstrained problem. Each of the unconstrained local optimiser methods have augmented Lagrangian variations that have the suffix _AL. Important: the augmented Lagrangian method does not normalise the constraints. To get the constraint functions to work with any of these methods, it is necessary to apply appropriate scaling to each constraint function. This is not the case with the COBYLA algorithm which supports constraints directly and does not require special scaling.