15 articles
Backtracking line search
Backtracking line search is an optimization procedure that repeatedly multiplies a trial step length by a constant factor until the Armijo sufficient-decrease condition holds, accepting the first adequate step.
Barrier method (optimization)
A barrier method solves inequality-constrained optimization by adding a logarithmic barrier that keeps iterates strictly feasible, and it underlies the interior-point methods used in modern conic optimization software.
DC programming
DC programming is an optimization framework for minimizing a function expressed as a difference of two convex functions, applied to nonconvex problems in machine learning and finance.
Direct search (optimization)
Direct search is a derivative-free optimization method that improves an objective function by sampling trial points and using function values alone, with no gradients or models.
Frank–Wolfe algorithm
The Frank–Wolfe algorithm, also called the conditional gradient method, is a projection-free first-order method for minimizing a smooth convex function over a compact convex set.
IPOPT
IPOPT, short for Interior Point OPTimizer, is an open-source software library for large-scale nonlinear optimization of continuous systems, developed under COIN-OR since 2002 by Andreas Wächter and Carl Laird.
Line search
A line search is a subroutine in numerical optimization that picks how far to step along a search direction, using conditions such as Armijo or Wolfe.
OSQP
OSQP (Operator Splitting Quadratic Program) is an open-source numerical optimization package, written in C, that solves convex quadratic programs using the alternating direction method of multipliers.
Pattern search (optimization)
Pattern search is a family of derivative-free direct search methods for nonlinear optimization that probes fixed directions with an adaptive step size, accepting steps only when the objective improves.
PDE-constrained optimization
PDE-constrained optimization is a class of optimization problems minimizing a cost functional subject to partial differential equations as constraints, underlying optimal control, inverse problems, and design, with the adjoint equation supplying gradients.
Penalty method
The penalty method is a numerical optimization technique that turns a constrained minimization problem into a sequence of unconstrained problems by penalizing constraint violations in the objective function.
Second-order cone programming
Second-order cone programming (SOCP) is a convex optimization method minimizing a linear function subject to Euclidean-norm constraints, used in portfolio optimization, engineering design, and machine learning.
Semidefinite programming
Semidefinite programming is a convex optimization technique that optimizes a linear function of a symmetric matrix under positive semidefinite constraints, generalizing linear programming and underpinning approximation algorithms, control theory, and machine learning.
Steepest descent method (optimization)
The steepest descent method is an iterative first-order method for minimizing a differentiable function by moving opposite the gradient; it dates to the mid-19th century and underlies gradient-based training.
Trust region
The trust region method is an iterative optimization technique that minimizes a local quadratic model within a bounded ball, adjusting the radius by how well the model predicts.