Course

Syllabus

Lab Experiments:

1. Identifying definiteness of matrices using eigenvalues and use of Hessian matrix to identify concavity of the surfaces (revision from Calculus and Linear Algebra).
2. Implementation of Golden Section Search, Fibonacci search for single variable optimization problems.
3. Evaluation of ordinary and partial derivatives numerically (in excel/MATLAB).
4. Implementation of Secant method and Newton’s method for single variable optimization problems.
5. Implementation of evolutionary search method for multivariable optimization problems.
6. Implementation of Cauchy Steepest method for solving multivariable optimization problems.
7. Implementation of Newton’s method for solving multivariable optimization problems.
8. Identifying whether a constrained optimization problem is convex or not and solutions using ‘cvx’

Unit 1

Introduction to optimization: classical optimization, Optimality criteria Necessary and sufficient conditions for existence of optimum point. Fundamental regionelimination rules to eliminate a region. One-dimensional Search methods: Golden search method, Fibonacci method, Newtons method, Secant method.

Unit 2

Unconstrained Multivariable optimization: Introduction, Necessary and sufficient conditions for existence of extreme point. Remarks on line search method. Gradient-based methods- introduction, the gradient based directions, the method of steepest descent, convergence. Analysis of Newtons method.

Unit 3

Newtons method for nonlinear least-squares. Introduction -The conjugate direction, the conjugate direction algorithm, The conjugate gradient algorithm for unconstrained optimization problems.

Unit 4

Nonlinear equality constrained optimization- Introduction, Problems with equality constraints: problem formulation, Regular point, Tangent and normal spaces, Direct substitution method, Lagrange multiplier method.

Unit 5

Nonlinear inequality constrained optimization -Introduction-Linear programming problem, Graphical method. Problems with inequality constraints: Kuhn-Tucker conditions. Linear and nonlinear stochastic programming. Specific search algorithms: Genetic algorithm and particle swarm optimization method.

Text Books / References

Textbooks:

1. Edwin K.P. Chong, Stanislaw H. Zak, “An Introduction to Optimization”, 2nd edition, Wiley, 2021.
2. S. Rao, “Optimization Theory and Applications”, Second Edition, New Age International (P) Limited Publishers, Fourth edition, 2009.

References:

1. Kalyanmoy Deb, “Optimization for Engineering Design Algorithms and Examples”, Prentice Hall of India, New Delhi, 2012.
2. Mokhtar S. Bazarra, Hamit D Sherali, C.M. Shetty, “Nonlinear programming Theory and applications”, 2nd edition, Wiley, 2004.
3. Mohan C.  Joshi, Kannan M.  Moudgalya, Optimization:  Theory and Practice, Narosa Publishing House, New Delhi, 2004.
4. Bertsimas, Dimitris, and John Tsitsiklis. Introduction to Linear Optimization. Belmont, MA: Athena Scientific, 1997.

This course deals with the fundamentals of single- and multivariable optimization using calculus and numerical-based optimization techniques. Topics include single variable optimization: optimality criteria, region elimination methods, and gradient-based methods. Multivariable optimization: including optimality criteria, gradient-based methods, conjugate methods, and multivariable optimization with equality and inequality constraints, along with their applications.

Course Outcomes:

• CO1 : Understand different types of single variable optimization techniques in science and engineering optimization problems. Apply single-variable optimization methods, such as calculus and region elimination methods, to find the optimum of single-variable optimization problems.
• CO2: Apply the gradient-based methods for single-variable and multivariable optimization problems.
• CO3 : Understand the unconstrained multivariable optimization problems with the calculus method. Additionally, apply various multivariable optimization methods to find the optimum of unconstrained optimization problems.
• CO4 : Study the multivariable optimization with equality and inequality constraints. Also, understand how to apply the graphical, Lagrange and Kuhn-Tucker techniques for finding the optimum constrained optimization problems.
• CO5 : Understand the linear and nonlinear stochastic optimization programming problem. Also, learn the special algorithms such as Genetic algorithm and particle swarm optimization

CO-PO Mapping