SF2852 Optimal Control, 2013, 7.5hp.

May exam 2013 can be found here:
exam a
solutions a

Homeworks and material from the lectures can be found on this page

Course home page address:
http://www.math.kth.se/optsyst/grundutbildning/kurser/SF2852/.

Examiner and lecturer:
Johan Karlsson, email: johan.karlsson@math.kth.se,
room 3710, Lindstedtsv 25, phone: 790 8440

Tutorial exercises:
Yuecheng Yang, yuecheng@kth.se,
room 3738, Lindstedtsv 25, phone: 790 7132.

Introduction
Optimal control is the problem of determining the control function for a dynamical system to minimize a performance index. The subject has its roots in the calculus of variations but it evolved to an independent branch of applied mathematics and engineering in the 1950s. The rapid development of the subject during this period was due to two factors. The first are two key innovations, namely the maximum principle by L. S. Pontryagin and the dynamic programming principle by R. Bellman. The second was the space race and the introduction of the digital computer, which led to the development of numerical algorithms for the solution of optimal control problems. The field of optimal control is still very active and it continues to find new applications in diverse areas such as robotics, finance, economics, and biology.

Course goals
The goal of the course is to provide an understanding of the main results in optimal control and how they are used in various applications in engineering, economics, logistics, and biology. After the course you should be able to

describe how the dynamic programming principle works (DynP) and apply it to discrete optimal control problems over finite and infinite time horizons,
use continuous time dynamic programming and the associated Hamilton-Jacobi-Bellman equation to solve linear quadratic control problems,
use the Pontryagin Minimum Principle (PMP) to solve optimal control problems with control and state constraints,
use Model Predictive Control (MPC) to solve optimal control problems with control and state constraints. You should also be able understand the difference between the explicit and implicit MPC control and explain their respective advantages,
formulate optimal control problems on standard form from specifications on dynamics, constraints and control objective. You should also be able to explain how various control objectives affect the optimal performance,
explain the principles behind the most standard algorithms for numerical solution of optimal control problems and use Matlab to solve fairly simple but realistic problems.

For the highest grade you should be able to integrate the tools you have learnt during the course and apply them to more complex problems. In particular you should be able to

explain how PMP and DynP relates to each other and know their respective advantages and disadvantages. In particular, you should be able to describe the difference between feedback control versus open loop control and you should use be able to compare PMP and DynP with respect to computational complexity.
explain the mathematical methods used to derive the results and combine them to derive the solution to variations of the problems studied in the course.

Course topics

Dynamic Programming Discrete dynamic programming, principle of optimality, Hamilton-Jacobi-Bellman equation, verification theorem.

Pontryagin minimum principle Several versions of Pontryagin Minimum Principle (PMP) will be discussed.

Infinite Horizon Optimal Control Optimal control over an infinite time horizon, stability, LQ optimal control.

Model Predictive Control Explicit and implicit model predictive control.

Applications Examples from economics, logistics, aeronautics, and robotics will be discussed.

Computational Algorithms The most common methods for numerical solution of optimal control problems are presented.

Course material
The required course material consists of the following lecture and exercise notes on sale at "institutionens elevexpedition", Lindstedtsv.

Ulf Jönsson et. al. Optimal Control, Lecture notes, KTH.
Peter Ögren et. al. Exercise Notes on Optimal Control , KTH.
Supplementary material will be handed out during the course.

Course requirements
The course requirements consist of an obligatory final written examination. There are also three optional homework sets that we strongly encourage you to do. The homework sets give you bonus credits in the examination.

Homework sets
Each homework set consists of three-five problems. The first two-three problems are methodology problems where you practice on the topics of the course and apply them to examples. The last problems are of more theoretical nature and helps you to understand the mathematics behind the course. It can, for example, be to derive an extension of a result in the course or to provide an alternative proof of a result in the course.
Each successfully completed homework set gives you maximally 2 bonus points for the exam. The bonus is only valid during the year it is acquired. The exact requirements will be posted on each separate homework set. The homework sets will be handed out in class roughly two weeks before the deadline. They will also be posted on the course homepage. You may email the solutions to the homework. If you choose to do so, the solution should be submitted as one pdf prepared in tex or comparable software.

Homework 1: This homework set covers problems on discrete dynamic programming and model predictive control. (Due on Wednesday April 17, at 15.14).

Here is the first homework set: [pdf]
[ Struktur.m] (Matlab structure for problem 3)

Homework 2: This homework set covers computational methods for solving optimal control problems. Results should be presented in seminar form on Wednesday May 8, at 15.15, and attendance is mandatory for receiving the bonus points. [pdf]

Homework 3: This homework set covers problems on PMP and related topics (Due on Wednesday May 15, at 15.14). [pdf]

Matlab code

Dynamic programming (Tracking with B747). Matlab files [fRic.m,fSys.m,PtoX.m,XtoP.m,mainRic.m]
Computational Algorithms (Shooting method for the orbit transfer problem). Matlab files [appr_dfdx.m,fh.m,main.m,newton_solver.m,plotresults.m,theta.m]

Written exam
You may use Beta Mathematics Handbook and the following formula sheet (pdf) . The exam will consist of five problems that give maximally 50 points. These problems will be similar to those in the homework assignments and the tutorial exercises. The preliminary grade levels are distributed according to the following rule, where the total score is the sum of your exam score and maximally 6 bonus points from the homework assignments (max credit is 56 points). These grade limits can only be modified to your advantage.

Total credit (points) Grade

45-56 A

39-44 B

33-38 C

28-32 D

25-27 E

23-24 FX

The grade FX means that you are allowed to make an appeal, see below.

The next exam will take place on Thursday May 23, 2013 at 08.00-13.00. You need to register for the exam during the period April 15 to May 5, 2012.

Appeal
If your total score (exam score + maximum 6 bonus points from the homework assignments and the computational exercises) is in the range 23-24 points then you are allowed to do a complementary examination for grade E. In the complementary examination you will be asked to solve two problems on your own. The solutions should be handed in to the examiner in written form and you must be able to defend your solutions in an oral examination. Contact the examiner no later than three weeks after the final exam if you want to do a complementary exam.

Course evaluation
At the end of the course you will be asked to complete a course evaluation form. The evaluation form will be posted on the course homepage and it can be handed in anonymously in the mailbox opposite to the entrance of "studentexpeditionen" on Lindstedtsv 25. We appreciate your candid feedback on lectures, tutorials, course materials, homeworks and computer exercises. This helps us to continuously improve the course.

Tentative schedule for 2013

Type Day Date Time Hall Topic Content

L1 Tue 19/3 10-12 E36 Introduction
Discrete dynamic programming
Pages 1-21
L2 Wed 20/3 15-17 E36 Discrete dynamic programming
Discrete PMP
Theorem 1, pages 22-23
Oven example, Knapsack problem
E1 Thu 21/3 13-15 E36 Discrete dynamic programming
Linear systems
Economic growth problem (Ex. 1.1), Discrete DynP
Recap Linear systems
L3 Mon 25/3 15-17 E53 Discrete dynamic programming
Infinite time horizon
Pages 24-28
L4 Tue 26/3 10-12 E36 Model predictive control
Pages 28-29
MPC (See handout)
E2 Thu 28/3 15-17 E53 Model predictive control
Infeasibility and
instability examples.
L5 Tue 9/4 10-12 D41 Dynamic programming
Pages 35-41
L6 Wed 10/4 15-17 D32 Dynamic Programming
Pages 41-44, 5-11
E3 Thu 11/4 15-17 E53 Dynamic Programming
Ex. 2.1, 2.2, 2.3
L7 Wed 17/4 13-15 D41 Mathematical preliminaries (ODE theory etc)
Pages 47-54
L8 Thu 18/4 15-17 E36 Pontryagins minimum principle (PMP) (using small variations)
Pages 59-65
E4 Fri 19/4 15-17 E36 PMP I
Ex. 3.2, 3.3
L9 Tue 23/4 10-12 D41 PMP (control constraints)
Pages 71-75, 62-63
L10 Wed 24/4 13-15 D42 PMP (optimal control to a manifold)
Pages 76-81,
nonregular example
E5 Fri 26/4 15-17 D32 PMP II: Time optimal control
Ex. 4.1. Exam problem.
L11 Mon 29/4 15-17 D32 PMP (generalizations)
Pages 82-87
E6 Tue 30/4 8-10 D42 PMP III
Ex. 5.1
L12 Thu 2/5 15-17 D32 PMP applications
Pages 90-99
L13 Mon 6/5 10-12 E33 Infinite time horizon optimal control
Pages 100-109
E7 Tue 7/5 10-12 D41 PMP IV

L14 Wed 8/5 15-17 E33 Computational methods - Seminar (student presentation)

L15 Mon 13/5 10-12 E33 Infinite time horizon optimal control

E8 Tue 14/5 15-17 E36 Infinite time horizon optimal control

E9 Wed 15/5 15-17 E36 Mixed Topics

Back-up Thu 16/5 15-17 E36 Back-up time

Welcome!