Stochastic Control Optimization

A.Y. 2020/2021
Overall hours
Learning objectives
Basic problems, methods and results in the theory of optimization of stochastic dynamical systems will be presented. Both discrete-time and continuous-time models will be considered, over finite and infinite horizon. Continuous-time models will be mostly described by stochastic differential equations. The main approaches will be dynamic programming, the study of the Hamilton-Jacobi-Bellman equation (including cases of solutions with low regularity), backward stochastic differential equations, the stochastic maximum principle (in the sense of Pontryagin). A brief introduction to other optimization problems will be given, such as optimal stopping or impulse control, and applications to basic models will be presented, for instance optimal investment problems in Mathematical Finance or linear quadratic optimal control.
Expected learning outcomes
Students attending the course will become acquainted with various classes of control and optimization problems for stochastic systems (with discrete time, with continuous time and formulated by stochastic differential equations, on finite and infinite horizon). They will learn the basic methods to solve such problems: dynamic programming and Hamilton-Jacobi-Bellman equations, backward stochastic differential equations, the stochastic maximum principle. They will also see how important models can be analyzed, such as optimal investment problems in Mathematical Finance and linear quadratic problems.
Course syllabus and organization

Single session

Lesson period
First semester

The programme and the teaching material are unchanged.


Lectures and exercise classes will be on line.

Usually on line lectures will be syncronous, i.e. with both teacher and attendees on line at the same time in the official scheduled time, connected by the Microsoft Teams platform. Lectures will be recorded (both audio and video). The use of other platforms, for instance Zoom, will be preferred for specific reasons.

Asyncronous lectures will also be given, namely audio and video recordings of the lectures will be made available in advance, and during the official timetable the teacher will review this material, adding comments, or answering questions.

The teaching method for the exercise classes will be the same.

On the website of the course, in the Ariel platform, audio and video recordings of all lectures and exercise classes will be uploaded and kept available for the whole academic year. All the related electronic material - slides, latex slides, electronic manuscripts etc. - will also be added to the website.


The exam will be oral and on line. Students will be connected by a computer and will be framed by its webcam. At the same time a smartphone connection will be used to frame the sheet where formulae or statements are written by the students.

MS Teams or Zoom platform will be used for the required connections, but other technical solutions are also admitted, for instance if a graphical tablet is available.


Depending on the evolution of the sanitary situation the previous indications are subject to possibile changes. All changes will be notified on the Ariel website of the course.
Course syllabus
1) Discrete-time stochastic optimal control.

Stochastic controlled dynamical systems, payoff functionals over finite or infinite horizon. Value function, dynamic programming and the Hamilton-Jacobi-Bellman (HJB) equation. Applications to reference models.

2) Optimal control of stochastic differential equations.

Controlled stochastic differential equations, payoff functionals over finite and infinite horizon. Value function and the dynamic programming principle. Hamilton-Jacobi-Bellman (HJB) equations of parabolic and elliptic type. Verification theorems for regular solutions of the HJB equation. Linear-quadratic stochastic optimal control. Introduction to generalized solutions to the HJB equation, in the viscosity sense. Application to optimal portfolio problems.

3) Backward stochastic differential equations.

Formulation, existence and uniqueness results. Probabilistic representation of solutions to partial differential equations of semilinear type and of the value function of an optimal control problem. Stochastic maximum principle in the sense of Pontryagin.

4) Overview on other problems and methods.

Under certain circumstances, various other topics may also be presented, for instance control with partial observation, ergodic control, optimal stopping problems, optimal switching, impulse control.
Prerequisites for admission
Students attending the course are expected to have a relatively advanced knowledge of probability theory (based on measure theory). Other prerequisite topics for the course are: some notions on stochastic processes (in particular martingales and Markov processes), stochastic integration with respect to Brownian motion and related stochastic calculus, stochastic differential equations driven by Brownian motion; on these topics only some remainders will be given during the lectures.
Teaching methods
Classroom lectures. Attendance is not compulsory, but it is strongly recommended.
Teaching Resources

H. Pham. Continuous-time Stochastic Control and Optimization with Financial Applications. Springer, 2009.

Remark: this textbook only covers a part of the programme of the course. Lecture notes written by the teacher will cover many other topics. They will be made freely available on the course website.
Assessment methods and Criteria
The final examination consists of an oral exam.

In the oral exam, students will be required to illustrate results as well as case studies presented during the course, in order to evaluate her/his knowledge and understanding of the covered topics as well as the ability to apply them.

The final positive grade is given in the numerical range 18-30, possibly cum laude, and will be communicated immediately after the oral examination.
MAT/06 - PROBABILITY AND STATISTICS - University credits: 6
Practicals: 12 hours
Lessons: 35 hours
Educational website(s)
Monday, 10:30 am - 1:30 pm (upon appointment, possibly suppressed for academic duties)
Department of Mathematics, via Saldini 50, office 1017.