Stochastic Controls - Hamiltonian Systems and HJB Equations

Textbook

Authors: Jiongmin Yong, Xun Yu Zhou | Publisher: Springer, 1999 | Series: Stochastic Modelling and Applied Probability 43

Overview

A comprehensive treatment of stochastic optimal control theory via Hamiltonian systems and Hamilton-Jacobi-Bellman equations. The book covers both deterministic and stochastic settings, with detailed development of the maximum principle, dynamic programming, and their interrelations. The linear-quadratic (LQ) framework and the associated Riccati equations receive thorough treatment, including the fundamentally different character of stochastic LQ problems where indefinite control weighting can be compensated by diffusion-control coupling.

Topics Studied

Deep BSDE methods for XVA (studied 22-03-2026)

Chapters read: Ch. 6 (pp. 281-342)

Key Definitions

• Definition 2.1 (p. 285): Finiteness and solvability of deterministic LQ problems — Problem (DLQ) is finite at $(s,y)$ if the infimum of the cost is finite; solvable if the infimum is attained
• Definition 3.1 (p. 301): Finiteness and solvability of stochastic LQ problems — Analogous to Definition 2.1 but in the weak formulation with adapted controls; includes pathwise unique solvability
• Definition 8.1 (p. 337): Efficient portfolio and efficient frontier for mean-variance portfolio selection

Key Theorems

• Theorem 2.2 (p. 286): Finiteness implies $N_r\ge 0$ for all $r\in [s,T]$; solvability equivalent to $N_s\bar{u}+H_s(y)=0$ with $N_s\ge 0$; if $N_s\gg 0$ the minimiser is $\bar{u}=-N_s^{-1}{H}_s(y)$
• Theorem 2.7 (p. 291): Under $R\gg 0$, solvability of deterministic LQ at $(s,y)$ is equivalent to solvability of the two-point boundary value problem for the linear Hamiltonian system (2.29)
• Theorem 2.8 (p. 294): If the Riccati equation (2.34) admits a solution $P(\cdot )$ on $[s,T]$, then Problem (DLQ) is uniquely solvable at $s$ with optimal feedback control $\bar{u}(t)=-R^{-1}[(B^{T}P(t)+S)\bar{x}(t)+B^T\phi (t)]$
• Theorem 2.9 (p. 296): Under $R\gg 0$, unique solvability of Problem (DLQ) at each $r\in [s,T]$ is equivalent to unique solvability of the Riccati equation on $[s,T]$
• Theorem 4.2 (p. 308): Stochastic analogue of Theorem 2.2 — finiteness and solvability conditions for Problem (SLQ) via the operator $N_s$ on the Hilbert space of adapted controls
• Theorem 5.1 (p. 309): Stochastic maximum principle for Problem (SLQ) — necessary conditions involve first and second adjoint equations; the condition $R(t)+D(t{)}^{T}P(t)D(t)\ge 0$ replaces $R(t)\ge 0$
• Corollary 5.2 (p. 310): If Problem (SLQ) is finite, then $R(T)+D(T{)}^{T}GD(T)\ge 0$ (necessary condition involving diffusion coefficient $D$)
• Corollary 5.6 (p. 312): Under $R^{-1}\in L^{\infty }$, solvability of Problem (SLQ) is equivalent to solvability of the coupled FBSDE (stochastic Hamiltonian system) (5.17), with optimal control $\bar{u}(t)=-R(t{)}^{-1}[S(t)\bar{x}(t)-B^T\bar{p}(t)-D^T\bar{q}(t)]$
• Theorem 6.1 (p. 315): If the stochastic Riccati equation (6.6) and associated equation (6.7) are solvable, then Problem (SLQ) is solvable with feedback control $\bar{u}(t)=-\Psi (t)x(t)-\Theta (t)$
• Theorem 7.2 (p. 320): Global existence and uniqueness of the stochastic Riccati equation in the standard case ($R\gg 0$, $Q-S^T{R}^{-1}S\ge 0$, $G\ge 0$), proved via a monotone iterative scheme with exponential convergence rate (Proposition 7.4)
• Theorem 7.5 (p. 325): For the case $C=0$, $S=0$, $Q,G\ge 0$: solvability of the stochastic Riccati equation is equivalent to the existence of a fixed point $\hat{R}=R+D^T\Gamma (\hat{R})D$ where $\Gamma$ is the solution operator of the deterministic Riccati equation
• Theorem 7.7 (p. 328): The stochastic Riccati equation (with $C=0$, $S=0$) admits a solution iff there exists $\hat{R}\in C([0,T];S_{+}^k)$ with $R+D^T\Gamma (\hat{R})D\ge \hat{R}$; in particular $R$ can be indefinite but not “too negative”
• Theorem 7.9 (p. 330): Complete characterisation of the maximal solvability interval for the one-dimensional stochastic Riccati equation with constant coefficients, via explicit formulas for $\Theta$ in all cases
• Theorem 8.2 (p. 338): Embedding of the mean-variance problem $P(\mu )$ into the auxiliary LQ problem $A(\mu ,\lambda )$; optimal portfolios of $P(\mu )$ are found via $A(\mu ,\lambda )$ with $\lambda =1+2\mu E\bar{x}(T)$
• Theorem 8.3 (p. 341): Efficient frontier: $\text{Var}\bar{x}(T)=\frac{e^{-{\int }_0^T\rho (t)dt}}{1-e^{-{\int }_0^T\rho (t)dt}}(E\bar{x}(T)-x_0{e}^{{\int }_0^{T}r(t)dt}{)}^2$ where $\rho (t)=B(t)(\sigma (t)\sigma (t{)}^T{)}^{-1}B(t{)}^T$

Atomic Notes

• stochastic LQ optimal control
• stochastic Riccati equation
• linear Hamiltonian system
• mean-variance portfolio selection
• finiteness and solvability of stochastic LQ problems
• feedback optimal control via Riccati equation

---

book