linear Hamiltonian system

A linear Hamiltonian system is a coupled forward-backward system of differential equations that characterises the optimal pair of a linear-quadratic control problem. In the deterministic case, it is a two-point boundary value problem coupling the state equation (forward) with the adjoint equation (backward). In the stochastic case, it becomes a forward-backward stochastic differential equation (FBSDE) whose solvability is equivalent to the solvability of the stochastic LQ problem.

In the deterministic setting with $R\gg 0$, the Hamiltonian system takes the form: $\dot{x}=(A-B{R}^{-1}S)x+B{R}^{-1}{B}^{T}p+b$, $\dot{p}=-(A-B{R}^{-1}S{)}^{T}p+(Q-S^T{R}^{-1}S)x$, with boundary conditions $x(s)=y$ and $p(T)=-Gx(T)$. The optimal control is recovered as $\bar{u}(t)=R^{-1}[B^{T}p(t)-Sx(t)]$. When the cost matrices satisfy the standard conditions ($R\gg 0$, $Q-S^T{R}^{-1}S\ge 0$, $G\ge 0$), this system has a unique solution that completely characterises the optimum.

In the stochastic setting, the Hamiltonian system becomes the coupled FBSDE: $dx=[Ax+Bu+b]dt+[Cx+Du+\sigma ]dW$, $dp=-[A^{T}p+C^{T}q-Qx-S^{T}u]dt+qdW$, with $x(s)=y$, $p(T)=-Gx(T)$, and the stationarity condition $Ru+Sx-B^{T}p-D^{T}q=0$. The process $q(\cdot )$ arises from the martingale representation in the backward equation and has no deterministic counterpart. When $R^{-1}$ exists, the stationarity condition can be substituted to produce a fully coupled linear FBSDE in $(x,p,q)$ alone. The optimal control is then $\bar{u}(t)=-R^{-1}[Sx(t)-B^{T}p(t)-D^{T}q(t)]$.

The passage from the Hamiltonian system to the stochastic Riccati equation is achieved by conjecturing the linear relationship $p(t)=-P(t)x(t)-\phi (t)$ between the adjoint and state processes. Substituting into the FBSDE and matching coefficients produces the Riccati equation for $P(\cdot )$ and an auxiliary linear equation for $\phi (\cdot )$. This is the key step that transforms the FBSDE characterisation into a feedback control representation.

Key Details

• Deterministic form: Two-point boundary value problem (2.29) coupling state $x$ and costate $p$
• Stochastic form: Coupled FBSDE (5.13)—(5.14) with additional martingale integrand $q(\cdot )$
• Stationarity condition: $Ru+Sx-B^{T}p-D^{T}q=0$ (stochastic analogue of the maximum condition)
• Equivalence: Under $N_s\ge 0$ and $R^{-1}\in L^{\infty }$, the FBSDE characterises optimality completely (Corollary 5.6)
• Connection to BSDE: The backward component $(p,q)$ is a linear BSDE; the nonlinear Feynman-Kac connection applies when moving to the value function PDE

Textbook References

Stochastic Controls - Hamiltonian Systems and HJB Equations (Yong & Zhou, 1999)

• Theorem 2.3 (p. 289): Necessary conditions from the maximum principle produce the deterministic Hamiltonian system with $R\ge 0$
• Theorem 2.7 (p. 291): Under $R\gg 0$ and $N_s\ge 0$, solvability of Problem (DLQ) at $(s,y)$ is equivalent to solvability of the two-point boundary value problem (2.29)
• Theorem 5.1 (p. 309): Stochastic maximum principle yields the Hamiltonian system (5.1)—(5.3) with first and second adjoint processes; the condition $R(t)+D(t{)}^{T}P(t)D(t)\ge 0$ replaces $R(t)\ge 0$
• Corollary 5.6 (p. 312): Under $R^{-1}\in L^{\infty }$ and $N_s\ge 0$, Problem (SLQ) is solvable iff the coupled FBSDE (5.17) admits an adapted solution
• Proposition 5.5 (p. 312): Full characterisation of optimality via FBSDE (5.13)—(5.14) with stationarity condition

---

concept