stochastic LQ optimal control

A stochastic linear-quadratic (LQ) optimal control problem seeks to minimise a quadratic cost functional over adapted controls that drive a linear stochastic differential equation. The state equation takes the form $dx=[Ax+Bu+b]dt+{\sum }_j[C_{j}x+D_{j}u+{\sigma }_j]d{W}_j$ with initial condition $x(s)=y$, and the cost functional is $J(s,y;u)=E\{{\int }_s^T[(Qx,x)+2(Sx,u)+(Ru,u)]dt+(Gx(T),x(T))\}$. The goal is to find an admissible control $\bar{u}(\cdot )$ that minimises $J$.

The stochastic LQ problem is fundamentally different from its deterministic counterpart. In the deterministic case, the control weighting matrix $R(t)$ must be nonnegative definite for the problem to be well-posed. In the stochastic case, because the control can enter the diffusion term (through $D_j$), the “uncertainty cost” from the diffusion can compensate for negative control weighting. Concretely, the relevant quantity becomes $R(t)+{\sum }_j{D}_j(t{)}^{T}P(t)D_j(t)$, where $P(t)$ solves the stochastic Riccati equation. This means stochastic LQ problems with indefinite — even negative definite — $R$ can be well-posed, provided the control’s influence on the noise creates sufficient implicit cost.

This observation has deep implications for financial applications. In the mean-variance portfolio selection problem, for instance, $R=0$ (no direct control cost) but the problem is well-posed because the portfolio’s influence on wealth volatility creates an implicit cost. In XVA hedging, the Riccati system for XVA hedging inherits this structure: the hedging rates emerge from a modified Riccati equation where the control simultaneously affects drift and diffusion of the XVA position.

The problem is solved in three equivalent ways: (1) the stochastic maximum principle, which leads to the linear Hamiltonian system (a coupled FBSDE); (2) dynamic programming via the HJB equation; (3) the completion of squares technique. All three approaches produce the stochastic Riccati equation, and the optimal control takes a state feedback form $\bar{u}(t)=-\Psi (t)x(t)-\Theta (t)$ when the Riccati equation is solvable.

Key Details

• State equation: $dx=[Ax+Bu+b]dt+[Cx+Du+\sigma ]dW$ (one-dimensional Brownian motion case for simplicity)
• Cost functional: Quadratic in state and control, with possibly indefinite weighting matrices $Q$, $R$, $G$
• Standard case: $R\gg 0$, $Q-S^T{R}^{-1}S\ge 0$, $G\ge 0$ — always uniquely solvable
• Key difference from deterministic: $R$ need not be nonnegative; the condition $R(t)+D(t{)}^{T}P(t)D(t)\ge 0$ replaces $R(t)\ge 0$
• Weak formulation: The probability space and Brownian motion are part of the control, i.e., the 5-tuple $(\Omega ,\mathcal{F},P,W(\cdot ),u(\cdot ))$ is optimised over
• Connection to XVA: The HJB equation for XVA under P under CARA utility with quadratic friction reduces to a stochastic LQ-type problem, whose Riccati equation yields closed-form hedging rates

Textbook References

Stochastic Controls - Hamiltonian Systems and HJB Equations (Yong & Zhou, 1999)

• Definition 3.1 (p. 301): Finiteness, solvability, and pathwise unique solvability of Problem (SLQ)
• Theorem 4.2 (p. 308): Finiteness implies $N_r\ge 0$; solvability equivalent to $N_s\bar{u}+H_s(y)=0$; if $N_s\gg 0$ the unique minimiser is $\bar{u}=-N_s^{-1}{H}_s(y)$
• Examples 3.2—3.4 (pp. 302—304): Demonstrations that $R=0$, $R<0$, and $G<0$ can each lead to well-posed stochastic LQ problems (contrasting with deterministic impossibility)
• Theorem 6.1 (p. 315): Solvability of the stochastic Riccati equation implies solvability of Problem (SLQ) with explicit state feedback control

---

concept