feedback optimal control via Riccati equation

The feedback optimal control for a linear-quadratic problem expresses the optimal control as a function of the current state rather than as a pre-committed open-loop trajectory. For both deterministic and stochastic LQ problems, when the associated Riccati equation is solvable, the optimal control takes the linear state feedback form $\bar{u}(t)=-\Psi (t)x(t)-\Theta (t)$, where the gain matrix $\Psi$ and offset $\Theta$ are deterministic functions determined by the Riccati solution $P(\cdot )$ and an auxiliary linear equation for $\phi (\cdot )$.

In the deterministic case with $R\gg 0$, the feedback control is $\bar{u}(t)=-R^{-1}[(B^{T}P(t)+S)x(t)+B^T\phi (t)]$, where $P(\cdot )$ solves the Riccati equation (2.34) and $\phi (\cdot )$ solves the linear equation (2.35). The value function is then $V(s,y)=\frac{1}{2}(P(s)y,y)+(\phi (s),y)+c(s)$, confirming that the optimal cost is quadratic in the initial state. The equivalence between the solvability of the Riccati equation and the solvability of the LQ problem (Theorem 2.9) makes this representation canonical.

In the stochastic case, the feedback control is $\bar{u}(t)=-\Psi (t)x(t)-\Theta (t)$ where $\Psi =(R+D^{T}PD{)}^{-1}(B^{T}P+S+D^{T}PC)$ and $\Theta =(R+D^{T}PD{)}^{-1}(B^T\phi +D^{T}P\sigma )$, with $P(\cdot )$ solving the stochastic Riccati equation (6.6). The gain matrix $\Psi$ now involves the diffusion coefficient $D$ through the term $D^{T}PD$, which is the signature of stochastic LQ control. The value function retains its quadratic structure.

The proof that this feedback control is optimal uses the completion of squares technique: $J(s,y;u)=J(s,y;\bar{u})+\frac{1}{2}E{\int }_s^T|(R+D^{T}PD{)}^{1/2}[u+\Psi x+\Theta ]{|}^{2}dt\ge J(s,y;\bar{u})$. This elegant argument does not require the maximum principle or HJB equation — it directly verifies optimality by exhibiting a non-negative remainder. Remarkably, the same Riccati equation emerges from all three approaches (maximum principle, dynamic programming, completion of squares), demonstrating the deep equivalence between these methods in the LQ setting.

Key Details

• Deterministic feedback: $\bar{u}=-R^{-1}[(B^{T}P+S)x+B^T\phi ]$ with $P$ solving (2.34) and $\phi$ solving (2.35)
• Stochastic feedback: $\bar{u}=-\Psi x-\Theta$ with $\Psi =(R+D^{T}PD{)}^{-1}(B^{T}P+S+D^{T}PC)$
• Value function: $V(s,y)=\frac{1}{2}(P(s)y,y)+(\phi (s),y)+f(s)$ — quadratic in initial state
• Completion of squares: Proof technique that exhibits $J(s,y;u)-J(s,y;\bar{u})$ as a non-negative integral
• Three derivations: Maximum principle (Hamiltonian system $\to$ $p=-Px-\phi$), HJB equation (quadratic ansatz for $V$), completion of squares — all yield the same Riccati equation
• Standard case: Under $R\gg 0$, $Q-S^T{R}^{-1}S\ge 0$, $G\ge 0$, the Riccati equation is globally solvable and $P(t)\ge 0$ for all $t$ (Corollary 2.10)
• Connection to XVA: The Riccati system for XVA hedging produces exactly such a feedback control — the mean-reverting hedging rates ${\nu }_t^{\ast }=-(M_{\theta \theta }/{\Lambda }_{\theta })({\theta }_t-{\theta }^{\text{tar}})$ are the stochastic LQ feedback form specialised to the XVA setting

Textbook References

Stochastic Controls - Hamiltonian Systems and HJB Equations (Yong & Zhou, 1999)

• Theorem 2.8 (p. 294): If the deterministic Riccati equation (2.34) is solvable on $[s,T]$, then Problem (DLQ) is uniquely solvable with feedback control (2.33) and quadratic value function (2.36)
• Theorem 2.9 (p. 296): Under $R\gg 0$, unique solvability of the LQ problem at each $r\in [s,T]$ is equivalent to unique solvability of the Riccati equation on $[s,T]$
• Corollary 2.10 (p. 297): Standard case — global solvability of Riccati with $P(t)\ge 0$
• Theorem 6.1 (p. 315): Stochastic Riccati solvability implies solvability of Problem (SLQ) with feedback (6.11); three equivalent derivations (maximum principle, HJB, completion of squares) all yield equation (6.6)
• Equations (6.14)—(6.19) (pp. 317—318): Dynamic programming derivation of the stochastic Riccati equation via quadratic ansatz for the value function

---

concept