HW 5

Question 1

A) find conditions on the parameter that make the system lyapunov function decrease exponentially fast during flows tha also in expectation during jumps:

The lyapunov function is given by:

\[ V(x) = \frac{1}{4}|x_2 - x_1|^2 + \frac{1}{4}|x_2 - x^*|^2 + cx_3^2 (\phi (x_1) - \phi^*)\]

This Lypanove function is

1. positive definite: for all terms are positive, and \(\phi\) is strong convex, therefore \(\phi(x_1) - \phi^* > 0\) for all \(x_1 \neq x^*\), and \(\phi(x^*) - \phi^* = 0\).
2. radically unbounded: as \(x \to \infty\), \(V(x) \to \infty\).
3. negative definite:

break down the Lyaunov function into three terms,

\[ V(x) = V_1 + V_2 + V_3\]

where \(V_1 = \frac{1}{4}|x_2 - x_1|^2\), \(V_2 = \frac{1}{4}|x_2 - x^*|^2\), and \(V_3 = cx_3^2 (\phi (x_1) - \phi^*)\).

\[\begin{aligned} \dot{V_1} &= \dot{\frac{1}{4}|x_2 - x_1|^2} = \frac{1}{4} \dot{((x_2 - x_1)^T(x_2 - x_1))} \\ &= \frac{1}{4} (\dot{((x_2-x_1)^T)} (x_2 - x_1) + (x_2 - x_1)^T \dot{(x_2-x_1)}) \\ &= \frac{1}{2} (x_2 - x_1)^T (\dot{x_2} - \dot{x_1}) \qquad \text{Because } \dot{x_2} - \dot{x_1} \text{ is scalar} \\ &= \frac{1}{2} (x_2 - x_1)^T \left(-2cx_3 \nabla \phi(x_1) - \frac{2}{x_3}(x_2 - x_1)\right) \\ &= \frac{1}{2} (x_2 - x_1)^\top (-2cx_3 \nabla \phi(x_1)) + \frac{1}{2} (x_2 - x_1)^\top \left(- \frac{2}{x_3}(x_2 - x_1)\right) \\ &= -cx_3 (x_2 - x_1)^\top \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) \end{aligned}\]

\[\begin{aligned} \dot{V_2} &= \dot{\frac{1}{4}|x_2 - x^*|^2} = \frac{1}{4} \dot{((x_2 - x^*)^T(x_2 - x^*))} \\ &= \frac{1}{4} (\dot{((x_2-x^*)^T)} (x_2 - x^*) + (x_2 - x^*)^T \dot{(x_2-x^*)}) \\ &= \frac{1}{2} (x_2 - x^*)^\top \dot{x_2} \\ &= \frac{1}{2} (x_2 - x^*)^\top (-2cx_3 \nabla \phi(x_1)) \\ &= -cx_3 (x_2 - x^*)^\top \nabla \phi(x_1) \end{aligned}\]

\[\begin{aligned} \dot{V_3} &= 2cx_3 \left(\frac{1}{2}\right) (\phi(x_1) - \phi^*) + cx_3^2 \nabla \phi(x_1)^\top \dot{x_1} \\ &= cx_3 (\phi(x_1) - \phi^*) + cx_3^2 \dot{(\phi(x_1) - \phi^*)} \\ &= cx_3 (\phi(x_1) - \phi^*) + cx_3^2 \nabla \phi(x_1)^\top \dot{x_1} \\ &= cx_3 (\phi(x_1) - \phi^*)+ cx_3^2 \nabla \phi(x_1)^\top \left(\frac{2}{x_3} (x_2 - x_1)\right) \\ &= cx_3 (\phi(x_1) - \phi^*)+ 2cx_3 \nabla \phi(x_1)^\top (x_2 - x_1) \end{aligned}\]

Combine:

\[\begin{aligned} \dot{V}(x) &= \dot{V_1}(x) + \dot{V_2}(x) + \dot{V_3}(x)\\ &= -cx_3 (x_2 - x_1)^\top \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) - cx_3 (x_2 - x^*)^\top \nabla \phi(x_1) + cx_3 (\phi(x_1) - \phi^*)+ 2cx_3 \nabla \phi(x_1)^\top (x_2 - x_1) \\ &= (2cx_3 (x_2 - x_1)^\top - cx_3 (x_2 - x^*)^\top - cx_3 (x_2 - x_1)^\top) \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) + cx_3 (\phi(x_1) - \phi^*) \\ &= cx_3 [(x_2 - x_1) - (x_2 - x^*)] ^\top \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) + cx_3 (\phi(x_1) - \phi^*) \\ &= cx_3 (x^* - x_1) ^\top \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) + cx_3 (\phi(x_1) - \phi^*) \end{aligned}\]

since \(\phi\) is strong convex, we have \(\phi(x_1) - \phi^* \leq \nabla \phi(x_1)^\top (x_1 - x^*) - \frac{\mu}{2} |x_1 - x^*|^2\). Therefore,

\[\begin{aligned} \dot{V}(x) &\leq -cx_3 (x^* - x_1) ^\top \nabla \phi(x_1) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) + cx_3 \left(\nabla \phi(x_1)^\top (x_1 - x^*) - \frac{\mu}{2} |x_1 - x^*|^2\right) \\ &= -cx_3 (x^* - x_1) ^\top \nabla \phi(x_1) + cx_3 (\nabla \phi(x_1)^\top (x_1 - x^*)) - \frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) - cx_3 \frac{\mu}{2} |x_1 - x^*|^2 \\ &= -\frac{1}{x_3} (x_2 - x_1)^\top (x_2 - x_1) - cx_3 \frac{\mu}{2} |x_1 - x^*|^2 \\ &< 0 \end{aligned}\]

therefore, the lyapunov function is negative definite during flow, and the system is stable during flow.

given \(x_3 \in [T_{\min}, T_{\max}]\), the first term reached maximum when \(x_3 = T_{\max}\) (since \(\frac{1}{x_3}\) is smallest), and the second term reached maximum when \(x_3 = T_{\min}\) (since \(x_3\) is smallest). Both bounds hold simultaneously for any \(x_3 \in [T_{\min}, T_{\max}]\), so we can combine them:

\[ \dot{V} \leq -\frac{c\mu T_{\min}}{2} |x_1 - x^*|^2 - \frac{1}{T_{\max}} |x_2 - x_1|^2\]

exponentially fast decrease in flow:

to show the lyapunov function decrease exponentially fast during flow, we want to show that \(\frac{\dot{V}(x)}{V(x)} \leq -\alpha\) for some \(\alpha > 0\).

\[\begin{aligned} V_2(x) &= \frac{1}{4} |x_2 -x^*|^2 \\ &= \frac{1}{4} |(x_2 - x_1) + (x_1 - x^*)|^2 \\ &\leq \frac{1}{4} (|x_2 - x_1|^2 + |x_1 - x^*|^2 + 2|x_2 - x_1||x_1 - x^*|) \\ &\leq \frac{1}{4} (|x_2 - x_1|^2 + |x_1 - x^*|^2 + |x_2 - x_1|^2 + |x_1 - x^*|^2) = \frac{1}{2}|x_2 - x_1|^2 + \frac{1}{2}|x_1 - x^*|^2 \end{aligned}\]

since \(\phi(x)\) is strong convex function that has a gradient that is globally lipschitz, we have \(\phi(x_1) - \phi^* \leq \frac{L}{2} |x_1 - x^*|^2\). therefore, we have:

\[\begin{aligned} V(x) &= \frac{1}{4}|x_2 - x_1|^2 + \frac{1}{4}|x_2 - x^*|^2 + cx_3^2 (\phi (x_1) - \phi^*) \\ &\leq \frac{1}{4}|x_2 - x_1|^2 + \frac{1}{2}|x_2 - x_1|^2 + \frac{1}{2}|x_1 - x^*|^2 + cT_{\max}^2 \cdot \frac{L}{2} |x_1 - x^*|^2 \\ &= \frac{3}{4}|x_2 - x_1|^2 + \left(\frac{1}{2} + \frac{cLT_{\max}^2}{2}\right)|x_1 - x^*|^2 \end{aligned}\]

therefore, we have:

\[ \frac{\dot{V}(x)}{V(x)} \leq \frac{-\frac{c\mu T_{\min}}{2} |x_1 - x^*|^2 - \frac{1}{T_{\max}} |x_2 - x_1|^2}{\frac{3}{4}|x_2 - x_1|^2 + \left(\frac{1}{2} + \frac{cLT_{\max}^2}{2}\right)|x_1 - x^*|^2} \]

the coefficients of all terms are constants, therefore it is bounded by a constant. therefore, this system is exponentially decreasing during flow.

the requirements for the parameters are:

Jump analysis

After jump, the lyapunov function is:

\[ V(x^+) = \frac{1}{4}|x_2^+ - x_1^+|^2 + \frac{1}{4}|x_2^+ - x^*|^2 + c(x_3^+)^2 (\phi (x_1^+) - \phi^*) \]

since \(x_2^+ = vx_1 + (1-v)x_2\)

**when v = 0 (with probability p), we have **

\[ V(x^+)_{v = 0} = \frac{1}{4}|x_2 - x_1|^2 + \frac{1}{4}|x_2 - x^*|^2 + cT_{\min}^2 (\phi (x_1) - \phi^*) \]

the first two terms are the same as \(V(x^-)\), and the third term is smaller than \(cT_{\max}^2 (\phi (x_1) - \phi^*)\). therefore, this lyapunov function is smaller than \(V(x^-)\). And since the coefficients are constant, this also decrease exponentially in expectation during jump.

when v = 1 (with probability 1-p), we have

\[ V(x^+)_{v = 1} = \frac{1}{4}|x_1 - x^*|^2 + cT_{\min}^2 (\phi (x_1) - \phi^*) \]

expectation

\[\begin{aligned} E[V(x^+)] &= p \cdot V(x^+)_{v = 0} + (1-p) \cdot V(x^+)_{v = 1} \\ &= p \cdot \left(\frac{1}{4}|x_2 - x_1|^2 + \frac{1}{4}|x_2 - x^*|^2 + cT_{\min}^2 (\phi (x_1) - \phi^*)\right) + (1-p) \cdot \left(\frac{1}{4}|x_1 - x^*|^2 + cT_{\min}^2 (\phi (x_1) - \phi^*)\right) \\ &= p \left(\frac{1}{4} |x_2 - x_1|^2 + \frac{1}{4} |x_2 - x^*|^2\right) + \frac{1 - p}{4} |x_1 - x^*|^2 + cT_{\min}^2 (\phi (x_1) - \phi^*) \end{aligned}\]

since \(\phi\) is strongly convex, we have \(|x_1 - x^*|^2 \leq \frac{2}{\mu} (\phi(x_1) - \phi^*)\). therefore, we have:

\[\begin{aligned} E[V(x^+)] &\leq p \left(\frac{1}{4} |x_2 - x_1|^2 + \frac{1}{4} |x_2 - x^*|^2\right) + \frac{1 - p}{4} \cdot \frac{2}{\mu} (\phi(x_1) - \phi^*) + cT_{\min}^2 (\phi (x_1) - \phi^*) \\ &= p \left(\frac{1}{4} |x_2 - x_1|^2 + \frac{1}{4} |x_2 - x^*|^2\right) + \left(cT_{\min}^2 + \frac{1-p}{2\mu}\right) (\phi(x_1) - \phi^*) \end{aligned}\]

we want \(E[V(x^+)] \leq \lambda V(x)\) for some \(\lambda < 1\). Comparing term by term with \(V(x) = \frac{1}{4}|x_2 - x_1|^2 + \frac{1}{4}|x_2 - x^*|^2 + cT_{\max}^2(\phi(x_1) - \phi^*)\):

• momentum terms: need \(p \leq \lambda\)
• potential term: need \(cT_{\min}^2 + \frac{1-p}{2\mu} \leq \lambda c T_{\max}^2\)

from the potential term: \(\lambda \geq \frac{T_{\min}^2}{T_{\max}^2} + \frac{1 - p}{2c\mu T_{\max}^2}\)

for this to be less than 1, we need to have

\[ T_{\max}^2 > T_{\min}^2 + \frac{1 - p}{2c\mu} \]

B) suppose the parameter T is selected as \(T = e \sqrt{\frac{p}{2c\mu}}\). find the condition on p that guarantees that this selection of T is consistent with the conditions discovered in the previous item

substitute \(T = T_{\max}\)

\[\begin{aligned} T_{\max}^2 &> T_{\min}^2 + \frac{1 - p}{2c\mu} \\ e^2 \frac{p}{2c\mu} &> T_{\min}^2 + \frac{1 - p}{2c\mu} \\ e^2 p &> 2c\mu T_{\min}^2 + 1 - p \\ p(e^2 + 1) &> 2c\mu T_{\min}^2 + 1 \\ p &> \frac{2c\mu T_{\min}^2 + 1}{e^2 + 1} \end{aligned}\]

C) use the results from previous to solve the parameter estimation question where the goal is to efficiently estimate the parameter \(\theta \in \mathbb{R}^n\) that satisfy \(y(t) = \phi(t)^\top \theta\).

The least square cost is:

\[ \Phi(x_1) = \frac{1}{2M} \sum_{k=1}^M (\phi(t_k)^\top x_1 - y(t_k))^2 \]

Let \(\Phi = [\phi(t_1), \phi(t_2), \ldots, \phi(t_M)] \in \mathbb{R}^{n \times M}\), and \(Y = [y(t_1), y(t_2), \ldots, y(t_M)]^\top\), then we have:

\[ \Phi(x_1) = \frac{1}{2M} \|\Phi^\top x_1 - Y\|^2 \]

since \(Y = \Phi^\top \theta\), we have:

\[\begin{aligned} \Phi(x_1) &= \frac{1}{2M} \|\Phi^\top x_1 - \Phi^\top \theta\|^2 \\ &= \frac{1}{2M} \|\Phi^\top (x_1 - \theta)\|^2 \\ &= \frac{1}{2M} (x_1 - \theta)^\top \Phi \Phi^\top (x_1 - \theta) \end{aligned}\]

its gradient is therefore \(\nabla \Phi(x_1) = \frac{1}{M} \Phi \Phi^\top (x_1 - \theta)\)

The hessian of \(\Phi\) is given by \(\nabla^2 \Phi(x_1) = \frac{1}{M} \Phi \Phi^\top = \frac{1}{M} \sum_{k=1}^M \phi(t_k) \phi(t_k)^\top\).

to make it strongly convex, it would require \(\frac{1}{M} \Phi \Phi^\top \succeq \mu I\), and therefore require \(\Phi \Phi^\top\) to be positive definite.

The gradient \(\nabla \Phi(x_1) = \frac{1}{M} \Phi \Phi^\top (x_1 - \theta)\) is globally lipschitz with constant

\[\begin{aligned} L &= \frac{1}{M} \lambda_{\max}(\Phi \Phi^\top) \\ &= \frac{1}{M} \left\| \sum_{k=1}^M \phi(t_k) \phi(t_k)^\top \right\| \end{aligned}\]

for a finite number of M, this L is a constant. therefore, the gradient \(\nabla \Phi(x_1)\) is globally lipschitz. since the gradient is linear, that means there is always a finite amount of M, and therefore the condition for strong convexity and global lipschitz is always satisfied. therefore, the system is stable, and the parameter estimation will converge to the true parameter \(\theta\).

combined with the result from A), we have

1. \(M \geq n\) to make sure \(\Phi \Phi^\top\) is positive definite, and therefore the system is stable.
2. with B), we require \(p > \frac{2c\mu T_{\min}^2 + 1}{e^2 + 1}\) to make sure the system is exponentially decreasing during flow, and also decrease exponentially in expectation during jump, and therefore the system is stable.
3. and \(\lambda_{\min} (\frac{1}{M} \sum_{k=1}^M \phi(t_k) \phi(t_k)^\top) > 0\) to make sure the system is strongly convex, and therefore the system is stable.

Question 2

A) given a task of designing a contol law u able to globally stabilize a point \(x^*\) on the unit circle \(\mathcal{S}^1\) , design with the idea of syneristic hybrid control

since \(\dot{x} = uSx\), that means the part \(Sx\) is the tagent to the circle \(\mathcal{S}^1\) at x. To make the system always flow towards a point, for example \(x^*\), we need defined the current point as \([c(x), s(x)]\):

\[ c(x) = {x^*}^\top x, \quad s(x) = (x^*)^\top Sx \]

Construct the lyapunov function as \(V(x) = 1 - c(x) = 1 - {x^*}^\top x\).

1. positive definite: since both \(x\) and \(x^*\) are on the unit circle, we have \(|x| = |x^*| = 1\), therefore, we have \(x^*^\top x \leq |x^*||x| = 1\), and therefore \(V(x) = 1 - {x^*}^\top x \geq 0\). And \(V(x) = 0\) if and only if \({x^*}^\top x = 1\), which means \(x = x^*\).
2. radically unbounded: since \(\mathcal{S}^1\) is compact, \(x\) is bounded, so the lyapunov function is also bounded, this conditions is not needed.
3. negative definite:

\[\begin{aligned} \dot{V}(x) &= -{x^*}^\top \dot{x} = -{x^*}^\top uSx \\ &= -u {x^*}^\top Sx = -u s(x) \end{aligned}\]

therefore, if \(u\) and \(s(x)\) have the same sign, then \(\dot{V}(x) < 0\) for all \(x \neq x^*\).

Therefore Define $ u(x) = s(x) + q(1 - c(x))$

flow set/map:

\[ C = \{(x, q) \in \mathcal{S}^1 \times \{-1, 1\} : q \cdot s(x) \geq 0\}, \quad \dot{x} = (s(x) + q(1 - c(x)))Sx, \quad \dot{q} = 0 \]

jump set/map:

\[ D = \{(x, q) \in \mathcal{S}^1 \times \{-1, 1\} : q \cdot s(x) \leq 0\}, \quad x^+ = x, \quad q^+ \in \text{sgn}(s(x)) \]

therefore, the lyapunov derivative is:

\[ \dot{V}(x) = -s(x)^2 - q(1 - c(x))s(x) \]

\(-s(x)^2\) is always negative, and \(-q(1 - c(x))s(x)\) is also negative during flow since \(q \cdot s(x) \geq 0\) and \(1 - c(x) \geq 0\) on \(\mathcal{S}^1\). Therefore \(\dot{V} \leq 0\) during flow.

The only points where \(\dot{V} = 0\) are where \(s(x) = 0\), i.e., \(x = x^*\) or \(x = -x^*\).

At \(x = x^*\): \(c = 1\), \(s = 0\), so \(u = 0 + q \cdot 0 = 0\). The target is an equilibrium.

At \(x = -x^*\): \(c = -1\), \(s = 0\), so \(u = 0 + q \cdot 2 = 2q \neq 0\). The antipode is NOT an equilibrium — the bias term pushes the state away.

By LaSalle's invariance principle, the only invariant set is \(\{x^*\}\), giving global asymptotic stability.

B) matlab simulation

This is the graph of the value of lyapunov function under different starting angle \(\theta\):

and this is the graph of the change in angular error \(\theta(t)\) under different starting angle \(\theta\):