Rotor imbalance suppression by optimal control
Matteo Gnuffi Dario Pighin Noboru Sakamoto
math.OC

	Fragmentation in Population III Galaxies formed through Ionizing Radiation
    [
    December 30, 2023
==========================================================================


	Matteo Gnuffi
	
		Marposs SpA
		
		40010 Bentivoglio, Italy
	 
	
	
	Dario Pighin^*
	
		Departamento de Matemáticas, Universidad Autónoma de Madrid
		
		28049 Madrid, Spain
	 
	
		Chair of Computational Mathematics, Fundación Deusto
		
		Avda. Universidades,
			24, 48007, Bilbao, Basque Country, Spain
	 

	

Noboru Sakamoto

	Faculty of Science and Engineering, Nanzan University
	
	Yamazato-cho, Showa-ku, Nagoya, 464-8673, Japan
 

	
	
	
	
	
	
	
	
An imbalanced rotor is considered. A system of moving balancing masses is given. We determine the optimal movement of the balancing masses to minimize the imbalance on the rotor. The optimal movement is given by an open-loop control solving an optimal control problem posed in infinite time. By methods of the Calculus of Variations, the existence of the optimum is proved and the corresponding optimality conditions have been derived. Asymptotic behavior of the control system is studied rigorously. By Łojasiewicz inequality, convergence of the optima as time t→ +∞ towards a steady configuration is ensured. An explicit estimate of the convergence rate is given. This guarantees that the optimal control stabilizes the system. In case the imbalance is below a computed threshold, the convergence occurs exponentially fast. This is proved by the Stable Manifold Theorem applied to the Pontryagin optimality system. Moreover, a closed-loop control strategy based on Reinforcement Learning is proposed. Numerical simulations have been performed, validating the theoretical results.
		
	
	
	
	

§ INTRODUCTION

	
    Imbalance vibration affects several rotor dynamic systems. Indeed, often times, rotor's mass distribution is not homogeneous, due to wear, damage and other reasons. The purpose of this paper is to present a control theoretical approach to rotors imbalance suppression. A balancing device, made of moving masses, is given. We look for the optimal movement of a system of balancing masses to minimize the vibrations.



The topic is very classical in engineering literature. Indeed, balancing devices are ubiquitous in rotor dynamic systems. 
For instance, grinding machines often get deteriorated during their operational life-cycle. This leads to dangerous imbalance vibrations, which affects their performances while shaping objects (see, for instance,  <cit.>). 
Imbalance is a significant concern for wind turbines as well. In this case, the imbalance may affect the efficiency of power production and the life-cycle of the turbine. If the vibrations become too large, the turbine may collapse. For this reason, vibration detection and correction systems have been developed (see the U.S. patent <cit.>). 
Balancing devices have been developed to stabilize CD-ROM drives, washing machines and spacecrafts (see <cit.>). 
Another classical topic in engineering is car's wheels balance. Indeed, easily the wheels can go out of alignment from encountering potholes and/or striking raised objects. Misalignment may cause irregular wear of the tyres. Suspensions components may be damaged as well. For this reason, refined machines have been designed for wheel balancing (see, e.g., <cit.>).
Vibrations suppression may also involve optimized fluids, like magnetorheological fluids <cit.>.
The classical engineering literature on imbalance suppression is concerned with imbalance detection and/or imbalance correction.

In the present work, we address the imbalance correction problem. The imbalance is an input. We consider an imbalanced rotor rotating about a fixed axis at constant angular velocity. We work in the general case of dynamical imbalance, where the imbalanced rotor exert both a force and a torque on the rotation axle. In this context, we suppose that two balancing heads are mounted along two planes orthogonal to the rotation axis. It is assumed that the balancing heads are integral with the rotor, i.e. they rotate together with the rotor. Each balancing head is made of two masses, free to rotate about the rotation axis. Their angular movements are measured with respect to a rotor-fixed reference frame. We can control the balancing masses by actuators transmitting the power with or without contact.

An initial configuration of the balancing masses is given. Our goal is to determine four angular trajectories steering the masses from their initial configuration to a steady configuration, where the balancing masses compensate the imbalance. Note that, differently from the classical wheel balancing machines, our balancing device rotates together with the rotor and the rotor is moving while the balancing procedure is accomplished. This motivates us to formulate the problem as a dynamic optimization problem so that transient responses are also taken into account. The method is designed for high speed applications.


A control problem is formulated. We exhibit an open-loop control strategy to move the balancing heads from their initial configuration to a steady configuration, where they compensate the imbalance of the rotor. First of all, viewing the problem in the framework of the Calculus of Variations, the existence of the optimum is proved and the related Euler-Lagrange optimality conditions have been derived. Then, asymptotic behavior of the control system is analyzed rigorously. By Łojasiewicz inequality, the stabilization of the optimal trajectories towards steady optima is proved. In any condition, explicit bounds of the convergence rate have been obtained. In case the imbalance is below a given threshold, we provide an exponential estimate of the stabilization. Such exponential decay is obtained, by seeing the problem as an optimal control problem, thus writing the Optimality Condition as a first order Pontryagin system. In this context, we prove the hyperbolicity of the Pontryagin system around steady optima, to apply the Stable Manifold Theorem (see <cit.> and <cit.>). Our conclusions fit in the general framework of Control Theory and, in particular, of stabilization, turnpike and controllability (see e.g. <cit.>). In addition, we propose a closed-loop approach using Reinforcement Learning <cit.>.

The remainder of the manuscript is organized as follows. In section <ref>, we conceive a physical model of the rotor together with the balancing device. In section <ref>, we formulate a control problem to determine stabilizing trajectories for the balancing masses. We summarize our achievements in Proposition <ref>. The steady problem is analyzed in subsection <ref>, where the steady optima are determined. In subsection <ref>, we prove some general results. In Proposition <ref>, the existence of the global minimizer is proved. In Proposition <ref>, the Optimality Conditions are deduced in the form of Euler-Lagrange equations or equivalently as a state-adjoint state Pontryagin system.

In Proposition <ref> and Proposition <ref> the asymptotic behaviour of the optima is analyzed in the spirit of stabilization and turnpike theory (see <cit.>). The Łojasiewicz inequality is employed to show that, in any condition, the optima stabilize towards a steady configuration. In case the imbalance does not violate a computed threshold, the stabilization is exponentially fast. This is shown as a consequence of the hyperbolicity of the Pontryagin system around steady optima and the Stable Manifold Theorem. Numerical simulations are performed in subsection <ref>. The exponential stabilization of the optima emerges, thus validating the theoretical results. In section <ref>, Reinforcement Learning is employed to design a feedback solution. The notation is introduced in table <ref>.





§ THE MODEL


Assume the rotor is a rigid body Ω⊂ℝ^3 rotating about an axis at a constant angular velocity ω. Often times the rotor mass distribution is not homogeneous, producing imbalance in the rotation. This leads to dangerous vibrations. Our goal is to find the optimal movement of a system of balancing masses in order to minimize the imbalance.

Consider (O;(x,y,z)) Ω-fixed reference frame.


By definition, the axes (x,y) rotate about axis z at a constant angular velocity ω.





The balancing device (see figures <ref> and <ref>) is made up two heads lying in two planes orthogonal to the rotation axis z. Each head is made of a pair of balancing masses, which are free to rotate on a plane orthogonal to the rotation axis z.  Namely, we have



	
  * two planes π_1{z=-a} and π_2{z=b}, with a, b≥ 0;
	
	
  * two mass-points (m_1,P_1,1) and (m_1,P_1,2) lying on π_1 at distance r_1 from the axis z, i.e.,

	in the reference frame (O;(x,y,z))
	
    P_1,1;x=   r_1cos(α_1-γ_1)
    
    	P_1,1;y=   r_1sin(α_1-γ_1)
    
    	P_1,1;z=   -a,
    
    
    
    	P_1,2;x=   r_1cos(α_1+γ_1)
    
    	P_1,2;y=   r_1sin(α_1+γ_1)
    
    	P_1,2;z=   -a;

	
  * two mass-points (m_2,P_2,1) and (m_2,P_2,2) lying on π_2 at distance r_2 from the axis z, namely, in the reference frame (O;(x,y,z))
	
    P_2,1;x=   r_2cos(α_2-γ_2)
    
    	P_2,1;y=   r_2sin(α_2-γ_2)
    
    	P_2,1;z=   b,
    
    
    
    	P_2,2;x=   r_2cos(α_2+γ_2)
    
    	P_2,2;y=   r_2sin(α_2+γ_2)
    
    	P_2,2;z=   b.


For any i=1,2, let b_i be the bisector of the angle generated by ⟶OP_i,1 and ⟶OP_i,2 (see figure <ref>). For any i=1,2, the intermediate angle α_i is the angle between the x-axis and the bisector b_i, while the gap angle γ_i is the angle between ⟶OP_i,1 and the bisector b_i.

Note that the angles α_i and γ_i are defined with respect to the Ω-fixed reference frame (O;(x,y,z)). Indeed, the balancing device described above is integral with the body Ω. Furthermore, we observe that on the one hand, in view of avoiding the generation of torque in each single head, the two balancing masses composing a single head are placed on a single plane. On the other hand, the available balancing heads are placed on two separate planes and torque may be generated by the composed action of the heads.


Following a classical approach, the imbalance may be described as the force F and the momentum N exerted by the imbalanced body Ω on the rotation axis. The force is applied at the origin O. The momentum is computed with respect to the pole O. Both the force and the momentum are supposed to be orthogonal to the rotation axis z. As we mentioned, F and N are given data.

In (O;(x,y,z)), set P_1 (0,0,-a), P_2 (0,0,b), F (F_x,F_y,0) and N (N_x,N_y,0). By imposing the equilibrium condition on forces and momenta, the force F and the momentum N can be decomposed into a force F_1 exerted at P_1 contained in plane π_1 and a force F_2 exerted at P_2 contained in π_2

    F_1=
    1/a+b[baseline=(current bounding box.center)]
     (b) [matrix of math nodes,nodes in empty cells,right delimiter=],left delimiter=[ ]
    	bF_x-N_y  
    
    	bF_y+N_x  
    
    	0  
     ;
    
    F_2=
    1/a+b[baseline=(current bounding box.center)]
     (b) [matrix of math nodes,nodes in empty cells,right delimiter=],left delimiter=[ ]
    	aF_x+N_y  
    
    	aF_y-N_x  
    
    	0  
     ;
    .


In each plane, we are able to generate a force to balance the system, by moving the balancing masses described in (<ref>) and (<ref>).

In particular, by trigonometric formulas

	
  * in plane π_1, we compensate force F_1 by the centrifugal force:
	
    B_1=2m_1r_1ω^2cos(γ_1)(cos(α_1),sin(α_1));

	
  * in plane π_2, we compensate force F_2 by the centrifugal force:
	
    B_2=2m_2r_2ω^2cos(γ_2)(cos(α_2),sin(α_2)).





The overall imbalance of the system is then given by the resulting force in π_1

    F_ris,1=B_1+F_1

and the resulting force in π_2

    F_ris,2=B_2+F_2.

Note that, if the balancing masses are moved incorrectly, we may increase the imbalance on the system.

We introduce the imbalance indicator

    GB_1+F_1^2+B_2+F_2^2.

The above quantity measures the imbalance on the overall system made of rotor and balancing heads.










By (<ref>) and (<ref>), we observe that

    G(α_1,γ_1,α_2,γ_2)=G_1(α_1,γ_1)+G_2(α_2,γ_2),

where

    G_1(α_1,γ_1)      [|2m_1r_1ω^2cos(γ_1)cos(α_1)+F_1,x|^2.
           .+|2m_1r_1ω^2cos(γ_1)sin(α_1)+F_1,y|^2]

and

    G_2(α_2,γ_2)      [|2m_2r_2ω^2cos(γ_2)cos(α_2)+F_2,x|^2.
           .+|2m_2r_2ω^2cos(γ_2)sin(α_2)+F_2,y|^2].




§ THE CONTROL PROBLEM



An initial configuration Φ_0=(α_1,0,γ_1,0;α_2,0,γ_2,0) for the balancing masses is given.

Our goal is to find a control strategy such that:

	
  * the balancing masses move from Φ_0 to a final configuration Φ=(α_1,γ_1;α_2,γ_2), where they compensate the imbalance;
	
  * the imbalance should not increase and velocities of the masses are kept small during the correction process. 
	

In this first part, we suppose that we do not have a real-time feedback concerning the imbalance of the system. For this reason, we design an open-loop control. A closed-loop strategy is designed in section <ref>.

Accordingly, we introduce a control problem to steer our system to a stable configuration, which minimizes the imbalance. In the context of the model described in section <ref>, we choose as state Φ(t) (α_1(t),γ_1(t);α_2(t),γ_2(t)), where α_i(t) and γ_i(t) are the angles regulating the position of the four balancing masses, as illustrated in (<ref>) and (<ref>).

The control ψ(t) (ψ_1(t),ψ_2(t);ψ_3(t),ψ_4(t)) is the time derivative of the state, i.e. its components are the time derivatives of the angles Φ_i(t). Namely, the state equation is

    d/dtΦ=ψ   t∈ (0,+∞)
    Φ(0)=Φ_0.

Note that we are in the particular case of the Calculus of Variations. The time interval is infinite and special attention has to be paid for the limiting behavior of the solution. 

The Lagrangian L:𝕋^4×ℝ^4⟶ℝ reads as

    L(Φ,ψ) 1/2[ψ^2+βĜ(Φ)],

where β>0 is a parameter to be fixed and Ĝ G-inf G, G being the imbalance indicator introduced in (<ref>). Note that for any Φ∈𝒮(G), Ĝ(Φ)=G(Φ)-inf G=inf G-inf G=0, namely 𝒮 coincides with the zero set of Ĝ. We have introduced Ĝ to guarantee the integrability of the Lagrangian along admissible trajectories over the half-line (0,+∞).

In the above Lagrangian, there is a trade-off between the
cost of controlling the system to a stable regime and the velocity of the balancing masses, with respect to the rotor. If β is large, the primary concern for the optimal strategy is to minimize the cost of controlling, while if β is small our priority is to minimize the velocities.

Let Φ_0∈𝕋^4 be an initial configuration. We introduce the space of admissible trajectories

    𝒜{Φ∈ H^1_loc([0,+∞);𝕋^4) | Φ(0)=Φ_0,L(Φ,Φ̇)∈ L^1(0,+∞)},

where the Sobolev space H^1_loc((0,+∞);𝕋^4) is defined in (<ref>) (section <ref>). Note that the requirement L(Φ,Φ̇)∈ L^1(0,+∞) is equivalent to

    Φ̇∈ L^2(0,+∞)G(Φ)-inf G∈ L^1(0,+∞).


Our goal is to minimize the functional J:𝒜⟶ℝ

    J(Φ) 1/2∫_0^∞[Φ̇^2+βĜ(Φ)] dt.





 §.§ Statement of the main result



We state now our main result.


	Consider the functional (<ref>). For i=1,2, set
	
    c^i1/2m_ir_iω^2(F_i,x,F_i,y),

	where the above notation has been introduced in table <ref>. Then,
	
		
  * there exists Φ∈𝒜 minimizer of J;
		
  * Φ=(α_1,γ_1;α_2,γ_2)
		
		∈ C^∞([0,+∞);𝕋^4)
		
		is C^∞ smooth and, for i=1,2, the following Euler-Lagrange equations are satisfied, for t>0
	
	
    -α̈_i=βcos(γ_i)[-c^i_1sin(α_i)+c^i_2cos(α_i)]
    
    	-γ̈_i=-βsin(γ_i)[c^i_1cos(α_i)+c^i_2sin(α_i)-cos(γ_i)]
    α_i(0)=α_0,i,γ_i(0)=γ_0,i, Φ̇(T)T→ +∞⟶0.

	
		
  (3) for any optimal trajectory Φ for (<ref>), there exists Φ∈𝒮 such that
		
    Φ(t)t→ +∞⟶Φ,

		
    Φ̇(t)t→ +∞⟶0.

		and
		
    |Ĝ(Φ(t))|t→ +∞⟶0.

		If, in addition
	
	
    m_1r_1 > √(F_1,x^2+F_1,y^2)/2ω^2m_2r_2> √(F_2,x^2+F_2,y^2)/2ω^2,

	
		
   we have the exponential estimate for any t≥ 0
	
	
    Φ(t)-Φ+Φ̇(t)+|G(Φ(t))|≤ Cexp(-μ t),

	
		
  
		with C, μ >0 independent of t.
	


In the following subsection, we analyze the corresponding steady problem. In subsection <ref>, we develop general tools to prove the above result. In subsection <ref>, we prove Proposition <ref>. In subsection <ref>, we perform some numerical simulations validating the theory. In section <ref> we present the feedback strategy.




 §.§ The steady problem




First of all, we address the steady problem:

	
  Find a 4-tuple of angles (α_1,γ_1;α_2,γ_2) such that the imbalance indicator G is minimized.


A solution to the above steady problem is called steady optimum. We recall that the set of steady optima is denoted by 𝒮= (G).


	We observe that by using (<ref>),
	
    𝒮=(G_1)×(G_2),

	namely we can reduce our 4-dimensional problem to a 2-dimensional problem.


Therefore, we have reduced to find minimizers of a function of the form:

    g(α,γ)|cos(γ)cos(α)-c_1|^2+|cos(γ)sin(α)-c_2|^2.

This task is accomplished in Lemma below.


	Let c=(c_1,c_2)∈ℝ^2. Set
	
    g(α,γ)|cos(γ)cos(α)-c_1|^2+|cos(γ)sin(α)-c_2|^2.

	Let (g) be the set of minimizers of g. Then,
	
		
  * if c=0, then
		
    (g)={(θ,π/2)  |  θ∈𝕋};

		
  * if c≠ 0, set dmin{1,c}. Then,
		
    (g)=((c_1+ic_2),arccos(d))∪

		
    ∪((c_1+ic_2)+π,arccos(-d)),

		where (c_1+ic_2) denotes the argument of the complex number c_1+ic_2.

		Moreover, if c≠ 0, there exists a unique (α,γ) minimizer of g, with 0≤α<2π and 0≤γ≤π/2;
		
  * inf g=0 if and only if c≤ 1;
		
  * inf g = 
		
		0,   c≤ 1
|c-1|^2,     c> 1.
	


This Lemma can be proved by trigonometric calculus.


Now, let Φ∈𝒮 be a minimizer of the imbalance indicator G. We highlight that two circumstances may occur:

	
  * inf G=0, namely, the overall system made of rotor and balancing masses can be fully balanced, by placing the four balancing masses as
	
    P_1,1   =   r_1(cos(α_1-γ_1),sin(α_1-γ_1))
    
    	P_1,2   =   r_1(cos(α_1+γ_1),sin(α_1+γ_1))

	and
	
    P_2,1   =   r_2(cos(α_2-γ_2),sin(α_2-γ_2))
    
    	P_2,2   =   r_2(cos(α_2+γ_2),sin(α_2+γ_2)).

	
  * inf G>0, i.e. the imbalance of the rotor is too large to be compensated by the available balancing masses. Despite that, (α_1,γ_1;α_2,γ_2)  is a minimizer of G. Hence, by locating the balancing masses in configuration (<ref>)-(<ref>), we do our best to balance the system, being aware full balance cannot be achieved.


In the Proposition below, we illustrate when the circumstance inf G=0 occurs.


	The imbalance indicator G admits zeros (inf G=0) if and only if
	
    m_1r_1≥√(F_1,x^2+F_1,y^2)/2ω^2m_2r_2≥√(F_2,x^2+F_2,y^2)/2ω^2.



	
	We have G(α_1,γ_1,α_2,γ_2)=0 if and only if
	
    2m_1r_1ω^2cos(γ_1)cos(α_1)   =F_1,x
    
    	2m_1r_1ω^2cos(γ_1)sin(α_1)   =F_1,y

	
    2m_2r_2ω^2cos(γ_2)cos(α_2)   =F_2,x
    
    	2m_2r_2ω^2cos(γ_2)sin(α_2)   =F_2,y.

	Note that the first two equations are decoupled with respect to the second ones. By Lemma <ref> (3), the above system admits a solution if and only if
	
    m_1r_1   ≥√(F_1,x^2+F_1,y^2)/2ω^2
    
    	m_2r_2   ≥√(F_2,x^2+F_2,y^2)/2ω^2,

	as required.


As we have seen at the beginning of section <ref>, an initial configuration Φ_0=(α_0,1,γ_0,1;α_0,2,γ_0,2) of the balancing masses is given. A key issue is to determine a trajectory Φ(t)=(α_0,1(t),γ_0,1(t);α_0,2(t),γ_0,2(t)) joining the initial configuration Φ_0 with a steady optimum Φ∈𝒮 minimizing the imbalance in the meanwhile. For this reason, the dynamical control problem has to be addressed. Our main result Proposition <ref> asserts the steady problem and the dynamical one are interlinked.




 §.§ General results


The purpose of this section is to provide some general tools to prove Proposition <ref>. We introduce a generalized version of our functional (<ref>).

Consider the Lagrangian L:𝕋^n×ℝ^n⟶ℝ

    L(Φ,ψ)1/2ψ^2+Q(Φ),

where Q:𝕋^n⟶ℝ^+ is real analytic.

Let Φ_0∈𝕋^n be an initial condition. Set the space of admissible trajectories

    𝒜{Φ∈ H^1_loc([0,+∞);𝕋^n) | Φ(0)=Φ_0L(Φ,Φ̇)∈ L^1(0,+∞)}.

The zero set of Q is denoted by 𝒵.

Our goal is to minimize the functional K:𝒜⟶ℝ

    K(Φ) ∫_0^∞1/2Φ̇^2+Q(Φ)  dt.



	If 𝒵≠∅, then the space of admissible trajectories 𝒜 is nonempty.


	
	Take Φ∈𝒵. Consider the trajectory
	
    Φ(t)
    	(1-t)Φ_0+tΦ   t∈ [0,1)
    Φ   t∈ [1,+∞).

	Now, Φ∈𝒜, thus showing that 𝒜≠∅.


In Proposition <ref>, we are concerned with the existence of minimizer of (<ref>). The proof can be found in the Appendix.


	There exists Φ∈𝒜 global minimizer of (<ref>).


We now derive to optimality conditions for (<ref>). Let Φ∈𝒜 be an admissible trajectory. We consider directions v∈ C^∞_c((0,+∞);ℝ^n). We can compute the directional derivative of K at Φ along the direction v, obtaining

    ⟨ dK(Φ),v⟩=∫_0^∞Φ̇v̇+∇ Q(Φ) vdt.


From the above computation of the directional derivative and Fermat's theorem, we derive the first order Optimality Conditions.

	Take Φ minimizer of (<ref>). Then, we have:
	
		
  * Φ∈ C^∞([0,+∞);𝕋^n);
		
  * the Euler-Lagrange equations are satisfied
		
    Φ̈=∇ Q(Φ)   t∈ (0,+∞)
    Φ(0)=Φ_0;

		
  * the energy is conserved, i.e.
		
    E(t)1/2Φ̇(t)^2-Q(Φ(t))≡ 0.

	


Now, in the spirit of stabilization-turnpike theory (see <cit.>), we show that the time-evolution optima converges as t→∞ to steady optima. As a byproduct, this will allows us to add to (<ref>) the final condition Φ̇(t)t→ +∞⟶0. We start by proving the following Lemma.


	Assume 𝒵≠∅ and Q is real analytic. Let Lmax_𝕋^n∇ Q be its Lipschitz constant. Let d>0 and N>0 be the constants appearing in the Łojasiewicz inequality (see, e.g. <cit.>),
	
    |Q(Φ)|≥ d (Φ,𝒵)^N, ∀ Φ∈𝕋^n.

	Let Φ_0∈𝕋^n be an initial condition. Consider Φ∈𝒜 global minimizer of (<ref>). Then, for any t>0, we have
	
    (Φ(t),𝒵)≤√(σ_1(Φ_0,𝒵)),

	and
	
    (Φ(t),𝒵)≤√(σ_2(Φ_0,𝒵)/t),

	where Ñmax{2,N},
	
    σ_1 (2π√(n)+L)/2max{2^Ñ+1(2π√(n))^Ñ-2,2^Ñ(2π√(n))^Ñ-N/d}

	and
	
    σ_2 (2π√(n)+L)σ_1^N/2d.


One of the consequences of (<ref>) is the validity of stabilization/turnpike problem for our control problem (<ref>). Indeed, the right hand-side of (<ref>) decays to zero as 1/t^NÑ. For this polynomial decay, no assumption are required on the hessian of Q.

	Step 1  Upper bound of K(Φ)=inf_𝒜K

	Since 𝕋^n is compact and Q is continuous, the zero set 𝒵 is compact as well, whence by Weierstrass Theorem, there exists Φ_0∈𝒵, such that Φ_0-Φ_0=(Φ_0,𝒵). Consider the trajectory
	
    Φ(t)
    	(1-t)Φ_0+tΦ_0    t∈ [0,1)
    Φ_0    t∈ [1,+∞).

	Now, on the one hand, for any t∈ [0,1]
	
    Q(Φ(t))   =   |Q(Φ(t))-Q(Φ_0)|
       ≤   L(1-t)Φ_0+tΦ_0-Φ_0
       =   L|1-t|Φ_0-Φ_0
       =   L|1-t|(Φ_0,𝒵).

	On the other hand, for any t>1, Φ(t)=Φ_0, whence
	
    Q(Φ(t)) = 0.

	Hence,
	
    ∫_0^+∞Q(Φ(t))  dt≤∫_0^1L|1-t|(Φ_0,𝒵)  dt=L/2(Φ_0,𝒵).

	Therefore,
	
    K(Φ)   ≤ K(Φ)≤1/2Φ_0-Φ_0^2+L/2(Φ_0,𝒵)
       ≤2π√(n)+L/2(Φ_0,𝒵).

	Step 2  Lower bound of K(Φ)=inf_𝒜K

	Arbitrarily fix t≥ 0. We are going to bound K(Φ) from below in terms of (Φ(t),𝒵), the distance of Φ(t) from the zero set 𝒵. Let s_t∈_[t,t+1]Q(Φ(·)) and set Φ_1Φ(s_t).
	
    K(Φ)   =∫_0^∞1/2Φ̇^2+Q(Φ)  ds
       ≥∫_t^t+11/2Φ̇^2+Q(Φ)  ds
       ≥∫_t^t+11/2Φ̇^2+min_ξ∈ [t,t+1] Q(Φ(ξ))  ds
       =∫_t^t+11/2Φ̇^2+Q(Φ_1)  ds
       =∫_t^t+11/2Φ̇^2  ds + Q(Φ_1)
       ≥∫_t^s_t1/2Φ̇^2  ds + Q(Φ_1)
       ≥inf_Γ∈𝒜_Φ(t),Φ_1∫_t^s_t1/2Γ̇^2  ds + Q(Φ_1)
       ≥Φ_1-Φ(t)^2/2(s_t-t) + Q(Φ_1)
       ≥Φ_1-Φ(t)^2/2 + Q(Φ_1),

	where (<ref>) is justified by |s_t-t|≤ 1 and in (<ref>) we minimize over the space of trajectories Γ linking Φ(t) and Φ_1 in time s_t-t
	
    𝒜_Φ(t),Φ_1{Γ∈ H^1((t,s_t);𝕋^n) | Γ(t)=Φ(t)Γ(s_t)=Φ_1}.

	We now employ the above inequality combined with (<ref>), getting
	
    K(Φ)   ≥   Φ_1-Φ(t)^2/2 + Q(Φ_1)
       ≥   Φ_1-Φ(t)^2/2+d (Q(Φ_1),𝒵)^N
       =   Φ_1-Φ(t)^2/2+d Φ_1-Φ_1^N
       ≥   Φ_1-Φ(t)^Ñ/2(2π√(n))^Ñ-2+d/(2π√(n))^Ñ-NΦ_1-Φ_1^Ñ
       ≥   1/max{2(2π√(n))^Ñ-2,(2π√(n))^Ñ-N/d}[Φ_1-Φ(t)^Ñ+Φ_1-Φ_1^Ñ]
       ≥   ζ^-1[Φ_1-Φ(t)+Φ_1-Φ_1]^Ñ
       ≥   ζ^-1Φ(t)-Φ_1^Ñ≥ζ^-1(Φ(t),𝒵)^Ñ,

	with Φ_1∈_𝒵·-Φ_1, Ñ= max{2,N} and
	
    ζmax{2^Ñ+1(2π√(n))^Ñ-2,2^Ñ(2π√(n))^Ñ-N/d}.

	
	Step 3  Proof of (<ref>)

	By using (<ref>) and (<ref>), we obtain
	
    2π√(n)+L/2(Φ_0,𝒵)   ≥   K(Φ)
       ≥   ζ^-1(Φ(t),𝒵)^Ñ,

	as required. We now aim at proving (<ref>).
	
	Step 4  Bound of min_τ∈ [0,t](Φ(τ),𝒵)

	Arbitrarily fix t>0. We have
	
    K(Φ)   =∫_0^∞1/2Φ̇^2+Q(Φ)  ds
       ≥∫_0^t Q(Φ)  ds
       ≥∫_0^t d (Φ,𝒵)^N ds
       ≥ d∫_0^tmin_τ∈ [0,t](Φ(τ),𝒵)^N ds
       = td min_τ∈ [0,t](Φ(τ),𝒵)^N.

	Therefore, by (<ref>),
	
    td min_τ∈ [0,t](Φ(τ),𝒵)^N≤ K(Φ)≤2π√(n)+L/2(Φ_0,𝒵),

	whence
	
    min_τ∈ [0,t](Φ(τ),𝒵)≤√(2π√(n)+L/2td(Φ_0,𝒵)).

	Step 5  Proof of (<ref>)

	Let τ_t∈_[0,t](Φ(·),𝒵). By applying (<ref>) with initial datum Φ(τ_t) at initial time τ_t, we get
	
    (Φ(t),𝒵)   ≤√(σ_1(Φ(τ_t),𝒵))
       =√(σ_1min_τ∈ [0,t](Φ(τ),𝒵)).

	Then, combining the above inequality with (<ref>), we obtain
	
    (Φ(t),𝒵)   ≤√(σ_1min_τ∈ [0,t](Φ(τ),𝒵))
       ≤√(σ_1√(2π√(n)+L/2td(Φ_0,𝒵)))
       =√((2π√(n)+L)σ_1^N/2d(Φ_0,𝒵)/t),

	as required.



	As kindly suggested by one of the referees, in practical applications, the control input Φ̇ is subject to saturation. Indeed, physical constraints must be taken into account, e.g. balancing masses cannot rotate too fast. Several rotors control strategies available in the literature consider saturation effects, such as <cit.>. In our case, we are able to guarantee that our control magnitude does not exceed an explicit threshold, constrains being intrinsically imposed in the functional definition (<ref>). Indeed, let us work in the framework of Lemma <ref>. By employing (<ref>) and energy conservation (<ref>), we obtain
	
    1/2Φ̇(t)^2=Q(Φ(t))≤ L(Φ(t),𝒵)≤ L√(σ_1(Φ_0,𝒵)),

	whence
	
    Φ̇(t)≤√(2L)√(σ_1(Φ_0,𝒵)),

	for any t≥ 0. This gives an upper bound for the magnitude of the optimal control.



	Assume 𝒵⊂𝕋^n is nonempty and finite and Q real analytic. Consider Φ∈𝒜 global minimizer of (<ref>). Then,
	
		
  * there exists Φ∈(Q) such that
		
    Φ(t)t→ +∞⟶Φ,

		
    Φ̇(t)t→ +∞⟶0

		and
		
    |Q(Φ(t))|t→ +∞⟶0.

		
  * the Euler-Lagrange equations can be complemented with final condition
		
    Φ̈=∇ Q(Φ)   t∈ (0,+∞)
    Φ(0)=Φ_0, Φ̇(T)T→ +∞⟶0.
		
	



	We start proving (<ref>).

	Let Φ be a minimizer of (<ref>). Let us prove (<ref>). prove Since 𝒵⊂𝕋^n is finite and Φ is continuous,
	
	there exists a unique Φ∈𝒵, such that
	
    (Φ(t),𝒵)=Φ(t)-Φ,∀ t≥t,

	with t large enough. By the above equality and (<ref>), we have
	
    Φ(t)-Φ= (Φ(t),𝒵)t→ +∞⟶0,

	as required. By the above convergence and (<ref>), we immediately have Φ̇(t)t→ +∞⟶0.
	
	(<ref>) follows from Proposition <ref> together with (<ref>).


Note that (<ref>) can be seen as a system of two coupled elliptic PDEs, with a Dirichlet condition at time t=0 and a Neumann condition at t=+∞.

Equivalently, we can formulate the first order optimality conditions as a state-adjoint state first order system.

    Φ̇=-q   t∈ (0,+∞)
    
    -q̇=∇ Q(Φ)   t∈ (0,+∞)
    Φ(0)=Φ_0,  q(T)T→ +∞⟶0.


By using the above optimality system, we can improve the decay rate estimate.

	Suppose 𝒵⊂𝕋^n is nonempty and finite and Q real analytic. In addition, assume
	
    ∇^2Q(Φ),

	Consider Φ∈𝒜 global minimizer of (<ref>). Then, we have the exponential estimate, for any t≥ 0
	
    Φ(t)-Φ+Φ̇(t)+|Q(Φ(t))|≤ Cexp(-μ t),

	with C, μ >0 independent of t.

The proof of Proposition <ref> is a consequence of Proposition <ref> and Lemma <ref> stated and proved below, inspired by <cit.> and <cit.>.

	Let Φ_0∈𝕋^n and Φ∈ C^∞(ℝ^+;𝕋^n) solution to
	
    Φ̈=∇ Q(Φ)   t∈ (0,+∞)
    Φ(0)=Φ_0.

	Suppose the existence Φ∈𝒵 such that
	
    Φ(t)t→ +∞⟶Φ

	and
	
    Φ̇(t)t→ +∞⟶0.

	Assume the condition (<ref>) holds. Then,
	
    Φ(t)-Φ+Φ̇(t)+|Q(Φ(t))|≤ Cexp(-μ t),∀ t≥ 0,

	with μ >0.


	Step 1   Reduction the a first order problem

	Take any Φ solution to (<ref>). Then, the function
	
    x[baseline=(current bounding box.center)]
    	 (b) [matrix of math nodes,nodes in empty cells,right delimiter=],left delimiter=[ ]Φ-Φ
    Φ̇
     ;

	
	
	
	
	
	
	
	
	solves the first order problem
	
    ẋ= f( x)   t∈ (0,+∞)
    x(T)T→ +∞⟶0.

	where
	
    f( x)[baseline=(current bounding box.center)]
    	 (b) [matrix of math nodes,nodes in empty cells,right delimiter=],left delimiter=[ ]
    		x_n+1
    
    
    
    
    
    		x_2n
    ∇ Q((x_1,…,x_n)+Φ) 
     ;
    	[loosely dotted] (b-2-1)–(b-6-1);
    	.

	Step 2   0 is an hyperbolic equilibrium point.

	We observe that f(0)=0, since Φ is a zero of Q. Moreover, the Jacobian of f at x=0 is a block matrix
	
    Df(0)=[       0     I_n; ∇^2Q(Φ)      0, ]

	where I_n is the n× n identity matrix. By assumption (<ref>), ∇^2Q(Φ) is positive definite. Then, there exists C symmetric positive definite, such that C^2=∇^2Q(Φ). Following <cit.>, we introduce the matrix
	
    Λ1/2[  2I_n -C^-1;    2C   I_n ].

	Since ∇^2Q(Φ) is (strictly) positive definite, Λ is invertible[
		
    Λ^-1=[   1/2I_2 1/2 C^-1;       -C     I_2. ]

	] and
	
    Λ^-1Df(0) Λ=[   C   0;   0 -C. ]

	Hence, the spectrum of the jacobian Df(0) does not intersect the imaginary axis, whence 0 is an hyperbolic equilibrium point for (<ref>), as required.
	

	Step 3   Conclusion by applying the Stable Manifold Theorem

	As we have seen in step 2, 0 is an hyperbolic equilibrium point for (<ref>). Then, by the Stable Manifold Theorem (see e.g. <cit.> or <cit.>), the stable and unstable manifolds for (<ref>) exist in a neighborhood of 0. Besides, thanks to (<ref>) and (<ref>), x=(Φ-Φ,Φ̇) belongs to the stable manifold of the above problem.
	
	
	
	
	
	Hence, by Stable Manifold theory (see, e.g. <cit.> or <cit.>), we have for some μ >0
	
    x(t)≤ Cexp(-μ t),∀ t≥ 0,

	which yields
	
    Φ(t)-Φ+Φ̇(t)≤ Cexp(-μ t),∀ t≥ 0.
		
	To conclude the proof, we observe that Q is globally Lipschitz
	
	
	
	and Q(Φ)=0. Then,
	
    |Q(Φ(t))|   =   |Q(Φ(t))-Q(Φ)|≤ LΦ(t)-Φ
       ≤    Cexp(-μ t).

	where in the last inequality we have employed (<ref>).




 §.§ Proof of Proposition <ref>



We prove Proposition <ref> employing the general results of subsection <ref>. The numbering (1), (2) and (3) refers to the numbered statements in Proposition <ref>.


	
	
	
	
	
	
	The existence of minimizers for (<ref>) follows from Proposition <ref>, with K= J.

	Step 1  Reduction to two angles

	By (<ref>), the imbalance indicator  splits as G(α_1,γ_1,α_2,γ_2)=G_1(α_1,γ_1)+G_2(α_2,γ_2), whence
	
    Ĝ(α_1,γ_1,α_2,γ_2)=Ĝ_1(α_1,γ_1)+Ĝ_2(α_2,γ_2),

	with Ĝ_1(α_1,γ_1) G_1(α_1,γ_1)-inf G_1 and Ĝ_2(α_2,γ_2) G_2(α_2,γ_2)-inf G_2. Then, the functional
	
    J(Φ)=J_1(α_1,γ_1)+J_2(α_2,γ_2),

	where
	
    J_1(α_1,γ_1)1/2∫_0^∞[|α̇_1|^2+|γ̇_1|^2+βĜ_1(α_1,γ_1)] dt

	and
	
    J_2(α_2,γ_2)1/2∫_0^∞[|α̇_2|^2+|γ̇_2|^2+βĜ_2(α_2,γ_2)] dt.

	This enables us to work on J_1 and J_2 separately. From the physical viewpoint, the functional J_1 is related to the first balancing head, while J_2 is related to the second balancing head. Both J_1 and J_2 fit in a general class of functionals (<ref>), defining
	
    Q_i(α_i,γ_i)      β/2[|cos(γ_i)cos(α_i)-c^i_1|^2. 
           .+|cos(γ_i)sin(α_i)-c^i_2|^2],

	possibly remaining β after the absorption of the coefficient 1/2m_ir_iω^2 and
	
    c^i=1/2m_ir_iω^2(F_i,x,F_i,y).

	Step 2   Proof of (2)

	For any Φ=(α_1,γ_1;α_2,γ_2) minimizer of (<ref>), (α_1,γ_1) minimizes J_1 and (α_2,γ_2) minimizes J_2. We apply Proposition <ref> to J_1 and J_2, computing the gradient of Q_i defined in (<ref>)
	
    ∂ Q_i/∂α_i(α_i,γ_i)   =   βcos(γ_i)[c^i_1sin(α_i)-c^i_2cos(α_i)]
    ∂ Q_i/∂γ_i(α_i,γ_i)   =   βsin(γ_i)[c^i_1cos(α_i)+c^i_2sin(α_i).
           .-cos(γ_i)].

	Step 3 
  Proof of (3) and (4)

	By Step 1, we reduce to prove the assertion for minimizers of J_1 and J_2. Let (α_i,γ_i) be a minimizer of J_i, for some i=1,2.
	
	
Case 1. (Q_i)⊂𝕋^2 is finite.
	
	If (Q_i)⊂𝕋^2 is finite, we directly apply Proposition <ref> to K J_i, getting the required convergences. If, in addition, (<ref>) is verified, we want to prove that the Hessian of Q_i at the steady optimum is positive definite. To this end, we compute ∇^2Q_i(α_i,γ_i)
	
    ∂^2 Q_i/∂α^2(α_i,γ_i)   =   βcos(γ_i)[c^i_1cos(α_i)+c^i_2sin(α_i)]
    ∂^2 Q_i/∂γ_i^2(α_i,γ_i)   =   βcos(γ_i)[c^i_1cos(α_i)+c^i_2sin(α_i).
           .-cos(γ_i)]+βsin(γ_i)^2
    ∂^2 Q_i/∂γ_i∂α_i(α_i,γ_i)   =   βsin(γ_i)[-c^i_1sin(α_i)+c^i_2cos(α_i)]

	Now, let Φ∈(Q_i).
	Since Φ∈(Q_i) and (<ref>) holds, by Lemma <ref>,
	
    c^i=cos(γ_i)(cos(α_i),sin(α_i)).

	and sin(γ_i)≠ 0. Hence, by (<ref>), c_1cos(α_i)+c_2sin(α_i)-cos(γ_i)=0. We plug these results into (<ref>), obtaining
	
    ∂^2 Q_i/∂α_i^2(α_i,γ_i)   =   βcos(γ_i)^2[cos(α_i)^2+sin(α_i)^2]
       =   βcos(γ_i)^2
    ∂^2 Q_i/∂γ_i^2(α_i,γ_i)   =   βsin(γ_i)^2
    ∂^2 Q_i/∂γ_i∂α_i(α_i,γ_i)   =   βcsin(γ_i)[-cos(α_i)sin(α_i).
           .+sin(α_i)cos(α_i)]=0,

	namely the Hessian of Q_i computed at (α_i,γ_i) is diagonal. Using once more (<ref>) and by Lemma <ref>, we have both cos(γ)≠ 0 and sin(γ)≠ 0. Then,
	
	the Hessian of Q_i computed at (α_i,γ_i) is (strictly) positive definite. We apply Proposition <ref> (2) to conclude.
	
	
Case 2. (Q_i)⊂𝕋^2 is a continuum.
	
	From the physical viewpoint, this occurs when in the plane π_i there is no imbalance, namely F_i=0. Now, by Lemma <ref>, (Q_i)⊂𝕋^2 is a continuum if and only if c^i=0, namely
	
    Q_i(α_i,γ_i)= β/2[|cos(γ_i)cos(α_i)|^2+|cos(γ_i)sin(α_i)|^2].

	and the Euler-Lagrange equations satisfied by (α_i,γ_i) read as
	
    α̈_i=0   t∈ (0,+∞)
    γ̈_i=-β/2sin(2γ)   t∈ (0,+∞)
    α(0)=α_0, α̇(T)T→ +∞⟶0
    γ(0)=γ_0, γ̇(T)T→ +∞⟶0.

	This entails that
	
    α(t)≡α_0.

	Furthermore, for any integer k, cos((2k+1)π)<0.
	Therefore, we are in position to conclude applying Proposition <ref> to the functional
	
    K(γ_i)1/2∫_0^∞[γ̇_i^2+β|cos(γ)|^2] dt.

	In case (Q_i)⊂𝕋^2 is a continuum, the above proof can be seen from the point of view of phase analysis. Indeed, the Euler-Lagrange equations reduce to the pendulum-like equation
	
    γ̈=-β/2sin(2γ)   t∈ (0,+∞)
    γ(0)=γ_0, γ̇(T)T→ +∞⟶0.

	
	We have the end condition γ̇(T)T→ +∞⟶0. Then, any solution γ of (<ref>) lies on the separatrix (the red curve in figure <ref>), so that it must stabilize towards some steady state.








 §.§ Numerical simulations



In order to perform some numerical simulations, we firstly discretize our functional (<ref>) and then we run - to minimize the resulting discretized functional.

For the purpose of the numerical simulations, it is convenient to rewrite (<ref>) as

    K(ψ,Φ) ∫_0^∞1/2Φ̇^2+Q(Φ)dt,

subject to the state equation

    d/dtΦ=ψ   t∈ (0,+∞)
    Φ(0)=Φ_0.




  §.§.§ Discretization


Let ε >0 and suppose we want to get ε-close to the stable configuration at some time T>0, i.e. (Φ(T),𝒵)<ε. Following (<ref>), such an ε-stabilization can be achieved by choosing

    T>σ_2/ε^ÑN(Φ_0,𝒵)

and Nt∈ℕ∖{0,1} large enough. Set Δ tT/Nt-1. The discretized state is (Φ_i)_i=0,…,Nt-1, whereas the discretized control (velocity) is

(ψ_i)_i=0,…,Nt-2. The discretized functional reads as

    K_d(ψ,Φ) Δ t∑_i=0^Nt-1[1/2ψ_i^2+Q(Φ_i)],

subject to the discretized state equation

    Φ_i-Φ_i-1/Δ t=ψ_i-1,i=1,…,Nt-1.



  §.§.§ Algorithm execution



By (<ref>) and (<ref>), the discretized minimization problem is

    K_d,.

We address the above minimization problem by employing the interior-point optimization routine  (see <cit.> and <cit.>) coupled with  <cit.>, which serves as modelling language and performs the automatic differentiation. The interested reader is referred to <cit.> and <cit.> for a survey on existing numerical methods to solve an optimal control problem.

In figures <ref>, <ref>, <ref> and <ref>, we plot the computed optimal trajectory for (<ref>), with initial datum Φ_0=(α_0,1,γ_0,1;α_0,2,γ_0,2)(2.6,0.6, 2.5,1.5). We choose F, N and m_i (see table <ref>), such that the condition (<ref>) is fulfilled. The exponential stabilization proved in Proposition <ref> emerges. In figure <ref>, we depict the imbalance indicator versus time, along the computed trajectories. As expected, it decays to zero exponentially.

































§ REINFORCEMENT LEARNING AND THE CLOSED-LOOP SOLUTION



So far, we presented an open-loop control strategy. The purpose of this section is to introduce a feedback strategy obtained by Reinforcement Learning <cit.>. All throughout the section, we will work in the context of section <ref>, under the assumption 𝒵≠∅ and Q real analytic. We start by defining the value function

    V(θ)=inf_𝒜_θJ=inf_𝒜_θ∫_0^∞1/2Φ̇^2+Q(Φ)  dt,

with

    𝒜_θ{Φ∈ H^1_loc([0,+∞);𝕋^n) | Φ(0)=θ.


    .L(Φ,Φ̇)∈ L^1(0,+∞)}.


Reasoning as in <cit.>, an optimal trajectory can be obtained by solving the (closed-loop) Ordinary Differential Equation

    Φ̇(t) = -∇ V(Φ(t)),t>0,

with initial condition Φ(0)=θ. The feedback law is -∇ V(Φ). Hence, the main task is to determine the value function V. We will show how to do this by using a value iteration algorithm of Reinforcement Learning. The convergence will be guaranteed by choosing an initial guess suggested by the stabilization/turnpike phenomenon (<ref>). Let us mention that another approach will be to employ the analytical approximation methods developed in <cit.>, which are designed to determine directly ∇ V.

To that end, let us write the Dynamic Programming Principle (DPP) for V forward in time (see e.g. <cit.>). Arbitrarily fix δ>0:

    V(θ) = inf_Φ∈𝒜_θ{∫_0^δ1/2Φ̇^2+Q(Φ)  ds + V(Φ(δ))},

where θ is an arbitrary initial configuration and 𝒜_θ has been defined in (<ref>).

We will approximate V as limit of a recursive sequence defined as

    V_i+1(θ) = inf_Φ∈𝒜_θ{∫_0^δ1/2Φ̇^2+Q(Φ)  ds + V_i(Φ(δ))},

with i∈ℕ and θ∈𝕋. As announced, the choice of the initial guess is a key point.



 §.§ Initial guess



We start by constructing an initial guess V_0∈ C^0(𝕋^n).

Define

    V_0:𝕋^n⟶ℝ^+,


    V_0(θ)1/2(θ,𝒵)^2+L/2(θ,𝒵),

with L=max_𝕋^n∇ Q.



 §.§ Approximating sequence



Let us define a sequence {V_i}_i∈ℕ⊂ L^∞(𝕋^n) approximating V. The initial guess has been defined in (<ref>), while for any i∈ℕ we set

    V_i+1(θ) inf_Φ∈𝒜_θ{∫_0^δ1/2Φ̇^2+Q(Φ)  ds + V_i(Φ(δ))},

for θ∈𝕋.



 §.§ Convergence of the algorithm



By using the stabilization/turnpike estimate (<ref>), we prove the convergence of the algorithm.


	In the notation of Lemma <ref>, suppose 𝒵≠∅ and Q is real analytic. For any ε >0 and for every
	
    i> 2π√(n)σ_2/δ(-L+√(L^2+8ε)/2)^ÑN

	we have
	
    |V_i(θ)-V(θ)|<ε, ∀ θ∈𝕋.



	Step 1   Prove V(θ)≤ V_0(θ), for any θ∈𝕋^n.

	Let θ∈𝒵, such that θ-θ=(θ,𝒵). Consider the trajectory
	
    Φ(t)
    	(1-t)θ+tθ   t∈ [0,1)
    θ   t∈ [1,+∞).

















Proceeding as in Step 1 of the proof of Lemma <ref>, we get



	
    V(θ)≤ K(Φ)≤(Φ_0,𝒵)^2+L/2(Φ_0,𝒵)=V_0(θ).

	Step 2   Prove V(θ)≤ V_i(θ), for each i∈ℕ and for any θ∈𝕋^n.

	We proceed by induction on i∈ℕ. By step 1, the assertion holds for i=0. Let us assume
	
    V(θ)≤ V_i(θ), ∀ θ∈𝕋^n

	and prove
	
    V(θ)≤ V_i+1(θ), ∀ θ∈𝕋^n.

	By definition, for any η > 0, there exists Φ_η∈𝒜_θ such that
	
    V_i+1(θ) + η > {∫_0^δ1/2Φ̇_η^2+Q(Φ_η)  ds + V_i(Φ_η(δ))}.

	Now, by induction assumption,
	
    {∫_0^δ1/2Φ̇_η^2+Q(Φ_η)  ds + V_i(Φ_η(δ))}
       ≥{∫_0^δ1/2Φ̇_η^2+Q(Φ_η)  ds + V(Φ_η(δ))},

	whence, by (<ref>), we obtain
	
    V_i+1(θ) + η > {∫_0^δ1/2Φ̇_η^2+Q(Φ_η)  ds + V(Φ_η(δ))}≥ V(θ),

	where in the last inequality we have used the Dynamic Programming Principle (DPP) (<ref>). The arbitrariness of η allows to conclude this step.
	
	Step 3   For any ε >0 there exists t_ε>0 such that
	
    |V_i(Φ_θ(t))-V(Φ_θ(t))|< ε,

	for any t> t_ε-iδ, for each i∈ℕ and θ∈𝕋.

	In the above expression and in the remainder of the proof, Φ_θ denotes an optimal trajectory for (<ref>) with initial configuration θ. Set
	
    t_ε2π√(n)σ_2/(-L+√(L^2+8ε)/2)^ÑN.

	We prove the assertion by induction. Let us start with i=0. By (<ref>), for any t> t_ε we have
	
    (Φ(t),𝒵)<-L+√(L^2+8ε)/2,

	whence
	
    V_0(θ)=1/2(θ,𝒵)^2+L/2(Φ_0,𝒵)<ε.

	We suppose the assertion for i and we prove it for i+1. By definition (<ref>), we have
	
    V_i+1(Φ_θ(t))≤∫_t^t+δ1/2Φ̇_θ^2+Q(Φ_θ)  ds + V_i(Φ_θ(t+δ)).

	Then, for any t> t_ε-(i+1)δ
	
    |V_i+1(Φ_θ(t))-V(Φ_θ(t))|   = V_i+1(Φ_θ(t))-V(Φ_θ(t))
       ≤∫_t^t+δ1/2Φ̇_θ^2+Q(Φ_θ)  ds + V_i(Φ_θ(t+δ))
       -V(Φ_θ(t))
       =∫_t^t+δ1/2Φ̇_θ^2+Q(Φ_θ)  ds + V_i(Φ_θ(t+δ))
       -∫_t^t+δ1/2Φ̇_θ^2+Q(Φ_θ)  ds
       -V(Φ_θ(t+δ))
       =V_i(Φ_θ(t+δ))-V(Φ_θ(t+δ))
       <ε,

	where in (<ref>) we have employed step 2, in (<ref>)  the Dynamic Programming Principle (DPP) (<ref>) and in (<ref>) the induction assumption together with t+δ> t_ε-iδ.
	
	Step 4   Conclusion.

	For any i∈ℕ satisfying (<ref>), we have t_ε-iδ <0, where t_ε has been defined in (<ref>). Then, we apply (<ref>) with t=0 thus concluding.




§ CONCLUSIONS AND PERSPECTIVES



In this manuscript, a problem of rotors imbalance suppression has been addressed. A physical model has been conceived. The control problem has been formulated in the context of the Calculus of Variations, in an infinite time horizon. A general class of variational problems has been introduced, containing imbalance suppression as a particular case. In this general framework, well-posedness in infinite-time has been proved and Optimality Conditions have been derived both as second order Euler-Lagrange equations and first order Pontryagin system. The Łojasiewicz inequality has been employed to prove convergence of the time optima towards the steady optima. Quantitative estimates of the rate of convergence have been obtained, without sign condition on the hessian of the imbalance indicator. In case the imbalance is below a given threshold, Stable Manifold theory has been used to obtain an exponential estimate of the speed of convergence. In case real-time feedback on the imbalance is available, a value iteration Reinforcement Learning algorithm has been proposed.

Both open-loop and feedback optimal controls have been designed. In the case of closed-loop, our Reinforcement Learning algorithm can be complemented by Hamilton-Jacobi theory (see e.g. <cit.>). The Hamilton-Jacobi equation for our functional (<ref>) reads as

    ∇ V(θ)^2=2Q(V(θ))θ∈𝕋^n,

where

    V(θ)=inf_𝒜_θJ=inf_𝒜_θ∫_0^∞1/2Φ̇^2+Q(Φ)  dt,

with

    𝒜_θ{Φ∈ H^1_loc([0,+∞);𝕋^n) | Φ(0)=θ.


    .L(Φ,Φ̇)∈ L^1(0,+∞)}.

As we have seen in section <ref>, the value function V can be approximated numerically by a value iteration algorithm of Reinforcement Learning

. Another approach could be to employ numerical solvers for the Hamilton-Jacobi equation like ROC-HJ <cit.>. Furthermore, we could employ the analytical methods illustrated in <cit.>, whose goal is to approximate directly ∇ V.
	


§ APPENDIX

The appendix is devoted to the proof of Lemma <ref>, Proposition <ref> and Proposition <ref>.









































































































§ PROOF OF PROPOSITION <REF>


Now, we prove the well posedness of the time-evolution problem, by employing the direct methods in the Calculus of Variations.

	Step 1  Boundedness of the minimizing sequence.

	
	Let {Φ^m}_m∈ℕ⊂𝒜 be a minimizing sequence for (<ref>). We wish to prove that {Φ̇^m}_m∈ℕ⊂ L^2((0,+∞);ℝ^n) is bounded.
	
	By definition of minimizing sequence, if m is large enough,
	
    1/2Φ̇^m_L^2^2≤ K(Φ^m)≤inf_𝒜K+1.

	Then, Φ̇^m_L^2≤ M for any natural m, as desired.

	Step 2  Weak convergence of the minimizing sequence in 𝒜.

	Now, for any t≥ 0,
	
    Φ^m(t)=Φ_0+∫_0^t Φ̇^m(s)ds.

	Then, by Cauchy-Schwarz inequality, for any T>0, Φ^m_L^2((0,T);𝕋^n)≤ M(√(T)+1). Hence, by Banach-Alaoglu Theorem, there exists Φ∈ H^1_loc((0,+∞);𝕋^n) with Φ̇∈ L^2((0,+∞);ℝ^n) such that, up to subsequences,
	
    Φ^mm→∞⟶Φ

	weakly in H^1((0,T);𝕋^n) for any T>0 and
	
    Φ̇^mm→∞⟶Φ̇,

	weakly in L^2((0,+∞);ℝ^n). Furthermore, the above convergence occurs point-wise. Indeed, for t≥ 0 and T≥ t, the linear operator
	
    δ_t:H^1((0,T);𝕋^n)⟶ℝ^n

	
    Φ⟼Φ(t)

	is continuous. Hence, by the definition of weak convergence,

	Φ^m(t)=δ_t(Φ^m)⟶δ_t(Φ)=Φ(t). Since, for any natural m, Φ^m(0)=Φ_0, we have Φ(0)=Φ_0, whence Φ∈𝒜, as required.

	Step 3  Conclusion

	
	
	
	
	
	By the lower semicontinuity of the norm with respect to the weak convergence
	
    ∫_0^∞Φ̇^2 dt≤lim inf_m→ +∞∫_0^∞Φ̇^m^2 dt.

	
	At this stage, we want to prove the inequality
	
    ∫_0^∞Q(Φ) dt≤lim inf_m→ +∞∫_0^∞Q(Φ^m) dt.

	Now, as we have shown in Step 2, Φ^m converges to Φ point-wise, whence
	
    Q(Φ^m(t))⟶ Q(Φ(t))

	for any t≥ 0.
	Furthermore, by Weierstrass theorem Q:𝕋^n⟶ℝ^+ is bounded. Then, for every T>0, by the Dominated Convergence Theorem,
	
    Q(Φ^m)⟶ Q(Φ)

	in the L^1((0,T);ℝ) norm, whence
	
    ∫_0^T Q(Φ) dt   =   lim_m→ +∞∫_0^T Q(Φ^m)dt
       ≤   lim inf_m→ +∞∫_0^∞ Q(Φ^m)dt.

	Hence, by arbitrariness of T>0,
	
    ∫_0^∞ Q(Φ) dt≤lim inf_m→ +∞∫_0^∞ Q(Φ^m)dt,

	i.e. (<ref>).
	
	In conclusion, by (<ref>) and (<ref>), we have
	
    K(Φ)   =   1/2∫_0^∞Φ̇^2+Q(Φ) dt
       ≤   lim inf_m→ +∞1/2∫_0^∞Φ̇^m^2+Q(Φ^m) dt
       =   inf_𝒜K,

	whence Φ∈𝒜 is the required minimizer. This finishes the proof.




§ PROOF OF PROPOSITION <REF>


After proving the existence of minimizers for <ref>, we derive the Optimality Conditions.


	Step 1   Regularity of Φ by the fundamental Lemma of the Calculus of Variations

	Take Φ a minimizer of (<ref>). By (<ref>) and Fermat's theorem, for any direction v∈ C^∞_c((0,+∞);ℝ^n), we have
	
    ∫_0^∞Φ̇v̇+∇ Q(Φ)· vdt=⟨ dK(Φ),v⟩=0.

	Then, by the fundamental Lemma in the Calculus of Variations (see <cit.>),
	
	Φ∈ C^2([0,T];𝕋^n).

	Step 2   Proof of (2)

	Since Φ∈ C^2, we are allowed to integrate by parts in (<ref>), getting
	
    0   =   ∫_0^∞Φ̇v̇+∇ Q(Φ) vdt
       =   lim_T→ +∞Φ̇(T)v(T)+∫_0^∞[-Φ̈+ ∇ Q(Φ) ]v  dt,

	which, thanks to the arbitrariness of v, leads to the differential equation in (<ref>). Furthermore, by bootstrapping in (<ref>), we have the C^∞ regularity of the minimizer Φ.

	
	
	
	Step 3   Proof of (3)

	Consider the energy
	
    E(t)=1/2Φ̇(t)^2-Q(Φ(t))

	and, take the time derivative
	
    Ė(t)=Φ̇(t)·Φ̈(t)-∇ Q(Φ(t))·Φ̇(t) = -[-Φ̈+ ∇ Q(Φ)]·Φ̇(t) =0,

	where in the last equality we have used the differential equation in (<ref>). Now, the integral ∫_0^∞Φ̇^2 dt is finite, whence there exists a sequence {T_q}⊂ (0,+∞) such that T_qq→∞⟶+∞ and
	
    Φ̇(T_q)q→ +∞⟶0 Q(Φ(t))q→ +∞⟶0.

	Therefore, the energy E(T_q)q→ +∞⟶0, whence, by using (<ref>), we have E(t)≡ 0.
	




§ GENERAL MATHEMATICAL NOTATION



The circumference is denoted by

    𝕋ℝ/∼,

where φ_1∼φ_2 if and only if there exists an integer k such that φ_2= φ_1+2kπ.

We introduce the following function spaces:








    L^2_loc((0,+∞);ℝ^n)⋂_T>0 L^2((0,T);ℝ^n).




    H^1((0,T);𝕋^n){Φ∈ L^2((0,T);𝕋^n)  | .


    . ΦΦ̇∈ L^2((0,T);𝕋^n)}.




    H^1_loc([0,+∞);𝕋^n){Φ∈ H^1((0,T);𝕋^n), ∀T>0};




    C^∞_c((0,+∞);ℝ^n){Φ:[0,+∞)⟶ℝ^n  |.


    Φ


    . (Φ)⊂⊂ (0,+∞)}.

	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	
	siam
	
