Data-based approaches to learning and control by similarity between heterogeneous systems
Chenchao Wang Deyuan Meng



                                              
                                              
                                              

 
                                                

[footnoteinfo]The material in this paper was not	presented at any conference.




Affi1,Affi3]Chenchao Wangchenchaow1999@163.com,    
Affi1,Affi2,Affi3]Deyuan Mengdymeng@buaa.edu.cn         

[Affi1]School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing 100191, PR China             
[Affi2]State Key Laboratory of CNS/ATM, Beijing 100191, PR China
[Affi3]The Seventh Research Division, Beihang University (BUAA), Beijing 100191, PR China  

                           
Similarity; similarity indexes; admissible behavior; sampled data; similarity-based learning.          
                             
                                          
                                          


§ ABSTRACT 
This paper proposes basic definitions of similarity and similarity indexes between admissible behaviors of heterogeneous host and guest systems and further presents a similarity-based learning control framework by exploiting offline sampled data. By exploring helpful geometric properties of the admissible behavior and decomposing it into the subspace and offset components, the similarity indexes between two admissible behaviors are defined as the principal angles between their corresponding subspace components. By reconstructing the admissible behaviors leveraging sampled data, an efficient strategy for calculating the similarity indexes is developed, based on which a similarity-based learning control framework is proposed. It is shown that the host system can directly accomplish the same control tasks by utilizing the successful experience from the guest system, without having to undergo the trial-and-error process.







§ INTRODUCTION



Learning-based control, as one of the most promising fields within control community, has attracted significant attention and popularity. Learning-based control takes direct inspirations from human's learning process (<cit.>). When individuals endeavor to acquire new skills, they repetitively engage in specific tasks and gather experience from past failures, ensuring their ability to better accomplish the same tasks in the future.
In a similar manner, dynamical systems can also recursively benefit from the past and correct the control errors, with the guarantee of the enhanced control performances (<cit.>). Such learning-based control mechanisms that learn from one's own past experience have been extensively investigated, and some well-established control frameworks have been presented (see, e.g., <cit.>). All of these aforementioned control frameworks, which either design controllers or adjust adaptive parameters based on past experience to rectify control errors, have found widespread and successful applications in real-world industrial systems (see, e.g., <cit.>).

Another characteristic of the human learning process entails its inherent strong interactivity, based on which a novice can efficiently acquire new skills through learning from the advanced experience of some skilled experts. Just as human inevitably require to interact with others to achieve common goals, the collaborative learning of multiple dynamic systems to achieve some unified objective is an essential and extensively discussed topic (see, e.g., <cit.>). A simple example may be the leader-follower formation problem in multi-agent systems (<cit.>), which has played a significant role in numerous engineering applications such as unmanned aerial vehicles (<cit.>), autonomous mobile robots (<cit.>), and high-speed trains (<cit.>). To ensure the achievement of the unified objective among the multiple systems, the idea of leveraging experience generated by other systems is extensively adopted, based on which one remarkable milestone is the distributed learning control (<cit.>). Nevertheless, the existing learning-based control strategies somewhat exhibit weaknesses in the following two aspects:

	
  W1) Existing learning-based control strategies simply collect experience from neighbors based on specific communication topology, failing to quantitively assess that which system's experience is more beneficial;
	
  W2) Existing learning-based control strategies directly employ the (weighted) relative information among systems to guarantee consensus, without fully exploiting the potential of the experience from other systems.

Therefore, in the scenarios where the host system is equipped with multiple external experience generated by guest systems, it is meaningful and urgent to develop an innovative learning-based control framework for the host system. A practical scenario is where the leader vehicle can proceed along a specified path through N iterations of trial-and-error  (<cit.>). We need to proposed a learning-based control framework that allows the follower vehicle to directly follow the leader's path without undergoing the trial-and-error process. This framework is required to address both shortcomings mentioned earlier in existing learning-based control strategies.


Simultaneously, with the advancements in computer science and storage technology, employing the sampled data generated during the system operation for control objectives has become increasingly reliable and convenient. There have been several results that presented learning-based control strategies within a data-driven framework (see, e.g., <cit.>, <cit.>). However, these results pay few attention to exploiting the successful experience of guest systems.

Motivated by aforementioned discussions, this paper is devoted to proposing the definitions of similarity and similarity indexes to address W1). Afterward, by exploiting the sampled data, a similarity-based learning control framework is developed to address W2), which focuses on how to efficiently learn from the successful experience of guest systems even in the absence of model information. The mechanism of the similarity-based learning control framework is depicted in Fig. <ref>.

Main contributions of this paper can be summarized as follows.

	
  C1) We innovatively propose the basic definitions of similarity and similarity indexes between two admissible behaviors, which can qualitatively and quantitatively measure the benifits of guest system's successful experience of to the control of the host system, respectively;
	
  C2) By designing offline input-output testing principles for linear time-varying (LTV) systems and exploiting the collected I/O data, we develop a data-based criterion for verifying the similarity and present a efficient data-based strategy for calculating the similarity indexes;
	
  C3) By leveraging the calculated similarity indexes and exploiting helpful projection techniques, we establish a similarity-based learning control framework from the offline sampled data. As a result, this framework allows the host system to directly leverage the successful experience of the guest system to accomplish the unified tasks, without resorting to other learning-based control strategies.



The rest of this paper is organized as follows. We present the preliminaries on admissible behavior of LTV systems and formulate the similarity-based learning control problems in Section <ref>. In Section <ref>, we design the offline input-output test principles and reconstruct the admissible behaviors from the sampled data. Afterward, we introduce the definitions of similarity and similarity indexes, and develop a data-based criterion for verifying the similarity and a data-based strategy for efficiently calculating the similarity indexes in section <ref>. In Section <ref>, by exploiting the calculated similarity indexes and projection techniques, the similarity-based learning control framework is presented exploiting the sampled data. Finally, Section <ref> provides illstrative simulations, and Section <ref> summarizes the contributions in this paper.

Notations: Let ℤ_N={0,1,⋯,N} and ℤ_+={0,1,2,⋯}. Let ℝ be the set involving all real numbers, and ℝ^n involves all n-dimensional real vectors whose entries locate in ℝ. For any matrix A, its transpose and kernel space are denoted as A^ T and ker(A), respectively. The linear space spanned by the columns of A is denoted as span(A). For arbitrary vectors a,b∈ℝ^n, the standard inner product ⟨ a,b⟩ refers to a^ Tb, and the induced norm is correspondingly defined as a=√(⟨ a,a ⟩). The identity and null matrices with appropriate dimensions are denoted as I and 0, respectively. Given s_1,s_2,⋯,s_n∈ℝ, the symbol diag(s_1,s_2,⋯,s_n) represents the diagonal matrix whose diagonal entries are s_1,s_2,⋯,s_n.





§ ADMISSIBLE BEHAVIOR AND PROBLEM STATEMENT
 
The preliminaries of admissible behavior are firstly introduced. We consider two unknown heterogeneous LTV systems whose dynamics within the time duration 𝕋 are represented as

    Σ_i,𝕋:{
    		x_i(t+1)   =A_i(t)x_i(t)+B_i(t)u_i(t) 
    
    		y_i(t)   =C_i(t)x_i(t)+D_i(t)u_i(t)
    	., t∈𝕋,  i∈{1,2}.

It is worth mentioning that the results proposed in this paper can be implemented equally to the scenarios where D_i(t)≡0 for all t∈𝕋, and the introduction of D_i(t) is solely for a generalized expression.
Here, the subscripts i=1 and i=2 refer to the host system and guest system, respectively, and the host system Σ_1,𝕋 can acquire experience from the guest system Σ_2,𝕋. Without loss of generality, the time duration is assumed to be 𝕋:=ℤ_T-1. The input and output are denoted as u_i(t)∈ℝ^n_u and y_i(t)∈ℝ^n_y, respectively. The internal state with unknown dimension is denoted as x_i(t)∈ℝ^∙, and the unknown time-varying model matrices with appropriate dimension is represented by {A_i(t),B_i(t),C_i(t),D_i(t)}. In order to investigate the input-output relationship over the entire time duration 𝕋, the following supervectors

    𝐮_i   =[ u_i^ T(0), u_i^ T(1), ⋯, u_i^ T(T-1) ]^ T, 
    𝐲_i   =[ y_i^ T(0), y_i^ T(1), ⋯, y_i^ T(T-1) ]^ T, 
    𝐱_i   =[ x_i^ T(0), x_i^ T(1), ⋯, x_i^ T(T-1) ]^ T

are introduced. For a vector w_i=col(𝐮_i,𝐲_i)∈ℝ^n_wT where n_w=n_u+n_y, if there exists some (may be non-unique) state supervector 𝐱_i such that (𝐮_i,𝐲_i,𝐱_i) satisfies (<ref>), then w_i is called as a T-length trajectory of Σ_i,𝕋. To capture the input-output transfer characteristics, the behavior of Σ_i,𝕋, denoted by ℬ_i, is defined as the set involving all T-length trajectories

    ℬ_i={w_i∈ℝ^n_wT|∃𝐱_i such that (𝐮_i,𝐲_i,𝐱_i) satisfies (<ref>).}.

It is worth mentioning that the above definition only focuses on the input-output transfer characteristics, but neglects the initially stored energy in the system Σ_i,𝕋, which can be characterized by x_i(0), and its influence on the system response. Without loss of generality, in this paper, we assume that the initial state of Σ_i,𝕋 is x_i(0)=x_i. Due to this consideration, we introduce a class of T-length admissible trajectories, denoted by w_i,x_i, which refer to those T-length trajectories who start from the initial state x_i(0)=x_i. Correspondingly, the admissible behavior is defined as follows.


 For the LTV system Σ_i,𝕋, its admissible behavior under the initial state x_i(0)=x_i, denoted by ℬ_i,x_i, is defined as the set involving all T-length admissible trajectories, i.e.,
	
    ℬ_i,x_i={w_i,x_i∈ℝ^n_wT|w_i,x_i∈ℬ_i and x_i(0)=x_0.}


 From the above definition, the admissible behavior is essentially a subset of the behavior, i.e., ℬ_i,x_i∈ℬ_i, since it is subject to extra constraints with respect to the initial energy. Admissible behavior also has specific engineering implications since practical systems always commence with some initially stored energy, and a direct example may be the RLC circuits where the the initial charge of the capacitor has an impact on the system response (see, e.g., <cit.>).

Based on the aforementioned preliminaries, we formulate the to-be-addressed problems in this paper as follows.


 For the unknown host system Σ_1,𝕋 with initial state x_1(0)=x_1 and unknown guest system Σ_2,𝕋 with initial state x_2(0)=x_2, let their admissible behaviors be denoted as ℬ_1,x_1 and ℬ_2,x_2, respectively. This paper focuses on dealing with the following problems:
	
		
  P1) Appropriate offline input-output test principles need to be designed, under which the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 can be reconstructed by exploiting the sampled data; 
		
  P2) The definitions of similarity and similarity indexes between two admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 need to be introduced. Moreover, a data-based criterion for verifying the similarity and a data-based strategy for calculating the similarity indexes need to be developed;
		
  P3)  Suppose that the guest system Σ_2,𝕋 has accomplished its task by leveraging some powerful learning-based control strategies and achieved the desired trajectory w_g∈ℬ_2,x_2. A similarity-based learning control framework for the host system Σ_1,𝕋 needs to be proposed such that it can accomplish the same control task by exploiting the successful experience of Σ_2,𝕋 and the sampled data. As a result, we will find a solution w_h∈ℬ_1,x_1 such that the difference ‖ w_g-w_h‖ is minimized.
	





§ DATA-BASED VERIFICATION FOR SIMILARITY AND SIMILARITY INDEXES
 


 §.§ Data-based reconstruction for admissible behaviors


Compared to the system matrices {A_i(t),B_i(t),C_i(t),D_i(t)}, the admissible behavior ℬ_i,x_i accurately captures the input-output transfer characteristics of the system Σ_i,𝕋, without involving the non-unique internal states.
Owing to the absence of model knowledge, the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 need to be identified by exploiting the sampled data. Before designing appropriate offline input-output test principles and collecting offline I/O data, several helpful geometric properties of the admissible behavior are explored.


 
	For the LTV system Σ_i,𝕋, its admissible behavior ℬ_i,x_i constitutes an affine set. Moreover, if the initial state is x_i=0, the admissible behavior ℬ_i,0 is a subspace.


	In order to investigate the relationship among the input, initial state, and output over the entire time duration 𝕋, we introduce the input-output transfer matrix G_i and and initial state-output transfer matrix L_i, both of which can be steadily constructed by employing the model matrices {A_i(t),B_i(t),C_i(t),D_i(t)} (see, e.g., <cit.>). Then by leveraging the supervectors in (<ref>), it is derived that
	
    𝐲_i=G_i𝐮_i+L_ix_i(0).

	Under the given initial state x_i(0)=x_i, any admissible trajectory of Σ_i,𝕋 must be the solution of the non-homogeneous linear algebraic equation (LAE) described by
	
    [ -G_i, I ]w_i,x_i=L_ix_i.

	By exploiting the properties of solutions to non-homogeneous LAEs, it directly follows that ℬ_i,x_i constitutes an affine set. Particularly, once the initial state is set as x_i=0, the admissible trajectory w_i,0 must locate within ker([ -G_i, I ]), and the admissible behavior ℬ_i,0 is exactly the subspace ker([ -G_i, I ]) in this case.

 
	Through exploring the geometric properties of the admissible behavior, Lemma <ref> reveals a helpful fact that the affine combination of two admissible trajectories remains an admissible behavior under the same initial state. To be specific, for two admissible trajectories w'_i,x_i∈ℬ_i,x_i and w”_i,x_i∈ℬ_i,x_i, their affine combination, defined as
	
    α w'_i,x_i+(1-α)w”_i,x_i,  ∀α∈ℝ

	is still an admissible trajectory in ℬ_i,x_i. Inspired by this fact, if a sufficient number of representative admissible trajectories are collected via I/O tests, then the entire admissible behavior ℬ_i,x_i can be reconstructed through affine combinations of these admissible trajectories. This provides theoretical support for recovering the admissible behavior based on sampled data.


As emphasized in Remark <ref>, the admissible bahaviors can be recovered via the affine combination of a sufficient number of representative admissible trajectories even in the absence of model knowledge. Therefore, it is necessary to design the appropriate offline input-output test principles to ensure the collection of required admissible trajectories. The following strategy can provide an alternative data-based representation for the admissible behavior. For the systems Σ_i,𝕋 where i∈{1,2}, at least n_uT+1 times I/O tests need to be conducted. In each I/O test, the system Σ_i,𝕋 starts from an unknown but fixed initial state x_i, and the k-th test input over the entire time duration 𝕋 is denoted as 𝐮_i^k∈ℝ^n_uT. Correspondingly, the k-th test output over 𝕋 is denoted as 𝐲_i^k∈ℝ^n_yT, and all test input/output data are collected as

    U^Test_i   =[ 𝐮^0_i, 𝐮^1_i, ⋯, 𝐮^n_uT_i ]∈ℝ^n_uT×(n_uT+1),
    
    		Y^Test_i   =[ 𝐲^0_i, 𝐲^1_i, ⋯, 𝐮^n_uT_i ]∈ℝ^n_yT×(n_uT+1).

To obatin a sufficient number of representative admissible trajectories, the test inputs need to be specifically designed.


 For the LTV system Σ_i,𝕋, the following offline test principles need to be conducted:
	
		
  * In the initial I/O test, the test input is designed as
		
    𝐮^0_i=[ 0^ T_n_u, 0^ T_n_u, ⋯, 0^ T_n_u ]^ T∈ℝ^n_uT;

		
  * In the later n_uT I/O tests, the test inputs are designed to satisfy the following rank condition
		
    rank([ 𝐮^1_i, 𝐮^2_i, ⋯, 𝐮^n_uT_i ])=n_uT.

	

By leveraging the sampled data collected building upon the above test principles, we can construct a data-based representation to reconstruct the admissible behavior.


 
	For the LTV system Σ_i,𝕋, let the sampled I/O data (U_i^Test,Y_i^Test) satisfy the offline test principles (<ref>) and (<ref>). A vector w_i,x_i=col(𝐮_i,𝐲_i)∈ℬ_i,x_i if and only if there exists some g_i∈ℝ^n_uT+1 such that
	
    [ 1^ T_n_uT+1;    U_i^Test;    Y_i^Test ]g_i=[   1; 𝐮_i; 𝐲_i ].



	Sufficiency: From the designed offline I/O test principles (<ref>) and (<ref>), it can be concluded that col(𝐮^j_i,𝐲^j_i)∈ℬ_i,x_i holds for all j∈ℤ_n_uT and i∈{1,2}, or equivalently, is essentially an admissible trajectory. The first equation in (<ref>) actually indicates that [ (U^Test_i)^ T, (Y^Test_i)^ T ]^ Tg_i represents the affine combination of the columns of the data matrix [ (U^Test_i)^ T, (Y^Test_i)^ T ]^ T. From Lemma <ref>, the affine combination of admissible trajectories remains an admissible trajectory. Therefore, w_i,x_i=col(𝐮_i,𝐲_i)∈ℬ_i,x_i if (<ref>) holds.
	
	Necessity: From the designed offline test principle (<ref>), the test inputs {𝐮^1_i, 𝐮^2_i, ⋯, 𝐮^n_uT_i } form a set of bases of ℝ^n_uT. That implies that, for arbitrary input 𝐮_i∈ℝ^n_uT, there must exist a series of real numbers g_i,k∈ℝ, k∈ℤ_n_uT such that
	
    𝐮_i=∑_k=1^n_uTg_i,k𝐮^k_i+g_i,0𝐮_i^0

	where 𝐮^0_i=0_n_uT is defined in test principle (<ref>).
	By designing g_i,0=1-∑_k=1^n_uTg_i,k and leveraging Lemma <ref>, it directly follows that the corresponding output satisfies
	
    𝐲_i   =∑_k=1^n_uT(g_i,k𝐲^k_i-g_i,kL_ix_i)+g_i.0𝐲_i^0-g_i,0L_ix_i+L_ix_i
       =∑_k=1^n_uTg_i,k𝐲^k_i+g_i,0𝐲_i^0.

	Therefore, for arbitrary w_i,x_i∈ℬ_i,x_i, there always exists a vector g_i=[ g_i,0, g_i,1, ⋯, g_i,n_uT ]^ T such that (<ref>) holds.


 
	Following the established data-based representation (<ref>) for the admissible behavior, some helpful geometric properties of the admissible behavior can be further explored. Let w_i^k=col(𝐮_i^k,𝐲_i^k) where k∈ℤ_n_uT be the k-th test admissible trajectory. By leveraging the offline I/O data, the admissible behavior can be decomposed into the sum of subspace and offset components as
	
    ℬ_i,x_i   =𝒲_i+w_i^0,
    𝒲_i   =span(w_i^1-w_i^0,w_i^2-w_i^0,⋯,w_i^n_uT-w_i^0).

	From the offline test principles (<ref>) and (<ref>), 𝒲_i is a subspace in Euclidean space ℝ^n_wT and is of dimension n_uT, which is exactly the number of free input channels  (see, e.g., <cit.>).
	For simplicity, we define
	
    H_i=[ α_i^1, α_i^2, ⋯, α_i^n_uT ]

	where the vectors {α_i^1, α_i^2, ⋯, α_i^n_uT} form a set of unit orthogonal bases of 𝒲_i, such that the subspace component can be simply expressed as 𝒲_i=span(H_i). Additionally, H_i can be readily obtained through the Gram–Schmidt process from the sampled I/O data (U_i^Test,Y_i^Test).


To further explore the geometric properties between two affine set, we present the definition of principal angles between two subspaces.
  

	(<cit.>) For two subspaces ℛ_1⊂ℝ^n and ℛ_2⊂ℝ^n with dim(ℛ_1)=dim(ℛ_2)=m, m<n, the principal angles
	
    Θ(ℛ_1,ℛ_2)=[ θ_1,θ_2,⋯,θ_m ], θ_k∈[0,π/2], k∈ℤ_m\{0}

	between ℛ_1 and ℛ_2 are recursively defined as
	
    s_k=cos(θ_k)=max_x∈ℛ_1max_y∈ℛ_2⟨ x,y⟩=⟨ x_k,y_k⟩

	subject to
	
    x=y=1, ⟨ x,x_i⟩=0, ⟨ y,y_i⟩=0, i∈ℤ_k-1\{0}

	Moreover, the vectors {x_1,x_2,⋯,x_m} and {y_1,y_2,⋯,y_m} are called the principal angles associated with ℛ_1 and ℛ_2.




 §.§ Similarity and similarity indexes


Regarding the host system Σ_1,𝕋 with x_1(0)=x_1 and the guest system Σ_2,𝕋 with x_2(0)=x_2, this subsection aims at introducing the definitions of similarity and similarity indexes. Moreover, by leveraging the collected offline I/O data, a data-based criterion for verifying the similarity and a data-based strategy for calculating the similarity indexes are developed. The definition of similarity between ℬ_1,x_1 and ℬ_2,x_2 is presented as follows.


 
	The admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are said to be similar if ℬ_1,x_1∩ℬ_2,x_2≠∅.

Even if ℬ_1,x_1∩ℬ_2,x_2≠∅ and ℬ_1,x_1∩ℬ_3,x_3≠∅, it does not necessarily follow that ℬ_2,x_2∩ℬ_3,x_3≠∅. From definition <ref>, when two admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are similar, there always exist some common admissible trajectories w_com∈ℬ_1,x_1∩ℬ_2,x_2 and common behavior ℬ_com involving all common admissible trajectories. Taking into account the fact that the admissible behavior can be decomposed as ℬ_i,x_i=span(H_i)+w_i^0, a data-based criterion for similarity can be readily derived as follows.



	Let the sampled I/O data (U_i^Test,Y_i^Test) satisfy the offline test principles (<ref>) and (<ref>), and let H_i be constructed as in (<ref>). Then admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are similar if and only if there exist vectors l_1∈ℝ^n_uT and l_2∈ℝ^n_uT such that
	
    [ H_1 H_2 ][ l_1; l_2 ]=w_2^0-w_1^0.

	Moreover, by solving the above non-homogeneous LAE, the common behavior can be expressed as
	
    ℬ_com={w_com∈ℝ^n_wT|w_com=H_1l_1+w_1^0.}.



	A consequence of Lemma <ref> and Definition <ref>.


 
	From Definition <ref> and Lemma <ref>, it is observed that the similarity is a rather loose concept, serving only as a qualitative assessment indicator. However, it can assist to address the Problem P3) in some special cases. To be specific, once the learned trajectory of the guest system satisfies w_g∈ℬ_com, the host system can directly adopt the successful experience w_g to accomplish the same tasks. In this situation, the difference w_g-w_h is equal to zero.

Building upon the definition of similarity, in order to further quantitatively assess the benefits of the successful experience of the guest system  on the control of host system, the definition of similarity indexes needs to be proposed. Since the admissible behavior can be decomposed into a sum of subspace and offset components, from a geometric perspective, the principal angles between two subspaces 𝒲_1 and 𝒲_2 can serve as a powerful tool. Based on Definition <ref>, the similarity indexes between two admissible behaviors are defined as follows.


 
	For similar admissible behaviors ℬ_1,x_1 and ℬ_2,x_2, let them be decomposed as (<ref>), the similarity indexes between ℬ_1,x_1 and ℬ_2,x_2, denoted by SI(ℬ_1,x_1,ℬ_2,x_2), refer to the cosine of the principal angles between 𝒲_1 and 𝒲_2, that is,
	
    SI(ℬ_1,x_1,ℬ_2,x_2):=cosΘ(𝒲_1,𝒲_2).





	
	The similarity indexes SI(·,·) are essentially the function with respect to two intersecting affine sets, and SI has the following properties:
	
		
  P1) SI(ℬ_1,x_1,ℬ_2,x_2)=SI(ℬ_2,x_2,ℬ_1,x_1), ∀ℬ_1,x_1, ℬ_2,x_2;
		
  P2) SI(ℬ_1,x_1,ℬ_1,x_1)-1_n_uT^ T;
		
  P3) Even if SI(ℬ_1,x_1,ℬ_2,x_2)=SI(ℬ_1,x_1,ℬ_3,x_3), it does not necessarily follow that ℬ_2,x_2=ℬ_3,x_3.
	



 
	By leveraging the offline test principles (<ref>) and (<ref>), the decomposition in (<ref>) can be constructed from sampled data, ensuring that the calculation of similarity indexes is model-independent. Additionally, the similarity indexes between two admissible behaviors are independent on the offset components w_i,off, which can be interpriated through a geometric perspective. Since two similar admissible behaviors are essentially intersecting affine hyperplanes in Euclidean spaces, the offset components only cause translations of the corresponding affine hyperplanes and does not affect their intersection and the principal angles.



 Compared to the concept of similarity proposed in Definition <ref>, the similarity indexes are quantitative assessment indicator. Specifically, the similarity indexes geometrically reflect the angle between two affine hyperplanes determined by the admissible behaviors. The similarity indexes closer to 1^ T_n_uT indicates that two admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are more similar.


Although Definition <ref> presents the concept of the similarity indexes, such a definition is unefficient to calculate the similarity indexes. In order to develop an efficient calculation strategy, we define the SVD of the matrix H_1^ TH_2 as

    H_1^ TH_2=UDV^ T

where

    D=diag(s_1,s_2,⋯,s_n_uT), s_1≥ s_2≥⋯≥ s_n_uT>0.

Through the designed offline I/O test principles (<ref>) and (<ref>), a data-based strategy can be proposed to efficiently calculate the similarity indexes, which is demonstrated in the following theorem.


 
	For admissible behaviors ℬ_1,x_1 and ℬ_2,x_2, let
	
		
  * The sampled I/O data (U_i^Test,Y_i^Test) satisfy the offline test principles (<ref>) and (<ref>);
		
  * The matrix H_i be constructed as in (<ref>);
		
  * The SVD of H_1^ TH_2 be given as (<ref>) and (<ref>).
	
	If (<ref>) is solvable,	then the similarity indexes between ℬ_1,x_1 and ℬ_2,x_2 can be calculated as
	
    SI(ℬ_1,x_1,ℬ_2,x_2)=[ s_1, s_2, ⋯, s_n_uT ].

	Moreover, the principal vectors associated with 𝒲_1 and 𝒲_2 are given by H_1U and H_2V.


	Since the LAE (<ref>) is solvable, the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are similar, from which the similarity indexes can be further calculated. Following the offline test principles (<ref>) and (<ref>) and the matrix H_i in (<ref>), it hold that ℬ_i,x_i=𝒲_i+w_i^0 and 𝒲_i=span(H_i). According to the definitions of singular values and singular vectors, the k-th biggest singular value of the matrix H_1^ TH_2 can be expressed as
	
    s_k=max_l=v=1 l^ TH_1^ TH_2v=l_k^ TH_1^ TH_2v_k, k∈ℤ_n_uT\{0}

	subject to
	
    ⟨ l,l_i⟩=⟨ v,v_i⟩=0, i∈ℤ_k-1\{0}

	where l_i∈ℝ^n_uT and v_i∈ℝ^n_uT. From the data-based construction of H_i in (<ref>), the matrices H_1 and H_2 are both orthogonal matrices. By introducing the following coordinate transformation
	
    x_i   =H_1l_i∈𝒲_1, x=H_1l∈𝒲_1
    
    			y_i   =H_2v_i∈𝒲_2, y=H_2v∈𝒲_2
    		, i∈ℤ_k-1\{0}

	it is directly concluded that
	
    x   =H_1l=l=1,
    y   =H_2v=v=1

	and
	
    ⟨ x,x_i⟩   =⟨ H_1l,H_1l_i⟩=⟨ l,l_i⟩=0
    ⟨ y,y_i⟩   =⟨ H_2v,H_2v_i⟩=⟨ v,v_i⟩=0
    		, i∈ℤ_k-1\{0}.

	Afterward, the k-th biggest singular value s_k defined in (<ref>) can be alternatively represented by
	
    s_k=max_x∈𝒲_1max_y∈𝒲_2⟨ x,y⟩=⟨ x_k,y_k⟩, k∈ℤ_n_uT\{0}

	subject to
	
    x=y=1, ⟨ x,x_i⟩=0, ⟨ y,y_i⟩=0, i∈ℤ_k-1\{0}.

	From Definition <ref>, the k-th biggest singular value of the matrix H_1^ TH_2 is exactly the cosine of k-th smallest principal angle between the subspaces 𝒲_1 and 𝒲_2, that is,
	
    s_k=cos(θ_k), k∈ℤ_n_uT\{0}.

	Therefore, the similarity indexes between ℬ_1,x_1 and ℬ_2,x_2 (or equivalently, the principal angles between subspaces 𝒲_1 and 𝒲_2) can be efficiently obtained through computing the singular values of the matrices H_1^ TH_2, that is,
	
    SI(ℬ_1,x_1,ℬ_2,x_2)=[ s_1, s_2, ⋯, s_n_uT ].

	Additionally, of note is that the vectors l_i and v_i in (<ref>) are essentially the i-th column of the orthogonal matrices U and V, that is,
	
    U   =[ l_1, l_2, ⋯, l_n_uT ],
     V   =[ v_1, v_2, ⋯, v_n_uT ].

	From Definition <ref>, the principal vectors associated with the subspaces 𝒲_1 and 𝒲_2 can be obtained from
	
    [ x_1, x_2, ⋯, x_n_uT ]   =H_1[ l_1, l_2, ⋯,  l_n_uT ]=H_1U,
    [ y_1, y_2, ⋯, y_n_uT ]   =H_2[ v_1, v_2, ⋯, v_n_uT ]=H_2V.

	Consequently, the similarity indexes between two admissible behaviors and the principal angles between their associated subspace components can be efficiently calculated through the SVD of H_1^ TH_2.

After presenting the definition of similarity indexes and calculating them from sampled data, we can pay our attention back to Problem P3), which is addressed in Section <ref>.





§ SIMILARITY-BASED LEARNING CONTROL FRAMEWORK
 
In this section, a similarity-based learning control framework is proposed by leveraging the sampled I/O data to address Problem P3). We suppose that, through some powerful control strategies, the guest system Σ_2,𝕋 has already accomplished its tasks and learned the admissible trajectory w_g. The core idea of the similarity-based learning control framework lies in that when the host system Σ_1,𝕋 is confronted with the unified tasks, the successful experience of the guest system can provide helpful guidance.
Moreover, the benefits of the successful experience of the guest system to the host system can be quantitively assessed via the similarity indexes introduced in Section <ref>,.

Specifically, as we revisit Problem P3), it is evident that the to-be-sought w_h is essentially the orthogonal projection of w_g onto ℬ_1,x_1. Existing learning-based control strategies depend on the model information of Σ_1,𝕋, adjusting the controller parameters through repetitive trial-and-error to ultimately find w_h. With respect to the mechanism of the existing learning-based control strategies, an illustrative example in the 3-dimensional Euclidean space ℝ^3 is depicted in Fig. <ref>. In contrast, the similarity-based learning control framework aims to directly obtain w_h via projection techniques by employing the similarity indexes and the successful experience of the guest system, and the trial-and-error processes are no longer needed. Likewise, an illustrative example is depicted in Fig. <ref>, where w_h can be efficiently calculated by exploiting the similarity indexes cosΦ and w_g, ensuring that w_h-w_g is minimized.


To present the similarity-based learning control framework precisely, the orthogonal projection operator onto the subspace 𝒲_1 is denoted by P_𝒲_1(·):ℝ^n_wT→𝒲_1. Correspondingly, the orthogonal projection operator onto the admissible behavior ℬ_1,x_1 is denoted by P_ℬ_1,x_1(·):ℝ^n_wT→ℬ_1,x_1. That is, for any w_g∈ℝ^n_wT, P_ℬ_1,x_1(w_g) refers to its orthogonal projection onto ℬ_1,x_1 that minimizes the difference w_g-P_ℬ_1,x_1(w_g). Before presenting the similarity-based learning control framework, the following lemma is introduced as the preliminary.


 
	(<cit.>)
	Let the sampled I/O data (U_i^Test,Y_i^Test) satisfy the offline test principles (<ref>) and (<ref>), and let H_i be constructed as in (<ref>). Then for all x∈ℝ^n_wT, the orthogonal projection onto ℬ_1,x_1 can be calculated by
	
    P_ℬ_1,x_1(x)=w_1^0+P_span(H_1)(x-w_1^0).


By leveraging the offline sampled data, Lemma <ref> calculates the orthogonal projection onto the admissible behavior via investigating another orthogonal projection onto the associated subspace component. In comparison to existing learning-based control strategies, the main superiority of the proposed similarity-based learning control framework lies in the fact that the term P_span(H_1)(x-w_1^0) can be efficiently obtained by exploiting the similarity indexes and projection techniques, which is demonstrated as follows.


 
	For admissible behaviors ℬ_1,x_1 and ℬ_2,x_2, let
	
		
  *  The sampled I/O data (U_i^Test,Y_i^Test) satisfy the offline test principles (<ref>) and (<ref>);
		
  *  The matrix H_i be constructed as in (<ref>);
		
  *  There exist l_1 and l_2 such that (<ref>) holds;
		
  *  The SVD of H_1^ TH_2 be given as (<ref>) and (<ref>).
	
	For the learned admissible trajectory given by w_g∈ℬ_2,x_2, the optimal admissible trajectory w_h∈ℬ_1,x_1 is calculated as
	
    w_h=H_1UDg+P_span(H_1)(w_2^0-w_1^0)+w_1^0

	where g satisfies
	
    w_g=H_2Vg+w_2^0.

	In this situation, the difference w_h-w_g is minimized, or equivalently, Problem P3) is addressed.


	From conditions (<ref>) and (<ref>), by leveraging the sampled data (U_i^Test,Y_i^Test), the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 can be decomposed as
	
    ℬ_i,x_i=span(H_i)+w_i^0.

	As previously emphasized, the to-be-sought w_h in Problem 3) is essentially the orthogonal projection of w_g onto ℬ_1,x_1, i.e., P_ℬ_1,x_1(w_g). From conditions (<ref>), the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 are similar, which allows for further calculating the similarity indexes. Additionally, by leveraging Theorem <ref>, the condition (<ref>) ensures that the similarity indexes between ℬ_1,x_1 and ℬ_2,x_2 can be obtained via calculating the singular values of H^ T_1H_2 as
	
    SI(ℬ_1,x_1,ℬ_2,x_2)=[ s_1, s_2, ⋯, s_n_uT ].

	Another helpful conclusion brought by the conditions (<ref>) is that the principal vectors associated with the subspaces 𝒲_1 and 𝒲_2 can be calculated as H_1U and H_2V. Consequently, the admissible behaviors can be equivalently expressed as
	
    ℬ_1,x_1   =span(H_1U)+w_1^0,
    ℬ_2,x_2   =span(H_2V)+w_2^0.

	For any admissible trajectory w_g∈ℬ_2,x_2, there always exists some vector g∈ℝ^n_uT such that (<ref>) holds, based on which we can conclude that
	
    w_h   =P_ℬ_1,x_1(w_g) 
       =P_ℬ_1,x_1(H_2Vg+w_2^0).

	However, the operator P_ℬ_1,x_1(·) is not linear, which results in difficulties in calculating this orthogonal projection. From Lemma <ref>, it can be concluded that the orthogonal projection onto ℬ_1,x_1 can be equivalently expressed as
	
    P_ℬ_1,x_1(H_2Vg+w_2^0)   =w_1^0+P_span(H_1U)(H_2Vg+w_2^0-w_1^0).

	Thanks to the linearity of the operator P_span(H_1U)(·), w_h can be further rewritten as
	
    w_h=w_1^0+P_span(H_1U)(H_2V)g+P_span(H_1U)(w_2^0-w_1^0).

	By leveraging the principal vectors H_1U and H_2V, the orthogonal projection in (<ref>) can be efficiently computed. This is also the prominent advantage of the similarity-based learning compared to existing learning-based methods. Let the i-th column of H_1U (or H_2V) be denoted as (H_1U)_i (or (H_2V)_i), then P_span(H_1U)(H_2V) can be expressed as
	
    P_span(H_1U)(H_2V)
    
    			=   [ P_span(H_1U)(H_2V)_1, ⋯,				P_span(H_1U)(H_2V)_n_uT ]

	where
	
    P_span(H_1U)(H_2V)_i=∑_j=1^n_uT⟨(H_2V)_i,(H_1U)_j⟩(H_1U)_i

	holds for all i∈ℤ_n_uT\{0}. Following Definitions <ref> and <ref> and the properties of SVD, it follows that
	
    s_k   =⟨(H_2V)_k,(H_1U)_k⟩,  ∀ k∈ℤ_n_uT\{0},
    
    			0   =⟨(H_2V)_i,(H_1U)_j⟩, ∀ i≠ j, ∀ i,j∈ℤ_n_uT\{0}.

	By leveraging (<ref>), the orthogonal projection P_span(H_1U)(H_2V) can be further expressed as
	
    P_span(H_1U)(H_2V)
    
    			=   [ P_span(H_1U)(H_2V)_1, ⋯,				P_span(H_1U)(H_2V)_n_uT ]
    
    			=   [ (H_1U)_1s_1, ⋯, (H_1U)_n_uTs_n_uT ]
    
    			=   H_1UD.

	Therefore, the to-be-sought admissible trajectory w_h∈ℬ_1,x_1 can be further expressed as
	
    w_h=w_1^0+H_1UDg+P_span(H_1)(w_2^0-w_1^0).

	Since w_h is essentially the orthogonal projection of w_g onto ℬ_1,x_1, the difference w_h-w_g must be minimal.


 The similarity-based learning control framework proposed in Theorem <ref> can provide  an innovative perspective on learning-based control. When seeking the optimal trajectory w_h∈ℬ_1,x_1, we no longer require to repeatedly execute certain learning-based control strategies for Σ_1,𝕋. Alternatively, we can directly obtain the optimal trajectory w_h leveraging the successful experience of guest systems. From Theorem <ref>, it is not difficult to observe that the closer cosΘ(𝒲_1,𝒲_2) is to 1^ T_n_uT, the smaller the difference ‖ w_h-w_d ‖. Additionally, the similarity indexes between ℬ_1,x_1 and ℬ_2,x_2 can be compensated by interconnecting the host system with another auxiliary system, such that the learning performance can be further enhanced. Based on this consideration, similarity-based learning control can be utilized in some high-similarity scenarios. For example, when there exist small uncertainties in the model parameters, the control of the real system can greatly benefit from the successful experience of the nominal model without having to re-identify the parameters and repeat the learning process.



 Theorem 2 minimizes the difference between the learned admissible trajectories of the host and guest systems, denoted as w_h-w_g. It is worth emphasizing that in many applications, we only require certain element of the admissible trajectories of the host and guest systems to be as close as possible. For example, in output tracking tasks, we expect the difference between the admissible outputs, denoted as 𝐲_h-𝐲_c, to be as small as possible. The similarity-based learning framework proposed in Theorem 2 can readily address such problems because all admissible outputs likewise span an affine set, as discussed in Remark <ref>. Therefore, by selecting appropriate affine sets, the similarity-based learning control framework can be applied to a wide range of application scenarios.



 Just like humans need to absorb a wide range of learning experiences from others, an increase in the number of successful experiences will improve the control performance of the similarity-based learning. This is because, as the number of guest systems increases, there always exists a guest system whose admissible behavior shares more similarity with that of the host system. By adopting the successful experience of the “most similar” guest system, the similarity-based learning control framework can eventually achieve better learning control performance.


Building upon the previously proposed results, the process of similarity-based learning control framework can be summarized in Algorithm <ref>.





§ SIMULATION EXAMPLES
 
For the illustration of the proposed similarity-based learning control frameworks, simulation examples are presented in this section. We provide a numerical example and simulation tests on the mobile robots simultaneously.


 Consider two heteronogeous discrete-time linear systems in the form of (<ref>), and their model matrices are given as follows:
	
    A_1(t)   =[       0.05t           1           0;           0       0.05t           1;       -0.09       -0.60 -1.40+0.05t ],
    
    			A_2(t)   =[      0.05t          1          0;          0      0.05t          1;      -0.08      -0.66 -1.5+0.05t ],
    
    			B_1(t)   =B_2(t)=[    6;    0; 0.50 ], C_1(t)=C_2(t)=[ 2; 1; 0 ]^ T,
    
    			D_1(t)   =D_2(t)=0, x_1(0)=[    0;    0; 1.02 ], x_2(0)=[ 0; 0; 1 ].

	Here, we present the model knowledge solely for clear illustration of the simulation settings, and it will not be utilized for the design and analysis. Of note is that this type of difference often arises in scenarios where there exist uncertainties between the host system Σ_1,𝕋 and guest system Σ_2,𝕋. Let the host and guest systems be given a consistent output tracking task over the time duration ℤ_34, with the reference output set as
	
    y_d(t)=e^-0.1tsin(π/5t), ∀ t∈ℤ_34.

	Owing to the absence of model knowledge, offline I/O tests are needed to collect a sufficient number of admissible trajectories. The offline I/O tests need to be executed for at least 36 times, and the test inputs are designed as
	
    𝐮_i^0=0_35, [ 𝐮_i^1, 𝐮_i^2, ⋯,𝐮_i^35 ]=I_35.

	With the designed test inputs, the proposed offline test principles (<ref>) and (<ref>) are satisfied, then the data-based representation (<ref>) can be constructed, and the admissible behaviors ℬ_1,x_1 and ℬ_2,x_2 can be decomposed by leveraging (<ref>).
	To address the output tracking problem of the guest system Σ_2,𝕋, iterative learning control (ILC) that is a learning-based strategy can serve as a powerful tool. After 300 iterations of the algorithm, the tracking problem of the guest system Σ_2,𝕋 is perfectly addressed. The output and input of the guest system, denoted as y_g(t)-ILC and u_g(t)-ILC, respectively, are depicted in Fig. <ref>.
	
	With the obtained admissible trajectory w_g∈ℬ_2,x_2, the tracking issue of the host system no longer needs to resort to ILC, which depends on repetitive learning and trial-and-error. By exploiting the proposed similarity-based learning control framework, the required w_h∈ℬ_1,x_1 can be obtained through Theorem <ref>. The learned output and input of the host system, denoted by y_h(t)-SBL and u_h(t)-SBL, respectively, are depicted in Fig. <ref>. From Fig. <ref>, it can be observed that the similarity-based learning control framework achieves satisfied learning performance, and the tracking issue of the host system can be directly addressed.
	
	As a comparison, another guest system, denoted by Σ_3,𝕋, that is less similar with the host system Σ_1,𝕋 is also provided. The model knowledge of Σ_3,𝕋 is presented as follows:
	
    A_3(t)   =[      0.05t          1          0;          0      0.05t          1;      -0.20      -0.20 -1.3+0.05t ],
    
    			B_3(t)   =[    6;    0; 0.50 ], C_3(t)=[ 2; 1; 0 ]^ T, D_3(t)=0.

	The initial state of Σ_3,𝕋 is set as x_3(0)=[ 0.2, 0, 1 ]^ T, and the tracking task for the reference output y_d(t) are taken into account again. By applying ILC to Σ_3,𝕋, the tracking issue of Σ_3,𝕋 can be addressed. The output and input of Σ_3,𝕋, denoted as y'_g(t)-ILC and u'_g(t)-ILC, respectively, are depicted in Fig. <ref>. After obatining the admissible trajectory w_3,x_3∈ℬ_3,x_3, the tracking problem of the host system can be directly addressed through the similarity-based learning control framework. The learned output and input of the system Σ_1,𝕋, denoted as y'_h(t)-SBL anf u'_h(t)-SBL, respectively, are depicted in Fig. <ref>. Since the guest system Σ_3,𝕋 is less similar with the host system Σ_1,𝕋, the performance brought by the similarity-based learning control is degraded.
	





	Consider the a class of mobile robots equipped with two independent driving wheels (<cit.>), whose physical models are illustrated in Fig. <ref>.
	
	The symbols v, ϕ, u_r, and u_l represents the velocity, azimuth, right driving input, and left driving input of the mobile robot, repectively. Let the state, input, and output of the robots be defined as x=[ v, ϕ, ϕ̇ ]^ T,
	u=[ u_r, u_l ]^ T,
	and y=[ v, ϕ ]^ T, respectively, and let the sampling time be T_s=0.05s. Through the discretization and linearization techniques, the dynamics of the mobile robots can be described by the state space representation as
	
    R_i:{
    			x_i(n+1)   =A_ix_i(n)+B_iu_i(n)
    
    			y_i(n)   =C_ix_i(n)
    		. .

	The symbol n∈ℤ_+ refers to the sampling points, thus the time interval between two adjacent sampling points is T_s.
	For the host robot R_1 and guest robot R_2 whose dynamics are represented by (<ref>), their model parameters are given as
	
    A_1   =[ 1.0100      0      0;      0      1 0.0520;      0      0 1.0100 ],			A_2=[ 0.9975      0      0;      0      1 0.0499;      0      0 0.9955 ],
    
    			B_1   =[  0.0130  0.0130; -0.0025 -0.0050; -0.0850 -0.1700 ], B_2=[  0.0125  0.0125; -0.0021 -0.0042; -0.0833 -0.1666 ],
     C_1   =C_2=[ 1 0; 0 1; 0 0 ], x_1(0)=[ 3; 0; 0 ], x_2(0)=[ 3.02;    0;    1 ].

	We provide the model parameters solely to illustrate the simulation settings clearly. In the scenarios where the model information is not available, we can still obtain the data-based representation for the admissible behavior by designing appropriate offline test principles, as discussed in Lemma <ref>.
	
	Two mobile robots are assigned the same task, which is to move along a preplanned circular path within the time duration 𝕋=[0,4]s. The circular path is specified by the velocity and azimuth references of the mobile robots. Specifically, within the time duration 𝕋=[0,4]s, the reference trajectories for velocity and azimuth are defined as
	
    y_d,v(n)   =3(m/s),  ∀ n∈ℤ_79,
    
    			y_d,ϕ(n)   ={   0 (rad), n∈ℤ_10
       -0.6875(n-11) (rad), n∈ℤ_79\ℤ_10. .

	Therefore, the equivalent objective is to track the reference y_d=[ y_d,v(n), y_d,ϕ(n) ]^ T over the specific time duration 𝕋. For the guest mobile robot, ILC can efficiently address the tracking problems. After 200 iterations, the tracking performances brought by ILC are shown in Fig. <ref>, where the learned velocity and azimuth are denoted as v_g(n)-ILC and ϕ_g(n)-ILC, respectively.
	
	As a result, the guest mobile robots gradually reaches the predefined circular trajectory. The learning process of the guest mobile robots is shown in Fig. <ref>.
	
	After 200 iterations of the algorithm, it can be observed that the mobile robot is able to proceed along the predefined circular trajectory.
	
	For the host mobile robot, it no longer relies on repetitive trial-and-error processes, but rather directly utilizes the successful experiences of the guest system to complete the control task. By leveraging the proposed similarity-based learning control framework, the learned velocity and azimuth of host mobile robot, denoted by v_h(n)-SBL and ϕ_h(n)-SBL, respectively, are depicted in Fig. <ref>. Consequently, the learned path of the host mobile robot is shown in Fig. <ref>. From Fig. 9, it can be concluded that the host mobile robot can move along the preplanned circular path by leveraging the successful experience of the guest mobile robots, and the effectiveness of the proposed similarity-based learning control framework is verified.
	




§ CONCLUSIONS
 
In this paper, we have innovatively proposed the definitions of similarity and similarity indexes between admissible behaviors, based on which a similarity-based learning control framework has been further developed. Owing to the absence of model knowledge, appropriate offline I/O test principles have been designed, based on which the admissible behaviors of LTV systems have been reconstructed from sampled data. By exploiting the sampled data, a data-based criterion for verifying the similarity and a data-based strategy for calculating the similarity indexes have been developed. Building upon the calculated similarity indexes and projection techniques, a similarity-based learning control framework has been developed by exploiting the sampled data. Consequently, the host system has accomplished the same tasks by leveraging the successful experience of the guest system, without repeatedly resorting to any existing learning-based control strategies.



plainnat        


