An extension of the second order dynamical system that models Nesterov's convex gradient method
Cristian Daniel Alecsa Szilárd Csaba László Titus Pinţa
math.OC math.FA 34G20, 47J25, 90C25, 90C30, 65K10

Squarefree monomial ideals with maximal depth
     Ahad Rahimi
    December 30, 2023
=============================================




Abstract.  In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Frèchet differentiable objective function. We show that inertial algorithms, such as Nesterov's algorithm, can be obtained via the natural explicit discretization from our dynamical system.  Our dynamical system can be viewed as a perturbed version of the heavy ball method with vanishing damping, however the perturbation is made in the argument of the gradient of the objective function. This perturbation seems to have a smoothing effect for the energy error and eliminates the oscillations obtained for this error in the case of the heavy ball method with vanishing damping, as some numerical experiments show. We prove that the  value of the objective function in a generated trajectory converges in order 𝒪(1/t^2) to the global minimum of the objective function. Moreover, we obtain that a  trajectory  generated by the dynamical system converges to a minimum point of the objective function.  



Key Words.  convex optimization,  heavy ball method, continuous second order dynamical system, convergence rate, inertial algorithm 


AMS subject classification.  34G20, 47J25, 90C25, 90C30, 65K10




§ INTRODUCTION




Since Su, Boyd and Candès  in  <cit.> showed that Nesterov's accelerated convex gradient method  has the exact limit the second order differential equation that governs the heavy ball system with vanishing damping, that is,
 
    ẍ(t)+/tẋ(t)+ g(x(t))=0,  x(t_0)=u_0, ẋ(t_0)=v_0, t_0>0, u_0,v_0∈^m,

with =3, the latter system has been intensively studied in the literature in connection to the minimization problem
inf_x∈^mg(x). Here g:^m is a convex Frèchet differentiable function with Lipschitz continuous gradient.

In <cit.> the authors proved that 
    g(x(t))-min g=𝒪(1/t^2)

for every ≥ 3, however they did not show the convergence of a generated trajectory to a minimum of the objective function g.

In  <cit.>, Attouch, Chbani,  Peypouquet and Redont considered the case >3 in (<ref>), and showed that  the  generated  trajectory x(t)  converges  to  a  minimizer  of g as t+∞. Actually in <cit.> the authors considered the perturbed version of the heavy ball system with vanishing damping, that is,

    ẍ(t)+/tẋ(t)+ g(x(t))=h(t),  x(t_0)=u_0, ẋ(t_0)=v_0, t_0>0, u_0,v_0∈^m,

where h:[t_0,+∞) is a small perturbation therm that satisfies ∫_t_0^+∞th(t)dt<+∞. Beside the convergence of a generated trajectory x(t) to a minimizer of g,  they showed that also in this case the convergence rate of the objective function along the trajectory, that is g(x(t))-min g, is of order 𝒪(1/t^2).

Another perturbed version of (<ref>) was studied by Attouch,   Peypouquet and Redont in <cit.>. They assumed that the objective g is twice continuously differentiable and the perturbation of their system is made at the damping therm. More precisely, they studied the dynamical system with Hessian driven damping

    ẍ(t)+/tẋ(t)+^2g(x(t))ẋ(t)+ g(x(t))=0,  x(t_0)=u_0, ẋ(t_0)=v_0, t_0>0, u_0,v_0∈^m,

where >0 and >̱0.
In case >3, >̱0 they showed the convergence of a generated trajectory to a minimizer of g. Moreover, they obtained that in this case the convergence rate of the objective function along the trajectory, that is g(x(t))-min g, is of order o(1/t^2).

Further,  Attouch, Chbani and Riahi in <cit.> studied the  subcritical  case ≤ 3 and  they proved that in this case the  convergence rates of the objective function  g along  the  trajectory generated by (<ref>), i.e g(x(t))-min g, is of order 𝒪(1/t^2/3).

Another approach is due to Aujol, Dossal and Rondepierre <cit.>, who assumed that beside convexity, the objective g in (<ref>) satisfies some geometrical conditions, such as the Łojasiewicz property. The importance of their results obtained in <cit.> is underlined by the fact that applying the classical Nesterov scheme on a convex objective function without studying its geometrical properties may lead to sub-optimal algorithms.

 It is worth mentioning the work of Aujol and Dossal  <cit.>,  who did not assumed the convexity of g, but the convexity of the function (g(x(t))-g(x^*))^$̱, where$̱ is strongly related to the damping parameter  and x^* is a global minimizer of g. Under these assumptions, they obtained some general convergence rates and also the convergence of the generated trajectories of (<ref>). In case =̱1 they results reduce to the results obtained in <cit.>.

However, the convergence of the  trajectories generated by  the continuous heavy ball system with vanishing damping (<ref>), in the general case when g is nonconvex is still an open question.
 Some important steps in this direction have been made in <cit.> (see also <cit.>), where convergence of the trajectories of a perturbed system,  have been obtained in a nonconvex setting. More precisely in <cit.>  is considered the system
  
    ẍ(t)+(+/t)ẋ(t)+ g(x(t))=0,  x(t_0)=u_0, ẋ(t_0)=v_0,

where t_0>0,u_0,v_0∈^m, >0, ∈.  Note that here  can take nonpositive values. For =0 we recover the dynamical system studied in <cit.>. According to <cit.>, the trajectory  generated by the dynamical system (<ref>) converges  to a critical point of g, if a regularization of g satisfies the Kurdyka-Łojasiewicz property.


Further results concerning the heavy ball method and its extensions can be found in <cit.>.






 §.§ An extension of the heavy ball method and the Nesterov type algorithms obtained via explicit discretization


What one can notice concerning the heavy ball system and its variants is, that despite of the result of Su et al. <cit.>, this system will never give through the natural implicit/explicit discretization the Nesterov algorithm. This is due to the fact that the gradient of g is evaluated in x(t), and this via discretization will become g(x_n), (or g(x_n+1)) and never of the form g(y_n), y_n=x_n+_n(x_n-x_n-1) as Nesterov's gradient method requires.
Another observation is that using the same approach as in <cit.>, one can show, see <cit.>, that  (<ref>), (and also (<ref>)),  models beside Nesterov's algorithm other algorithms too. In this paper we overcome the deficiency emphasized above, by introducing a dynamical system that via explicit discretization leads to inertial algorithms of gradient type. To this end, let us consider the optimization problem

    (P)  inf_x∈^mg(x)

where g:^m is a convex Fréchet differentiable, function with L_g-Lipschitz continuous gradient, i.e. there exists L_g≥ 0 such that g(x)- g(y)≤ L_gx-y for all x,y∈^n.

We associate to (<ref>) the following second order dynamical system:

    {[ ẍ(t)+/tẋ(t)+∇ g(x(t)+(+/t)ẋ(t))=0;           x(t_0)=u_0, ẋ(t_0)=v_0, ].

where u_0,v_0∈^m, t_0≥0 and >0, ∈̱, ≥ 0.

The connection of (<ref>) with the  heavy ball system with vanishing damping  (<ref>) is obvious, the latter one can be obtained from (<ref>) for ==̱0.
 The study of the dynamical system (<ref>) in connection to the optimization problem (<ref>) is motivated by the following facts:
 
  1. The dynamical system (<ref>)  leads via explicit discretization to inertial algorithms.  In particular Nesterov's algorithm can be obtained via this natural discretization. 
  
  2. A generated trajectory and the objective function value in this trajectory in general have a better convergence behaviour than a trajectory generated by the heavy ball system with vanishing damping (<ref>), as some numerical examples shows.
  
  3. The same numerical experiments reveal that the perturbation term (+/t)ẋ(t) in the argument of the  gradient of the objective function g has a smoothing effect and annihilates the oscillations obtained in case of the dynamical system (<ref>) for the errors 
  g(x(t)-min g and x(t)-x^*, where x^* is a minimizer of g.
  
  4. A trajectory  x(t) generated by the dynamical system (<ref>) ensures the convergence rate of order 𝒪(1/t^2) for the decay g(x(t)+(+/t)ẋ(t))-min g, provided it holds that >3, > 0, ∈̱  or ≥3, g=0, ≥̱0.
 
 5. A trajectory  x(t) generated by the dynamical system (<ref>) ensures the same convergence rate of order 𝒪(1/t^2) for the decay g(x(t))-min g  as the heavy ball method with vanishing damping, for the cases >3, > 0, ∈̱ and >3, =0, ≥̱0. 
    
 6. The convergence of a generated trajectory x(t) to a minimizer of the objective function g can be obtained in case >3, >0 and ∈̱ and also in the case >3, =0 and ≥̱0. 
  
  
  
  
  Nevertheless, in case =0 and <̱0 the dynamical system (<ref>) can generate periodical solutions, hence the convergence of a generated trajectory to a minimizer of the objective is hopeless. To illustrate this fact, for <̱0, >0, =0 consider the strongly convex objective function g:, g(x)=/-2 x^2. Then, taking into account that =0 the dynamical system (<ref>) becomes
 
    {[      ẍ(t)+/-x(t)=0,; x(0)=0, ẋ(0)=√(/-). ].

Now, the periodical function x(t)=sin√(/-) t is a solution of (<ref>), consequently do not exist the limit lim_t+∞x(t).


We emphasize, that despite the novelty of the dynamical system (<ref>), its formulation is natural since by explicit discretization leads to inertial gradient methods, in particular the famous Polyak and Nesterov numerical schemes can be obtained from (<ref>). For other inertial algorithms of gradient type we refer to <cit.>.

Indeed, explicit discretization of (<ref>), with the constant stepsize h,  t_n=nh,  x_n=x(t_n) leads to

    x_n+1-2x_n+x_n-1/h^2+/nh^2(x_n-x_n-1)+ g(x_n+(/h+/nh^2)(x_n-x_n-1))=0.


Equivalently, the latter equation can be written as

    x_n+1=x_n+(1-/n)(x_n-x_n-1)-h^2 g(x_n+(/h+/nh^2)(x_n-x_n-1)).


Now, setting h^2=s and denoting the constants /h  and /h^2 still with  and $̱, we get the following general inertial algorithm:

Letx_0,x_-1∈^mand for alln∈consider the sequences

    {[ y_n=x_n+(1-/n)(x_n-x_n-1);  z_n=x_n+(+/n)(x_n-x_n-1);       x_n+1=y_n-s g(z_n). ].

However, from a practical point of view it is more convenient to work with the following equivalent formulation:
Letx_0,x_-1∈^mand for alln∈consider the sequences

    {[   y_n=x_n+n/n+(x_n-x_n-1);                          ; z_n=x_n+ n+/n+(x_n-x_n-1);                          ;       x_n+1=y_n-s g(z_n), ].

where>0, ∈̱and≥0.Notice that for ==̱0 we obtain a variant of Polyak's algorithm <cit.> and for =1 and =̱0 we obtain Nesterov's algorithm <cit.>. An interesting fact is that Algorithm (<ref>) allows different inertial steps and this approach seems to be new in the literature.


Independently to us, very recently, a  system similar to (<ref>) was studied by Muehlebach and Jordan in <cit.> and they show that Nesterov's accelerated gradient method can be obtained from their system via a semi-implicit Euler discretization scheme. They considered a constant damping instead of /t and also took =̱0. Further they also treated the ODE
ẍ(t)+3/t+2ẋ(t)+s∇ g(x(t)+t-1/t+2ẋ(t))=0 which for s=1 is obviously equivalent to the particular case of the governing ODE from (<ref>), obtained for =3, =1, =̱-3.
However, the freedom of controlling the parameters $̱ andin (<ref>) is essential as the next numerical experiments show.


 §.§ Some numerical experiments


In this section we consider two numerical experiments for the trajectories generated by the dynamical system (<ref>) for a strongly convex and for a convex but not strongly convex objective function.

Everywhere in the following numerical experiments we consider the continuous time dynamical system (<ref>), solved numerically with a Runge Kutta 4-5 (ode45) adaptive method in MATLAB. We solved the dynamical system with ode45 on the interval[1, 100]and the plot in Fig. 1 - Fig. 2 show the energy error|g(x(t)) - g(x^*)|on the left, and the iterate errorx(t) - x^*on the right.

We show the evolution of the two errors with respect to different values forα,βandγ, including the case that yields the Heavy Ball with Friction. One can observe that the best choice is not==̱0which is the case of heavy ball system with vanishing damping (<ref>).

 1. Consider the functiong:^2,g(x,y)=2x^2 + 5y^2 -4x+10y+7.Then,gis a strongly convex function, g(x,y)=(4x-4,10y+10), furtherx^*=(1,-1)is the unique minimizer ofgandg^*=g(1,-1)=0.We compare the convergence behaviour of the generated trajectories of (<ref>) by taking into account the following instances.


    α   β   γ   Color

      
      3       0       0    blue

      3.1     1       0    red

      3.1     0     0.5    yellow

      3.1     1     1    purple

      3.1     -1     1    green

    


The result are depicted in Figure.1 for the starting pointsu_0=v_0=(-5,30)andu_0=v_0=(5,-30), respectively.


Finally, for the starting points u_0=v_0=(-5,-30) then we obtain the graphs depicted at Figure 4.

 2. In the next experiment we consider the convex, but not strongly convex functiong:^2,g(x,y)=x^4 + 5y^2 -4x-10y+8.Then, g(x,y)=(4x^3-4,10y-10), furtherx^*=(1,1)is the unique minimizer ofgandg^*=g(1,1)=0.We compare the convergence behaviour of the generated trajectories of (<ref>) by taking into account the following instances.


    α   β   γ   Color

      
      3.1       0       0    blue

      3.1     2       0    red

      3.1     0     1    yellow

      3.1     0.5     0.5    purple

      3.1     -0.5     1    green

    


The result are depicted in Figure.2 for the starting pointsu_0=v_0=(-1,5)andu_0=v_0=(2,-2), respectively.

 Observe that in all cases the trajectories generated by dynamical system (<ref>) have a better behaviour than the trajectories generated by the heavy ball system with vanishing damping (<ref>). As we have emphasized before, it seems that the perturbation (+/t)ẋ(t) in the argument of the gradient of the objective function has a smoothing effect. The choice of the parameters  and $̱ will be validated by the theoretical results from Section 3, where we show that in case>3, >0, ∈̱and also in the case>3, =0, ≥̱0the energy errorg(x(t)-min gis of order𝒪(1/t^2)just as the case of heavy ball method. Further, for the values>3, >0, ∈̱and for>0, =0, ≥̱0we are able to show that a generated trajectoryx(t)converges to a minimum of the objective functiong.§.§ The organization of the paper


The outline of the paper is the following. In the next section we show the existence and uniqueness of the trajectories generated by the system (<ref>). Further we show that the third order derivative exists almost for everyt≥ t_0and we  give some estimates of the third order derivative in terms of velocity and acceleration. We do not assume the convexity of the objective functiongin these results. However, it seems that the assumption thatghas Lipschitz continuous gradient is essential in obtaining existence and uniqueness of the generated trajectories of the dynamical system (<ref>). In Section 3 we deal with the convergence analysis of the generated trajectories. We introduce a general energy functional, which will play the role of a Lyapunov function associated to the dynamical system (<ref>). We show convergence of the generated trajectories and also a rate of order𝒪(1/t^2)for the decayg(x(t)+(+/t)ẋ(t))-min g.Further, we show that also the errorg(x(t))-min ghas a rate of order𝒪(1/t^2). Finally, we show the convergence of the generated trajectories to a minimum of the objective functiong.In section 4 we conclude our paper and we present some possible related future researches.
 

§ EXISTENCE AND UNIQUENESS



 §.§ On strong global solutions

The first step toward our existence and uniqueness result obtained in the present section concerns the definition of a strong global solution of the dynamical system (<ref>).


 We call the function x:[t_0, + ∞) →^m a strong global solution of the dynamical system (<ref>) if  satisfies the following properties:

    (i)    x, ẋ : [t_0, + ∞) →^m  are locally absolutely continuous; 
        (ii)   ẍ(t) + /tẋ(t) + ∇ g ( x(t)+ (+/t)  ẋ(t) )=0  for almost every  t ≥ t_0; 
        (iii)    x(t_0) = u_0  and ẋ(t_0) = v_0.

For brevity reasons, we recall that a mappingx : [t_0, + ∞) →^mis called locally absolutely continuous if it is absolutely continuous on every compact interval[t_0, T], whereT > t_0. Further, we have the following equivalent characterizations for an absolutely continuous functionx:[t_0,T]^m, (see, for instance, <cit.>):

  (a) there exists y : [t_0, T] →^m an integrable function, such that
 x(t) = x(t_0) + ∫_t_0^t y(t) ds, ∀ t ∈ [t_0, T];
  (b)x is a continuous function and its distributional derivative is Lebesque integrable on the interval [t_0, T];
  (c) for every ε > 0, there exists η > 0, such that for every finite family I_k = (a_k, b_k) from [t_0, T], the following implication is valid :

    [ I_k ∩ I_j = ∅ and ∑_k |b_k-a_k| < η] ⇒[ ∑_k  x(b_k) - x(a_k)  < ε].

Let x : [t_0, +∞) →^m be a locally absolutely continuous function. Then x is differentiable almost everywhere and its derivative coincides with its distributional derivative almost everywhere. On the other hand, we have the equality ẋ(t) = y(t) for almost every t ∈ [t_0, +∞), where y = y(t) is defined at the integration formula (a).


The first result of the present section concerns the existence and uniqueness of the trajectories generated by the dynamical system (<ref>). We prove existence and uniqueness of a strong global solution of (<ref>) by making use of the Cauchy-Lipschitz-Picard Theorem for absolutely continues trajectories (see for example <cit.>, <cit.>). The key argument is that one can rewrite (<ref>) as a particular first order dynamical system in a suitably chosen product space (see also <cit.>).



Let (u_0, v_0) ∈^m ×^m. Then, the dynamical system (<ref>) admits a unique strong global solution.

By making use of the notation X(t)=(x(t),ẋ(t)) the system (<ref>) can be rewritten as a first order dynamical system:

    {[    Ẋ(t)=F(t,X(t)); X(t_0)=(u_0,v_0), ].

where F:[t_0,+∞)×^m×^m^m×^m,  F(t,u,v)=(v, -/tv-∇ g(u+(+/t)v)).


 The existence and uniqueness of a strong global solution of (<ref>) follows according to the Cauchy-Lipschitz-Picard Theorem applied to the first order dynamical system (<ref>). In order to prove the existence and uniqueness of the trajectories generated by (<ref>) we show the following:

 (I) For every t∈[t_0,+∞) the mapping F(t, ·, ·) is L(t)-Lipschitz continuous and L(·)∈ L^1_loc([t_0,+∞)).

 (II) For all u,v∈^m one has F(·,u,v)∈ L^1_loc([t_0,+∞),^m×^m) .



Let us prove (I). Let t ∈ [t_0, +∞) be fixed and consider the pairs (u,v) and (u̅, v̅) from ^m ×^m. Using the Lipschitz continuity of g and the obvious inequality A+B^2≤ 2A^2+2B^2 for all A,B∈^m, we make the following estimations :

    F(t,u,v)-F(t, u, v)   =√(v- v^2+/t( v-v)+∇ g( u+(+/t) v)-∇ g(u+(+/t)v)^2)
       ≤√((1+2(/t)^2)v- v^2+2L_g^2(u- u)+(+/t)(v- v)^2)
       ≤√((1+2(/t)^2+4L_g^2(+/t)^2)v- v^2+4L_g^2u- u^2)
       ≤√(1+4L_g^2+2(/t)^2+4L_g^2(+/t)^2)√(v- v^2+u- u^2)
       =√(1+4L_g^2+2(/t)^2+4L_g^2(+/t)^2) (u,v)-( u, v).



By employing the notation L(t)= √(1+4L_g^2+2(/t)^2+4L_g^2(+/t)^2), we have that
 F(t,u,v) - F(t, u̅, v̅) ≤ L(t) · (u,v) - (u̅, v̅) .
Obviously the function t  L(t) is  continuous  on [t_0, +∞), hence L(·) is integrable on [t_0,T] for all t_0<T<+∞.

For proving (II) consider (u,v) ∈^m ×^m  a fixed pair of elements and let T > t_0. We consider the following estimations:

    ∫_t_0^TF(t,u,v)dt    =∫_t_0^T √(v^2+/tv+∇ g(u+(+/t)v)^2)dt  
       ≤∫_t_0^T √((1+2(/t)^2)v^2+4∇ g(u+(+/t)v)- g(u)^2+4 g(u)^2)dt  
       ≤∫_t_0^T √((1+2(/t)^2+4L_g^2(+/t)^2)v^2+4 g(u)^2)dt  
       ≤√(v^2+∇ g(u)^2)∫_t_0^T √(5+2(/t)^2+4L_g^2(+/t)^2)dt

and the conclusion follows by the continuity of the function t √(5+2(/t)^2+4L_g^2(+/t)^2) on [t_0,T].

The Cauchy-Lipschitz-Picard theorem guarantees existence and uniqueness of the trajectory of the first order dynamical system (<ref>) and thus of the second order dynamical system (<ref>).
 Note that we did not use the convexity assumption imposed on g in the proof of Theorem <ref>. However, we emphasize that according to the proof of Theorem <ref>, the assumption that g has a Lipschitz continuous gradient is essential in order to obtain existence  and uniqueness of the trajectories generated by the dynamical system (<ref>).


 §.§ On the third order derivatives


In this section we show that the third order derivative of a strong global solution of the dynamical system (<ref>) exists almost everywhere on[t_0, + ∞). Further we give an upper bound estimate for the third order derivative in terms of velocity and acceleration of a strong global solution of (<ref>). For simplicity, in the proof of the following sequel we employ the1-norm on^m ×^m, defined as (x_1, x_2) _1 =  x_1  +  x_2 , for allx_1,x_2 ∈^m. Obviously one has1/√(2) (x_1, x_2) _1≤ (x_1, x_2) =√(x_1^2+x_2^2)≤ (x_1, x_2) _1,x_1,x_2∈^m.For the starting points (u_0,v_0) ∈^m ×^m let x be the unique strong global solution of the dynamical system (<ref>). Then, ẍ is locally absolutely continuous on [t_0, + ∞), consequently the third order derivative x^(3) exists almost everywhere on [t_0, + ∞).

We  show that Ẋ(t) = (ẋ(t), ẍ(t)) is locally absolutely continuous, hence ẍ is also locally absolutely continuous. This implies by Remark <ref> that x^(3) exists almost everywhere on [t_0, + ∞).


Let T > t_0 and s,t ∈ [t_0, T]. We consider the following chain of inequalities :

    Ẋ(s) - Ẋ(t) _1   =  F(s,X(s)) - F(t,X(t)) _1
       = ( ẋ(s) - ẋ(t) , /tẋ(t)-/sẋ(t) + g( x(t)+ (+/t) ẋ(t) ) -  g( x(s) + (+/s) ẋ(s) )) _1
       ≤ẋ(s) - ẋ(t)  + /sẋ(s) - /tẋ(t) 
       + g( x(s) + (+/s) ẋ(s) ) -  g( x(t) + (+/t) ẋ(t) )

and by using the L_g-Lipschitz continuity of g we obtain

    Ẋ(s) - Ẋ(t) _1   ≤ẋ(s) - ẋ(t)  + /sẋ(s) - /tẋ(t)  + L_g  x(s) - x(t) +L_g(+/s) ẋ(s) - (+/t) ẋ(t)
       ≤ẋ(s) - ẋ(t)  + αsẋ(s) - ẋ(t)  + | αs - /t| ẋ(t) + L_g  x(s) - x(t) 
         + L_g  (+/s)ẋ(s) - ẋ(t)  + L_g | /s-/t| ·ẋ(t) 
       = ( 1 + αs + L_g (+/s)) ẋ(s) -ẋ(t)  + L_g  x(s) - x(t)  + (  + L_g ||̱) ·| 1s - 1t|  ẋ(t) .

Further, let us introduce the following additional notations:
L_1 := max_s ∈ [t_0, T]( 1 + αs + L_g (+/s))= 1 + αt_0 + L_g (+/t_0)L_2 :=  (  + L_g ||̱) max_t ∈ [t_0,T]ẋ(t) .
Then, one has

    Ẋ(s) - Ẋ(t) ≤Ẋ(s) - Ẋ(t) _1≤ L_1 ẋ(s) - ẋ(t)  + L_g  x(s) - x(t)  +  L_2  | 1s - 1t|.

By the fact that x is the strong global solution for the dynamical system (<ref>), it follows that x and ẋ are  absolutely continuous on the interval [t_0,T]. Moreover, the function t 1t belongs to C^1([t_0, T], ℝ), hence it is also absolutely continuous on the interval [t_0,T]. Let ε > 0.  Then, there exists η > 0, such that for I_k = (a_k,b_k) ⊆ [t_0,T] satisfying I_k ∩ I_j = ∅ and ∑_k |b_k-a_k| < η, we have that

∑_kẋ(b_k) - ẋ(a_k)  < ε3L_1,  ∑_k x(b_k) - x(a_k)  < ε3L_g∑_k| 1b_k - 1a_k| < ε3 L_2.
Summing all up, we obtain

    ∑_kẊ(b_k) - Ẋ(a_k) ≤ L_1 ∑_kẋ(b_k) - ẋ(a_k)  + L_g ∑_k x(b_k) - x(a_k)  + L_2  ∑_k| 1b_k - 1a_k| < ε,

consequently Ẋ is absolutely continuous on [t_0,T]. and the conclusion follows.


Concerning an upper bound estimate of the third order derivativex^(3)the following result holds.

For the initial values (u_0,v_0) ∈^m ×^m consider x the unique strong global solution of the second-order dynamical system (<ref>). Then, there  exists K> 0 such that for almost every t ∈ [t_0, +∞), we have that :

    x^(3)(t) ≤ K( ẋ(t)  +  ẍ(t) ).

For  h > 0 we consider the following inequalities :

    Ẋ(t+h) - X(t) _1   =  F(t+h, X(t+h)) - F(t,X(t)) _1
       ≤( ẋ(t+h) - ẋ(t) )  + α1t+hẋ(t+h) - 1tẋ(t) 
        + ∇ g ( x(t+h) + (+t+h) ẋ(t+h) ) - ∇ g ( x(t) +(+ t) ẋ(t) ) 
       ≤( ẋ(t+h) - ẋ(t) )  + α1t+hẋ(t+h) - 1tẋ(t) 
       + L_g  x(t+h)-x(t) +L_g(+t+h) ẋ(t+h)  - (+t) ẋ(t) .

Now, dividing by h > 0 and taking the limit h  0, it follows that

    Ẍ(t) _1≤ẍ(t)  + α( 1tẋ(t) )^' + L_g ẋ(t)  + L_g ( (+t) ẋ(t) )^'.

Consequently,

    x^(3)(t) ≤α -1t^2ẋ(t) + 1tẍ(t)  + L_g ẋ(t)  + L_g  -t^2ẋ(t) + (  + t) ẍ(t) .

Finally,
 x^(3)(t) ≤( L_g + +L_g||̱t^2) ẋ(t)  + ( /t + L_g | + t| ) ẍ(t) ≤ K(ẋ(t) +ẍ(t) ),
where K=max{max_t≥ t_0( L_g + +L_g||̱t^2),max_t≥ t_0( /t + L_g | + t|)}.


§ CONVERGENCE ANALYSIS


 §.§ On a general energy functional associated to the dynamical system (<ref>)

In order to obtain convergence rates for the function values in the trajectories generated by the dynamical system (<ref>), we need to introduce an appropriate energy functional which will play the role of a Lyapunov function. The form of such an energy  functional associated to heavy ball system with vanishing damping and its extensions is well known in the literature, see for instance <cit.>, <cit.>, <cit.> and <cit.>. However, the novelty of the dynamical system (<ref>), compared with the extended/perturbed variants of the heavy ball system studied in the above mentioned papers, consists in the fact that in system (<ref>) the perturbation is carried out in the argument of the gradient of the objective function. This seems to be a new approach in the literature, therefore the previously mentioned energy functionals are not suitable for a valuable convergence analysis of the dynamical system (<ref>).
in this section we   consider  the general dynamical system
 
    {[ ẍ(t)+(t)ẋ(t)+∇ g(x(t)+(̱t)ẋ(t))=0;           x(t_0)=u_0, ẋ(t_0)=v_0, ].

where u_0,v_0∈ H and ,:̱[t_0+∞) (0,+∞),  t_0≥0 are  continuously derivable functions.

Hence, let us denote(t)=/tand(̱t)=+/t, and assume  that g≠∅.Further, letg^*=min g=g(x^*), x^*∈ g.In connection to the dynamical system (<ref>), we  introduce the general energy  functional

    ℰ(t)=a(t)(g(x(t)+(̱t)ẋ(t))-g^*)+1/2b(t)(x(t)-x^*)+c(t)ẋ(t)^2+d(t)/2x(t)-x^*^2,

which can be seen as an extension of the energy function studied in <cit.> in connection to the heavy ball system with vanishing damping.

Our purpose is to define the non-negative functionsa(t),b(t),c(t),d(t)such thatℰ̇(t)≤ 0, that is, the functionℰ(t)is non-increasing after at_1≥ t_0.Indeed, ifℰ(t)is non-increasing fort≥ t_1,thena(t)(g(x(t)+(̱t)ẋ(t))-g^*)≤ℰ(t)≤ℰ(t_1),in other wordsg(x(t)+(̱t)ẋ(t))-g^*≤ℰ(t_1)/a(t),t≥ t_1.In what follows we derive  the conditions which must be imposed on the positive functionsa(t),b(t),c(t),d(t)in order to obtainℰ̇(t)≤ 0for everyt≥ t_1.We have,ℰ̇(t)=a'(t)(g(x(t)+(̱t)ẋ(t))-g^*)+a(t) g(x(t)+(̱t)ẋ(t)),(̱t)ẍ(t)+('̱(t)+1)ẋ(t)+b'(t)(x(t)-x^*)+(b(t)+c'(t))ẋ(t)+c(t)ẍ(t),b(t)(x(t)-x^*)+c(t)ẋ(t)+d'(t)/2x(t)-x^*^2+d(t)ẋ(t),x(t)-x^*.Now, from (<ref>) we getẍ(t)=-(t)ẋ(t)- g(x(t)+(̱t)ẋ(t)),henceℰ̇(t)=a'(t)(g(x(t)+(̱t)ẋ(t))-g^*)+a(t) g(x(t)+(̱t)ẋ(t)),-(̱t) g(x(t)+(̱t)ẋ(t))+(-(̱t)(t)+'̱(t)+1)ẋ(t)+b'(t)(x(t)-x^*)+(b(t)+c'(t)-c(t)(t))ẋ(t)-c(t) g(x(t)+(̱t)ẋ(t)),b(t)(x(t)-x^*)+c(t)ẋ(t)+d'(t)/2x(t)-x^*^2+d(t)ẋ(t),x(t)-x^*.Consequently,

    ℰ̇(t)=a'(t)(g(x(t)+(̱t)ẋ(t))-g^*)+-a(t)(̱t) g(x(t)+(̱t)ẋ(t))^2+(-a(t)(t)(̱t)+a(t)'̱(t)+a(t)-c^2(t)) g(x(t)+(̱t)ẋ(t)),ẋ(t)+(b'(t)b(t)+d'(t)/2)x(t)-x^*^2+(b^2(t)+b(t)c'(t)+b'(t)c(t)-b(t)c(t)(t)+d(t))ẋ(t),x(t)-x^*+c(t)(b(t)+c'(t)-c(t)(t))ẋ(t)^2-b(t)c(t) g(x(t)+(̱t)ẋ(t)),x(t)-x^*.Butg(x(t)+(̱t)ẋ(t)),x(t)-x^*= g(x(t)+(̱t)ẋ(t)),x(t)+(̱t)ẋ(t)-x^*- g(x(t)+(̱t)ẋ(t)),(̱t)ẋ(t)and by the convexity ofgwe haveg(x(t)+(̱t)ẋ(t)),x(t)+(̱t)ẋ(t)-x^*≥ g(x(t)+(̱t)ẋ(t))-g^*,hence-b(t)c(t) g(x(t)+(̱t)ẋ(t)),x(t)-x^*≤-b(t)c(t)( g(x(t)+(̱t)ẋ(t))-g^*)+b(t)c(t)(̱t) g(x(t)+(̱t)ẋ(t)),ẋ(t).Therefore, (<ref>) becomes

    ℰ̇(t)≤ (a'(t)-b(t)c(t))(g(x(t)+(̱t)ẋ(t))-g^*)-a(t)(̱t) g(x(t)+(̱t)ẋ(t))^2+(-a(t)(t)(̱t)+a(t)'̱(t)+a(t)-c^2(t)+b(t)c(t)(̱t)) g(x(t)+(̱t)ẋ(t)),ẋ(t)+(b'(t)b(t)+d'(t)/2)x(t)-x^*^2+(b^2(t)+b(t)c'(t)+b'(t)c(t)-b(t)c(t)(t)+d(t))ẋ(t),x(t)-x^*+c(t)(b(t)+c'(t)-c(t)(t))ẋ(t)^2In order to haveℰ̇(t)≤ 0for allt≥ t_1, t_1≥ t_0,one must assume that for allt≥ t_1the following inequalities hold:

    a'(t)-b(t)c(t)≤ 0,   
    
    -a(t)(̱t)≤ 0,   
    
    -a(t)(t)(̱t)+a(t)'̱(t)+a(t)-c^2(t)+b(t)c(t)(̱t)= 0,   
    
    b'(t)b(t)+d'(t)/2≤ 0,   
    
    b'(t)c(t)+b(t)(b(t)+c'(t)-c(t)(t))+d(t)= 0,   
    
    c(t)(b(t)+c'(t)-c(t)(t))≤ 0.
 Observe that (<ref>) implies that (̱t)≥ 0 for all t≥ t_1, t_1≥ t_0 and  this shows that in dynamical system (<ref>), one must have ≥̱0 whenever =0. Further, (<ref>) is satisfied whenever b(t) and d(t) are constant functions. It is obvious that there exists t_1 such that for all t≥ t_1 we have  -(t)(̱t)+'̱(t)+1=1-/t-+̱/t^2>0, hence from (<ref>) we get
a(t)=c^2(t)-b(t)c(t)(̱t)/-(t)(̱t)+'̱(t)+1.

Since (<ref>) and (<ref>) do not depend by (̱t), it seem natural to choose c(t), b(t) and d(t) the same as in case of heavy ball system with vanishing damping (see <cit.>), that is,
c(t)=t,  b(t)=b∈(0,-1] and d(t)=b(-1-b), for all t≥ t_0, provided >1. Now, an easy computation shows that in this case
a(t)=t^2+(-b)t++̱(+̱^2)(-b)+(+(+̱2+^2^3)(-b))t+(+̱)̱(+̱(+̱^2)(-b))/t^2- t-(+̱)̱,
hence (<ref>) is satisfied whenever b>2, which implies that >3.

However, if =0 then
a(t)=t^2++̱(̱-b)+((+̱)̱(+̱(̱-b))/t^2-(+̱)̱,
hence (<ref>) holds also for b=2 and =3.

 §.§ Error estimates for  the values

In this section we obtain convergence rate of order𝒪(1/t^2), t+∞for the differenceg(x(t)+(+/t)ẋ(t))-g^*whereg^*=g(x^*)=min g, x^*∈ g≠∅.From here we are able to show thatg(x(t))-g^*also has a convergence rate of order𝒪(1/t^2), t+∞.However, just as in the case of heavy ball system with vanishing damping, in order to obtain these rates, it is necessary to assume≥ 3in our system (<ref>).
We have the following result.


Let x be the unique strong global solution of (<ref>) and assume that g≠∅.
Assume further that > 3, >0 and ∈̱.

Then, there exists K≥ 0 and t_1≥ t_0 such that

    g(x(t)+(+/t)ẋ(t))-min g≤K/t^2, t≥ t_1.

Further,

    ∫_t_0^+∞t(g(x(t)+(+/t)ẋ(t))-min g)dt<+∞,

and

    ∫_t_0^+∞t^2 g(x(t)+(+/t)ẋ(t))^2dt<+∞.

 Let min g=g^*=g(x^*), x^*∈ g. Consider the energy functional (<ref>)
 with
 b(t)=-1>2, c(t)=t, d(t)=0.
 According to Remark <ref> we have
 a(t)=t^2+ t+2+̱^2+(3+^2^3+)t+(̱+1)(2+̱^2)/t^2- t-(̱+1) and
 the conditions  (<ref>)-(<ref>) are satisfied with equality for every t≥ t_0.

Obviously,  if t is big enough on has that a(t)>0 and since >0 it holds that +/t>0 for t big enough, even if <̱0. Hence, there exists t'≥ t_0 such (<ref>) is satisfied for every t≥ t'.

Now, a'(t)-b(t)c(t)=(3-)t++𝒪(1/t^2) and by taking into account that >3 we obtain that there exists t”≥ t_0 such that (<ref>) holds for every t≥ t”.

Let t”'=max(t',t”). We conclude that, ℰ̇(t)≤0 for all t≥ t”', hence ℰ(t) is nonincreasing, i.e.
a(t)(g(x(t)+(+/t)ẋ(t))-min g)≤ℰ(t)≤ℰ(t”'),t≥ t”'.

 But, obviously there exists t_1≥ t”' such that a(t)≥ t^2 for all t≥ t_1 and by denoting K=ℰ(t”') we obtain

 g(x(t)+(+/t)ẋ(t))-min g≤K/t^2,t≥ t_1.

 Next we prove (<ref>). Since a'(t)-b(t)c(t)=(3-)t++𝒪(1/t^2) there exist t_2≥ t_1 such that
 a'(t)-b(t)c(t)≤(3-)t/2,t≥ t_2.

 Hence,
 ℰ̇(t)≤(3-)t/2(g(x(t)+(+/t)ẋ(t))-min g),t≥ t_2
 and by integrating from t_2  to T> t_2  we get
 ∫_t_2^Tt(g(x(t)+(+/t)ẋ(t))-min g)dt≤2/3-(ℰ(T)-ℰ(t_2)).
Now, by letting T+∞ and taking into account that ℰ is nonincreasing, the conclusion follows.

For proving (<ref>) observe that there exists t_3≥ t_1 such that -a(t)(+/t)≤-/2 t^2 for all t≥ t_3.
Consequently ℰ̇(t)≤ -/2 t^2  g(x(t)+(+/t)ẋ(t))^2,t≥ t_3.

Integrating from t_3 to T>t_3 and letting T+∞ we obtain the desired conclusion.
 
Note that according to Remark <ref> the condition (<ref>) holds even if =0 and ≥̱0.
Consequently, even in the case  >3, =0, ≥̱0 the conclusions (<ref>) and (<ref>) in Theorem <ref> hold.
However, if =̱0 then we cannot obtain (<ref>). Nevertheless, if >̱0 then (<ref>) becomes:
∫_t_0^+∞t g(x(t)+/tẋ(t))^2dt<+∞. Concerning the case =3, note that (<ref>) is  satisfied if one assumes that =0 but ≥̱0. Moreover, in this case (<ref>) is also satisfied since, one has
a'(t)-b(t)c(t)=- 4^̱2(+1)/(t^2-(̱+1))^2≤ 0. Hence, also in the case =3, =0, ≥̱0(<ref>) in the conclusion of Theorem <ref> holds.

Moreover,  if we assume >̱0 then (<ref>) becomes:
∫_t_0^+∞t g(x(t)+/tẋ(t))^2dt<+∞.

Next we show that also the  errorg(x(t))-min gis of order𝒪(1/t^2).For obtaining this result we need the Descent Lemma, see <cit.>.

 Let g:^m be a Frèchet differentiable function with L_g Lipschitz continuous gradient. Then one has
g(y)≤ g(x)+ g(x),y-x+L_g/2y-x^2, ∀ x,y∈^m.
Let x be the unique strong global solution of (<ref>) and assume that g≠∅.
If > 3, >0 and ∈̱, then x is bounded and there exists K≥ 0 and t_1≥ t_0 such that

    ẋ(t)≤K/t,t≥ t_1.

Further,

    ∫_t_0^+∞tẋ(t)^2 dt≤+∞

and there exists K_1≥ 0 and t_2≥ t such that

    g(x(t))-min g≤K_1/t^2,t≥ t_2.

For x^*∈ g let g^*=g(x^*)=min g and  consider the energy function (<ref>)
 with
 b(t)=b, where 2<b<-1, c(t)=t, d(t)=b(-1-b)>0 and
 a(t)=c^2(t)-b(t)c(t)(+/t)/1-/t^2-/t(+/t)=(t^2-b  t-b)̱t^2/t^2- t-(̱+1)=(1+(-b) t-(̱+1-b)/t^2- t-(̱+1))t^2.
 According to Remark <ref> there exists t_1≥ t_0 such that the conditions (<ref>)-(<ref>) are satisfied.

From the definition of ℰ one has
b(t)(x(t)-x^*)+c(t)ẋ(t)≤√(2ℰ(t)),
and
x(t)-x^*≤√(2/d(t)ℰ(t))
that is
b(x(t)-x^*)+tẋ(t)≤√(2ℰ(t)),
and
x(t)-x^*≤√(2/b(-1-b)ℰ(t)).
By using the fact that ℰ nonincreasing on an interval [t_1,+∞) the latter inequality assures that x is bounded.

Now, by using the inequality X-Y≥X-Y we get b(x(t)-x^*)+tẋ(t)≥ tẋ(t)-b(x(t)-x^*), hence for all t≥ t_1 one has
tẋ(t)≤√(2ℰ(t))+b(x(t)-x^*)≤(1+√(1/-1-b))√(2ℰ(t))≤ K,
where K=(1+√(1/-1-b))√(2ℰ(t_1)).


Further, (<ref>) becomes (b+1-)t<0, hence for all t≥ t_1 one has
ℰ̇(t)≤(b+1-)tẋ(t)^2.

By integrating from t_1 to T>t_1 one gets
∫_t_1^Ttẋ(t)^2dt≤1/-1-b(ℰ(t_1)-ℰ(T))≤1/-1-bℰ(t_1).
By letting T+∞ we obtain
∫_t_0^+∞tẋ(t)^2 dt≤+∞.

Now, by using Lemma <ref> with y=x(t) and x=x(t)+(+/t)ẋ(t) we obtain

    g(x(t))- g(x(t)+(+/t)ẋ(t))≤
 g(x(t)+(+/t)ẋ(t)),-(+/t)ẋ(t)+L_g/2(+/t)ẋ(t)^2≤| t+|̱/t g(x(t)+(+/t)ẋ(t))ẋ(t)+L_g/2(+/t)^2ẋ(t)^2.

Now according to (<ref>) there exists t'≥ t_0 and K'>0 such that
 g(x(t)+(+/t)ẋ(t))≤K'/t,t≥ t'.
Further (<ref>) assures that there exists t_1≥ t_0 and K>0 such  that
ẋ(t)≤K/t,t≥ t_1.
Obviously, |+/t|≤max(,+/t_0),t≥ t_0,
hence (<ref>) assures that there exists K_1'>0 and t_2'≥ t_0 such that

    g(x(t))- g(x(t)+(+/t)ẋ(t))≤K_1'/t^2,t≥ t_2'.

Now, by adding (<ref>) and (<ref>) we get that there exists K_1>0 and t_2≥ t_0 such that
(g(x(t)+(+/t)ẋ(t))-min g)+(g(x(t))- g(x(t)+(+/t)ẋ(t)))≤K_1/t^2,t≥ t_2 that is

g(x(t))-min g≤K_1/t^2,t≥ t_2. Note that if we assume that ≥̱0 then one can allow =0 in the hypotheses of Theorem <ref>, since in this case condition (<ref>) is satisfied and the conclusions of Theorem <ref> hold.
 Note that (<ref>), which holds whenever >3 and >0, assures that
t g(x(t)+(+/t)ẋ(t))∈ L^2([t_0,+∞),).
Further, (<ref>) assures that

    √(t)ẋ(t)∈ L^2([t_0,+∞),).


Consequently, the system (<ref>) leads to
tẍ(t)=ẋ(t)+t g(x(t)+(+/t)ẋ(t))≤ẋ(t)+t g(x(t)+(+/t)ẋ(t))
hence,

    tẍ(t)∈ L^2([t_0,+∞),).

Now, from (<ref>) we have
 √(t) x^(3)(t) ≤ K(√(t)ẋ(t)  + √(t)ẍ(t) )
 for some K>0 and for almost every t≥ t_0, which combined with (<ref>) and (<ref>) gives

    √(t) x^(3)(t) ∈ L^2([t_0,+∞),).
 Notice that (<ref>), (<ref>) and (<ref>) assure in particular that

    lim_t+∞ẋ(t)=lim_t+∞ẍ(t)=lim_t+∞x^(3)(t)=0.
 In the case =0 and ≥̱0 according to Remark <ref> one has √(t) g(x(t)+(+/t)ẋ(t))∈ L^2([t_0,+∞),). Hence, as in Remark <ref> we derive that
√(t)ẍ(t), √(t) x^(3)(t) ∈ L^2([t_0,+∞),)
and consequently (<ref>) holds.


 §.§ The convergence of the generated trajectories


In this section we show convergence of the generated trajectories to a minimum point of the objectiveg.The main tool that we use in order to attain this goal will be the following continuous version of Opial Lemma, (see <cit.>.


Let H to be a separable Hilbert space and S ⊆ H, with S ≠∅. Further, consider x : [t_0, +∞) → H a given map. Suppose that the following conditions are satisfied :

    (i)   ∀ z ∈ S,  ∃lim_t  +∞ x(t) - z 
        (ii)   every weak sequentially limit point of  x(t)  belongs to the set S.

Then, x(t) converges weakly to a point of S as t → + ∞.

In the setting of the proof concerning the convergence of the generated trajectories, we consider the set S to be argmin(g). Moreover, the Hilbert space H is  ℝ^m, and this implies that we actually deduce strong convergence of a strong global solution of the dynamical system (<ref>) to a minimum of the objective function g.
The following central results will be used when proving the convergence of the trajectories generated by the dynamical system (<ref>); see, for example, <cit.> and <cit.>, respectively.

 Suppose that F:[0,+∞)→ is locally absolutely continuous and bounded below and that
there exists G∈ L^1([0,+∞)) such that for almost every t ∈ [0,+∞)d/dtF(t)≤ G(t).
Then there exists lim_t→ +∞ F(t)∈.
  If 1 ≤ p < ∞, 1 ≤ r ≤∞, F:[0,+∞)→[0,+∞) is
locally absolutely continuous, F∈ L^p([t_0,+∞)), G:[t_0,+∞)→, G∈  L^r([0,+∞)) and
for almost every t ∈ [0,+∞)d/dtF(t)≤ G(t), then lim_t→ +∞ F(t)=0.


We show that whenever x is a strong global solution of the dynamical system (<ref>) one has lim_t+∞ẍ(t)=0.
Consider now the set of limit points of the trajectoryx, that isω (x) ={ x∈^m: ∃ (t_n)_n∈⊆,  t_n+∞, n+∞ such that  x(t_n) x,  n+∞}.We show thatω(x)⊆ g.We emphasize that sincegis convex one hasg= g:={ x∈^m: g( x)=0.}.We have the following result.
  Let x be the unique strong global solution of (<ref>) and assume that g≠∅.
If > 3, >0 and ∈̱, then the following assumptions hold.

  (i)ω(x)=ω(x(·)+(+/t)ẋ(·));

  (ii)ω(x)⊆ g.
 Indeed (<ref>) assures  that lim_t +∞( γ + βt) ẋ(t) = 0, which immediately  proves (i).


For proving (ii) consider x∈ω(x). Then, there exists (t_n)_n∈⊆,  t_n+∞, n+∞ such that
 lim_n+∞x(t_n) = x. Now, since g is continuous and
 lim_n+∞(x(t_n)+(+/t_n)ẋ(t_n))= x
 one has
lim_n+∞ g(x(t_n)+(+/t_n)ẋ(t_n))= g( x).

Further, according to (<ref>)lim_n+∞(ẍ(t_n)+/t_nẋ(t_n))=0.
Now, the system (<ref>) gives
0=lim_n+∞(ẍ(t_n)+/t_nẋ(t_n)+ g(x(t_n)+(+/t_n)ẋ(t_n)))= g( x)
that is x∈ g and this proves (ii).
 Obviously, according to Remark <ref> the conclusion of Lemma <ref> remains valid also in the case  >3, =0 and ≥̱0.
Note that (ii) from the conclusion of Lemma <ref> is actually the condition (ii) from Lemma <ref>. For proving (i) from Lemma <ref>, that is, the limit lim_t+∞x(t)-x^* exists for every x^*∈ g, we need the following result from <cit.>.
(Lemma A.4. <cit.>) Let t_0>0, and let w:[t_0,+∞) be a continuously differentiable function which is bounded from below. Assume that
 tẅ(t) + ẇ(t) ≤ G(t)
for some > 1,  almost every t >t_0, and some nonnegative function G∈ L^1(t_0,+∞). Then, the positive part [ẇ]_+ of ẇ belongs
to L^1(t_0,+∞) and limit lim_t+∞ w(t) exists.

Now we can prove the following.

 Let x be the unique strong global solution of (<ref>) and assume that g≠∅.
If > 3, >0 and ∈̱, then for every x^*∈ g there exists the limit
lim_t+∞x(t)-x^*.
Let x^*∈ g and define the function h_x^*:[t_0,+∞), h_x^*(t)= 12 x(t) - x^* ^2. Using the chain rule with respect to the differentiation of h_x^*, we obtain that

    ḣ_x^*(t) = ẋ(t), x(t)-x^*

and that

    ḧ_x^*(t) = ẍ(t), x(t)-x^*  + ẋ(t) ^2.

Let us denote g^∗ = g(x^∗) = min(g). Using the dynamical system (<ref>), one has that

    ḧ_x^*(t) + αtḣ_x^*(t) = -∇ g ( x(t) + ( γ + βt) ẋ(t) ), x(t) - x^∗ + ẋ(t) ^2.

This means that

    ḧ_x^*(t) + αtḣ_x^*(t) =- ∇ g ( x(t) + ( γ + βt) ẋ(t) ), x(t) + ( γ + βt) ẋ(t)- x^*  + ẋ(t) ^2

 + ∇ g ( x(t) + ( γ + βt) ẋ(t) ), ( γ + βt) ẋ(t) .

By the convexity of the mapping g and taking into account that g^*=g(x^*)=min g, we obtain
- ∇ g ( x(t) + ( γ + βt) ẋ(t) ), x(t) + ( γ + βt) ẋ(t)- x^* ≤ g^* - g ( x(t) + ( γ + βt) ẋ(t) )≤ 0,
hence  by using (<ref>) the inequality (<ref>) becomes

    ḧ_x^*(t) + αtḣ_x^*(t) ≤ g^* - g ( x(t) + ( γ + βt) ẋ(t) ) + ( 1 - αt( γ + βt) ) ẋ(t) ^2 - ( γ + βt) ẋ(t), ẍ(t) .


Consequently, one has

    tḧ_x^*(t) + ḣ_x^*(t) ≤( t - ( γ + βt) ) ẋ(t) ^2 - (  t + ) ẋ(t), ẍ(t) .


Now, according to (<ref>) one has
tẋ(t)^2∈ L^1([t_0,+∞),).
Consequently,

    ( t - ( γ + βt) ) ẋ(t) ^2∈ L^1([t_0,+∞),).



Further,
- (  t + ) ẋ(t), ẍ(t) ≤1/2ẋ(t)^2+1/2(  t + )^2ẍ(t)^2
and  (<ref>) assures that
t^2ẍ(t)^2∈ L^1([t_0,+∞),),
hence

    - (  t + ) ẋ(t), ẍ(t) ∈ L^1([t_0,+∞),).


According to (<ref>) and (<ref>) we get that the function
G(t)= ( t - ( γ + βt) ) ẋ(t) ^2 +1/2ẋ(t)^2+1/2(  t + )^2ẍ(t)^2∈ L^1([t_0,+∞),).  Moreover, it is obvious that there exists t_1≥ t_0 such that G is non negative for every t≥ t_1. From (<ref>)
one has

    tḧ_x^*(t) + ḣ_x^*(t) ≤ G(t),t≥ t_0,

hence Lemma <ref> leads to the existence of the limit
lim_t+∞h_x^*(t)
and consequently the limit
lim_t+∞x(t)-x^*
also exists.


Now, we present the main result of this subsection regarding the convergence of the solution of the dynamical system (<ref>) ast → + ∞.


Let x be the unique strong global solution of the dynamical system (<ref>) and assume that g≠∅.
If > 3, >0 and ∈̱, then x(t) converges to a point in g as t  +∞.

 Taking into account that the conclusion (ii) in Lemma <ref> and the conclusion of Lemma <ref> are exactly the conditions in the hypothesis of Lemma <ref> with S= g, the conclusion of the theorem is straightforward.
 As it was expected, Theorem <ref> remains true if in its hypothesis we assume only that >3, =0 and ≥̱0. Indeed, note that under these assumptions the conclusion of Lemma <ref> holds, since G from its proof becomes
G(t)= ( t -t) ẋ(t) ^2 +1/2ẋ(t)^2+1/2^̱2ẍ(t)^2
and according to Remark <ref>G(t)∈ L^1([t_0,+∞),).  Moreover, it is obvious that there exists t_1≥ t_0 such that G is non negative for every t≥ t_1.

This fact combined with Remark <ref> lead to the desired conclusion.


§ CONCLUSIONS


In this paper we study a second order dynamical system which can be viewed as an extension of the heavy ball system with vanishing damping. This dynamical system is actually a perturbed version of the heavy ball system with vanishing damping, but the perturbation is made in the argument of the gradient of the objective function. Numerical experiments show that this perturbation brings a smoothing effect in the behaviour of the energy errorg(x(t))-min gand also in the behaviour of the absolute errorx(t)-x^*, wherex(t)is a generated trajectory by our dynamical system andx^*is a minimum of the objectiveg.Another novelty of our system that via explicit discretization leads to inertial algorithms. A related future research is the convergence analysis of  algorithm (<ref>), since this algorithm contains as particular case the famous Nesterov algorithm. However, since Algorithm (<ref>) may allow  different inertial steps, its area of applicability can be considerable.

We have shown existence and uniqueness of the trajectories generated by our dynamical system even for the case the objective functiongis non-convex. Further, we treated the cases when the energy errorg(x(t))-min gis of order𝒪(1/t^2)and we obtained the convergence of a generated trajectory to a minimum of the objective functiong.Another related research is the convergence analysis of the generated trajectories in the case when the objective functiongis possible non-convex. This would be a novelty in the literature even  for the case>3, ==̱0, that is, for the case of the heavy ball system with vanishing damping.


We underline that the dynamical system (<ref>) can easily be extended to proximal-gradient dynamical systems (see <cit.> and the references therein). Indeed, letf:^mbe a  proper convex and lower semicontinuous function and letg:^mbe a possible non-convex smooth function withL_gLipschitz continuous gradient.

We recall that the proximal point operator of the convex functionł  fis defined as

    _ł  f : ^m→^m,   _ł f(x)=_y∈^m{f(y)+1/2ły-x^2}.


Consider the optimization probleminf_x∈^mf(x)+g(x).One can associate to this optimization problem the following second order proximal-gradient dynamical system.


    {[ ẍ(t)+(++/t)ẋ(t)+x(t)=_ł f((x(t)+(+/t)ẋ(t))-ł g(x(t)+(+/t)ẋ(t))),;                                                                 ;                                          x(t_0)=u_0, ẋ(t_0)=v_0, ].

where>0, >0, ł>0, ∈̱andt_0>0.Obviously, whenf≡ 0andł=1then (<ref>) becomes the dynamical system (<ref>) studied in the present paper.

The discretization of the the dynamical system (<ref>) leads to proximal-gradient inertial algorithms, similar to the modified FISTA algorithm studied by Chambolle and Dossal in <cit.>, (see also <cit.>).

Indeed, the explicit discretization of (<ref>) with respect to the time variablet, with step sizeh_nand initial pointsx_0:=u_0, x_1:=v_0yields the iterative schemex_n+1 - 2x_n + x_n-1/h_n^2 + (++/n h_n) x_n - x_n-1/h_n + x_n+1 =_ł f( x_n+(+/n h_n) x_n - x_n-1/h_n -ł∇ g(x_n+(+/n h_n) x_n - x_n-1/h_n))   ∀ n ≥ 1.The latter can be expressed as(1+h_n^2)x_n+1=x_n+(1- h_n-+/n)(x_n-x_n-1)+h_n^2 _ł f( x_n+(+/n h_n) x_n - x_n-1/h_n -ł∇ g(x_n+(+/n h_n) x_n - x_n-1/h_n)).Consequently, the dynamical system (<ref>) leads to the algorithm:
Forx_0, x_1∈^mconsider

    {[                    y_n=x_n+(1- h_n-+/n)(x_n-x_n-1),;                                                    ;              z_n=x_n+(/h_n+/n h_n^2) (x_n - x_n-1),;                                                    ; x_n+1=1/1+h_n^2y_n+h_n^2/1+h_n^2_ł f(z_n-ł g(z_n)), ].

where≥ 0, ≥ 0and∈̱.Now, the simplest case is obtained by taking in (<ref>) constant step sizeh_n≡ 1. By denoting+$̱ still with ∈, the Algorithm
(<ref>) becomes:
For x_0,x_1∈^m and for every n≥ 1 consider

    {[        y_n=x_n+(1-)n-/n(x_n-x_n-1),;                                    ;        z_n=x_n+ n+/n (x_n - x_n-1),;                                    ; x_n+1=1/2y_n+1/2_ł f(z_n-ł g(z_n)), ].

where ≥,̱ ≥ 0 and ∈̱.  
The convergence of the sequences generated by Algorithm (<ref>) to a critical point of the objective function f+g  would open the gate for the study of FISTA type algorithms with nonidentical inertial terms.


99AD J.F. Aujol, Ch. Dossal,  Optimal rate of convergence of an ODE associated
to the Fast Gradient Descent schemes for b > 0, HAL preprint, https://hal.inria.fr/hal-01547251v2/document
ADR J.F. Aujol, Ch. Dossal, A. Rondepierre,  Optimal convergence rates for Nesterov acceleration, arxiv.org/abs/1805.05719
alvarez-attouch2001 F. Alvarez, H. Attouch,  An inertial proximal method for maximal monotone operators via discretization
of a nonlinear oscillator with damping, Set-Valued Analysis, 9, 3-11, 2001
AAD V. Apidopoulos, J.F. Aujol, Ch. Dossal,  Convergence rate of inertial Forward–Backward algorithm beyond Nesterov's rule, Math. Program. (2018). https://doi.org/10.1007/s10107-018-1350-9

abbas-att-sv B. Abbas, H. Attouch, B.F. Svaiter,  Newton-like dynamics and forward-backward methods for
structured monotone inclusions in Hilbert spaces, Journal of Optimization Theory and its Applications 161(2), 331-360, 2014

alv-att-bolte-red F. Alvarez, H. Attouch, J. Bolte, P. Redont,  A second-order gradient-like dissipative dynamical system
with Hessian-driven damping. Application to optimization and mechanics, Journal de Mathématiques Pures et Appliquées (9) 81(8),
747-779, 2002

AC H. Attouch, Z. Chbani, Fast inertial dynamics and fista algorithms in
convex optimization. Perturbation aspects, arXiv preprint arXiv:1507.01367,
2015
att-c-r-arx2017 H. Attouch, Z. Chbani, H. Riahi,  Rate of convergence of the Nesterov accelerated gradient method in the
subcritical case α≤ 3, ESAIM: COCV (2017). https://doi.org/10.1051/cocv/2017083
att-c-r H. Attouch, Z. Chbani, H. Riahi,  Fast convex optimization via time  scaling of damped inertial gradient dynamics, https://hal.archives-ouvertes.fr/hal-02138954
att-c-p-r-math-pr2018 H. Attouch, Z. Chbani, J. Peypouquet, P. Redont,  Fast convergence of inertial dynamics and algorithms with asymptotic
vanishing viscosity, Math. Program. 168(1-2) Ser. B, 123-175, 2018

AGR H. Attouch, X. Goudou,  P. Redont,   The heavy ball with
friction method, I. The continuous dynamical system: global exploration of the
local minima of a real-valued function by asymptotic analysis of a dissipative
dynamical system, Communications in Contemporary Mathematics 2, 1-34, 2000

att-p-r-jde2016 H. Attouch, J. Peypouquet, P. Redont,  Fast convex optimization via inertial dynamics with Hessian driven damping,
J. Differential Equations, 261(10), 5734-5783, 2016

att-sv2011 H. Attouch, B.F. Svaiter,  A continuous dynamical Newton-like approach to solving monotone inclusions,
SIAM Journal on Control and Optimization 49(2), 574-598, 2011
BM M. Balti, R. May, Asymptotic for the perturbed heavy ball system
with vanishing damping term, arXiv preprint arXiv:1609.00135, 2016.
bete09 A. Beck, M. Teboulle,  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems, SIAM Journal on Imaging Sciences, 2(1), 183-202, 2009
BBJ P. Bégout, J. Bolte, M.A. Jendoubi,  On damped second-order gradient systems, Journal of Differential Equations, (259), 3115-3143, 2015

BCL1 R.I. Boţ¸, E.R. Csetnek, S.C. László,  An inertial forward-backward algorithm for minimizing
the sum of two non-convex functions, Euro Journal on Computational Optimization, 4(1), 3-25, 2016
BCL R.I. Boţ,  E.R. Csetnek,  S.C. László,   Approaching nonsmooth nonconvex minimization through
second-order proximal-gradient dynamical systems, Journal of Evolution Equations, 18(3), 1291-1318, 2018
ALV C. Alecsa, S.C. László, A. Viorel,  A gradient type algorithm with backward inertial steps associated to a nonconvex minimization problem, https://www.researchgate.net/publication/331832475
BCL-AA R.I. Boţ,  E.R. Csetnek,  S.C. László,  A second order dynamical approach with variable damping to nonconvex smooth minimization, Applicable Analysis (2018). https://doi.org/10.1080/00036811.2018.1495330

BCL-MM R.I. Boţ,  E.R. Csetnek,  S.C. László,  A primal-dual dynamical approach to structured convex minimization problems, https://arxiv.org/abs/1905.08290
CAG1 A. Cabot, H. Engler, S. Gadat,  On the long time behavior of second order differential equations with asymptotically small dissi-
pation, Trans. Amer. Math. Soc. 361, pp. 5983-6017, 2009
CAG2 A. Cabot, H. Engler, S. Gadat, Second order differential equations with asymptotically small dissipation and piecewise
at potentials, Electronic Journal of Differential Equations 17, 33-38, 2009
ch-do2015 A. Chambolle, Ch. Dossal,  On the convergence of the iterates of the "fast iterative shrinkage/thresholding algorithm”,
J. Optim. Theory Appl., 166(3), 968-982, 2015

haraux A. Haraux,  Systèmes Dynamiques Dissipatifs et Applications,
Recherches en Mathématiques Appliquéées 17, Masson, Paris, 1991

L S.C. László,  Convergence rates for an inertial algorithm of gradient type associated to a smooth nonconvex minimization, https://arxiv.org/pdf/1811.09616.pdf

MJ M. Muehlebach, M. I. Jordan,  A Dynamical Systems Perspective on Nesterov Acceleration, arXiv:1905.07436v1

Nest Y. Nesterov ,  Introductory lectures on convex optimization: a basic course. Kluwer Academic Publishers, Dordrecht, 2004

nesterov83 Y.E. Nesterov,  A method for solving the convex programming problem with convergence rate O(1/k^2),
(Russian) Dokl. Akad. Nauk SSSR 269(3), 543-547, 1983

Opial Z. Opial,  Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Amer. Math. Soc., 73,
591-597, 1967
poly B. T. Polyak,  Some methods of speeding up the convergence of iteration methods, U.S.S.R. Comput. Math. Math. Phys., 4(5), 1-17, 1964

sontag E.D. Sontag,  Mathematical control theory. Deterministic finite-dimensional systems, Second edition,
Texts in Applied Mathematics 6, Springer-Verlag, New York, 1998

su-boyd-candes W. Su, S. Boyd, E.J. Candes,  A differential equation for modeling Nesterov's
accelerated gradient method: theory and insights, Journal of Machine Learning Research, 17, 1-43, 2016
Bruck R.E. Bruck,  Asymptotic convergence of nonlinear contraction semigroups in Hilbert spaces, J. Funct. Anal., 18 ,  15-26, 1975