Physics-Informed Neural Networks for Prognostics and Health Management
  of Lithium-Ion Batteries
For Prognostics and Health Management (PHM) of Lithium-ion (Li-ion)
batteries, many models have been established to characterize their degradation
process. The existing empirical or physical models can reveal important
information regarding the degradation dynamics. However, there are no general
and flexible methods to fuse the information represented by those models.
Physics-Informed Neural Network (PINN) is an efficient tool to fuse empirical
or physical dynamic models with data-driven models. To take full advantage of
various information sources, we propose a model fusion scheme based on PINN. It
is implemented by developing a semi-empirical semi-physical Partial
Differential Equation (PDE) to model the degradation dynamics of Li-ion
batteries. When there is little prior knowledge about the dynamics, we leverage
the data-driven Deep Hidden Physics Model (DeepHPM) to discover the underlying
governing dynamic models. The uncovered dynamics information is then fused with
that mined by the surrogate neural network in the PINN framework. Moreover, an
uncertainty-based adaptive weighting method is employed to balance the multiple
learning tasks when training the PINN. The proposed methods are verified on a
public dataset of Li-ion Phosphate (LFP)/graphite batteries.
Pengfei Wen, Zhi-Sheng Ye, Yong Li, Shaowei Chen, Pu Xie, Shuai Zhao
Fusing Models for Prognostics and Health Management of Lithium-Ion Batteries Based on Physics-Informed Neural Networks

###### Abstract

For Prognostics and Health Management (PHM) of Lithium-ion (Li-ion) batteries, many models have been established to characterize their degradation process. The existing empirical or physical models can reveal important information regarding the degradation dynamics. However, there is no general and flexible methods to fuse the information represented by those models. Physics-Informed Neural Network (PINN) is an efficient tool to fuse empirical or physical dynamic models with data-driven models. To take full advantage of various information sources, we propose a model fusion scheme based on PINN. It is implemented by developing a semi-empirical semi-physical Partial Differential Equation (PDE) to model the degradation dynamics of Li-ion-batteries. When there is little prior knowledge about the dynamics, we leverage the data-driven Deep Hidden Physics Model (DeepHPM) to discover the underlying governing dynamic models. The uncovered dynamics information is then fused with that mined by the surrogate neural network in the PINN framework. Moreover, an uncertainty-based adaptive weighting method is employed to balance the multiple learning tasks when training the PINN. The proposed methods are verified on a public dataset of Li-ion Phosphate (LFP)/graphite batteries.

 Battery Degradation, Information Fusion, Prognostics and Health Management, Physics-informed Machine Learning, Remaining Useful Life.

## I Introduction

Global sales of Electric Vehicles (EVs) doubled in 2021 from 2020 to a new record of 6.6 million according to [1] released by International Energy Agency (IEA). It keeps rising strongly and 2 million were sold in the first quarter of 2022, up 75% from the same period in the previous year. As their power source, Lithium-ion (Li-ion) batteries have become one of the most important industrial consumables. The demand for batteries reached 340 Gigawatt-hours (GWh) in 2021, which also doubled from the previous year. In the same year, the average battery price is USD 132/kWh. The cost of battery replacement due to its degradation is still high, and the Battery Management System (BMS) is developed to perform Prognostics and Health Management (PHM). The primary tasks of BMS include both State of Health (SoH) estimation and Remaining Useful Life (RUL) prognostics. Fused usage information collected from various sources via BMS is expected to significantly improve the performance of PHM.

Generally, information fusion has been achieved in various flexible forms, including data-level, feature-level, decision-level and model-level [2, 3]. The prognostic models can be categorized as physics-based models, experience-based models, and data-driven models [4, 5, 6]. Physics-based and experience-based models represent abundant prior domain knowledge of the monitored systems condensed by experts that have been widely accepted [7]. There is no insurmountable gap between these two categories of models since many physics models originate from empirical models, such as the Paris-Erdogan equation. On the contrary, building data-driven models is a process of mining latent information in data [8]. Dynamic models are commonly built to represent the governing dynamics during degrading, which can provide critical insights into the internal change of monitored systems.

Dynamic models for Li-ion-batteries degradation can be established based on the classic Pseudo-Two-dimensional (P2D) model, which possesses high accuracy but is severely restricted due to the subsequent complexity and computation burden [9]. The Single-Particle (SP) model is then proposed to simplify the P2D model, which assumes that both electrodes consist of multiple uniform-sized spherical particles. Based on the SP model, a widely-accepted degradation mechanism was proposed. Since electrolytes can be reduced in the presence of lithiated carbon, Solid Electrolyte Interphase (SEI) formation is caused by the lithium carbonate. The newly exposed surface of the particles is then covered by SEI during the cycling. Under this mechanism, the degradation rate considering both Diffusion Induced Stresses (DIS), crack growth, and SEI thickness growth was derived in [10, 11]. There are many parameters to estimate and verify in this dynamic model proposed fully based on prior electrochemical knowledge, which restricts its generalizability. Another category of degradation models focuses more on the utilization of available degradation data with monitoring time or cycles, and then the degradation trends are modeled by analytical functions under the given statistical assumptions [12]. These models
commonly oversimplify the physics that the degradation trends of Li-ion batteries should follow. Semi-physics and semi-empirical dynamic models are proposed to take both factors into account [13]. For instance, an exponential-law model can be derived based on battery Coulombic efficiency [14]. This exponential model can also be integrated from an empirically given differential form and then its parameters are estimated based on temperature stress, State of Charge (SoC) stress, time stress, and Depth of Discharge (DoD) stress [13]. In view of the fact that there is an upper bound for the capacity loss, the Verhulst model has been proposed to quantify the rate of capacity loss [15]. As can be seen, there usually exists more or less available information from physics for SoH estimation, while this is not always true for RUL prognostics. To fill this gap, discovery for dynamic models with a preset fixed form or completely arbitrary form has been explored by developing Partial Differential Equation (PDE)-Net [16, 17] and Deep Hidden Physics Models (DeepHPM) [18, 19] respectively.

According to the No-Free-Lunch (NFL) theorem, information from different models fits distinct problems well. Model fusion can leverage the advantages of combining the information from different categories of available models for PHM. It can be difficult to categorize fusion forms of model fusion. Five mostly-applied forms of fusing models are reviewed according to the combination and interfaces of distinct types of prognostic models [4]. Fusing physics-based models and data-driven PHM methods drags much attention [20]. Among these fusion forms, transition equations built based on physics or experience have been widely integrated into the framework of Bayesian filtering [21, 22]. These transition equations are then used to iterate the degradation state. This implementation of fusing physics-informed dynamic models and data-driven prognostic models inspires the frontiers of Physics-Informed Machine Learning (PIML) [23] in PHM.

The principle of PIML is to fuse the physics (experience)-based models and data-driven models. As reviewed in [4], this category of model fusion can be instantiated in various forms as well. Typical implementations of PIML are reviewed in [23] in terms of their applicable scenarios, principles, frameworks, and applications. As a formalized framework among PIML methods, Physics-Informed Neural Network (PINN) gains momentum since dynamic models established in various forms can be integrated. This high-level flexibility making it popular in multiple frontiers such as epidemiology [24], power electronics [25, 26], acoustics [27], and electromagnetics [28].

In this paper, we propose a semi-physics and semi-empirical dynamic model based on the Verhulst model [15] to capture the fading trends of the battery SoH. This model is further generalized as a PDE considering observable features given charging and discharging profiles, and a new parameter that represents the level of initial SEI formation is introduced. To estimate the SoH of Li-ion batteries, we introduce PINN to fuse the prior information formulated as the dynamic model and the information extracted from monitoring data. Besides, DeepHPM is also employed to discover the dynamics as a comparison with the improved Verhulst model for SoH estimation. It is also used for RUL prognostics without any explicit dynamic model. This implementation is a fusion of information mined by two data-driven models with different functions [4]. During the training of PINN, we use an uncertainty-based method to adaptively weigh losses of subsequently proposed multi-task learning. The proposed methods are then verified on a public experimental cycling dataset of Li-ion Phosphate (LFP)/graphite batteries. The contributions and advantages of this paper include: 1) We propose a new scheme to fuse prior information of physical or empirical degradation models and that of monitoring data in PHM for Li-ion batteries. 2) We propose a new semi-physical semi-empirical dynamic degradation to provide more insight into capacity loss. 3) We propose a scheme of fusing information of two distinct-designed data-driven models when there is little prior information. Moreover, the proposed models can be trained with an adaptive weighting method, which solves the challenge of combining losses appropriately in training PINNs. The code and data accompanying this paper are available on GitHub at1.

Footnote 1: [Online]. Available: [https://github.com/WenPengfei0823/PINN-Battery-Prognostics](https://github.com/WenPengfei0823/PINN-Battery-Prognostics)

The rest of this article is organized as follows. Section II setups the focused problem and the establishment of two categories of dynamic models is introduced, including an improved Verhulst model and DeepHPM-based automatically discovered models. Section III presents the framework of PINN for model fusion and its training method, where an adaptively uncertainty-based multi-tasks balancing technique is included. Section IV and V verify the proposed methods on a public experimental dataset, and several details are provided in Appendix A. Finally, Section VI concludes this article.

## II Problem Statement

Battery aging results from a combined action of both times went and cycling [13, 29]. Fading in a Li-ion battery upon charging and discharging can be attributed to a coupled mechanical-chemical degradation within the cell. The DISs can boost the growth of the cracks on the electrode surfaces during cycling and thus leading to the growth of SEI [11, 10]. This degradation mechanism assembles the fatigue process of materials upon cyclic loading [13], where the capacity loss due to the stress accumulates independently and ultimately causes life loss. Such a process can be categorized as cycle aging. At the same time, a battery also degrades over time because of the inherent slow electrochemical reaction, forming the calendar aging. Since charging and discharging processes can often be completed in dozens of minutes while calendar aging becomes observable usually on the scales of months or even years, cycle aging contributes the most to capacity and life loss. As a result, we assume that calendar aging can be neglected here when modeling the degradation dynamics of batteries.

The failure time of a Li-ion battery is commonly defined as the time when its currently usable capacity reduces below a pre-specified threshold. SoH of a battery is consequently related to its current capacity and is typically defined as the ratio of current capacity over the nominal one [30]. Here
we focus on the Percentage Capacity Loss (PCL) \(u\) which represents the relative gap between the usable capacity and the nominal one, and it is denoted as:

\[u_{k}=1-SoH_{k}=1-\frac{Q_{k}}{Q_{Nom}}\times 100\%, \tag{1}\]

where \(Q_{Nom}\) represents the nominal capacity and \(Q_{k}\) represents the usable capacity in \(k^{\mathrm{th}}\) cycle.

Without loss of generality, PCL \(u\) can be set as a univariate function of a virtual continuous independent time variable \(t\)[31] whose unit is still the cycles. When a battery is operated upon identical conditions upon each cycle, a series of observations can be acquired at discrete time \(t_{k}\):

\[u_{k}=u\left(t_{k}\right). \tag{2}\]

To handle the degradation dynamics of the battery degradation trend, the fading rate of its capacity can be denoted as

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t}=\mathcal{G}\left(t,u;\Theta \right). \tag{3}\]

Eq. (3) is an explicit Ordinary Differential Equation (ODE) parameterized by set \(\Theta\), and \(\mathcal{G}\) denotes a nonlinear function of \(t\) and \(u\). For instance, based on SEI formation and the growth process on both initial and cracked surface [10, 11], eq. (3) can be specified as

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t}\bigg{|}_{t=t_{k}} =\theta_{1}\theta_{5}\left(1+\theta_{2}t_{k}\right)^{\frac{\theta _{3}}{2-\theta_{3}}}+\theta_{4}t_{k}^{-\frac{1}{2}}, \tag{4}\] \[+\theta_{1}\theta_{6}\sum_{l=1}^{k-1}\left[1+\theta_{2}\left(t_{ k}-t_{l}\right)\right]^{\frac{\theta_{3}}{2-\theta_{3}}}\left(t_{k}-t_{l}\right)^{- \frac{1}{2}},\] \[l<k, \tag{5}\]

where \(t_{l}\) represents the monitoring time earlier than \(t_{k}\) and \(\Theta=\left\{\theta_{1},\theta_{2},\ldots,\theta_{6}\right\}\) are composite parameters. Each of them should be set according to dozens of electrochemical parameters including the geometric area of the graphite electrode film, the activation energy for crack propagation, and the solid phase porosity of the electrode film, etc. This theoretical model details molecular-level aging mechanism but it can be hardly employed in practical applications due to the lack of detailed cell conditions. Such a dynamic model describes a degradation trend reported in [32] with a decelerated trend of \(u\) during the early cycles and a moderate linear trend [33] during the latter cycles. Based on the observed trend, a simplified semi-empirical model was proposed in [13] by using regression analysis:

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t}=\theta\left[1-u\left(t\right) \right], \tag{6}\]

where \(\theta\) denotes a basic linearized degradation rate that is co-determined by SoC, DoD, and cell temperature. An exponential-function solution with a decreasing absolute value of derivative can be acquired by integrating (6) to \(u\). It can be observed that this model may not fit the battery degradation with an accelerated trend during the early cycles reported in [34, 35].

### _Improved Verhulst Dynamic Model_

Here the function \(\mathcal{G}\left(t,u;\Theta\right)\) is also made dependent on the health state of the cell, and takes the simplest linear form as an instance:

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t} =ru\left(t\right), \tag{7}\] \[s.t.\ u\left(t\right),\ r>0.\]

where \(r\) is the degradation constant. Eq. (7) and partial term of (6) mark that the rate at which the capacity loss changes is proportional to the usable capacity at that time. The solution of (7) is the exponential function \(u\left(t\right)=u_{0}\exp\left(rt\right)\) with an initial loss \(u_{0}\), representing an accelerated degradation trend. This initial loss \(u_{0}\) is commonly set as 10% [10]. Considering the capacity loss results from SEI growth and so on is always finite, a constant \(K\) representing the upper bound of capacity loss is introduced to constrain the increasing rate of loss:

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t} =ru\left(t\right)\left[1-\frac{u\left(t\right)}{K}\right], \tag{8}\] \[\mathrm{s.t.\ }r>0,\] \[0<u\left(t\right)<K.\]

Eq. (8) is also known as the logistic differential equation proposed by Pierre Verhulst to model the population growth process [15]. When the lost capacity increases close to \(K\), the increasing rate \(\mathrm{d}u\left(t\right)/\mathrm{d}t\) will decline until it is close to 0. Since a battery is commonly considered as failed when its \(u\) increases exceed 20%, it can be _a priori_ roughly known that \(K\) has a range of 20% to 100%. Furthermore, SEI will naturally form after a brand-new battery is manufactured and before use. In this process, the capacity loss caused by the SEI formation is supposed not to be governed by these dynamic models. Instead, a parameter \(C\) that denotes such initial capacity loss is introduced to modify the model (8) as

\[\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t} =r\left[u\left(t\right)-C\right]\left[1-\frac{u\left(t\right)-C} {K-C}\right]. \tag{9}\] \[\mathrm{s.t.\ }r >0,\] \[0<u\left(t\right)<K,\] \[0<C\leq u_{0}.\]

Even though under identical operation conditions, different cells manufactured in the same batch may show distinct degradation characteristics. With the setup that \(u\) is a univariate function of \(t\), this cell-to-cell heterogeneity can be modeled by the difference in degradation model parameters [36, 37]. Considering the sole time-independent variable \(t\) cannot distinguish specific trajectory when different batteries are fading, other variables that can indicate the latent health state can be introduced. In this setup, \(u\) is expanded to a multivariate function of both \(\mathbf{x}=\left[x_{1},x_{2},\ldots,x_{S}\right]^{\mathrm{T}}\in\mathbb{R}^{S}\) and \(t\). An \(S\)-dimensional Health Indicator (HI) \(\mathbf{x}\) can characterize the health state during the battery fading, and both monitoring data and designed representative features can act as health indicators [33, 38]. It is feasible that cell-to-cell heterogeneity is quantified by different specific combinations of values in the
feature space. The degradation rate given \(\mathbf{x}\) and monitoring time \(t\) is subsequently modeled by a PDE as:

\[\frac{\partial u\left(\mathbf{x},t\right)}{\partial t} =r\left[u\left(\mathbf{x},t\right)-C\right]\left[1-\frac{u\left(\mathbf{x},t\right)-C}{K-C}\right]. \tag{10}\] \[\mathrm{s.t.} r>0,\] \[0<u\left(t\right)<K,\] \[0<C\leq u_{0}.\]

The parameters of eq. (10) are of the same meanings as those in eq. (9).

### _Data-Driven Dynamic Model_

The dynamic model (10) depicts one of many forms of SoH degrading. Beyond SoH prognostics, it can be difficult to distill degradation mechanisms when predicting other variables such as RUL. Following (3), we define more generalized nonlinear dynamics parameterized by \(\Theta\) to distill the mechanisms governing the evolution [18] of given data of HIs and time as

\[u_{t}-\mathcal{G}\left(\mathbf{x},t,u,u_{\mathbf{x}},u_{\mathbf{x}\mathbf{x}},u_{\mathbf{x}\mathbf{x} \mathbf{x}},\ldots;\Theta\right)=0. \tag{11}\]

In eq. (11), \(u_{\mathbf{x}}=\left[\frac{\partial u}{\partial x_{1}},\frac{\partial u}{\partial x _{2}},\ldots,\frac{\partial u}{\partial x_{S}}\right]^{\mathrm{T}}\) denotes the first-order partial derivative of \(u\) with respect to \(\mathbf{x}\) and so on. The nonlinear function \(\mathcal{G}\) poses more flexible relations on \(t\), \(u\), and their any-order partial derivatives. Subsequently, an infinite dimensional dynamical system can be represented by \(\mathcal{G}\)[18]. As can be seen, (10) is a specific implementation of (11) where \(\mathcal{G}\left(u;\Theta\right)=r\left(u-C\right)\left(1-\frac{u-C}{K-C}\right)\). In most cases of health monitoring, it is challenging to define an explicit dynamic model. With scattered and noisy monitoring observations, bias still exists in the PDE model (10) and the more-complicated latent dynamics. We construct a Neural Network (NN) as a function approximator [39] to fill this gap beyond a particular family of basis functions [19], forming a DeepHPM [18]:

\[u_{t}-\mathrm{DeepHPM}\left(\mathbf{x},t,u,u_{\mathbf{x}},u_{\mathbf{x}\mathbf{x}},u_{\mathbf{x} \mathbf{x}},\ldots;\Theta\right)=0. \tag{12}\]

Here we denote the NN used to approximating \(\mathcal{G}\) as the DeepHPM. The parameter set \(\Theta\) represents the trainable network parameters of the function approximator DeepHPM.

## III Methodology

The PDE dynamic model established based on prior knowledge (10) or approximated by NN (12) can be quite hard to solve. Based on the well-known capability of NNs as universal function approximators, we employ another NN parameterized by \(\Phi\) to approximate the hidden solution \(u\left(\mathbf{x},t;\Phi\right)\) of the system. Solution \(u\left(\mathbf{x},t;\Phi\right)\) is also called the _surrogate_ network. With this NN solver, the direct access or approximations to the involved partial derivatives are unnecessary [40, 18]. Model fusion of the surrogate NN with the explicit PDE models or DeepHPM formulates a PINN.

### _Model Fusion by PINN_

A typical structure of PINN consists of three modules including a dynamic model, a surrogate NN, and an automatic differentiator. The dynamic model \(\mathcal{G}\left(\mathbf{x},t,u,u_{\mathbf{x}},u_{\mathbf{x}\mathbf{x}},u_{\mathbf{x}\mathbf{x}}, \ldots;\Theta\right)\) distills the mechanisms governing the dynamics of a degrading system, which can be either explicitly defined as (10) or approximated by a DeepHPM as (12). The surrogate NN \(u\left(\mathbf{x},t;\Phi\right)\) is used to approximate the hidden solution \(u\left(\mathbf{x},t\right)\) of the dynamic model, and the Automatic Differentiator (AutoDiff) [41] is used to calculate values of all involved partial differentials input to the dynamic model.

Except for solving the latent \(u\left(\mathbf{x},t\right)\) to build the prognostic model, discovering the dynamic model is realized by identifying the parameter set \(\Theta\) of either specific PDE or DeepHPM. Identifying the unknown parameters forms a high-dimensional inverse problem describing the dynamical system. With the surrogate NN \(u\left(\mathbf{x},t;\Phi\right)\), we define the left-hand side of eq. (12) as a function \(f\left(\mathbf{x},t;\Phi,\Theta\right)\):

\[f\left(\mathbf{x},t;\Phi,\Theta\right)\coloneqq u_{t}\left(\mathbf{x},t;\Phi\right)- \mathcal{G}\left(\mathbf{x},t,u,u_{\mathbf{x}\mathbf{x}},u_{\mathbf{x}\mathbf{x}\mathbf{x}},\ldots; \Theta\right).\]

The parameters \(\Phi\) of surrogate NN \(u\left(\mathbf{x},t;\Phi\right)\) and \(\Theta\) of dynamic model \(\mathcal{G}\left(\mathbf{x},t,u,u_{\mathbf{x}},u_{\mathbf{x}\mathbf{x}},u_{\mathbf{x}\mathbf{x}\mathbf{x} },\ldots;\Theta\right)\) can be trained by minimizing the mean squared error losses:

\[\mathcal{L}=\lambda_{u}\mathcal{L}_{u}+\lambda_{f}\mathcal{L}_{f}+\lambda_{f_{ f}}\mathcal{L}_{f_{f}}, \tag{13}\]

where

\[\mathcal{L}_{u}= \sum_{i=1}^{N}\left[u\left(\mathbf{x}_{i},t_{i};\Phi\right)-u_{i} \right]^{2}, \tag{14}\] \[\mathcal{L}_{f}= \sum_{i=1}^{N}\left[f\left(\mathbf{x}_{i},t_{i};\Phi,\Theta\right) \right]^{2},\] (15) \[\mathcal{L}_{f_{t}}= \sum_{i=1}^{N}\left[f_{t}\left(\mathbf{x}_{i},t_{i};\Phi,\Theta \right)\right]^{2}. \tag{16}\]

Loss function (13) actually represents multi-tasks learning. Weight coefficients \(\lambda_{u}\), \(\lambda_{f}\), and \(\lambda_{f_{t}}\) can be tuned to balance the loss terms in the training process. The loss term \(\mathcal{L}_{u}\) corresponds to the regression fitting error at the collected observations, while \(\mathcal{L}_{f}\) and \(\mathcal{L}_{f_{t}}\) enforce the structure of PINN imposed by (11) and its first-order partial derivative with respect to \(t\)[42]. Here \(\mathcal{D}_{Train}=\left\{\mathbf{x}_{i},t_{i},u_{i}\right\}_{i=1}^{N}\) denotes the training data and \(u_{i}\) represent the label to be predicted corresponding to the input data \(\left\{\mathbf{x}_{i},t_{i}\right\}\), which can be actual SoH or RUL in PHM of batteries for the instance.

### _Framework of PINN_

Here the involved frameworks include a data-driven vanilla NN (baseline), a PINN with the Verhulst equation as the dynamic model (PINN-Verhulst), and a PINN with a DeepHPM as the dynamic model (PINN-DeepHPM). These frameworks are shown in Fig. 1, 2, and 3, respectively. To compare the respective learning framework clearer, the vanilla NN in the baseline is also denoted as a surrogate NN although there is no PDE to be solved. The basic structure of the surrogate NNs is illustrated in Fig. 4. The hidden layers of the surrogate
NNs are all composed of Fully Connected (FC) layers, and the hyperbolic tangent is employed as the activation function due to its differentiability. Structures of all the involved surrogate NNs in both Fig. 1, 2, and 3 are set as this if there are no special notes. The structure of DeepHPM is set the same as that of the surrogate NNs.

### _Weight Coefficients Tuning in Training PINN_

Focusing on the loss function (13) of training a PINN, it can be observed that PINN is constrained or regularized by the loss imposed by the given set of PDEs beyond a regression task at the given data set. It is observed that networks is sensitive to the relative weights of multiple objective functions from desired training tasks [43]. Manually tuning the weighting coefficients \(\lambda\) can be quite expensive, and methods for adaptive tuning during the training are desirable. Numerical stiffness will cause unbalanced back-propagated gradients in the PINN training process, and the weighting coefficients can be accordingly tuned based on the gradient statistics [44]. Other mechanisms have also been utilized to design the adaptive balancing such as random lookback [45] and homoscedastic aleatoric uncertainty [43]. Following [43], the likelihood of the observed data labels is assumed as Gaussian. The mean of likelihood is set as the output of the surrogate NNs and the variance is set according to the weighting coefficient:

\[p\left(\left.\boldsymbol{u}\right|u\left(\boldsymbol{X},\boldsymbol{t}; \Phi\right)\right)=\mathcal{N}\left(u\left(\boldsymbol{X},\boldsymbol{t}; \Phi\right),\frac{1}{\lambda_{u}}\right), \tag{17}\]

where \(\boldsymbol{u}=\left[u_{1},u_{2},\ldots,u_{N}\right]^{\mathrm{T}}\), \(\boldsymbol{X}=\left[\boldsymbol{x}_{1},\boldsymbol{x}_{2},\ldots,\boldsymbol {x}_{N}\right]^{\mathrm{T}}\), and \(\boldsymbol{t}=\left[t_{1},t_{2},\ldots,t_{N}\right]^{\mathrm{T}}\) consist of labels and samples of the training set respectively. The output corresponding to the loss \(\mathcal{L}_{u}\) is used as the instance and the other objectives in (13) can be accordingly defined. The observation noise scalar is set related to the weighting coefficient \(\lambda\) corresponding to the output in the loss function. The log-likelihood can be written as

\[\log p\left(\left.\boldsymbol{u}\right|u\left(\boldsymbol{X},\boldsymbol{t}; \Phi\right)\right)\propto-\frac{\lambda_{u}}{2}\left\|\boldsymbol{u}-u\left( \boldsymbol{X},\boldsymbol{t};\Phi\right)\right\|^{2}+\frac{1}{2}\log\lambda_{ u}. \tag{18}\]

Based on (14), (15), (16), and (18), the multi-task minus log-likelihood can be obtained as (19). It can be seen that the loss function (13) is regularized as:

\[\mathcal{L}=\lambda_{u}\mathcal{L}_{u}+\lambda_{f}\mathcal{L}_{f}+\lambda_{f_ {t}}\mathcal{L}_{f_{t}}-\log\lambda_{u}\lambda_{f}\lambda_{f_{t}}. \tag{20}\]

With these settings, relative weighting coefficients \(\lambda_{u}\), \(\lambda_{f}\), and \(\lambda_{f_{t}}\) of the loss terms can be learned by minimizing the regularized loss function (20) adaptively. The output weighted noise is constrained by the penalty term \(\log\lambda_{u}\lambda_{f}\lambda_{f_{t}}\). In practice, we define \(\lambda^{\prime}\coloneqq-\log\lambda\) and train \(\lambda^{\prime}\) for numerical stability. The loss function (20) can be subsequently rewritten as:

\[\mathcal{L} =\exp\left(-\lambda^{\prime}_{u}\right)\mathcal{L}_{u}+\exp \left(-\lambda^{\prime}_{f}\right)\mathcal{L}_{f}+\exp\left(-\lambda^{\prime} _{f_{t}}\right)\mathcal{L}_{f_{t}}\] \[+\log\lambda^{\prime}_{u}+\log\lambda^{\prime}_{f}+\log\lambda^{ \prime}_{f_{t}}. \tag{21}\]

Training PINN by minimizing (21) instead of (13) is expected to balance the multiple losses, which is denoted as AdpBal here.

## IV Dataset Description and Preprocessing

### _Dataset Description_

The dataset involved in this paper consists of three batches of commercial LFP/graphite battery cells manufactured by A123 Systems, where a total of 124 cells were cycled to failure under dozens of different fast-charging protocols [35]. The nominal capacity of the cells is 1.1 Ah, and the unit charging and discharging rate 1C equals 1.1 A subsequently. Each charging protocol is marked as a string formatted as "C1(Q1)-C2", where the corresponding cell was first charged with the current C1 from 0% SoC (unit: %) to the SoC Q1. When charged at Q1, the charging current was then switched to C2 with which the cell was charged to 80% SoC. All cells were finally charged from 80% to 100% SoC with a Constant-Current Constant-Voltage (CC-CV) form to the 3.6V upper cutoff potential and the C/50 cutoff current. Moreover, all cells were discharged also with a CC-CV form at 4C to 2.0V lower cutoff potential and the C/50 cutoff current. Detailed information on the studied cells and the experimental settings can be accessed in [35].

### _Feature Extraction_

As introduced in Section II and III, appropriately designed health features \(\boldsymbol{x}=\left[x_{1},x_{2},\ldots,x_{S}\right]^{\mathrm{T}}\) can effectively characterize the exact degradation process that each cell undergoes. Extracting features varying to the number of cycles for which the battery has operated attracts much attention. A critical principle is that the extracted features are expected to be of strong correlation with SoH [30]. Point features can be simply extracted based on the charging/discharging profiles during cycling, such as the peaks of Incremental Capacity (IC) curves during the CC charging [46]. Point features can be conveniently extracted mainly characterizing the transient state in each cycle. Interval features can represent characteristics in a period, and they can also be easily extracted by intercepting values on the profiles at certain intervals. Therefore, information related to SoH may be over-simplified since only values on the endpoints of the intervals are paid attention to. Interval features can be extracted as the charge time during the CC charging [34] and time intervals of equispaced charge current/voltage changing [47, 48] for the instance. To capture the trend variation of charging/discharging profiles during cycling, trend features are designed focusing on parameters varying upon a specific prior function modeling the profiles during a cycle. The trend of charging current during CV charging is approximately exponential, and the

Fig. 1: Framework of typical data-driven vanilla NN.
parameter of the fitted exponential function can be used as a trend feature [46]. Within a certain range of discharging voltage, differences in discharging capacity show a quadratic-function-formed trend to discharging capacity. Parameters of the quadratic polynomial can be accordingly estimated as the trend feature as well [30]. Other statistical features computed based on the whole profile of a cycle can also be employed such as the mean [49, 30], energy [50], Skewness [50], and Kurtosis [50], etc.

Following [30], we also extracted the trend feature based on the quadratic model:

\[Q_{i+1}\left(V_{i+1}\right)-Q_{i}\left(V_{i}\right)=-\omega\left[Q_{i}\left(V_{ i}\right)\right]^{2}+b+\varepsilon. \tag{22}\]

Eq. (22) is defined for the profile of each cycle. \(Q_{i}\left(V_{i}\right)\) is a measurement of the discharging capacity with respect to the discharging voltage between 2.7V and 3.3V, and \(\varepsilon\sim\mathcal{N}\left(0,\sigma^{2}\right)\) represents the normal measuring error. Undetermined coefficients \(\omega\) and \(b\) are estimated as two features in that cycle. Maximum, minimum, and the variance of the IC curve when the discharging voltage is between 2.7V and 3.3V are also used as features. Other employed features include average temperature, internal resistance, and charging time. To reduce the measurement noise, the moving average is then applied to the extracted features for better representative capability. All the extracted features from cell #124 are illustrated in Fig. 5

Figure 4: Basic structure of NNs.

Figure 3: Framework of PINN-DeepHPM.

Figure 2: Framework of PINN-Verhulst.
as an example, and it should be noted that the features are 0-1 normalized in this figure for better illustration.

### _Standardization_

Note that there are significant variations among different channels of inputs and outputs, which may severely impact the performance. Standardization is consequently implemented. The derivatives of outputs to the inputs can be kept unchanged in this way when using AutoDiff, and the standardized data are processed by the NN. On the other hand, the partial derivatives of the before- or after-standardization outputs to the before- or after-standardization inputs can be obtained. Here we employ the widely-used z-score standardization to scale inputs. Specifically, the standardization factors are set as the corresponding mean and standard deviation calculated from the training data:

\[\widetilde{x}_{i}=\frac{x_{i}-\mathrm{Mean}\left(\mathbf{X}_{Train} \right)}{\mathrm{Std}\left(\mathbf{X}_{Train}\right)}, \tag{23}\] \[\widetilde{u}_{i}=\frac{u_{i}-\mathrm{Mean}\left(\mathbf{u}_{Train} \right)}{\mathrm{Std}\left(\mathbf{u}_{Train}\right)}. \tag{24}\]

In eq. (23) and (24), \(\mathbf{u}=\left[u_{1},u_{2},\dots,u_{N}\right]^{\mathrm{T}}\) and \(\mathbf{X}=\left[\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{N}\right]^{\mathrm{T}}\) consist of labels and samples of the training set as denoted in Section III-C. Monitoring time \(t\) is also standardized in this way since it can be regarded as part of inputs.

## V Case Study

The proposed prognostic framework is verified for both SoH estimation and RUL prediction. For better comparison with the existing methods, the training set and test set are formed in different ways, which are marked as cases A, B, and C respectively. In case A, data from cells #91 and #100 are used for training/validation and cell #124 is used as the test set [30]. Their charging protocol is 4.8C(80%)-4.8C. In case B, data from cells #101, #108, and #120 are used for training/validation and cell #116 is used as the test set [30]. Their charging protocol follows 5.3C(54%)-4C. In case C, data from batch 2 are selected and 20% of them are randomly chosen as the test set [49]. These cells follow various charging protocols.

Here the widely used Root Mean Square Error (RMSE) metric is adopted to measure the performance. RMSE is formulated as:

\[\mathrm{RMSE}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left(\hat{u}_{i}-u_{i}\right)^{ 2}}, \tag{25}\]

where \(\hat{u}_{i}\) and \(u_{i}\) denote the output of the prediction model and the actual data label corresponding to \(i^{\mathrm{th}}\) input sample. When it is used for SoH estimation, the output PCL should be transformed as SoH based on eq. (1). There are a total of \(N\) samples in the test set. Moreover, we also adopt the Root Mean Square Percentage Error (RMSPE) as the metric since the scale of SoH and RUL can be quite distinct. RMSPE is formulated as:

\[\mathrm{RMSPE}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left(\frac{\hat{u}_{i}-u_{i}}{u _{i}}\right)^{2}}\times 100\%. \tag{26}\]

Division of training and test data have been detailed above. Then 20% of training-validation data are divided as the validation data in cases A and B, and 25% of training-validation data are divided as the validation data in case C. The rest of the training-validation data is used for training ultimately. Several general settings are listed in Table I. Other critical hyper-parameters such as the number of hidden layers and neurons are further tuned depending on the performance of validation data. To reduce the number of hyper-parameters to be tuned,

Figure 5: Extracted features of cell #124.

Figure 6: SoH estimation of cell #124 in case A.

Figure 7: SoH estimation of cell #116 in case B.
we tune them based on baseline models as far as possible. Then the obtained optimal parameters for the baseline models are applied to the proposed models. It is worth mentioning that those parameters may not be the optima for the proposed models. For instance, optimal network structures (the number of hidden layers and neurons) are determined based on the vanilla data-driven NN illustrated in Fig. 1, and then they are used for both PINN-Verhulst and PINN-DeepHPM. It has been revealed that the performance of discovering dynamics by using DeepHPM can be sensitive to input terms [18, 16, 17], so which terms should be involved needs to be further explored. We organize various combinations of input features, monitoring time as well as partial derivatives to them as a candidate library. Similarly, simply summing the multiple losses in eq. (13) for training PINNs is further set as a baseline. The optimal combination of input terms in the library for DeepHPM is determined with this baseline, and then it is used to validate AdpBal for the multiple losses. Calculated RMSPEs for SoH estimation and RMSEs for RUL prognostics on the validation set in terms of the number of hidden layers and neurons per layer are listed in Tables IV, V, VI, VII, and VIII in the Appendix. Validation RMSPEs and RMSEs in terms of the combinations of inputs to DeepHPM are listed in Table IX. Accordingly, settings for different cases are determined and summarized in Table II. With these settings, the proposed models are all trained in a regular computation environment (Intel Xeon E5-2630 CPU, 2.2 GHz; Nvidia GeForce GTX 1080 Ti GPU). The training-test process is repeated for 5 rounds in each case, and the results are averaged to reduce the randomness. The results of SoH estimation and RUL prognostics are summarized in Table III.

### _SoH Estimation_

The two categories of dynamic models, the Verhulst model, and DeepHPM are used to estimate the SoH. This represents two cases that whether there are available explicit dynamic models respectively.

In case A, the calculated RMSPEs of SoH estimation on the validation data in terms of different combinations of the number of hidden layers and neurons are listed in Table IV. The optimal network structure for the data-driven baseline model is 2 hidden layers and 128 neurons per hidden layer. With this setting, the baseline model provides a 0.42% estimation error in terms of RMSPE on the test data. PINN-Verhulst provides a 1.41%-RMSPE estimation when simply summing the multiple losses. With AdpBal for tuning the weight coefficients of the losses, a 0.49% RMSPE can be obtained. When there is no prior explicit dynamic model, the inputs of DeepHPM are set as \(\mathbf{x},t\) when discovering the dynamic model via DeepHPM according to Table IX. These validation errors are calculated by simply summing the multiple losses. the estimation errors are 0.43% and 0.47% respectively without and with AdpBal.

Figure 8: Variation of relative weighting coefficients by using adaptive balancing in the model training. (a)–(e) Respective variation of relative weighting coefficients in each round.
In case B, the network structure is set as 2 hidden layers with 64 neurons per hidden layer according to Table V. With this setting, the baseline estimation error is 0.56%. PINN-Verhulst (Sum), PINN-Verhulst (AdpBal), PINN-DeepHPM (Sum), and PINN-DeepHPM (AdpBal) provide 0.56%, 0.44%, 0.51%, and 0.42% estimation errors in terms of RMSPEs.

The SoH estimation is further illustrated in Fig. 6 and 7. The data-driven baseline method without model fusion can perform satisfactorily for SoH estimation, especially in case A. With fusing the improved Verhulst model, the performance of simply summing the losses has no significant improvement over the baseline. The estimation accuracy is even significantly declined in case A. By using the AdpBal, the performance is improved instead. Compared with case A, there is a better improvement in case B. The reason can be that the numbers of hidden layers and neurons are optimal for baseline, but probably not for the proposed methods. When DeepHPM is used to discover the governing dynamics, similar results are observed. It verifies the ability of DeepHPM to discover dynamics and the robustness of using AdpBal for training PINN. With AdpBal, relative values of the weighting coefficients changing in terms of the training epochs of case B in all 5 rounds are illustrated in Fig. 8. It can be seen that relative values of the weighting coefficients vary in similar ways during different rounds of training. This also shows the robustness of AdpBal. Moreover, the proposed methods are compared with a Gaussian Process Regression (GPR)-based method [30]. It is worth mentioning that partial onboard test data are further used to calibrate this GPR-based method (GPR Onboard) in [30]. This comparison is shown in Table III. As can be seen, the proposed methods all perform better than this comparative study even without using any onboard test data.

### _RUL Prognostics_

Since there is no prior explicit dynamic PDE model correlating the predicted RUL to the input features as well as monitoring time, only DeepHPM is employed here to discover the dynamics. The results are summarized in Table III.

Predicted and actual RUL of the test cells #124 and #116 are illustrated in Fig. 9 and 10. In case A, the data-driven prognostic model provides a 46.38 cycles error in terms of RMSE. With physics-informed losses based on the discovery of DeepHPM, the prediction error is 48.81 cycles. After AdpBal is turned on, the predictive RMSE is 45.86 cycles. The prognostic error is slightly reduced by 6.04% ((45.86\(-\)48.81)/48.81\(\times\)100%) after weighting coefficients are tuned automatically. In case B, the prediction RMSE of baseline, PINN-DeepHPM (Sum), and PINN-DeepHPM (AdpBal) are 64.48, 65.52, and 56.29 cycles respectively. The prognostic error is significantly reduced by 14.09% ((56.29\(-\)65.52)/65.52\(\times\)100%) after weighting coefficients are tuned automatically. Similar to that of SoH estimation, the proposed method shows a minor improvement in case A and a major improvement in case B. As a result, the generalizability of the proposed methods from SoH estimation to RUL prognostics is verified. In case C, RUL prognostics error is reduced by 6.7% ((14.19\(-\)15.21)/15.21\(\times\)100%) with introducing DeepHPM. However, the performance deteriorated by using AdpBal. A possible reason can be that there

Fig. 10: RUL prognostics of cell #116 in case B.

Fig. 9: RUL prognostics of cell #124 in case A.
are multiple operation conditions for the cells in case C, which severely impacts the homoscedastic uncertainty among measurements of cells. Moreover, the proposed methods are compared with a Long-Short Term Memory (LSTM) NN [49] and a Bayesian Deep Learning (DL) framework incorporating uncertainty quantification and calibration [49]. It can be seen from Table III that the fused PINN-DeepHPM outperforms those LSTM-based methods. LSTM commonly fits temporal information better than vanilla NN since it can capture the dynamics by using difference equations. The fusion with DeepHPM enables the vanilla NN to capture the dynamics as well.

## VI Conclusion

In this article, we propose a new model fusion framework for Li-ion batteries PHM based on Physics-Informed Neural Network. A flexible form is provided by PINN to fuse empirical or physical dynamic models and data-driven surrogate NNs. With the established dynamic model, the rate of capacity degrading can be quantified with respect to the operating cycles and the current SoH of monitored cells. The generalization of the dynamic model to a PDE form fits different cells with the same combination of parameters. The added extra parameter can represent the initial SEI formation, which is a common reason of capacity loss in the initial cycles. Under this setting, the model is more consistent with the physical characteristics of battery degradation. DeepHPM provides a specialized model to discover the governing dynamics of battery degradation when lacking prior information. With the uncertainty-based weighting method, losses of multiple learning tasks can be adaptively balanced when training the PINN. Implementation of a public dataset verifies the effectiveness of the proposed methods. The results show that the proposed model fusion scheme can improve the performance of PHM for Li-ion batteries, but an appropriate weighting method is compulsory to balance the multi-task losses for training PINN.

## Appendix A Performance of Different Settings on the Validation Data

### _NN Structures for SoH Estimation_

In terms of the different number of hidden layers and neurons per layer, the calculated RMSPEs for SoH estimation on the validation set in respective cases are listed in Tables IV and V. These results are calculated by using the vanilla NN (Baseline).
are determined according to Appendices A-A and A-B, these results are calculated by using the PINN-DeepHPM by simply summing the losses (Sum).
