# Broken Neural Scaling Laws

Ethan Caballero
Mila, McGill University
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>&Kshitij Gupta
Mila, University of Montreal &Irina Rish
Mila, University of Montreal & David Krueger
University of Cambridge

###### Abstract

We present a smoothly broken power law functional form that accurately models
and extrapolates the scaling behaviors of deep neural networks (i.e. how the
evaluation metric of interest varies as the amount of compute used for
training, number of model parameters, training dataset size, or upstream
performance varies) for various architectures and for various tasks within a
large and diverse set of upstream and downstream tasks, in zero-shot,
prompted, and fine-tuned settings. This set includes large-scale vision,
language, audio, video, diffusion generative modeling, multimodal learning,
contrastive learning, AI alignment, robotics, arithmetic, unsupervised/self-
supervised learning, and reinforcement learning (single agent and multi-
agent). When compared to other functional forms for neural scaling behavior,
this functional form yields extrapolations of scaling behavior that are
considerably more accurate on this set. Moreover, this functional form
accurately models and extrapolates scaling behavior that other functional
forms are incapable of expressing such as the non-monotonic transitions
present in the scaling behavior of phenomena such as double descent and the
delayed, sharp inflection points present in the scaling behavior of tasks such
as arithmetic. Lastly, we use this functional form to glean insights about the
limit of the predictability of scaling behavior. See arXiv for longer version.
Code is available at
https://github.com/ethancaballero/broken_neural_scaling_laws

##  1 Introduction

The amount of compute used for training, number of model parameters, and
training dataset size of the most capable artificial neural networks keeps
increasing and will probably keep rapidly increasing for the foreseeable
future. However, no organization currently has direct access to these larger
resources of the future; and it has been empirically verified many times that
methods which perform best at smaller scales often are no longer the best
performing methods at larger scales (e.g., one of such examples can be seen in
Figure 2 (right) of tolstikhin2021mlp). To work on, identify, and steer the
methods that are most probable to stand the test-of-time as these larger
resources come online, one needs a way to predict how all relevant performance
evaluation metrics of artificial neural networks vary in all relevant settings
as scale increases.

Neural scaling laws (cortes1994learning; 2017arXiv171200409H;
DBLP:journals/corr/abs-1909-12673; icm2020arXiv200108361K;
DBLP:journals/corr/abs-2106-04560; abnar2021exploring;
Alabdulmohsi2022revisiting; brown2020language) aim to predict the behavior of
large-scale models from smaller, cheaper experiments, allowing to focus on the
best-scaling architectures, algorithms, datasets, and so on. The upstream/in-
distribution test loss typically (but not always!) falls off as a power law
with increasing data, model size and compute. However, the downstream/out-of-
distribution performance, and other evaluation metrics of interest (even
upstream/in-distribution evaluation metrics) are often less predictable,
sometimes exhibiting inflection points (on a linear-linear plot) and non-
monotonic behaviors. Discovering universal scaling laws that accurately model
a wide range of potentially unexpected behaviors is clearly important not only
for identifying that which scales best, but also for AI safety, as predicting
the emergence of novel capabilities at scale could prove crucial to
responsibly developing and deploying increasingly advanced AI systems. The
functional forms of scaling laws evaluated in previous work are not up to this
challenge.

One salient defect is that they can only represent monotonic functions. They
thus fail to model the striking phenomena of double-descent
(nakkiran2021deep), where increased scale temporarily decreases test
performance before ultimately leading to further improvements. Many also lack
the expressive power to model inflection points (on a linear-linear plot),
which can be observed empirically for many downstream tasks, and even some
upstream tasks, such as our $N$-digit arithmetic task, or the modular
arithmetic task introduced by power2022grokking in their work on “grokking”.

To overcome the above limitations, we present broken neural scaling laws
(BNSL) \- a functional form that generalizes power laws (linear in log-log
plot) to “smoothly broken” power laws, i.e. a smoothly connected piecewise
(approximately) linear function in a log-log plot. An extensive empirical
evaluation demonstrates that BNSL accurately model and extrapolate the scaling
behaviors for various tasks within a large and diverse set of upstream and
downstream tasks, in zero-shot, prompted, and fine-tuned settings. This set
includes large-scale vision, language, audio, video, diffusion generative
modeling, multimodal learning, contrastive learning, AI alignment, robotics,
arithmetic, unsupervised/self-supervised learning, and reinforcement learning
(single agent and multi-agent). When compared to other functional forms for
neural scaling behavior, this functional form yields extrapolations of scaling
behavior that are considerably more accurate on this set. It captures well the
non-monotonic transitions present in the scaling behavior of phenomena such as
double descent and the delayed, sharp inflection points present in the scaling
behavior of tasks such as arithmetic.

##  2 The Functional Form of Broken Neural Scaling Laws

Figure 1: A Broken Neural Scaling Law (BNSL) (dark black solid line) (with 3
breaks where purple dotted lines intersect with dark black solid line) that
contains 4 individual power law segments (where the dashed lines that are
yellow, blue, red, and green overlap with the dark black solid line). The 1st
and 2nd break are very smooth; the 3rd break is very sharp. See Section 2 for
more details.

The general functional form of a broken neural scaling law (BNSL) is given as
follows:

$\vspace{-1.2mm}y=a+\bigg{(}bx^{-c_{0}}\bigg{)}\prod_{i=1}^{n}\left(1+\left(\frac{x}{d_{i}}\right)^{1/f_{i}}\right)^{-c_{i}*f_{i}},$
(1)

where $y$ represents the performance evaluation metric (e.g. prediction error,
cross entropy, BLEU score percentage, reward, Elo rating, or FID score)
(downstream or upstream) and $x$ represents a quantity that is being scaled
(e.g. number of model parameters, amount of compute used for training,
training dataset size, or upstream performance). The remaining parameters
$a,b,c_{0},c_{1}...c_{n},d_{1}...d_{n},f_{1}...f_{n}$ are unknown constants
that must be estimated by fitting the above functional form to the $(x,y)$
data points. (In our experiments, SciPy curve-fitting library
(virtanen2020scipy) was used.)

The constants in equation 1 are interpreted as follows. Constant $n$
represents the number of (smooth) “breaks” (i.e. transitions) between $n+1$
consecutive approximately linear (on a log-log plot) segments, for a total of
$n+1$ approximately linear segments (on a log-log plot). Constant $a$
represents the limit as to how far the value of $y$ (performance evaluation
metric) can be reduced (or maximized) even if $x$ (the quantity being scaled)
goes to infinity. Constant $b$ represents the offset of functional form on a
log-log plot (analogous to the intercept $b$ in $y=mx+b$ on a linear-linear
plot). Constant $c_{0}$ represents the slope of the first approximately linear
region on a log-log plot. Constant $c_{i}$ represents the difference in slope
of the $(i)$th approximately linear region and $(i+1)$th approximately linear
region on a log-log plot. Constant $d_{i}$ represents where on the x-axis the
break between the $(i)$th and the $(i+1)$th approximately linear region (on a
log-log plot) occurs. Constant $f_{i}$ represents the sharpness of break
between the $(i)$th and the $(i+1)$th approximately linear region on a log-log
plot; smaller (nonnegative) values of $f_{i}$ yield a sharper break and
intervals (before and after the $(i)$th break) that are more linear on a log-
log plot; larger values of $f_{i}$ yield a smoother break and intervals
(before and after the $(i)$th break) that are less linear on a log-log plot.

For mathematical analysis and explanation of why Equation 1 is smoothly piece-
wise (approximately) linear function on a log-log plot, see Appendix A.2. For
mathematical decomposition of Equation 1 into the power law segments it is
composed of (e.g. as in Figure 1), see Appendix A.3.

Note that, while an intuition for using such smoothly connected approximately
piece-wise linear (in log-log plot) function was that, with enough segments,
it could fit well any smooth univariate scaling function, it remained unclear
whether BNSL would also extrapolate well; as we demonstrate below, it does
extrapolate quite accurately in our experiments. Additionally, we find that
the number of breaks needed to accurately model an entire scaling behavior is
often quite small.

##  3 Empirical Results: Fits and Extrapolations of Functional Forms

We now show the fits and extrapolations of various functional forms. In all
plots here and onward and in the appendix, black points are points used for
fitting a functional form, green points are the held-out points used for
evaluating extrapolation of a functional form fit to the black points, and a
red line is the BNSL that has been fit to black points. See Section A.7 for
further experimental details on fitting BNSL and determining the number of
breaks.

Except when stated otherwise, each plot contains a single break of a BNSL fit
to black points that are smaller (along the x-axis) than the green points.

In the tables and elsewhere, M1 refers to functional form $y=ax^{b}$, M2
refers to functional form $y=ax^{b}+c$, M3 refers to functional form
$y=a(x^{-1}+d)^{-b}+c$, M4 refers to functional form
$(y-\epsilon_{\infty})/((\epsilon_{0}-y)^{a})=bx^{c}$ . For a detailed history
and related work concerning these functional forms, see section A.1. See
section A.8 for a mathematical explanation of the theoretical limitations of
previously proposed scaling laws.

All the extrapolation evaluations reported in the tables are reported in terms
of root mean squared log error (RMSLE) ± root standard log error. See Appendix
A.4 for definition of RMSLE and Appendix A.5 for definition of root standard
log error.

See appendix for plots of extrapolation results on large-scale vision,
language, audio, video, diffusion generative modeling, multimodal learning,
contrastive learning, AI alignment, robotics, arithmetic, unsupervised/self-
supervised learning, and reinforcement learning (single agent and multi-
agent). See section A.12 for example of BNSL modeling and extrapolating a non-
monotonic scaling behavior with 2 breaks.

Domain |  | M1 $\uparrow$ | M2 $\uparrow$ | M3 $\uparrow$ | M4 $\uparrow$ | BNSL $\uparrow$
---|---|---|---|---|---|---
Downstream Image Classification |  | 2.78% | 4.17% | 9.72% | 13.89% | 69.44%
Language |  | 10% | 5% | 10% | 0% | 75%
Table 1:  Percentage of tasks by domain where each functional form is the best
for extrapolation of scaling behavior. Numbers for M1, M2, M3, and M4 were
obtained via correspondence with authors of Alabdulmohsi2022revisiting. See
Sections A.9 and A.10 for more details.

###  3.1 Inflection Points

We show that BNSL is capable of modeling and extrapolating the scaling
behavior of tasks that have an inflection point on a linear-linear plot such
as the task of arithmetic (4-digit addition). Here we model and extrapolate
the scaling behavior of a transformer model (vaswani2017attention) with
respect to the training dataset size on the 4-digit addition task. Various
other functional forms are mathematically incapable of expressing inflection
points on a linear-linear plot (as shown in Section A.8) and as a result, are
mathematically incapable of expressing and modeling inflection points (on a
linear-linear plot) that are present in the scaling behavior of 4-digit
addition. In Figure 2 left, we show that BNSL expresses and accurately models
the inflection point present in the scaling behavior of 4-digit addition and
as a result accurately extrapolates the scaling behavior of 4 digit addition.
For further details about the hyperparameters please refer to the Appendix
Section A.6.

Figure 2:  Extrapolation of BNSL on 4 Digit Addition. Note that these plots
are linear-linear. Each point in the left plot is the mean of greater than
1000 seeds at that dataset size. In the left plot, each point is gathered from
a model trained to do the task of 4 digit addition. In the right plot, each
point is gathered from a noiseless simulation of the BNSL of the task of 4
digit addition. See Sections 3.1, A.6, 4, for more details.

##  4 The Limit of the Predictability of Scaling Behavior

We use BNSL to glean insights about the limit of the predictability of scaling
behavior. Recent papers (ganguli2022predictability; wei2022emergent) have
advertised many tasks as having “unpredictable” “emergent” “phase
transition/change” scaling behavior, the most famous of which is the task of
arithmetic. In the previous section and in Figure 2 left, we successfully
predicted (i.e. extrapolated) the scaling behavior of 4-digit addition
(arithmetic). However, we are only able to accurately extrapolate the scaling
behavior if given some points from training runs with a training dataset size
of at least 720, and the break in which the scaling behavior of 4-digit
addition transitions from one power law to another steeper power-law happens
at around training dataset size of 415.

Ideally, one would like to be able to extrapolate the entire scaling behavior
by fitting only points from before the break. In Figure 2 right, we use a
noiseless simulation of the BNSL of 4-digit addition to show what would happen
if one had infinitely many training runs / seeds to average out all the noisy
deviation between runs such that one could recover (i.e. learn via a curve-
fitting library such as SciPy (virtanen2020scipy)) the learned constant of the
BNSL as well as possible. When using this noiseless simulation, we find that
we are only able to accurately extrapolate the scaling behavior if given some
points from training runs with a training dataset size of at least 415, which
is very close to the break.

This has a few implications:

1) When the scaling behavior exhibits greater than zero breaks that are
sufficiently sharp, there is a limit as to how small the maximum (along the
x-axis) of the points used for fitting can be if one wants to perfectly
extrapolate the scaling behavior, even if one has infinitely many seeds /
training runs. 2) If an additional break of sufficient sharpness happens at a
scale that is sufficiently larger than the maximum (along the x-axis) of the
points used for fitting, there does not (currently) exist a way to extrapolate
the scaling behavior after that additional break. 3) If a break of sufficient
sharpness happens at a scale sufficiently smaller than the maximum (along the
x-axis) of the points used for fitting, points smaller (along the x-axis) than
that break are often useless for improving extrapolation.

##  5 Conclusions

Summary. We have presented a smoothly broken power law functional form that
accurately models and extrapolates the scaling behaviors of artificial neural
networks for various architectures and for various tasks from a very large and
diverse set of upstream and downstream tasks. This set includes large-scale
vision, language, audio, video, diffusion generative modeling, multimodal
learning, contrastive learning, AI alignment, robotics, arithmetic,
unsupervised/self-supervised learning, and reinforcement learning (single
agent and multi-agent). When compared to other functional forms for neural
scaling behavior, this functional form yields extrapolations of scaling
behavior that are considerably more accurate on this set. Additionally, this
functional form accurately models and extrapolates scaling behavior that other
functional forms are incapable of expressing such as the non-monotonic
transitions present in the scaling behavior of phenomena such as double
descent and the delayed, sharp inflection points present in the scaling
behavior of tasks such as arithmetic. Finally, we used this functional form to
obtain insights about the limit of the predictability of scaling behavior. See
arXiv for longer version of this paper.

##  Appendix A Appendix

###  A.1 Related Work

To the best of our knowledge, cortes1994learning was the first paper to model
the scaling of multilayer neural network’s performance as a power law (also
known as a scaling law) (plus a constant) of the form $y=ax^{b}+c$ in which
$x$ refers to training dataset size and $y$ refers to test error; we refer to
that functional form as M2. 2017arXiv171200409H showed that the functional
form, M2, holds over many orders of magnitude.
DBLP:journals/corr/abs-1909-12673 demonstrated that the same functional form,
M2, applies when $x$ refers to model size (number of parameters).
icm2020arXiv200108361K brought “neural” scaling laws to the mainstream and
demonstrated that the same functional form, M2, applies when $x$ refers to the
amount of compute used for training. abnar2021exploring proposed to use the
same functional form, M2, to relate downstream performance to upstream
performance. DBLP:journals/corr/abs-2106-04560 introduced the functional form
$y=a(x+d)^{b}+c$, (referred to by us as M3) where d represents the scale at
which the performance starts to improve beyond the random guess loss (a
constant) and transitions to a power law scaling regime.
Alabdulmohsi2022revisiting proposed functional form
$(y-\epsilon_{\infty})/((\epsilon_{0}-y)^{a})=bx^{c}$, (referred to by us as
M4) where $\epsilon_{\infty}$ is irreducible entropy of the data distribution
and $\epsilon_{0}$ is random guess performance, for relating scale to
performance and released a scaling laws benchmark dataset that we use in our
experiments.

2021arXiv210201293H described a smoothly broken power law functional form
(consisting of 5 constants after reducing redundant variables) in equation 6.1
of their paper, when relating scale and downstream performance. While this
functional form can be summed with an additional constant representing
unimprovable performance to obtain a functional form that is mathematically
equivalent to our BNSL with a single break, it is important to note that (i)
2021arXiv210201293H describes this form only in the specific context, when
exploring how fine-tuning combined with transfer learning scales as a function
of the model size - thus, their functional form contains a break only with
respect to number of model parameters but not with respect to other input
quantities which we do explore such as dataset size, amount of compute, and
upstream performance; (ii) 2021arXiv210201293H mentioned this equation in
passing and as a result did not try to fit or verify this functional form on
any data; (iii) they arrived at it simply via combining the scaling law for
transfer (that was the focus of their work) with a scaling law for pretraining
data; (iv) they did not identify it as a smoothly broken power law, or note
any qualitative advantages of this functional form; (v) they did not discuss
the family of functional forms with multiple breaks.

Finally, we would like to mention that smoothly broken power law functional
forms, equivalent to equation 1, are commonly used in the astrophysics
literature (e.g. dampe2017direct) as they happen to model well a variety of
physical phenomena. This inspired us to investigate their applicability to a
wide range of deep neural scaling phenomena as well.

###  A.2 Analysis and Explanation of why BNSL is smoothly connected piecewise
(approximately) linear function on a log-log plot

Analysing Equation 1 reveals why BNSL is smoothly connected piecewise
(approximately) linear function on a log-log plot. Considering $y$ as a
function of $z:=\log(x)$, applying logarithms to both sides and setting $a=0$
yields:

$\log(y)=\log(b)-c_{0}z-\sum_{i=1}^{n}c_{i}f_{i}\log\left(1+\left(\frac{\exp(z)}{d_{i}}\right)^{1/f_{i}}\right).$
(2)

We can now see the terms in the sum resemble the well-known
$\mathrm{softplus}$ function: $\mathrm{softplus}(x):=\log(1+exp(x))$, which
smoothly interpolates between the constant 0 function and the identity. By
plotting one such term for different values of $c_{i},d_{i},f_{i}$, it is easy
to confirm that they influence the shape of the curve as described in Section
2.

###  A.3 Decomposition of Broken Neural Scaling Law into power law segments
that it is composed of

We now show a way to decompose a BNSL (Equation 1) with 3 breaks into the
power law segments that it is composed of. This decomposition is what we used
to produce segments 1-4 overlaid in Figure 1 and is usable when values of $f$
in Equation 1 are not too large. This decomposition pattern is straight-
forward to extend to $n$ breaks.

$segment_{1}=b*(x)^{-(c_{0})}$

$segment_{2}=b*(d_{1})^{-(c_{0})}*(x\hskip 3.98337pt/d_{1})^{-(c_{1}+c_{0})}$

$segment_{3}=b*(d_{1})^{-(c_{0})}*(d_{2}/d_{1})^{-(c_{1}+c_{0})}*(x\hskip
3.98337pt/d_{2})^{-(c_{2}+c_{1}+c_{0})}$

$segment_{4}=b*(d_{1})^{-(c_{0})}*(d_{2}/d_{1})^{-(c_{1}+c_{0})}*(d_{3}/d_{2})^{-(c_{2}+c_{1}+c_{0})}*(x\hskip
3.98337pt/d_{3})^{-(c_{3}+c_{2}+c_{1}+c_{0})}$

###  A.4 Definition of Root Mean Squared Log Error

$Root\\_Mean\\_Squared\\_Log\\_Error=RMSLE=\sqrt{(\sum_{i=1}^{n}(log(y_{i})-log(\hat{y}_{i}))^{2})/n}$

###  A.5 Definition of Root Standard Log Error

$error=(log(y_{i})-log(\hat{y}_{i}))^{2})$
$\mu_{error}=\frac{1}{N}\sum_{i=1}^{N}error$
$\sigma_{error}=\sqrt{\frac{1}{N-1}\sum_{i=1}^{N}(error_{i}-\mu_{error})^{2}}$
$Root\\_Standard\\_Log\\_Error=\sqrt{\mu_{error}+\frac{\sigma_{error}}{\sqrt{len(\hat{y})}}}-\sqrt{\mu_{error}}$

###  A.6 Experimental details of Section 3.1

We perform an extensive set of experiments to model and extrapolate the
scaling behavior for the 4-digit arithmetic addition task with respect to the
training dataset size. Our code is based on the minGPT implementation
(Karpathy2020). We set the batch size equal to the training dataset size. We
do not use dropout or a learning rate decay here. Each experiment was run on a
single V100 GPU and each run took less than 2 hours. For our experiments we
train the transformer model using the following set of hyperparameters:

$D_{model}$ | 128
---|---
$D_{MLP}$ | 512
Number of heads | 2
Number of transformer blocks (i.e. layers) | 1
Learning rate | 0.0001
Weight Decay | 0.1
Dropout Probability | 0.0
Dataset sizes | 144-1008
Vocab Size | 10
Table 2: Hyperparameters for 4-digit addition task

###  A.7 Experimental details of fitting BNSL and determining the number of
breaks

We fit BNSL as follows: We first use scipy.optimize.brute to do a grid search
of the values of the constants
($a,b,c_{0},c_{1}...c_{n},d_{1}...d_{n},f_{1}...f_{n}$) of BNSL that best
minimize the mean squared log error (MSLE) between the real data and the
output of BNSL. We then use the values obtained from the grid search as the
initialization of the non-linear least squares algorithm of
scipy.optimize.curve_fit. We then use the non-linear least squares algorithm
of scipy.optimize.curve_fit to minimize the mean squared log error (MSLE)
between the real data and the output of BNSL.

The version of MSLE we use for such optimization is the following numerically
stable variant:

$Numerically\\_Stable\\_MSLE=\sum_{i=1}^{N}((log(y_{i}+1)-log(\hat{y}_{i}+1))^{2})/N$

With regards to determining the number of breaks $n$ in the BNSL, one way to
go about doing so is to hold out the last few largest (along the x-axis)
points used for fitting (not the green points used for evaluating
extrapolation) as a validation set. The value of $n$ with lowest validation
error when fitting on the remaining smaller (along the x-axis) points is then
used. As mentioned in implication 3 of Section 4, there are many scenarios in
which a break of sufficient sharpness happens at a scale sufficiently smaller
than the maximum (along the x-axis) of the points used for fitting such that
points smaller (along the x-axis) than that break are often useless for
improving extrapolation. For example, if a full scaling behavior contains a
very sharp break and then a very smooth break, one can crop out the smaller
(along the x-axis) points that contain the sharp break and fit a BNSL with a
single break (i.e. with $n=1$) if one only cares about extrapolation accuracy.
To determine where the crop point is, one can employ the same validation set
strategy mentioned at the beginning of this paragraph, but selecting for crop
point instead of number of breaks.

###  A.8 Theoretical Limitations of Previously Proposed Scaling Laws

Our use of BNSLs is inspired by the observation that scaling is not always
well predicted by a simple power law; nor are many of the modifications which
have been applied in previous works sufficient to capture the qualitative
properties of empirical scaling curves. Here we show mathematically two
qualitative defects of these functional forms:

  1. 1.

They are strictly monotonic (first-order derivative does not change its sign)
and thus unable to fit double descent phenomena.

  2. 2.

They cannot express inflection points (second-order derivative does not change
its sign), which are frequently observed empirically. An exception to this is
M4, proposed by Alabdulmohsi2022revisiting.

Note that these functional forms can exhibit inflection points on the log-log
axes which are commonly used for plotting scaling data (as it was observed in
several prior works). However, for inflection points on a linear-linear plot,
the extra expressiveness of broken neural scaling laws appears to be necessary
(and sufficient). Figure 4 and Figure 2, provide examples of BNSLs producing
non-monotonic behavior and inflection points, respectively, establishing the
capacity of this functional form to model these phenomena that occur in real
scaling behavior.

name | $f(x)$ | $f^{\prime}(x)$ | $f^{\prime\prime}(x)$
---|---|---|---
M1 | $ax^{b}$ | $abx^{b-1}$ | $ab(b-1)x^{b-2}$
M2 | $ax^{b}+c$ | $abx^{b-1}$ | $ab(b-1)x^{b-2}$
M3 | $a(x^{-1}+d)^{-b}+c$ | $\frac{ab}{x(1+dx)(d+1/x)^{b}}$ | $abx^{(b-2)}(1+dx)^{(-2-b)}(b-1-2dx)$
Table 3:  Previously proposed functional forms M1, M2, M3 and their (first and
second order) derivatives. See Equation 3 for M4.

M1, M2, M3 functional forms cannot model non-monotonic behavior or inflection
points: First, recall that expressions of the form $m^{n}$ can only take the
value $0$ if $m=0$. We now examine the expressions for the first and second
derivatives of M1, M2, M3, provided in Table 3, and observe that they are all
continuous and do not have roots over the relevant ranges of their variables,
i.e. $x>0$ in general and $b<0$ in the case of M3 (we require $x>0$ because
model size, dataset size, and compute are always non-negative). This implies
that, for any valid settings of the parameters $a,b,c,d,x$, these functional
forms are monotonic (as the first derivative never changes sign), and that
they lack inflection points (since an inflection point must have
$f^{\prime\prime}(x)=0$).

M4 functional form cannot model non-monotonic behavior. The case of M4 is a
bit different, since the relationship between $y$ and $x$ in this case is
expressed as an inverse function, i.e.

$\displaystyle
x=g(y)=\left(\frac{y-\epsilon_{\infty}}{b(\epsilon_{0}-y)^{a}}\right)^{1/c}$
(3)

However, non-monotonicity of $y$ as an inverse function $y=g^{-1}(x)$ is ruled
out, since that would imply two different values of $x=g(y)$ can be obtained
for the single value of $y$ – this is impossible, since $f(y)$ maps each $y$
deterministically to a single value of $x$. As a result, M4 cannot express
non-monotonic functions.

M4 functional form can model inflection points. It is easy to see that if
$y=g^{-1}(x)$ had an inflection point, then $x=g(y)$ would have it as well.
This is because an inflection point is defined as a point $x$ where $f(x)$
changes from concave to convex, which implies that $g(y)$ changes from convex
to concave, since the inverse of a convex function is concave; the root(s) of
$g^{\prime\prime}(y)$ are the point(s) at which this change occurs. Using
Wolfram Alpha111https://www.wolframalpha.com/ and matplotlib (Hunter:2007), we
observe that M4 is able to express inflection points, e.g.
$(a,b,c,\epsilon_{0},\epsilon_{\infty},x,y)=(1,1,-2,3/4,1/4,1/\sqrt{3},5/8$),
or
$(a,b,c,\epsilon_{0},\epsilon_{\infty},x,y)=(2,1,-3,2/3,1/3,(-5/6+\sqrt{3}/2)^{1/3},1/\sqrt{3})$.

###  A.9 Vision

Using the scaling laws benchmark of Alabdulmohsi2022revisiting, we evaluate
how well various functional forms extrapolate performance on vision tasks as
training dataset size increases. In this large-scale vision subset of the
benchmark, the tasks that are evaluated are error rate on each of various few-
shot downstream image classification (IC) tasks; the downstream tasks are:
Birds 200 (welinder2010caltech), Caltech101 (fei2004learning), CIFAR-100
(krizhevsky2009learning), and ImageNet (deng2009imagenet). The following
architectures of various sizes are pretrained on subsets of JFT-300M
(sun2017revisiting): big-transfer residual neural networks (BiT)
(kolesnikov2020big), MLP mixers (MiX) (tolstikhin2021mlp), and vision
transformers (ViT) (dosovitskiy2020image). As can be seen in Tables 1 and 4,
BNSL yields extrapolations with the lowest RMSLE (Root Mean Squared
Logarithmic Error) for 69.44% of tasks of any of the functional forms, while
the next best functional form performs the best on only 13.89% of the tasks.

To view all plots of BNSL on each of these tasks, see figures 16, 17, 18, 19
in Appendix A.24. To view all plots of M1, M2, M3, and M4 on each of these
tasks, see Appendix A.4 of Alabdulmohsi2022revisiting.

In Section A.13, we additionally show that BNSL yields accurate extrapolations
of performance on large-scale downstream vision tasks when amount of compute
used for (pre-)training is on the x-axis and compute is scaled in the manner
that is Pareto optimal with respect to the performance evaluation metric on
the y-axis (downstream accuracy in this case).

In Section A.14, we additionally show that BNSL yields accurate extrapolations
of the scaling behavior of diffusion generative models of images when amount
of compute used for (pre-)training is on the x-axis and compute is scaled in
the manner that is Pareto optimal with respect to the performance evaluation
metric on the y-axis (NLL and FID score in this case).

In Section A.15, we additionally show that BNSL yields accurate extrapolations
of the scaling behavior of generative models of video.

In Section A.21, we show that BNSL accurately extrapolates the scaling
behavior of the downstream performance of multimodal contrastive learning
(i.e. non-generative unsupervised learning).

In Section A.16, we additionally show that BNSL yields accurate extrapolations
of the scaling behavior when data is pruned Pareto optimally (such that each
point along the x-axis uses the subset of the dataset that yields the best
performance (y-axis value) for that dataset size (x-axis value)).

In Section A.17, we additionally show that BNSL yields accurate extrapolations
when upstream performance is on the x-axis and downstream performance is on
the y-axis.

In Section A.23, we additionally show that BNSL accurately extrapolates to
scales that are an order of magnitude larger than the maximum (along the
x-axis) of the points used for fitting.

Task |  Model |  |  M1 $\downarrow$ |  M2 $\downarrow$ |  M3 $\downarrow$ |  M4 $\downarrow$ |  BNSL $\downarrow$
---|---|---|---|---|---|---|---
Birds 200 10-shot |  BiT/101/3 |  |  9.13e-2 ± 2.8e-3 |  9.13e-2 ± 2.8e-3 |  9.13e-2 ± 2.8e-3 |  2.95e-2 ± 1.3e-3 |  1.76e-2 ± 1.1e-3
Birds 200 10-shot |  BiT/50/1 |  |  6.88e-2 ± 7.5e-4 |  6.88e-2 ± 7.5e-4 |  5.24e-2 ± 6.2e-4 |  2.66e-2 ± 5.3e-4 |  1.19e-2 ± 3.5e-4
Birds 200 10-shot |  MiX/B/16 |  |  9.15e-2 ± 1.1e-3 |  9.15e-2 ± 1.1e-3 |  3.95e-2 ± 7.0e-4 |  4.62e-2 ± 8.2e-4 |  3.04e-2 ± 6.9e-4
Birds 200 10-shot |  MiX/L/16 |  |  5.51e-2 ± 1.4e-3 |  5.51e-2 ± 1.4e-3 |  5.51e-2 ± 1.4e-3 |  5.15e-2 ± 1.7e-3 |  1.85e-2 ± 8.9e-4
Birds 200 10-shot |  ViT/B/16 |  |  6.77e-2 ± 1.1e-3 |  6.77e-2 ± 1.1e-3 |  3.52e-2 ± 8.1e-4 |  1.51e-2 ± 6.2e-4 |  1.69e-2 ± 7.0e-4
Birds 200 10-shot |  ViT/S/16 |  |  3.95e-2 ± 1.2e-3 |  3.95e-2 ± 1.2e-3 |  3.74e-2 ± 1.1e-3 |  1.85e-2 ± 7.9e-4 |  1.09e-2 ± 6.1e-4
Birds 200 25-shot |  BiT/101/3 |  |  9.41e-2 ± 3.2e-3 |  9.41e-2 ± 3.2e-3 |  9.41e-2 ± 3.2e-3 |  6.38e-2 ± 2.0e-3 |  1.55e-2 ± 1.3e-3
Birds 200 25-shot |  BiT/50/1 |  |  1.10e-1 ± 1.0e-3 |  7.29e-2 ± 8.0e-4 |  1.52e-2 ± 4.9e-4 |  1.97e-2 ± 5.6e-4 |  1.33e-2 ± 4.4e-4
Birds 200 25-shot |  MiX/B/16 |  |  1.40e-1 ± 1.9e-3 |  1.40e-1 ± 1.9e-3 |  6.93e-2 ± 1.2e-3 |  2.11e-2 ± 6.9e-4 |  1.64e-2 ± 6.6e-4
Birds 200 25-shot |  MiX/L/16 |  |  1.12e-1 ± 2.0e-3 |  1.12e-1 ± 2.0e-3 |  1.12e-1 ± 2.0e-3 |  5.44e-2 ± 1.8e-3 |  2.08e-2 ± 1.1e-3
Birds 200 25-shot |  ViT/B/16 |  |  9.02e-2 ± 1.6e-3 |  9.02e-2 ± 1.6e-3 |  3.75e-2 ± 1.0e-3 |  1.51e-2 ± 5.7e-4 |  1.62e-2 ± 6.1e-4
Birds 200 25-shot |  ViT/S/16 |  |  5.06e-2 ± 1.4e-3 |  5.06e-2 ± 1.4e-3 |  4.96e-2 ± 1.4e-3 |  4.02e-2 ± 1.2e-3 |  1.03e-2 ± 6.6e-4
Birds 200 5-shot |  BiT/101/3 |  |  8.17e-2 ± 2.0e-3 |  8.17e-2 ± 2.0e-3 |  8.17e-2 ± 2.0e-3 |  3.38e-2 ± 1.3e-3 |  1.81e-2 ± 8.2e-4
Birds 200 5-shot |  BiT/50/1 |  |  5.44e-2 ± 5.6e-4 |  5.44e-2 ± 5.6e-4 |  5.44e-2 ± 5.6e-4 |  2.59e-2 ± 5.4e-4 |  1.34e-2 ± 3.7e-4
Birds 200 5-shot |  MiX/B/16 |  |  8.27e-2 ± 1.0e-3 |  8.27e-2 ± 1.0e-3 |  5.49e-2 ± 7.8e-4 |  2.14e-2 ± 5.3e-4 |  1.39e-2 ± 4.1e-4
Birds 200 5-shot |  MiX/L/16 |  |  5.68e-2 ± 1.4e-3 |  5.68e-2 ± 1.4e-3 |  5.68e-2 ± 1.4e-3 |  3.20e-2 ± 9.7e-4 |  1.85e-2 ± 6.4e-4
Birds 200 5-shot |  ViT/B/16 |  |  3.40e-2 ± 8.9e-4 |  3.40e-2 ± 8.9e-4 |  3.40e-2 ± 8.9e-4 |  1.65e-2 ± 6.7e-4 |  1.36e-2 ± 5.8e-4
Birds 200 5-shot |  ViT/S/16 |  |  2.75e-2 ± 7.9e-4 |  2.75e-2 ± 7.9e-4 |  2.75e-2 ± 7.9e-4 |  1.20e-2 ± 5.2e-4 |  7.39e-3 ± 4.5e-4
CIFAR-100 10-shot |  BiT/101/3 |  |  8.57e-2 ± 3.8e-3 |  8.57e-2 ± 3.8e-3 |  8.25e-2 ± 3.7e-3 |  4.77e-2 ± 3.0e-3 |  2.58e-2 ± 2.3e-3
CIFAR-100 10-shot |  BiT/50/1 |  |  7.44e-2 ± 1.5e-3 |  1.24e-2 ± 5.8e-4 |  2.08e-2 ± 7.2e-4 |  1.24e-2 ± 5.8e-4 |  1.83e-2 ± 8.3e-4
CIFAR-100 10-shot |  MiX/B/16 |  |  8.77e-2 ± 1.9e-3 |  8.77e-2 ± 1.9e-3 |  2.71e-2 ± 1.2e-3 |  2.37e-2 ± 9.9e-4 |  2.44e-2 ± 9.5e-4
CIFAR-100 10-shot |  MiX/L/16 |  |  1.05e-1 ± 3.1e-3 |  1.05e-1 ± 3.1e-3 |  4.85e-2 ± 2.6e-3 |  4.97e-2 ± 1.6e-3 |  4.75e-2 ± 2.6e-3
CIFAR-100 10-shot |  ViT/B/16 |  |  8.98e-2 ± 2.0e-3 |  8.98e-2 ± 2.0e-3 |  8.98e-2 ± 2.0e-3 |  4.98e-2 ± 1.7e-3 |  3.71e-2 ± 1.4e-3
CIFAR-100 10-shot |  ViT/S/16 |  |  6.84e-2 ± 1.1e-3 |  2.11e-2 ± 6.6e-4 |  3.35e-2 ± 8.6e-4 |  2.54e-2 ± 7.5e-4 |  2.57e-2 ± 7.5e-4
CIFAR-100 25-shot |  BiT/101/3 |  |  8.77e-2 ± 5.6e-3 |  8.77e-2 ± 5.6e-3 |  4.44e-2 ± 3.5e-3 |  3.40e-2 ± 2.7e-3 |  2.88e-2 ± 3.0e-3
CIFAR-100 25-shot |  BiT/50/1 |  |  7.31e-2 ± 2.0e-3 |  2.35e-2 ± 1.5e-3 |  3.65e-2 ± 1.8e-3 |  2.35e-2 ± 1.5e-3 |  1.89e-2 ± 1.1e-3
CIFAR-100 25-shot |  MiX/B/16 |  |  1.08e-1 ± 2.3e-3 |  4.75e-2 ± 1.6e-3 |  2.10e-2 ± 9.4e-4 |  2.24e-2 ± 9.9e-4 |  2.67e-2 ± 1.1e-3
CIFAR-100 25-shot |  MiX/L/16 |  |  9.79e-2 ± 2.2e-3 |  9.79e-2 ± 2.2e-3 |  3.67e-2 ± 1.7e-3 |  2.98e-2 ± 1.4e-3 |  3.45e-2 ± 1.6e-3
CIFAR-100 25-shot |  ViT/B/16 |  |  1.07e-1 ± 1.9e-3 |  1.07e-1 ± 1.9e-3 |  6.54e-2 ± 1.6e-3 |  4.80e-2 ± 1.4e-3 |  3.02e-2 ± 4.5e-3
CIFAR-100 25-shot |  ViT/S/16 |  |  8.03e-2 ± 1.2e-3 |  2.19e-2 ± 7.4e-4 |  3.13e-2 ± 8.4e-4 |  2.27e-2 ± 7.1e-4 |  2.14e-2 ± 6.9e-4
CIFAR-100 5-shot |  BiT/101/3 |  |  5.94e-2 ± 3.2e-3 |  5.94e-2 ± 3.2e-3 |  5.94e-2 ± 3.2e-3 |  3.30e-2 ± 2.4e-3 |  3.78e-2 ± 2.6e-3
CIFAR-100 5-shot |  BiT/50/1 |  |  4.87e-2 ± 1.3e-3 |  4.87e-2 ± 1.3e-3 |  1.69e-2 ± 8.8e-4 |  1.87e-2 ± 8.9e-4 |  1.45e-2 ± 8.7e-4
CIFAR-100 5-shot |  MiX/B/16 |  |  7.07e-2 ± 1.2e-3 |  7.07e-2 ± 1.2e-3 |  2.78e-2 ± 8.4e-4 |  1.76e-2 ± 6.6e-4 |  1.70e-2 ± 6.3e-4
CIFAR-100 5-shot |  MiX/L/16 |  |  7.06e-2 ± 1.6e-3 |  7.06e-2 ± 1.6e-3 |  4.17e-2 ± 1.4e-3 |  3.32e-2 ± 1.2e-3 |  2.77e-2 ± 1.0e-3
CIFAR-100 5-shot |  ViT/B/16 |  |  6.27e-2 ± 1.6e-3 |  6.27e-2 ± 1.6e-3 |  6.27e-2 ± 1.6e-3 |  4.30e-2 ± 1.3e-3 |  2.82e-2 ± 1.0e-3
CIFAR-100 5-shot |  ViT/S/16 |  |  6.93e-2 ± 1.2e-3 |  2.84e-2 ± 8.2e-4 |  3.88e-2 ± 8.0e-4 |  3.16e-2 ± 7.5e-4 |  3.50e-2 ± 9.2e-3
Caltech101 10-shot |  BiT/101/3 |  |  3.07e-1 ± 2.0e-2 |  3.07e-1 ± 2.0e-2 |  1.51e-1 ± 1.3e-2 |  1.00e-1 ± 1.1e-2 |  4.75e-2 ± 8.1e-3
Caltech101 10-shot |  BiT/50/1 |  |  3.29e-1 ± 1.6e-2 |  7.68e-2 ± 5.0e-3 |  1.13e-1 ± 6.0e-3 |  6.01e-2 ± 4.4e-3 |  1.77e-2 ± 2.5e-3
Caltech101 10-shot |  MiX/B/16 |  |  1.35e-1 ± 1.4e-2 |  1.35e-1 ± 1.4e-2 |  1.35e-1 ± 1.4e-2 |  1.92e-1 ± 1.6e-2 |  2.04e-1 ± 9.7e-3
Caltech101 10-shot |  MiX/L/16 |  |  1.25e-1 ± 1.3e-2 |  1.25e-1 ± 1.3e-2 |  1.25e-1 ± 1.3e-2 |  1.30e-1 ± 1.2e-2 |  2.13e-1 ± 1.5e-2
Caltech101 10-shot |  ViT/B/16 |  |  7.76e-2 ± 4.3e-3 |  7.76e-2 ± 4.3e-3 |  3.11e-2 ± 3.0e-3 |  5.75e-2 ± 4.4e-3 |  4.02e-2 ± 3.9e-3
Caltech101 10-shot |  ViT/S/16 |  |  1.95e-1 ± 6.0e-3 |  3.41e-2 ± 2.9e-3 |  2.40e-2 ± 2.0e-3 |  3.41e-2 ± 2.9e-3 |  2.40e-2 ± 2.0e-3
Caltech101 25-shot |  BiT/101/3 |  |  1.15e-1 ± 6.5e-3 |  1.15e-1 ± 6.5e-3 |  1.15e-1 ± 6.5e-3 |  1.15e-1 ± 6.5e-3 |  9.86e-2 ± 8.0e-3
Caltech101 25-shot |  BiT/50/1 |  |  3.60e-1 ± 1.9e-2 |  8.80e-2 ± 5.5e-3 |  1.43e-1 ± 7.6e-3 |  4.76e-2 ± 3.6e-3 |  1.55e-2 ± 1.6e-3
Caltech101 25-shot |  MiX/B/16 |  |  8.28e-2 ± 1.2e-2 |  8.28e-2 ± 1.2e-2 |  8.28e-2 ± 1.2e-2 |  1.65e-1 ± 1.7e-2 |  1.93e-1 ± 1.3e-2
Caltech101 25-shot |  MiX/L/16 |  |  9.66e-2 ± 1.0e-2 |  9.66e-2 ± 1.0e-2 |  9.66e-2 ± 1.0e-2 |  9.66e-2 ± 1.0e-2 |  1.49e-1 ± 1.3e-2
Caltech101 25-shot |  ViT/B/16 |  |  1.03e-1 ± 5.6e-3 |  3.33e-2 ± 2.5e-3 |  4.46e-2 ± 3.6e-3 |  3.33e-2 ± 2.5e-3 |  3.95e-2 ± 5.4e-3
Caltech101 25-shot |  ViT/S/16 |  |  1.77e-1 ± 5.4e-3 |  3.79e-2 ± 3.1e-3 |  2.80e-2 ± 1.8e-3 |  3.79e-2 ± 3.1e-3 |  3.29e-2 ± 2.1e-3
Caltech101 5-shot |  BiT/101/3 |  |  2.12e-1 ± 1.2e-2 |  2.12e-1 ± 1.2e-2 |  2.12e-1 ± 1.2e-2 |  1.65e-1 ± 9.4e-3 |  1.87e-2 ± 4.3e-3
Caltech101 5-shot |  BiT/50/1 |  |  2.34e-1 ± 6.1e-3 |  4.13e-2 ± 2.1e-3 |  1.61e-2 ± 1.3e-3 |  4.69e-2 ± 2.1e-3 |  4.10e-2 ± 2.1e-3
Caltech101 5-shot |  MiX/B/16 |  |  2.43e-1 ± 1.2e-2 |  2.43e-1 ± 1.2e-2 |  2.35e-1 ± 1.1e-2 |  7.28e-2 ± 4.3e-3 |  1.92e-2 ± 1.9e-3
Caltech101 5-shot |  MiX/L/16 |  |  1.38e-1 ± 9.7e-3 |  1.38e-1 ± 9.7e-3 |  1.38e-1 ± 9.7e-3 |  1.37e-1 ± 9.9e-3 |  1.63e-1 ± 1.1e-2
Caltech101 5-shot |  ViT/B/16 |  |  1.10e-1 ± 6.3e-3 |  1.10e-1 ± 6.3e-3 |  6.02e-2 ± 4.7e-3 |  6.81e-2 ± 4.8e-3 |  3.87e-2 ± 3.4e-3
Caltech101 5-shot |  ViT/S/16 |  |  1.90e-1 ± 4.7e-3 |  3.82e-2 ± 2.6e-3 |  5.04e-2 ± 2.9e-3 |  3.82e-2 ± 2.6e-3 |  2.78e-2 ± 1.8e-3
ImageNet 10-shot |  BiT/101/3 |  |  1.27e-1 ± 2.0e-3 |  1.27e-1 ± 2.0e-3 |  7.36e-2 ± 1.1e-3 |  3.06e-2 ± 7.0e-4 |  6.65e-3 ± 3.8e-4
ImageNet 10-shot |  BiT/50/1 |  |  9.54e-2 ± 7.2e-4 |  9.54e-2 ± 7.2e-4 |  5.75e-3 ± 2.0e-4 |  1.86e-2 ± 2.8e-4 |  3.84e-3 ± 1.5e-4
ImageNet 10-shot |  MiX/B/16 |  |  9.34e-2 ± 7.9e-4 |  9.34e-2 ± 7.9e-4 |  3.37e-2 ± 2.9e-4 |  2.32e-2 ± 3.0e-4 |  4.22e-3 ± 1.5e-4
ImageNet 10-shot |  MiX/L/16 |  |  9.83e-2 ± 1.3e-3 |  9.83e-2 ± 1.3e-3 |  9.83e-2 ± 1.3e-3 |  4.01e-3 ± 1.9e-4 |  4.33e-3 ± 1.8e-4
ImageNet 10-shot |  ViT/B/16 |  |  4.62e-2 ± 7.1e-4 |  4.62e-2 ± 7.1e-4 |  4.62e-2 ± 7.1e-4 |  1.44e-2 ± 3.0e-4 |  5.70e-3 ± 2.0e-4
ImageNet 10-shot |  ViT/S/16 |  |  4.74e-2 ± 5.6e-4 |  4.74e-2 ± 5.6e-4 |  1.66e-2 ± 2.5e-4 |  7.18e-3 ± 2.0e-4 |  3.71e-3 ± 1.4e-4
ImageNet 25-shot |  BiT/101/3 |  |  1.42e-1 ± 2.3e-3 |  1.42e-1 ± 2.3e-3 |  6.67e-2 ± 9.1e-4 |  3.31e-2 ± 8.7e-4 |  4.76e-3 ± 2.8e-4
ImageNet 25-shot |  BiT/50/1 |  |  1.17e-1 ± 9.2e-4 |  1.17e-1 ± 9.2e-4 |  4.06e-3 ± 1.7e-4 |  1.84e-2 ± 2.6e-4 |  4.67e-3 ± 1.6e-4
ImageNet 25-shot |  MiX/B/16 |  |  9.59e-2 ± 9.3e-4 |  9.59e-2 ± 9.3e-4 |  5.39e-2 ± 4.9e-4 |  2.04e-2 ± 3.1e-4 |  4.17e-3 ± 1.7e-4
ImageNet 25-shot |  MiX/L/16 |  |  1.03e-1 ± 1.3e-3 |  1.03e-1 ± 1.3e-3 |  1.03e-1 ± 1.3e-3 |  6.33e-3 ± 2.2e-4 |  7.60e-3 ± 2.6e-4
ImageNet 25-shot |  ViT/B/16 |  |  5.17e-2 ± 8.8e-4 |  5.17e-2 ± 8.8e-4 |  5.17e-2 ± 8.8e-4 |  1.52e-2 ± 3.8e-4 |  4.96e-3 ± 2.0e-4
ImageNet 25-shot |  ViT/S/16 |  |  5.52e-2 ± 4.4e-4 |  4.12e-2 ± 3.4e-4 |  9.65e-3 ± 2.3e-4 |  7.78e-3 ± 2.1e-4 |  6.11e-3 ± 2.4e-4
ImageNet 5-shot |  BiT/101/3 |  |  9.24e-2 ± 1.4e-3 |  9.24e-2 ± 1.4e-3 |  9.24e-2 ± 1.4e-3 |  2.09e-2 ± 7.9e-4 |  8.05e-3 ± 5.0e-4
ImageNet 5-shot |  BiT/50/1 |  |  8.95e-2 ± 6.7e-4 |  8.95e-2 ± 6.7e-4 |  1.53e-2 ± 2.2e-4 |  1.11e-2 ± 2.3e-4 |  7.94e-3 ± 2.1e-4
ImageNet 5-shot |  MiX/B/16 |  |  9.09e-2 ± 7.2e-4 |  9.09e-2 ± 7.2e-4 |  3.01e-2 ± 2.8e-4 |  1.95e-2 ± 2.7e-4 |  6.49e-3 ± 2.2e-4
ImageNet 5-shot |  MiX/L/16 |  |  7.99e-2 ± 9.7e-4 |  7.99e-2 ± 9.7e-4 |  7.99e-2 ± 9.7e-4 |  9.92e-3 ± 4.5e-4 |  5.68e-3 ± 2.4e-4
ImageNet 5-shot |  ViT/B/16 |  |  4.11e-2 ± 6.3e-4 |  4.11e-2 ± 6.3e-4 |  4.11e-2 ± 6.3e-4 |  1.55e-2 ± 2.8e-4 |  1.29e-2 ± 2.7e-4
ImageNet 5-shot |  ViT/S/16 |  |  4.20e-2 ± 4.1e-4 |  4.20e-2 ± 4.1e-4 |  2.40e-2 ± 2.6e-4 |  8.02e-3 ± 1.9e-4 |  4.72e-3 ± 1.6e-4
Table 4:  Extrapolation Results on scaling behavior of Downstream Vision
Tasks. See Section A.9 for more details. Numbers for M1, M2, M3, and M4
obtained via correspondence with authors of Alabdulmohsi2022revisiting.

###  A.10 Language

Using the scaling laws benchmark of Alabdulmohsi2022revisiting, we evaluate
how well various functional forms extrapolate performance on language tasks as
the training dataset size increases. In this large-scale language subset of
the benchmark, the tasks that are evaluated are error rates on each of the
various downstream tasks from the BIG-Bench (BB) (srivastava2022beyond)
benchmark and upstream test cross-entropy of various models trained to do
language modeling (LM) and neural machine translation (NMT). All LM and BB
tasks use a decoder-only language model. As can be seen in Tables 1 and 5,
BNSL yields extrapolations with the lowest RMSLE (Root Mean Squared
Logarithmic Error) for 75% of tasks of any of the functional forms, while the
next best functional form performs the best on only 10% of the tasks.

To view all plots of the BNSL on each of these tasks, see Figures 20, 21, 22
in Appendix A.24.

To view plots of M1, M2, M3, and M4 on these tasks, see Figure 8 of
Alabdulmohsi2022revisiting.

In Section A.18, we additionally show that BNSL yields accurate extrapolations
of performance on large-scale downstream language tasks when number of model
parameters is on the x-axis.

In Section A.22, we additionally show that BNSL yields accurate extrapolations
of performance on large-scale downstream audio (speech recognition) tasks.

Domain |  Task |  Model |  |  M1 $\downarrow$ |  M2 $\downarrow$ |  M3 $\downarrow$ |  M4 $\downarrow$ |  BNSL $\downarrow$
---|---|---|---|---|---|---|---|---
BB |  date understanding, 1-shot |  2.62e+8 Param |  |  3.19e-2 ± 9.6e-4 |  3.19e-2 ± 9.6e-4 |  4.67e-3 ± 1.4e-4 |  3.19e-2 ± 9.6e-4 |  3.40e-3 ± 7.9e-5
BB |  date understanding, 2-shot |  2.62e+8 Param |  |  2.86e-2 ± 6.2e-4 |  2.86e-2 ± 6.2e-4 |  4.83e-3 ± 4.1e-4 |  2.86e-2 ± 6.2e-4 |  4.38e-3 ± 4.0e-4
BB |  linguistic mappings, 1-shot |  2.62e+8 Param |  |  1.66e-2 ± 5.5e-4 |  1.62e-2 ± 5.4e-4 |  1.66e-2 ± 5.5e-4 |  1.33e-2 ± 3.8e-4 |  1.13e-2 ± 2.2e-4
BB |  linguistic mappings, 2-shot |  2.62e+8 Param |  |  1.70e-2 ± 6.5e-4 |  1.70e-2 ± 6.5e-4 |  1.70e-2 ± 6.5e-4 |  1.06e-2 ± 5.1e-4 |  9.51e-3 ± 5.1e-4
BB |  mult data wrangling, 1-shot |  2.62e+8 Param |  |  1.07e-2 ± 1.0e-3 |  1.07e-2 ± 1.0e-3 |  1.07e-2 ± 1.0e-3 |  6.66e-3 ± 7.3e-4 |  6.39e-3 ± 4.6e-4
BB |  mult data wrangling, 2-shot |  2.62e+8 Param |  |  1.57e-2 ± 1.5e-3 |  1.57e-2 ± 1.5e-3 |  1.57e-2 ± 1.5e-3 |  5.79e-3 ± 7.0e-4 |  2.67e-3 ± 2.7e-4
BB |  qa wikidata, 1-shot |  2.62e+8 Param |  |  4.27e-3 ± 8.9e-4 |  4.32e-3 ± 8.2e-4 |  4.27e-3 ± 8.9e-4 |  4.32e-3 ± 8.2e-4 |  4.68e-3 ± 7.3e-4
BB |  qa wikidata, 2-shot |  2.62e+8 Param |  |  4.39e-3 ± 7.0e-4 |  4.66e-3 ± 6.4e-4 |  4.39e-3 ± 7.0e-4 |  9.02e-3 ± 6.9e-4 |  8.05e-3 ± 7.3e-4
BB |  unit conversion, 1-shot |  2.62e+8 Param |  |  8.30e-3 ± 4.4e-4 |  8.30e-3 ± 4.4e-4 |  1.48e-3 ± 2.7e-4 |  4.79e-3 ± 3.4e-4 |  1.07e-2 ± 2.5e-4
BB |  unit conversion, 2-shot |  2.62e+8 Param |  |  1.07e-2 ± 4.4e-4 |  1.07e-2 ± 4.4e-4 |  7.50e-3 ± 5.5e-4 |  7.55e-3 ± 5.1e-4 |  7.02e-3 ± 3.9e-4
LM |  upstream test cross-entropy |  1.07e+9 Param |  |  1.71e-2 ± 6.0e-4 |  1.66e-3 ± 5.1e-5 |  4.50e-3 ± 5.9e-5 |  1.28e-3 ± 3.9e-5 |  9.71e-4 ± 3.2e-5
LM |  upstream test cross-entropy |  4.53e+8 Param |  |  1.65e-2 ± 6.6e-4 |  7.41e-4 ± 9.8e-5 |  6.58e-4 ± 6.6e-5 |  7.41e-4 ± 9.8e-5 |  5.86e-4 ± 7.7e-5
LM |  upstream test cross-entropy |  2.62e+8 Param |  |  1.55e-2 ± 7.2e-4 |  9.20e-4 ± 9.7e-5 |  3.97e-3 ± 1.3e-4 |  9.20e-4 ± 9.7e-5 |  7.90e-4 ± 5.1e-5
LM |  upstream test cross-entropy |  1.34e+8 Param |  |  1.43e-2 ± 4.8e-4 |  1.46e-3 ± 6.8e-5 |  6.46e-4 ± 5.1e-5 |  1.46e-3 ± 6.8e-5 |  9.01e-4 ± 5.5e-5
LM |  upstream test cross-entropy |  1.68e+7 Param |  |  6.37e-3 ± 9.4e-5 |  3.03e-4 ± 1.2e-5 |  1.56e-3 ± 3.5e-5 |  3.03e-4 ± 1.2e-5 |  4.34e-4 ± 1.8e-5
NMT |  upstream test cross-entropy |  28 Enc, 6 Dec |  |  1.71e-1 ± 0 |  5.64e-2 ± 0 |  3.37e-2 ± 0 |  1.81e-2 ± 0 |  1.69e-2 ± 0
NMT |  upstream test cross-entropy |  6 Enc, 28 Dec |  |  2.34e-1 ± 0 |  5.27e-2 ± 0 |  1.65e-2 ± 0 |  4.44e-2 ± 0 |  1.56e-2 ± 0
NMT |  upstream test cross-entropy |  6 Enc, 6 Dec |  |  2.62e-1 ± 0 |  3.84e-2 ± 0 |  8.92e-2 ± 0 |  2.05e-2 ± 0 |  1.37e-3 ± 0
NMT |  upstream test cross-entropy |  Dec-only, LM |  |  2.52e-1 ± 0 |  1.03e-2 ± 0 |  3.28e-2 ± 0 |  8.43e-3 ± 0 |  7.33e-3 ± 0
NMT |  upstream test cross-entropy |  Transformer-Enc, LSTM-Dec |  |  1.90e-1 ± 0 |  1.26e-2 ± 0 |  6.32e-2 ± 0 |  1.26e-2 ± 0 |  8.30e-3 ± 0
Table 5:  Extrapolation Results on scaling behavior of Language Tasks. See
Section A.10 for more details. Numbers for M1, M2, M3, and M4 were obtained
via correspondence with authors of Alabdulmohsi2022revisiting. BB stands for
BIG-Bench (srivastava2022beyond). NMT stands for Neural Machine Translation.
LM stands for Language Modeling.

###  A.11 Reinforcement Learning

We show that BNSL accurately models and extrapolates the scaling behaviors of
various multi-agent and single-agent reinforcement learning algorithms trained
in various environments. In the top left plot and top right plot and bottom
left plot of Figure 3, BNSL accurately models and extrapolates the scaling
behavior of the AlphaZero algorithm trained to play the game Connect Four from
Figure 4 and Figure 5 and Figure 3 respectively of neumann2022scaling; the
x-axes respectively are compute (FLOPs) used for training, training dataset
size (states), and number of model parameters. In Figure 3 bottom right, BNSL
accurately models and extrapolates the scaling behavior of the Proximal Policy
Optimization (PPO) algorithm (schulman2017proximal) trained to play the
Procgen (cobbe2020leveraging) game called Heist.

In Section A.19, we find BNSL accurately extrapolates the scaling behavior of
a pretrained language model finetuned (i.e. aligned) via Reinforcement
Learning from Human Feedback (RLHF) to be helpful from Figure 1 of
bai2022training.

Figure 3:  Extrapolation of BNSL on Reinforcement Learning Scaling
Experimental Data. Experimental data of the top left plot and top right plot
and bottom left plot is from Figure 4 and Figure 5 and Figure 3 respectively
of neumann2022scaling. Experimental Data of the bottom right plot is from
Figure 2 of cobbe2020leveraging. Top left plot is the compute-optimal Pareto
frontier. See Section A.11 for more details.

###  A.12 Non-Monotonic Scaling

We show that BNSL accurately models and extrapolates non-monotonic scaling
behaviors that are exhibited by Transformers (vaswani2017attention) in double
descent (nakkiran2021deep) in Figure 4. Various other functional forms are
mathematically incapable of expressing non-monotonic behaviors (as shown in
Section A.8).

Figure 4:  Extrapolation of BNSL on Double Descent. Both plots are of
transformers trained to do neural machine translation via minimizing cross-
entropy. Experimental data of left figure is obtained from Figure 8 top of
nakkiran2021deep; “Model Width” on the x-axis refers to embedding dimension
$d_{model}$ of the transformer. Experimental data of the right figure is
obtained from Figure 11b of nakkiran2021deep. The plot on the left contains
two breaks of a BNSL fit to the black points. See Section A.12 for more
details.

###  A.13 Extrapolation Results for Downstream Vision Tasks when training runs
are scaled to be compute-optimal.

Task | Model | M3 $\downarrow$ | BNSL $\downarrow$
---|---|---|---
ImageNet 10-Shot | ViT | 1.91e-2 ± 6.48e-3 | 9.79e-3 ± 4.70e-3
ImageNet Finetune | ViT | 1.14e-2 ± 2.42e-3 | 9.37e-3 ± 2.60e-3
Table 6:  Extrapolation Results for Downstream Vision Tasks when training runs
are scaled using the compute-optimal scaling (i.e. Pareto frontier) with
respect to downstream performance. Experimental data obtained from Figure 2 of
DBLP:journals/corr/abs-2106-04560. See Section A.13 for more details.

Figure 5:  Extrapolation Results of BNSL for Downstream Vision Tasks when
training runs are scaled to be compute-optimal. Experimental data obtained
from Figure 2 of DBLP:journals/corr/abs-2106-04560. See Section A.13 for more
details.

In Figure 5 via fitting BNSL, we additionally obtain accurate extrapolations
of scaling behavior of large-scale downstream vision tasks when compute
(FLOPs) used for (pre-)training is on the x-axis and compute is scaled in the
manner that is Pareto optimal with respect to the performance evaluation
metric (downstream accuracy in this case). The experimental scaling data was
obtained from Figure 2 of DBLP:journals/corr/abs-2106-04560, and a result in
Table 6 we compare extrapolation of BNSL to the extrapolation of M3 (which was
proposed in DBLP:journals/corr/abs-2106-04560); we find that BNSL that yields
extrapolations of scaling behavior that are more accurate on these tasks.

###  A.14 Extrapolation Results for Diffusion Generative Models of Images

Figure 6:  Extrapolation Results of BNSL for scaling behavior of Diffusion
Generative Models of Images. Frechet Inception Distance (FID) score is on the
y-axis in the left plot. Negative log-likelihood (NLL) is the y-axis in the
right plot. For both plots, compute used for training is on the x-axis and
Imagenet 64x64 is the evaluation dataset. Experimental data of scaling
behavior obtained from Figure 10 of nichol2021improved. See Section A.14 for
more details.

In Figure 6, we show that BNSL accurately extrapolates the scaling behavior of
Diffusion Generative Models of Images from Figure 10 of nichol2021improved
when Negative Log-likelihood (NLL) or Frechet Inception Distance (FID) score
is on the y-axis and compute used for training is on the x-axis; compute is
scaled in the manner that is Pareto optimal with respect to the performance
evaluation metric on the y-axis.

###  A.15 Extrapolation Results for Generative Models of Video

Figure 7:  Extrapolation Results of BNSL for scaling behavior of Generative
Models of Video. Upstream Test Cross-Entropy is on the y-axis. Videos scraped
from the web are the evaluation dataset. During training, compute (used for
training autoregressive transformer) on the x-axis is scaled in the manner
that is Pareto optimal with respect to the performance evaluation metric on
the y-axis. Experimental data of scaling behavior obtained from top right plot
of Figure 5 of 2020arXiv201014701H. See Section A.15 for more details.

In Figure 7, we show that BNSL accurately extrapolates the scaling behavior of
generative models of video. Each frame is downsampled to a pixel resolution of
64x64; each frame is then tokenized via a pretrained 16x16 VQVAE
(van2017neural) to obtain 256 tokens per frame. 16 consecutive frames are then
input to an autoregressive transformer as a length 4096 (16x16x16) sequence.
The dataset is 100 hours of videos scraped from the web. See section 2 of
2020arXiv201014701H for more details.

###  A.16 Extrapolation Results when Data is Pruned Pareto Optimally

Figure 8:  Extrapolation Results of BNSL for scaling behavior when data is
pruned Pareto optimally (such that each point along the x-axis uses the subset
of the dataset that yields the best performance (y-axis value) for that
dataset size (x-axis value)). Experimental data of scaling behavior obtained
from Figure 3D of https://doi.org/10.48550/arxiv.2206.14486. See Section A.16
for more details.

In Figure 8, we show that BNSL accurately extrapolates the scaling behavior
when data is pruned Pareto optimally (such that each point along the x-axis
uses the subset of the dataset that yields the best performance (y-axis value)
for that dataset size (x-axis value)) from Figure 3D of
https://doi.org/10.48550/arxiv.2206.14486.

###  A.17 Extrapolation Results when Upstream Performance is on the x-axis

Figure 9:  Extrapolation Results of BNSL for scaling behavior when Upstream
Performance is on the x-axis and Downstream Performance is on the y-axis.
Experimental data of scaling behavior obtained from Figure 5 of
abnar2021exploring. The upstream task is supervised pretraining of ViT
(dosovitskiy2020image) on subsets of JFT-300M (sun2017revisiting). The
Downstream Task is 20-shot ImageNet classification. See Section A.17 for more
details.

In Figure 9, we show that BNSL accurately extrapolates the scaling behavior
when upstream performance is on the x-axis and downstream performance is on
the y-axis. The upstream task is supervised pretraining of ViT
(dosovitskiy2020image) on subsets of JFT-300M (sun2017revisiting). The
downstream task is 20-shot ImageNet classification. The experimental data of
this scaling behavior is obtained from Figure 5 of abnar2021exploring.

###  A.18 Extrapolation Results for Downstream Language Tasks when Number of
Model Parameters is on the x-axis.

Figure 10:  Extrapolation Results of BNSL for Downstream Language Tasks when
Number of Model Parameters is on the x-axis. “Few-Shot” in plot title means
few-shot prompting is used (and “One-Shot” in plot title means one-shot
prompting is used) for that downstream evaluation as described in GPT-3 arXiv
paper (brown2020language). Experimental data obtained from Table H.1 of the
GPT-3 arXiv paper (brown2020language). See Section A.18 for more details.

We find in general for each of every modality that the variance between seeds
is higher when number of model parameters is on x-axis (as opposed to e.g.
training dataset size on the x-axis). Table H.1 of the GPT-3 arXiv paper
(brown2020language) release includes results for 8 numbers of model
parameters. In Figure 10, we include examples of when 8 numbers of model
parameters (7 for fitting, and largest held-out to evaluate extrapolation) are
sufficient for obtaining accurate downstream extrapolation from BNSL due to
variance between seeds being low enough. For many other downstream tasks with
number of model parameters on the x-axis, the variance between seeds is much
higher such that a number considerably larger than 7 points along the curve is
needed to obtain an accurate extrapolation.

###  A.19 Extrapolation Results for AI Alignment via RLHF

Figure 11:  Extrapolation Results for Downstream AI Alignment when Number of
Model Parameters is on the x-axis. Experimental data obtained from the Static
HH RLHF results from Figure 1 of bai2022training. See Section A.19 for more
details.

In Figure 11, we find BNSL accurately extrapolates the scaling behavior of a
pretrained language model finetuned (i.e. aligned) via Reinforcement Learning
from Human Feedback (RLHF) to be helpful from Figure 1 of bai2022training. The
y-axis is Elo score based on crowdworker preferences. The x-axis is the number
of model parameters that the language model contains.

###  A.20 Extrapolation Results for Robotics

Figure 12:  Extrapolation Results for Robotic control. Experimental data
obtained from the transporter (zeng2021transporter) model results from Table 1
of shridhar2021cliport. X-axis is number training demonstrations. Y-axis is
task success score (mean percentage) obtained via 100 evaluations. See Section
A.20 for more details.

In Figure 12, we find BNSL accurately extrapolates the scaling behavior of a
transporter (zeng2021transporter) model trained via imitation learning to do
robotic control tasks.

###  A.21 Extrapolation Results for Downstream Performance of Multimodal
Contrastive Learning (i.e. Non-Generative Unsupervised Learning)

Figure 13:  Extrapolation Results of BNSL for Downstream Performance of
Multimodal Contrastive Learning (i.e. Non-Generative Unsupervised Learning).
Experimental data of scaling behavior obtained from Table 10 and Table 11 in
arXiv version of radford2021learning. The upstream task is “Contrastive Image
Language Pretraining” (a.k.a. CLIP) of ResNets on a training dataset
consisting of hundreds of millions of image-text pairs. The x-axis is
GFLOPs/image (GigaFLOPs/image) of the forward-pass of model. The Downstream
Task is ImageNet classification (i.e. the y-axis of plot). The y-axis of left
plot is Zero-Shot Downstream. The y-axis of right plot is performance of model
with finetuned linear probe on it. See Section A.21 for more details.

In Figure 14, we show that BNSL accurately extrapolates the scaling behavior
of the Downstream Performance of Multimodal Contrastive Learning (i.e. Non-
Generative Unsupervised Learning).

###  A.22 Extrapolation Results for Downstream Performance on Audio Tasks

Figure 14:  Extrapolation Results of BNSL for Downstream Audio Tasks when
Number of Model Parameters is on the x-axis. Experimental data obtained from
the second plot of Figure 6 of Whisper paper (radford2022robust). The
downstream task in the plot is downstream zero shot multilingual speech
recognition performance on the FLEURS dataset of “Whisper” speech recognition
model pretrained on a dataset of 681,070 hours of audio. See Section A.22 for
more details.

In Figure 14, we show that BNSL accurately extrapolates the scaling behavior
of the Downstream Performance on Audio Tasks.

###  A.23 Extrapolation to Scales that are an Order of Magnitude larger than
the maximum (along the x-axis) of the points used for fitting

Figure 15:  Extrapolation Results of BNSL to Scales that are an Order of
Magnitude larger than the maximum (along the x-axis) of the points used for
fitting. Experimental data of scaling behavior obtained from Figure 5 of
abnar2021exploring. The upstream task is supervised pretraining of MLP mixers
(MiX) (tolstikhin2021mlp) on subsets (i.e. the x-axis of plot) of JFT-300M
(sun2017revisiting). The Downstream Task is n-shot ImageNet classification
(i.e. the y-axis of plot). See Section A.23 for more details.

In Figure 15, we show that BNSL accurately extrapolates to scales that are an
order of magnitude larger than the maximum (along the x-axis) of the points
used for fitting. The upstream task is supervised pretraining of MLP mixers
(MiX) (tolstikhin2021mlp) on subsets (i.e. the x-axis of plot) of JFT-300M
(sun2017revisiting). The downstream task is n-shot ImageNet classification
(i.e. the y-axis of plot). The experimental data of this scaling behavior is
obtained from Alabdulmohsi2022revisiting.

###  A.24 Plots of BNSL Extrapolations on Scaling Laws Benchmark of
Alabdulmohsi2022revisiting

Figure 16:  Birds 200

Figure 17:  CIFAR-100

Figure 18:  Caltech101. From eyeballing, we think the subset of Caltech101
with unsatisfactory extrapolations has unsatisfactory extrapolations due to
the maximum (along the x-axis) of the black point used for fitting being near
or before a break; this is accentuated by not having enough points for fitting
for the SciPy fitter to be able to determine whether the break is an actual
break or just noisy deviation. See Section 4 for more details on this
explanation.

Figure 19:  ImageNet

Figure 20:  BIG-Bench (BB). From eyeballing, we think the subset of BIG-Bench
with unsatisfactory extrapolations has unsatisfactory extrapolations due to
the maximum (along the x-axis) of the black point used for fitting being near
or before a break; this is accentuated by not having enough points for fitting
for the SciPy fitter to be able to determine whether the break is an actual
break or just noisy deviation. See Section 4 for more details on this
explanation.

Figure 21:  Neural Machine Translation (NMT)

Figure 22:  Language Modeling (LM)


# Some results on Ricatti Equations, Floquet Theory and Applications

Anderson L. A. de Araujo, Abílio Lemos, Alexandre Miranda Alves
Universidade Federal de Viçosa, CCE, Departamento de Matemática
Avenida PH Rolfs, s/n
CEP 36570-900, Viçosa, MG, Brasil
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
and

Kennedy Martins Pedroso
Universidade Federal de Juiz de Fora, Departamento de Matemática
Rua José Lourenço Kelmer, s/n, Bairro São Pedro
CEP: 36036-900, Juiz de Fora, MG, Brasil
E-mail<EMAIL_ADDRESS>
Abstract. In this paper, we present two new results to the classical Floquet
theory, which provides the Floquet multipliers for two classes of the planar
periodic system. One these results provides the Floquet multipliers
independently of the solution of system. To demonstrate the application of
these analytical results, we consider a cholera epidemic model with phage
dynamics and seasonality incorporated.

AMS Subject Classification 2010. Floquet theory; stability; epidemic models.

Keywords. 34C25; 34D20; 92D30.

##  1 Introduction

The Floquet theory is concerned with the study of the linear stability of
differential equations with periodic coefficients, see [2]. This theory
focuses on the concept of Floquet multipliers and offers a powerful means to
analyze nonautonomous, periodic differential equations. However, it is very
difficult to determine the Floquet multipliers of general linear periodic
systems. Except for a few special cases, which include second-order scalar
equations and systems of Hamiltonian type or canonical forms, very little is
known about the analysis of Floquet multipliers.

For a nonlinear periodic system, if it has a nonconstant periodic solution,
its stability can be analyzed by linearization about the periodic solution.
The variational system then becomes a linear periodic system, and its Floquet
multipliers provide useful information on the stability of the periodic
solution.

Consider the planar system

(1) $\displaystyle{\begin{array}[]{ccc}\dot{u}&=&p_{11}(t)u+p_{12}(t)v\\\
\dot{v}&=&p_{21}(t)u+p_{22}(t)v,\\\ \end{array}}$

where the $p_{ij}$ are continuous real valued $T$-periodic functions with
$T>0$. By Floquet theory, see [2, 5], there are solutions to the system (1),
say, $\varphi_{1}=(u_{1},v_{1})$ and $\varphi_{2}=(u_{2},v_{2})$, and real
numbers $\lambda_{1}$ and $\lambda_{2}$ (not necessarily distinct) that
satisfy

$\displaystyle{\begin{array}[]{ccc}\varphi_{1}(t+T)&=&\lambda_{1}\varphi_{1}(t)\\\
\varphi_{2}(t+T)&=&\lambda_{2}\varphi_{2}(t).\end{array}}$

The solutions $\varphi_{1}$ and $\varphi_{2}$ are called normal solutions and
the numbers $\lambda_{1}$ and $\lambda_{2}$ are called Floquet multipliers.
Set

$\phi(t)=\left[\begin{array}[]{cc}u_{1}(t)&u_{2}(t)\\\ v_{1}(t)&v_{2}(t)\\\
\end{array}\right]\,\,\mbox{and}\,\,P(t)=\left[\begin{array}[]{cc}p_{11}(t)&p_{12}(t)\\\
p_{21}(t)&p_{22}(t)\\\ \end{array}\right],$

it is a well-known fact that $\lambda_{1}$ and $\lambda_{2}$ are the
eigenvalues of the matrix $\phi^{-1}(0)\phi(T)$ and that

(2) $\lambda_{1}\lambda_{2}=\exp\int_{0}^{T}\mbox{trace}P(t)dt.$

We observe that, if $\lambda_{1}\neq\lambda_{2}$, then $\varphi_{1}$ and
$\varphi_{2}$ are linearly independent. In this case, knowledge of the Floquet
multipliers and the values of $\varphi_{1}$ and $\varphi_{2}$ for $0<t<T$
gives information for every solution of (1) for all $t$. The calculation of
the Floquet multipliers is not routine, since one does not generally know even
one nontrivial solution of (1). A procedure for obtaining the Floquet
multipliers and the corresponding normal solutions for (1) is possible when an
associate Riccati equation to the system (1) presents a periodic solution.
Following Proctor, see [7], making the change of coordinate $u=z_{1}+\sigma
z_{2}$ and $v=z_{2}$ in (1), where $\sigma$ is a solution of the Riccati
equation

(3) $\dot{x}=c(t)+b(t)x+a(t)x^{2}$

with $a(t)=-p_{21}(t)$, $b(t)=p_{11}(t)-p_{22}(t)$ and $c(t)=p_{12}(t)$, that
is,

(4) $\dot{x}=p_{12}(t)+[p_{11}(t)-p_{22}(t)]x-p_{21}(t)x^{2},$

the differential equation in $z$ can be integrated, thus providing the
following result:

###### Proposition 1.1

[7, Theorem 3.2]

  1. (1)

If $\varphi=(\varphi_{1},\varphi_{2})$ is a solution of (1), then
$\sigma=\frac{\varphi_{1}}{\varphi_{2}}$ is a solution of (3) on any interval
on which $\varphi$ does not vanish.

  2. (2)

If $\sigma$ is a solution of (3) on an interval $I$ containing the number $k$,
then

(5)
$\displaystyle{\begin{array}[]{ccc}u(t)&=&\sigma(t)\exp{\int_{k}^{t}p_{21}(s)\sigma(s)+p_{22}(s)ds}\\\
v(t)&=&\exp{\int_{k}^{t}p_{21}(s)\sigma(s)+p_{22}(s)ds}\\\ \end{array}}$

is a solution of (1)on $I$.

If $\sigma$ is a solution of (3) with period $nT$ and $f$ is the mean value of
$p_{21}\sigma+p_{22}$ over the period $nT$, where $n$ is a positive integer,
then $\exp\left(nTf\right)$ is a Floquet multiplier for (1) for the period
$nT$ and (5) is a normal solution of (1) corresponding to this multiplier.

Thus, for $n=1$,

$\lambda=\exp{\int_{0}^{T}p_{21}(s)\sigma(s)+p_{22}(s)ds}$

is a Floquet multiplier associated to solution $\varphi$. We observe that the
formula given in (2) provides us another multiplier.

Now the question is: how to guarantee the existence of a periodic solution to
the equation (3)? The next result, proved by Mokhtarzadeh, Pounark and Razani,
provides an answer to the question above. Before enunciating it, we define the
following function.

$G:[0,T]\times[0,T]\longrightarrow\mathbb{R}$, such that

$\displaystyle{G(t,s)=\left\\{\begin{array}[]{rr}\frac{1}{1-\exp\int_{0}^{T}b(r)dr}\exp\int_{s}^{t}b(r)dr,&0\leq
s\leq t\leq T\\\
\frac{\exp\int_{0}^{T}b(r)dr}{1-\exp\int_{0}^{T}b(r)dr}\exp\int_{s}^{t}b(r)dr,&0\leq
t\leq s\leq T.\end{array}\right.}$

###### Proposition 1.2

[Theorem $3.2$, [6]] Let $a(t),b(t)$ and $c(t)$ be continuous $T$-periodic
functions with $\int_{0}^{T}b(t)dt\neq 0$. Set

(6) $M=\displaystyle\sup_{0\leq t,s\leq
T}|G(t,s)|\,\,\mbox{and}\,\,N=\displaystyle\sup_{0\leq t\leq
T}\left|\int_{0}^{T}G(t,s)c(s)ds\right|$

and suppose

(7) $\int_{0}^{T}\left|a(\xi)\right|d\xi\leq\displaystyle\frac{1}{4MN}.$

Then, $x^{\prime}=c(t)+b(t)x+a(t)x^{2}$ has at least a $T$-periodic solution.

In [6, Theorem 3.2], the authors defined the following Banach space,

$X=\\{\phi;\phi\,\,\mbox{is a $T$-periodic continuous real functions on
}\mathbb{R}\\}.$

For $\phi\in X$, defined $|\phi|_{\infty}=\sup_{0\leq t\leq T}|\phi(t)|$ and

(8) $\Omega=\\{\phi\in X;|\phi-\psi|_{\infty}\leq N\\},$

where $\psi:[0,T]\longrightarrow\mathbb{R}$ is defined by

(9) $\psi(t)=\int_{0}^{T}G(t,s)c(s)ds.$

It is easy to see that $\Omega$ is closed, bounded and convex subset of $X$.
They also defined the operator $S:\Omega\longrightarrow X$ by

(10) $S(\phi)(t)=\int_{0}^{T}G(t,s)\left[a(s)\phi^{2}(s)+c(s)\right]ds.$

By assumptions (6), (7) and using Schauder Fixed Point Theorem, the authors
proved the existence of $x\in\Omega$, such that, $S(x)=x$, that is, for all
$t\in[0,T]$

$x(t)=\int_{0}^{T}G(t,s)\left[a(s)x^{2}(s)+c(s)\right]ds.$

As a consequence of the Propositions 1.1 and 1.2 we state the following
result.

###### Theorem 1.3

Suppose

  1. (i)

$\int_{0}^{T}p_{11}(t)-p_{22}(t)dt\neq 0$;

  2. (ii)

$\int_{0}^{T}\left|p_{21}(t)\right|dt\leq 1/4MN$.

Then, the Floquet multipliers of the system (1) are

$\lambda_{1}=\exp{\int_{0}^{T}p_{22}(t)+p_{21}(t)\sigma(t)dt}\,\,\mbox{and}\,\,\lambda_{2}=\exp{\int_{0}^{T}p_{11}(t)-p_{21}(t)\sigma(t)dt},$

where $\sigma(t)$ is a $T$-periodic solution of equation (4).

Now, we state one of the main results of this work, which is related to the
existence of a periodic solution to the Ricatti equation, which satisfies
$\int_{0}^{T}a(t)x(t)dt=0$, as a consequence of the coincidence degree theory,
proposed by R. E. Gaines and J. L. Mawhin [3].

###### Theorem 1.4

Consider $0<T$ and let $a(t),b(t)$ and $c(t)$ be continuous $T$-periodic
functions with

(11) $\int_{0}^{T}a(t)dt=0,$ (12) $|a(t)|\leq A,0<b\leq
b(t),\,\,\int_{0}^{T}c(t)dt=0$

where $b,A>0$ are constants. Then, the Ricatti equation

(13) $x^{\prime}=c(t)+b(t)x+a(t)x^{2}$

has at least one nontrivial $x$ $T$\- periodic solution that satisfies

(14) $\int_{0}^{T}a(t)x(t)dt=0~{}\mbox{and}~{}|x|_{\infty}\leq\frac{b}{2A}.$

As a consequence of the Theorem 1.4, we obtain the Floquet multipliers
explicitly. In this case, they do not depend on the solution of the Riccati
equation.

###### Theorem 1.5

Suppose

  1. (i)

$\int_{0}^{T}p_{21}(t)dt=\int_{0}^{T}p_{12}(t)dt=0$ and $|p_{21}(t)|\leq A$,
where $A>0$ is a constant;

  2. (ii)

$p_{11}(t)-p_{22}(t)\geq b>0$, where $b$ is a constant.

Then the Floquet multipliers of the system (1) are

$\lambda_{1}=\exp{\int_{0}^{T}p_{11}(t)dt}\,\,\mbox{and}\,\,\lambda_{2}=\exp{\int_{0}^{T}p_{22}(t)dt}.$

We observe that the Proposition 1.2 guarantees the existence of a $T$-periodic
solution, but does not explicitly provide such a solution. Therefore, the
Floquet multipliers provided by Theorem 1.3 cannot be obtained explicitly. On
the other hand, the Theorem 1.5 provides Floquet multipliers that can be
obtained explicitly.

In what follows, starting from section $2$, we present some concepts and
results of coincidence degree theory. In section $3$, we prove the theorem
1.3, 1.4 and 1.5. In section $4$, we present some stability results of linear
and nonlinear systems in terms of Floquet multipliers. Finally, in section
$5$, we apply Theorem 1.3 in a mathematical model for cholera.

##  2 Some concepts and results about coincidence degree

The method to be used in this paper involves the applications of the
continuation theorem of coincidence degree. In order to make this presentation
as self-contained as possible, we introduce a few concepts and results about
the coincidence degree. For further details, see R. E. Gaines and J. L. Mawhin
[3].

###### Definition 2.1

Let $X$, $Y$ be real Banach spaces, $L:DomL\subset X\rightarrow Y$ be a linear
mapping. The mapping $L$ is said to be a Fredholm mapping of index zero, if

$\dim KerL=codimImL<+\infty$

and $ImL$ is closed in $Y$.

If $L$ is a Fredholm mapping of index zero, then there are continuous
projectors $P:X\rightarrow X$ and $Q:Y\rightarrow Y$, such that

$ImP=KerL$

and

$KerQ=ImL=Im(I-Q).$

It follows that the restriction $L_{P}$ of $L$ to $DomL\cap
KerP:(I-P)X\rightarrow ImL$ is invertible. Denote the inverse of $L_{P}$ by
$K_{P}$.

###### Definition 2.2

A continuous mapping $N:X\rightarrow Y$ is said to be $L$-compact on
$\overline{\Omega}$, if $\Omega$ is an open bounded subset of $X$,
$QN(\overline{\Omega})$ is bounded and
$K_{P}(I-Q)N:\overline{\Omega}\rightarrow X$ is compact.

Since $ImQ$ is isomorphic to $KerL$, there is an isomorphism $J:ImQ\rightarrow
KerL$. We shall be interested in proving the existence of solutions for the
operator equation

(15) $Lx=Nx,$

where a solution is an element of $DomL\cap\overline{\Omega}$ which verifies
(15).

The following results is due to R. E. Gaines and J. L. Mawhin [3].

######  Proposition 2.3 (Mawhin’s Continuation Theorem)

Let $L$ be a Fredholm mapping of index $0$ and let $N$ be $L$-compact on
$\bar{\Omega}$. Suppose that

  1. (i)

For each $\lambda\in(0,1)$, $x\in\partial\Omega$

$Lx\neq\lambda Nx.$

  2. (ii)

$QNx\neq 0$ for each $x\in KerL\cap\partial\Omega$ and

$deg(JQN,\Omega\cap KerL,0)\neq 0,$

where $J:ImQ\rightarrow KerL$ is an isomorphism.

Then, the equation $Lx=Nx$ has at least one solution in
$DomL\cap\bar{\Omega}$.

##  3 Proof of Theorems 1.3, 1.4 and 1.5

###  3.1 Proof of Theorem 1.3

Proof. Due to $(i)$ and $(ii)$, the Proposition 1.2 guarantees that the
equation (4) has a $T$-periodic solution $\sigma(t)$. Thus, by Proposition
1.1, a Floquet multiplier of the system (1) is

$\lambda_{1}=\exp{\int_{0}^{T}p_{22}(t)+p_{21}(t)\sigma(t)dt}.$

Now, we use the equation (2) to determine the other multiplier, which is

$\lambda_{2}=\exp{\int_{0}^{T}p_{11}(t)-p_{21}(t)\sigma(t)dt}.$

###  3.2 Proof of Theorem 1.4

Proof. Consider the following Banach spaces

$X=\\{x|x\in C(\mathbb{R},\mathbb{R}),x(t+T)=x(t),\textup{for all}\
t\in\mathbb{R}\\}\cap\left\\{x|\int_{0}^{T}a(t)x(t)dt=\int_{0}^{T}c(t)x(t)dt=0\right\\},$

and

$Y=\\{x|x\in C(\mathbb{R},\mathbb{R}),x(t+T)=x(t),\textup{for all}\
t\in\mathbb{R}\\},$

with the norm

$\|x\|_{X}=\|x\|_{Y}=|x|_{\infty},$

where $|x|_{\infty}=\displaystyle\max_{t\in[0,T]}|x(t)|$.

Define a linear operator $L:DomL\subset X\rightarrow Y$ by setting

$DomL=\\{x|x\in X,x^{\prime}\in C(\mathbb{R},\mathbb{R})\\}$

and for $x\in DomL$,

$Lx=x^{\prime}.$

We also define a nonlinear operator $N:X\rightarrow Y$ by setting

$Nx=c(t)+b(t)x+a(t)x^{2}.$

It is not difficult to see that, by (11),

$KerL=\mathbb{R},\ \textup{and}\ ImL=\left\\{y|y\in
Y,\int_{0}^{T}y(s)ds=0\right\\}.$

Thus the operator $L$ is a Fredholm operator with index zero.

Define the continuous projector $P:X\rightarrow KerL$ and the averaging
projector $Q:Y\rightarrow Y$ by setting

$Px(t)=x(0)$

and

$Qy(t)=\frac{1}{T}\int_{0}^{T}y(s)ds.$

Hence, $ImP=KerL$ and $KerQ=ImL$. Denoting by $K_{P}:ImL\rightarrow DomL\cap
KerP$ the inverse of $L|_{DomL\cap KerP}$, we have

$K_{P}y(t)=\int_{0}^{t}y(s)ds.$

Then $QN:X\rightarrow Y$ and $K_{P}(I-Q)N:X\rightarrow X$ read

$QNx=\frac{1}{T}\int_{0}^{T}b(s)x(s)ds+\frac{1}{T}\int_{0}^{T}a(s)x^{2}(s)ds,$
$K_{P}(I-Q)Nx(t)=\int_{0}^{t}c(s)ds+\int_{0}^{t}b(s)x(s)ds+\int_{0}^{t}a(s)x^{2}(s)ds-
tQNx.$

Clearly, $QN$ and $K_{P}(I-Q)N$ are continuous. By using Arzela-Ascoli
theorem, it is not difficult to prove that
$\overline{K_{P}(I-Q)N(\overline{\Omega})}$ is compact for any open bounded
set $\Omega\subset X$. Indeed, let $L>0$ such that $|x|_{\infty}\leq L$, for
each $x\in\overline{\Omega}$. Notice that $QN(\overline{\Omega})$ is bounded
and $|QNx|_{\infty}\leq|b|_{\infty}L+|a|_{\infty}L^{2}$, for all
$x\in\overline{\Omega}$. Hence, $K_{P}(I-Q)N(\overline{\Omega})$ is uniformly
bounded and

$|K_{P}(I-Q)Nx|_{\infty}\leq
2T(|c|_{\infty}+|b|_{\infty}L+|a|_{\infty}L^{2}),\forall
x\in\overline{\Omega}.$

Now, let $t_{1},t_{2}\in[0,T]$, if we suppose $t_{2}<t_{1}$, we obtain

$\begin{array}[]{l}|K_{P}(I-Q)Nx(t_{1})-K_{P}(I-Q)Nx(t_{2})|\\\
=\left|\int_{t_{2}}^{t_{1}}c(s)ds+\int_{t_{2}}^{t_{1}}b(s)x(s)ds+\int_{t_{2}}^{t_{1}}a(s)x^{2}(s)ds-(t_{1}-t_{2})QNx\right|\\\
\leq 2(|c|_{\infty}+|b|_{\infty}L+|a|_{\infty}L^{2})|t_{1}-t_{2}|,\forall
x\in\overline{\Omega}.\end{array}$

Therefore, $K_{P}(I-Q)N(\overline{\Omega})$ is a equicontinuous set of
$C([0,T])$(hence of $X$). By using Arzela-Ascoli theorem,
$\overline{K_{P}(I-Q)N(\overline{\Omega})}$ is compact. Therefore, $N$ is
$L$-compact on $\overline{\Omega}$ with any open bounded set $\Omega\subset
X$.

As $\frac{b}{2A}>0$, we consider

(16) $\Omega_{d}:=\\{x\in X||x|_{\infty}<\frac{b}{2A}\\},$

that is an open set in $X$.

Notice that

(17) $b(t)-a(t)\frac{b}{2A}>0.$

Indeed,

$b(t)-a(t)\frac{b}{2A}\geq b-A.\frac{b}{2A}=b-\frac{b}{2}=\frac{b}{2}>0.$

Notice that

(18) $b(t)+a(t)\frac{b}{2A}>0.$

Indeed,

$b(t)+a(t)\frac{b}{2A}\geq b-A\frac{b}{2A}=b-\frac{b}{2}=\frac{b}{2}>0.$

Let $0<\lambda<1$ and $x$ such that

$x^{\prime}=\lambda c(t)+\lambda b(t)x+\lambda a(t)x^{2}.$

By multiplying by $x$ and integrand of $0$ to $T$, we have

$0=\int_{0}^{T}x^{\prime}xdt=\lambda\int_{0}^{T}c(t)xdt+\lambda\int_{0}^{T}b(t)x^{2}+a(t)x^{3}dt.$

That is,

$0=\int_{0}^{T}c(t)xdt+\int_{0}^{T}x^{2}(b(t)+a(t)x)dt=\int_{0}^{T}x^{2}(b(t)+a(t)x)dt.$

By (16), if $x\in\partial\Omega_{d}$, we have $|x|_{\infty}=\frac{b}{2A}$, and
we obtain

$0\geq\int_{0}^{T}x^{2}(b(t)-|a(t)||x|_{\infty})dt.$

By (12), we have

$0\geq\int_{0}^{T}x^{2}(b-A\frac{b}{2A})dt$
$\geq\int_{0}^{T}x^{2}\frac{b}{2}dt>0.$

But this is a contradiction. Therefore, the condition ($1$) of Proposition 2.3
holds for $\Omega_{c}$. Take $x\in\partial\Omega_{d}\cap KerL$. Thus, we have
$x=-\frac{b}{2A}$ or $x=\frac{b}{2A}$.

If $x=-\frac{b}{2A}$, by (17), we have

$b(t)-a(t)\frac{b}{2A}>0.$

Hence,

(19)
$QNx=\frac{1}{T}\int_{0}^{T}-\frac{b}{2A}\left(b(t)-a(t)\frac{b}{2A}\right)dt<0.$

If $x=\frac{b}{2A}$, by (18), we have

$b(t)+a(t)\frac{b}{2A}>0.$

Hence,

(20)
$QNx=\frac{1}{T}\int_{0}^{T}\frac{b}{2A}\left(b(t)+a(t)\frac{b}{2A}\right)dt>0.$

Then, for each $x\in\partial\Omega_{d}\cap KerL$, we have

(21) $QNx=\frac{1}{T}\int_{0}^{T}x\left(b(t)+a(t)x\right)dt\neq 0.$

Therefore, the condition ($2$) of Proposition 2.3 holds for $\Omega_{d}$.

Define a continuous function $H(x,\mu)$ by setting

$H(x,\mu)=(1-\mu)x+\mu\frac{1}{T}\int_{0}^{T}x\left(b(t)+a(t)x\right)dt,\,\,\,\,\mu\in[0,1].$

It follows from (19), (20) and (21) that

$H(x,\mu)\neq 0,\ \textup{for all}\ x\in\partial\Omega_{d}\cap KerL.$

Hence, using the homotopy invariance theorem, we have

$deg(QN,\Omega_{d}\cap
KerL,0)=deg\left(\frac{1}{T}\int_{0}^{T}x\left(b(t)+a(t)x\right)dt,\Omega_{d}\cap
KerL,0\right)$ $=deg\left(x,\Omega_{d}\cap KerL,0\right)=-1\neq 0.$

In view of all the discussions above, we conclude from Proposition 2.3 that
the equation (13) has a solution in $DomL\cap\bar{\Omega}_{d}$.

###  3.3 Proof of Theorem 1.5

Proof. Due to $(i)$ and $(ii)$, the Theorem 1.4 guarantees that the equation
(4) has a $T$-periodic solution $\sigma(t)$ and
$\int_{0}^{T}p_{21}(t)\sigma(t)dt=0$. Thus, by Proposition 1.1, a Floquet
multiplier of the system (1) is

$\lambda_{1}=\exp{\int_{0}^{T}p_{11}(t)dt}.$

Now, we use the equation (2) to determine the other multiplier, which is

$\lambda_{2}=\exp{\int_{0}^{T}p_{22}(t)dt}.$

##  4 Stability of linear and nonlinear systems

The following theorem provides details about the stability of the system (1)
in terms of Floquet exponents. We refer to Theorem 7.2 on page 120 of Hale’s
book [4].

###### Theorem 4.1

  1. (i)

A necessary and sufficient condition that the system (1) is uniformly stable
is that the Floquet multipliers of the system (1) have modulii $\leq 1$ and
the ones with modulii $=1$ have multiplicity $1$.

  2. (ii)

A necessary and sufficient condition that the system (1) is uniformly
asymptotically stable is that all Floquet multipliers of the system (1) have
modulii $<1$.

Now, consider the system

(22) $\dot{X}=A(t)X+F(t,X),$

where $A(t)$ is an $n\times n$ continuous matrix function, and $F(t,X)$ is
continuous in $t$ and $X$ and Lipschitz-continuous in $X$ for all
$t\in\mathbb{R}$ and $X$ in a neighbourhood of $X=0$. Moreover, we assume that

(23) $\lim\limits_{|X|\to 0}\frac{|F(t,X)|}{|X|}=0\mbox{ uniformly in }t.$

Notice that the condition in (23) implies that $X=0$ is a solution to system
(22). Then, we have the following theorem on the behaviour of the trivial
solution $X=0$. This result is an extended version of [4, Theorem 2.4] or [9,
Theorem 7.2].

###### Theorem 4.2

If the trivial solution $X=0$ of the system $\dot{X}=A(t)X$ is uniformly
asymptotically stable for $t\geq 0$, then the trivial solution of (22) is also
uniformly asymptotically stable. If the trivial solution $X=0$ of the system
$\dot{X}=A(t)X$ is unstable, then the trivial solution of (22) is also
unstable.

In the following example, we study the stability of a planar system, whose
Floquet multipliers are obtained by Theorem 1.5.

###### Example 1

Consider the planar system

(24) $\displaystyle{\begin{array}[]{ccc}\dot{u}&=&(m(t)-A)u+\sin(t)v\\\
\dot{v}&=&a(t)u+(m(t)-B)v,\\\ \end{array}}$

where $m(t)$ is a continuous function and $2\pi$-periodic,

$a(t)=\dfrac{(\alpha+1)\sin(t)}{\beta+\cos(t)},$

$\beta>1$ and $A-B>0$. The Riccati equation associated with this system is

$y^{\prime}=-a(t)y^{2}+(A-B)y+\sin(t).$

Since $\int_{0}^{2\pi}a(t)dt=\int_{0}^{2\pi}\sin(t)dt=0$, $b(t)=A-B>0$ and
$a(t)$ is limited, according to Theorem 1.4, that there is a $2\pi$-periodic
solution $\sigma(t)$ with $\int_{0}^{2\pi}a(t)\sigma(t)dt=0$. Now, by Theorem
1.5, the Floquet multipliers are

$\displaystyle{\begin{array}[]{ccc}\lambda_{1}&=&\displaystyle\exp\int_{0}^{2\pi}p_{11}(t)dt\\\
\lambda_{2}&=&\displaystyle\exp\int_{0}^{2\pi}p_{22}(t)dt,\\\ \end{array}}$

that is,

$\displaystyle{\begin{array}[]{ccc}\lambda_{1}&=&\displaystyle\exp\int_{0}^{2\pi}(m(t)-A)dt\\\
\lambda_{2}&=&\displaystyle\exp\int_{0}^{2\pi}(m(t)-B)dt.\\\ \end{array}}$

Therefore,

$\lambda_{1}=e^{-2\pi
A}e^{\int_{0}^{2\pi}m(t)dt}\quad\mbox{and}\quad\lambda_{2}=e^{-2\pi
B}e^{\int_{0}^{2\pi}m(t)dt}.$

Observe that, if $\int_{0}^{2\pi}m(t)dt<2\pi A$ and
$\int_{0}^{2\pi}m(t)dt<2\pi B$, then the system (24), according to Theorem
4.1, is uniformly asymptotically stable.

##  5 An application to cholera modeling

In this section, we propose the analysis of a mathematical model for cholera
dynamics with seasonal oscillation, studied by [8]. Cholera is a severe
intestinal infection caused by the bacterium Vibrio cholerae. Many epidemic
models have been published, but Codeço [1] was the first to explicitly
incorporate bacterial dynamics into a SIR epidemiological model.

The following new model is a significant extension of Codeço’s model [1] that
incorporates the phage dynamics and the seasonal oscillation of cholera
transmission, as proposed in [8]:

(25) $\left\\{\begin{array}[]{lcl}\frac{dS}{dt}&=&n(H-S)-d\frac{B}{K+B}S,\\\
\frac{dI}{dt}&=&d\frac{B}{K+B}S-rI,\\\ \frac{dB}{dt}&=&eI-
mB-\delta\frac{B}{\tilde{K}+B}P,\\\
\frac{dP}{dt}&=&\xi\,I+\kappa\frac{B}{\tilde{K}+B}P-\nu\,P,\end{array}\right.$

where

  * •

$S$ is the susceptible human population,

  * •

$I$ is the infectious human population,

  * •

$B$ and $P$ are the concentrations of the pathogen (i.e. vibrio) and the
phage, respectively in the contaminated water,

  * •

the total human population, $H$, is assumed to be a constant,

  * •

$n$ denotes the natural human birth/death rate,

  * •

$d$ denotes the human contact rate to the vibrio,

  * •

$\delta$ is the death rate of the bacteria due to phage predation,

  * •

$\kappa$ is the growth rate of the phage due to feeding on the vibrio,

  * •

$e$ and $\xi$ are the rates of human contribution (e.g. by shedding) to the
pathogen and the phage, respectively,

  * •

$m$ and $\nu$ are the natural death rates of the vibrio and the phage,
respectively.

In addition, $r=n+\gamma$ with $\gamma$ as the recovery rate, and $K$ and
$\tilde{K}$ as the half saturation rates of the vibrio in the interaction with
human and phage, respectively.

In [8], the authors investigated the impact of seasonality on cholera
dynamics, particularity when they examined the periodic variation of three
parameters, $m$, $e$ and $d$, and applied the results from the Floquet theory
in the analysis, as follows. Firstly, the authors consider that the parameter
$m$ is a positive periodic function of time, $m(t)$, which represents a
seasonal variation of the extinction rate of the vibrio. On the second
scenario, the parameter $e$ is set as a positive periodic function, $e(t)$,
which represents a seasonal oscillation of the per capita contamination rate,
i.e., the unit rate of human contribution (e.g. shedding) to the pathogen in
the environment. On the third scenario, the parameter $d$ is set as a positive
periodic function, $d(t)$, which represents a seasonal variation of the
contact rate. For an accurate analysis of the dynamics in each previous case,
see [8].

In this paper, we consider the following scenario, by setting the parameters
$m$, $e$ and $d$ as positive periodic functions $m(t)$, $e(t)$ and $d(t)$
simultaneously.

It is clear that $E_{0}=(H,0,0,0)$ is the unique disease free equilibrium
(DFE) of the system. For ease of discussion, we translated the DFE to the
origin via a change of variable by $\overline{S}=H−S$. Then, with
linearization at $(0,0,0,0)$, the original system becomes

(26)
$\left\\{\begin{array}[]{lcl}\frac{d\overline{S}}{dt}&=&-n\overline{S}+\frac{d(t)H}{K}B+\left(\frac{d(t)B(H-\overline{S})}{K+B}-\frac{d(t)H}{K}B\right),\\\
\frac{dI}{dt}&=&-rI+\frac{d(t)H}{K}B+\left(\frac{d(t)B(H-\overline{S})}{K+B}-\frac{d(t)H}{K}B\right),\\\
\frac{dB}{dt}&=&e(t)I-m(t)B-\delta\frac{B}{\tilde{K}+B}P,\\\
\frac{dP}{dt}&=&\xi\,I+\kappa\frac{B}{\tilde{K}+B}P-\nu\,P.\end{array}\right.$

Thus, system (26) can be written in a compact form

$\dot{X}=A(t)X+F(t,X),$

with $X=(\bar{S},I,B,P)^{T}$ and the matrix

(27) $A(t)=\left(\begin{array}[]{cccc}-n&0&\frac{d(t)H}{K}&0\\\
0&-r&\frac{d(t)H}{K}&0\\\ 0&e(t)&-m(t)&0\\\ 0&\xi&0&-\nu\end{array}\right).$

By [8], it is straightforward to check that $\lim\limits_{|X|\to
0}\frac{|F(t,X)|}{|X|}=0$ uniformly in $t$. Based on Theorem 4.2, we only need
to consider the periodic linear system

(28) $\dot{X}=A(t)X,$

where the matrix $A(t)$ is given by (27).

Notice that the matrix $A(t)$ has a block tridiagonal structure. From [8,
Theorem 2.5 and Corollary 2.5], two Floquet exponents of the system (28) are
given by $−n$ and $−\nu$; the other two Floquet exponents are determined by
the matrix block $\left(\begin{array}[]{cc}-r&\frac{d(t)H}{K}\\\ e(t)&-m(t)\\\
\end{array}\right).$ Hence, its stability depends on the $2\times 2$ sub-
system

(29) $\dot{Y}=\left(\begin{array}[]{cc}-r&\frac{d(t)H}{K}\\\ e(t)&-m(t)\\\
\end{array}\right)Y.$

We consider $m(t)$, $e(t)$ and $d(t)$, such that

(30) $A=\max\limits_{0\leq t\leq T}d(t),$ (31) $E=\max\limits_{0\leq t\leq
T}e(t)$

and

(32) $m_{1}=\min\limits_{0\leq t\leq T}m(t),$

satisfying

(33) $2EMAT\frac{H}{K}<\min\\{m_{1},r\\}$

and

(34) $\int_{0}^{T}e(t)dt\leq\frac{1}{4MN},$

where $M$ and $N$ are defined in (6).

We have the following lemma to describe the stability of the sub-system (29).

###### Lemma 5.1

Suppose (33), (34) and $r\neq\frac{1}{T}\int_{0}^{T}m(t)dt$. Then, the trivial
solution of (29) is asymptotically stable.

Proof. We observe that $m(t)$, $e(t)$ and $a(t)$ satisfies the assumptions of
Theorem 1.3, with $\int_{0}^{T}(p_{11}-p_{22})dt=-Tr+\int_{0}^{T}m(t)dt\neq 0$
and $\int_{0}^{T}p_{21}(t)dt=\int_{0}^{T}e(t)dt\leq\frac{1}{4MN}$. Then, it
follows from Theorem 1.3 that the Floquet multipliers of the system (29) are

$\lambda_{1}=\exp{\left(\int_{0}^{T}(-r-e(t)\sigma(t))dt\right)}$

and

$\lambda_{2}=\exp{\left(\int_{0}^{T}(-m(t)+e(t)\sigma(t))dt\right)},$

where $\sigma(t)$ is a $T$-periodic solution of equation (4). Since
$\sigma\in\Omega$ ($\Omega$ defined in (8)), we obtain

$\left|\sigma(t)-\int_{0}^{T}G(t,s)a(s)\frac{H}{K}ds\right|<N.$

Therefore,

$|\sigma(t)|\leq\int_{0}^{T}|G(t,s)||a(s)\frac{H}{K}|ds+N\leq
2MAT\frac{H}{K}.$

Hence, by (33), we obtain

$-r-e(t)\sigma(t)\leq-r+2EMAT\frac{H}{K}<0$

and

$-m(t)+e(t)\sigma(t)\leq-m_{1}+2EMAT\frac{H}{K}<0.$

Notice that all Floquet multipliers of the system (29) have modulii $<1$. By
Theorem 4.1-($ii$), the trivial solution $X=0$ of the system (29) is uniformly
asymptotically stable for $t\geq 0$.

We can summarize our analysis above in the theorem below.

###### Theorem 5.2

Assume that system (25) satisfies (33), (34) and
$r\neq\frac{1}{T}\int_{0}^{T}m(t)dt$. Then, the disease free equilibrium
$E_{0}=(H,0,0,0)$ is uniformly asymptotically stable.

## References

  * [1] Codeço, C.T. _Endemic and epidemic dynamics of cholera: the role of the aquatic reservoir_ , BMC Infectious diseases, v. 1, n. 1, p. 1, 2001.
  * [2] Floquet, G. _Sur les équations différentielles linéaires à coefficients périodiques [On linear differential equations with periodic coefficients]_ , Annales scientifiques de l’École normale supérieur, p. 47–88, 1883.
  * [3] Gaines, R. E. and Mawhin, J. L. _Coincidence Degree and Nonlinear Differential Equations_ , Lecture Notes in Math., 586, Berlin, New York:Springer-Verlag, 1977.
  * [4] Hale, J. K. _Ordinary differential equations_ , vol. XXI, Pure and applied mathematics. New York (NY):Wiley-Interscience; 2ª Ed., 1980.
  * [5] Hartman, P. _Ordinary differential equations_ , Wiley, New York, 1964.
  * [6] Mokhtarzadeh, M.R., Pournaki, M.R. and Razani, A. _A note on periodic solutions of Riccati equations_ , Nonlinear Dynamics, v. 62, n. 1–2, p. 119–125, 2010.
  * [7] Proctor, T. G. _Characteristic multipliers for some periodic differential equations_ , Proceedings of the American Mathematical Society, v. 22, n. 2, p. 503–508, 1969.
  * [8] Tian, M.P. and Wang, J. _Some results in Floquet, with application to periodic epidemic models_ , Applicable Analysis, v. 94, n. 6, p. 1128–1152, 2015.
  * [9] Verhulst, F. _Nonlinear differential equations and dynamical systems_ , 2nd ed. Berlin: Springer, 1996\.