# Bayesian nonparametric dynamic hazard rates in evolutionary life tables

Luis E. Nieto-Barajas
Department of Statistics, ITAM, Mexico
<EMAIL_ADDRESS>

###### Abstract

In the study of life tables the random variable of interest is usually assumed
discrete since mortality rates are studied for integer ages. In dynamic life
tables a time domain is included to account for the evolution effect of the
hazard rates in time. In this article we follow a survival analysis approach
and use a nonparametric description of the hazard rates. We construct a
discrete time stochastic processes that reflects dependence across age as well
as in time. This process is used as a bayesian nonparametric prior
distribution for the hazard rates for the study of evolutionary life tables.
Prior properties of the process are studied and posterior distributions are
derived. We present a simulation study, with the inclusion of right censored
observations, as well as a real data analysis to show the performance of our
model.

Keywords: Beta process, discrete time processes, latent variables, moving
average process, stationary process.

##  1 Introduction

Modelling and forecasting mortality rates has been of interest since the last
decade of previous century. A mortality rate is basically a hazard rate
function of age for a random variable $X$ that indicates the age of death in a
population. Non-negative random variables is the subject of study of survival
analysis (e.g. Klein and Moeschberger,, 2003). Random behaviour is
characterised by the survival, $S(x)$, and the hazard, $h(x)$, functions, for
a specific age $x>0$. To distinguish the change of mortality in time, we
denote by $X_{t}$ the age of death at time $t$ (year or decade) in a
population, for $t\in\mathbb{T}$, with $\mathbb{T}$ the set of periods under
study. Again, characterization of the set of random variables
$\\{X_{t}\\}_{t\in\mathbb{T}}$ can be done by including the time domain in the
survival and hazard functions, $S_{t}(x)$ and $h_{t}(x)$, respectively.

Typical models assume that a transformation of the hazard rate function,
usually logarithmic or logistic, can be written in terms of age and time
effects. For example, Lee and Carter, (1992) consider
$\log{h_{t}(x)}=a_{x}+b_{x}k_{t}+\epsilon_{x,t}$, with $\epsilon_{x,t}$ an
error term with zero mean and constant variance. They impose some estimability
constraints and use minimum squares to estimate the unknown parameters
$a_{x}$, $b_{x}$ and $k_{t}$. Several other authors have further study this
model in different directions: disaggregation by sex (Lee,, 2000);
improvements in the forecasting step by including two sets of singular value
decomposition vectors (Renshaw and Haberman,, 2003); including a cohort effect
(Renshaw and Haberman,, 2006); and modelling of multiple populations via
neural networks (Richman and Wüthrich,, 2019).

On a different perspective, Currie et al., (2004) propose a Poisson regression
model with P-splines to smooth mortality tables over age and time. Debón et
al., (2008) consider dynamic tables, and after removing trends, model the
residuals via a Gaussian random field in age and time. Debón et al., (2010)
summarises several geostatistical approaches and includes cohort effects.

All previous approaches consider that the hazard rate, $h_{t}(x)$, has been
observed and focus in modelling and smoothing its variability in age and time.
However they are ignoring that the “observed” $h_{t}(x)$ is actually a
nonparametric estimate of the hazard function, which in discrete age, is
defined as $h_{t}(x)=\mbox{P}(X_{t}=x\mid X_{t}\geq x)$. Therefore a
frequentist nonparametric estimate of $h_{t}(x)$ is the number of people in
the population who died at age $x$ and time $t$ divided by the number of
people who are still alive at age $x$ for the same time. What we propose is to
follow a survival analysis approach and consider each individual a realisation
of the random variable $X_{t}$ and estimate $h_{t}(x)$ via a bayesian
nonparametric approach. For this we introduce a stochastic process that
includes dependence across age and time. This process is then used as a
nonparametric prior distribution for the hazard rates, which allows us to
estimate future behaviours of the hazards in time. An advantage of following a
survival analysis approach is that we can include partial information, as
censored observations, into the analysis.

The description of the rest of the paper is as follows: In Section 2 we define
the bayesian nonparametric model, sampling and prior distributions, and
characterise its properties. Section 3 shows how to obtain posterior inference
for the hazard rates under the presence of exact and right censored
observations. Section 4 considers numerical analysis with simulated and real
data. We conclude with some remarks in Section 5.

##  2 Bayesian nonparametric model

Let $X_{t}$ be a non-negative discrete random variable, with support
$\mathcal{X}=\\{1,\ldots,N_{X}\\}$, that denotes the age of death at time
$t\in\mathbb{T}$, with $\mathbb{T}=\\{1,\ldots,N_{T}\\}$, and $N_{X}$ and
$N_{T}$ the maximum age and time under study, respectively. The probability
law of $X_{t}$ is described be the density function
$f_{t}(x)=\mbox{P}(X_{t}=x)$ for $x\in\mathcal{X}$, the survival function
$S_{t}(x)=\mbox{P}(X_{t}>x)$ for $x>0$, and by the hazard rate function
$h_{t}(x)=\mbox{P}(X_{t}=x\mid X_{t}\geq x)$ for $x\in\mathcal{X}$, and for
$t\in\mathbb{T}$.

Les us denote the hazard rate for age $x$ at time $t$ by the parameter
$\pi_{x,t}$, that is, $h_{t}(x)=\pi_{x,t}$. Since a discrete hazard rate is a
probability, we require that $\pi_{x,t}\in(0,1)$, for all $x\in\mathcal{X}$
and $t\in\mathbb{T}$. Then we can re-write the survival and density functions
in terms of the parameters $\boldsymbol{\pi}=\\{\pi_{x,t}\\}$ as (e.g. Klein
and Moeschberger,, 2003)

$S_{t}(x\mid\boldsymbol{\pi})=\prod_{z\leq
x}(1-\pi_{z,t})I(x>0)\quad\text{and}\quad
f_{t}(x\mid\boldsymbol{\pi})=\pi_{x,t}\prod_{z<x}(1-\pi_{z,t})I(x\in\mathcal{X}),$
(1)

where $I(x\in A)$ is the indicator function that takes the value of one if
$x\in A$ and zero otherwise. Functions (1) can be seen as nonparametric models
defined by $\boldsymbol{\pi}$.

The idea is to define a convenient nonparametric prior for $\boldsymbol{\pi}$
that recognises a dependence across different ages $x$’s as well as dependence
across different times $t$’s. Nieto-Barajas and Walker, (2002) proposed a
Markov beta process indexed by a single dimension, say $x$, through the use of
a latent process. Later on, Jara et al., (2013) generalised the beta process
to allow for dependencies on more than one lag. In both cases, for a
particular specification of the parameters, the marginal distribution of the
processes are beta. We now extend these ideas to indexes in two dimensions.

Let $\\{\pi_{x,t}\\}_{(x,t)\in\mathcal{X}\times\mathbb{T}}$ be a discrete age-
time stochastic process defined for the cross product of sets $\mathcal{X}$
and $\mathbb{T}$. For each $(x,t)\in\mathcal{X}\times\mathbb{T}$ we require a
latent variable $\upsilon_{x,t}$, plus a single latent variable $\omega$,
common to all indexes. The idea is to relate the hazards at age $x$ and time
$t$ to the hazards of previous ages, say $x-1,\ldots,x-p$, as well as to the
hazards of previous years, say $t-1,\ldots,t-q$, where $p$ and $q$ define the
order of the dependence. Instead of linking the parameters of interest
$\pi_{x,t}$ directly, we link them through the latent $\upsilon_{x,t}$’s. The
extra latent parameter $\omega$ plays the role of an anchor needed to obtain a
desired marginal distribution.

In general, we denote by $\partial_{x,t}$ the set of indexes (neighbours) that
will influence the location $(x,t)$. Apart from the neighbours, we also need
to include the actual location. That is, for an order–$(p,q)$ dependence
model, with $p,q>1$, we define
$\partial_{x,t}=\\{(x,t),(x-1,t),\ldots,(x-p,t),(x,t-1),\ldots,(x,t-q)\\}$. To
illustrate, in Figure 1 we show a graphical representation of dependence of
orders $p=1$ and $q=1$ for a scenario with $N_{X}=4$ and $N_{T}=2$.

$\omega$$\upsilon_{1,1}$$\upsilon_{2,1}$$\upsilon_{3,1}$$\upsilon_{4,1}$$\pi_{1,1}$$\pi_{2,1}$$\pi_{3,1}$$\pi_{4,1}$$\upsilon_{1,2}$$\upsilon_{2,2}$$\upsilon_{3,2}$$\upsilon_{4,2}$$\pi_{1,2}$$\pi_{2,2}$$\pi_{3,2}$$\pi_{4,2}$
Figure 1: Graphical representation of dependence of orders $p=1$ and $q=1$.

Let $\mbox{Be}(a,b)$ denotes a beta distribution with mean $a/(a+b)$, and
$\mbox{Bin}(c,\omega)$ denotes a binomial distribution with number of
Bernoulli trials $c$ and success probability $\omega$. Let
$\boldsymbol{\upsilon}=\\{\upsilon_{x,t}\\}$ be the whole set of latent
variables, then the law of the process $\\{\pi_{x,t}\\}$ is defined
hierarchically by

$\displaystyle\omega$ $\displaystyle\sim\mbox{Be}(a,b)$
$\displaystyle\upsilon_{x,t}\mid\omega$
$\displaystyle\stackrel{{\scriptstyle\mbox{\scriptsize{ind}}}}{{\sim}}\mbox{Bin}(c_{x,t},\omega)$
(2) $\displaystyle\pi_{x,t}\mid\boldsymbol{\upsilon}$
$\displaystyle\stackrel{{\scriptstyle\mbox{\scriptsize{ind}}}}{{\sim}}\mbox{Be}\left(a+\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}\,,\>b+\sum_{(z,s)\in\partial_{x,t}}(c_{z,s}-\upsilon_{z,s})\right)$

where $a$, $b$ and ${\bf c}=\\{c_{x,t}\\}$ are non-negative parameters for all
$x\in\mathcal{X}$ and $t\in\mathbb{T}$. We define $\upsilon_{x,t}=0$ with
probability one (w.p.1) and $c_{x,t}=0$ for any $x,t<0$. In notation we say
$\boldsymbol{\pi}\sim\mbox{BeP}_{p,q}(a,b,{\bf c})$, that is, an order–$(p,q)$
beta process with parameters $a$, $b$ and ${\bf c}$, defined by (2).
Properties of this beta process are given in Proposition 1. In particular, the
correlation induced and the marginal distribution can be obtained in closed
form.

###### Proposition 1

Let $\\{\pi_{x,t}\\}_{(x,t)\in\mathcal{X}\times\mathbb{T}}$ be a discrete age-
time stochastic process $\mbox{BeP}_{p,q}(a,b,{\bf c})$, defined by equations
(2).

  1. (i)

The marginal distribution of $\pi_{x,t}$ for all
$(x,t)\in\mathcal{X}\times\mathbb{T}$, is $\mbox{Be}(a,b)$,

  2. (ii)

The correlation between any pair $(\pi_{x,t},\pi_{x^{\prime},t^{\prime}})$ is
given by

$\mbox{Corr}(\pi_{x,t},\pi_{x^{\prime},t^{\prime}})=\frac{(a+b)\left(\sum_{(z,s)\in\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}}}c_{z,s}\right)+\left(\sum_{(z,s)\in\partial_{x,t}}c_{z,s}\right)\left(\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}c_{z,s}\right)}{\left(a+b+\sum_{(z,s)\in\partial_{x,t}}c_{z,s}\right)\left(a+b+\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}c_{z,s}\right)},$

  3. (iii)

If $c_{x,t}=0$ for all $(x,t)\in\mathcal{X}\times\mathbb{T}$ then the
$\pi_{x,t}$’s become independent.

Proof For $(i)$ we note that conditionally on $\omega$ the $\upsilon_{x,t}$
are independent binomials, so the sum is again binomial, i.e.,
$\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}\mid\omega\sim\mbox{Bin}\left(\sum_{(z,s)\in\partial_{x,t}}c_{z,s},\omega\right)$.
Integrating out $\omega$, the marginal distribution of
$\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}$ is beta-binomial with parameters
$(a,b,\sum_{(z,s)\in\partial_{x,t}}c_{z,s})$. Using conjugacy properties
between the beta and the binomial distributions (e.g. Bernardo and Smith,,
2000) we obtain $\pi_{x,t}\sim\mbox{Be}(a,b)$ marginally. For $(ii)$ we rely
on conditional independence properties and the iterative covariance formula.
Then
$\mbox{Cov}(\pi_{x,t},\pi_{x^{\prime},t^{\prime}})=\mbox{E}\\{\mbox{Cov}(\pi_{x,t},\pi_{x^{\prime},t^{\prime}}\mid\boldsymbol{\upsilon})\\}+\mbox{Cov}\\{\mbox{E}(\pi_{x,t}\mid\boldsymbol{\upsilon}),\mbox{E}(\pi_{x^{\prime},t^{\prime}}\mid\boldsymbol{\upsilon})\\}$.
The first term in the sum becomes zero since $\pi_{x,t}$’s are conditional
independent given $\boldsymbol{\upsilon}$. The second term, after removing the
additive constants of the expected values, is rewritten as
$\mbox{Cov}\left(\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s},\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}\upsilon_{z,s}\right)$
divided by
$(a+b+\sum_{(z,s)\in\partial_{x,t}}c_{z,s})(a+b+\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}c_{z,s})$.
Concentrating in the numerator, and using the iterative covariance formula for
a second time, we get
$\mbox{E}\\{\mbox{Cov}(\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s},\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}\upsilon_{z,s}\mid\omega)\\}+\mbox{Cov}\\{\mbox{E}(\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}\mid\omega),$
$\mbox{E}(\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}\upsilon_{z,s}\mid\omega)\\}$.
The first term, after separating the sums in the common part, reduces to
$\mbox{E}\\{\mbox{Var}(\sum_{(z,s)\in\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}}}\upsilon_{z,s}\mid\omega)\\}$
which becomes
$(\sum_{(z,s)\in\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}}}c_{z,s})\mbox{E}\\{\omega(1-\omega)\\}$.
After computing the expected value, this can be re-written as
$(\sum_{(z,s)\in\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}}}c_{z,s})(a+b)\mbox{Var}(\omega)$.
The second term, after computing the expectations becomes
$(\sum_{(z,s)\in\partial_{x,t}}c_{z,s})(\sum_{(z,s)\in\partial_{x^{\prime},t^{\prime}}}c_{z,s})\mbox{Var}(\omega)$.
Finally, using $(i)$, we note that
$\mbox{Var}(\pi_{x,t})=\mbox{Var}(\pi_{x^{\prime},t^{\prime}})$
$=\mbox{Var}(\omega)$, so dividing the covariance between the product of the
standard deviations we obtain the result. For $(iii)$ we note that if
$c_{x,t}=0$ then $\upsilon_{x,t}=0$ w.p.1. If we do this for all
$(x,t)\in\mathcal{X},\mathbb{T}$ then it is straightforward to see that the
$\pi_{x,t}^{\prime}s$ become independent. $\diamond$

Many things can be concluded from Proposition 1. Parameters $a$ and $b$ are
the two shape parameters of the marginal beta distribution of the process.
Parameters ${\bf c}$ control the strength of the correlation between any two
elements $\pi_{x,t}$ and $\pi_{x^{\prime},t^{\prime}}$. The correlation is
stronger, if the two elements share more parameters. Some locations can be
more influential than others if their corresponding $c_{x,t}$ parameter is
larger. Moreover, the process $\\{\pi_{x,t}\\}$ becomes strictly stationary if
the parameters ${\bf c}$ are all equal. That is, if $c_{x,t}=c$ for all $x$
and $t$, then the correlation for $x,x^{\prime}>p$ and $t,t^{\prime}>q$
simplifies to

$\mbox{Corr}(\pi_{x,t},\pi_{x^{\prime},t^{\prime}})=\frac{(a+b)\\{\\#(\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}})\\}c+(p+q+1)^{2}c^{2}}{\\{a+b+(p+q+1)c\\}^{2}},$

where $\\#(\partial_{x,t}\cap\partial_{x^{\prime},t^{\prime}})$ denotes the
number of elements in the intersection set.

Therefore, our bayesian nonparametric model is defined by the nonparametric
sampling distribution (1) together with process $\mbox{BeP}_{p,q}(a,b,{\bf
c})$, defined by (2), as nonparametric prior for $\boldsymbol{\pi}$. We now
proceed to show how posterior inference is done with this model.

##  3 Posterior analysis

Let us assume that, for each year $t\in\mathbb{T}$, we observe a sample of
size $n_{t}$, that is, $X_{1,t},X_{2,t},\ldots,$ $X_{n_{t},t}$, with possible
right censored observations. Let $\delta_{i,t}$ be the censored indicator that
takes the value of $1$ if the observation is exact and zero if it is right-
censored. Then, the joint distribution of the sample is

$f({\bf
x}\mid\pi)=\prod_{t\in\mathbb{T}}\prod_{i=1}^{n_{t}}f(x_{i,t}\mid\boldsymbol{\pi})^{\delta_{i,t}}S(x_{i,t}\mid\boldsymbol{\pi})^{1-\delta_{i,t}}.$

Using expressions (1), we can substitute the survival and density functions in
terms of $\boldsymbol{\pi}$ to obtain

$f({\bf
x}\mid\boldsymbol{\pi})=\prod_{t\in\mathbb{T}}\prod_{x\in\mathcal{X}}\pi_{x,t}^{r_{x,t}}(1-\pi_{x,t})^{m_{x,t}-r_{x,t}},$

where $r_{x,t}=\sum_{i=1}^{n_{t}}I(X_{i,t}=x,\,\delta_{i,t}=1)$ is the number
of people dying and $m_{x,t}=\sum_{i=1}^{n_{t}}I(X_{i,t}\geq x)$ is the number
of people at risk, at age $x$ and time $t$, respectively.

The prior distribution for $\boldsymbol{\pi}$ is characterised by (2) and its
marginal distribution could be obtained via integration with respect to the
latent variables. However, inference is simpler if we consider the joint prior
distribution for the whole vector that includes the latent variables
$\boldsymbol{\upsilon}$ and $\omega$. This has the form

$f(\boldsymbol{\pi},\boldsymbol{\upsilon},\omega)=\mbox{Be}(\omega\mid
a,b)\prod_{x\in\mathcal{X}}\prod_{t\in\mathbb{T}}\mbox{Bin}(\upsilon_{x,t}\mid
c_{x,t},\omega)$ $\hskip
170.71652pt\times\mbox{Be}\left(\pi_{x,t}\left|a+\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}\,,\>b+\sum_{(z,s)\in\partial_{x,t}}(c_{z,s}-\upsilon_{z,s})\right.\right),$

where the distributions with an argument upfront denote the corresponding
density function.

The posterior distribution for the extended vector of parameters is simply
$f(\boldsymbol{\pi},\boldsymbol{\upsilon},\omega\mid{\bf x})\propto
f(x\mid\boldsymbol{\pi})f(\boldsymbol{\pi},\boldsymbol{\upsilon},\omega)$. To
obtain summaries from it we suggest to implement a Gibbs sampler (Smith and
Roberts,, 1993) with the following posterior conditional distributions.

  1. i.

Conditional distribution of the latent $\omega$

$f(\omega|\boldsymbol{\upsilon})=\mbox{Be}\left(\omega\left|\,a+\sum_{x\in\mathcal{X}}\sum_{t\in\mathbb{T}}\upsilon_{x,t}\,,\,b+\sum_{x\in\mathcal{X}}\sum_{t\in\mathbb{T}}(c_{x,t}-\upsilon_{x,t})\right.\right).$

  2. ii.

Conditional distribution of the latent variables $\upsilon_{x,t}$, for
$x\in\mathcal{X}$ and $t\in\mathbb{T}$

$f(\upsilon_{x,t}|\boldsymbol{\pi},\omega)\propto\frac{{c_{x,t}\choose\upsilon_{x,t}}\left\\{\left(\frac{\omega}{1-\omega}\right)\prod_{(z,s)\in\varrho_{x,t}}\left(\frac{\pi_{z,s}}{1-\pi_{z,s}}\right)\right\\}^{\upsilon_{x,t}}I(\upsilon_{x,t}\in\\{0,1,\ldots,c_{x,t}\\})}{\prod_{(z,s)\in\varrho_{x,t}}\Gamma\left(a+\sum_{(y,u)\in\partial_{z,s}}\upsilon_{y,u}\right)\Gamma\left(b+\sum_{(y,u)\in\partial_{z,s}}\left(c_{y,u}-\upsilon_{y,u}\right)\right)}.$

  3. iii.

Conditional distribution of the parameters $\pi_{x,t}$, for $x\in\mathcal{X}$
and $t\in\mathbb{T}$

$f(\pi_{x,t}\mid\boldsymbol{\upsilon},{\bf
x})=\mbox{Be}\left(\pi_{x,t}\left|\,a+\sum_{(z,s)\in\partial_{x,t}}\upsilon_{z,s}+r_{x,t}\,,\>b+\sum_{(z,s)\in\partial_{x,t}}(c_{z,s}-\upsilon_{z,s})+m_{x,t}-r_{x,t}\right.\right).$

Sampling from conditional distributions (i) and (iii) is straightforward,
since they have standard form (a beta distribution). Sampling from (ii) is not
complicated due to the discreteness of the density in a bounded support, so it
can be evaluated and normalised.

Although we do not have a complete analytical expression for the posterior
distribution of the hazard rate parameters $\pi_{x,t}$, we can have an idea
how the data and the prior dependence come to play. Taking iterative
expectation from (iii) we have that the posterior mean of $\pi_{x,t}$ has the
form

$\mbox{E}(\pi_{x,t}\mid{\bf
x})=\frac{a+\sum_{(z,s)\in\partial_{x,t}}\mbox{E}(\upsilon_{z,s}\mid{\bf
x})+r_{x,t}}{a+b+\sum_{(z,s)\in\partial_{x,t}}c_{z,s}+m_{x,t}}.$ (3)

Recalling that if $c_{x,t}=0$ for all $x$ and $t$ this implies that
$\upsilon_{x,t}=0$ w.p.1, so regardless of the data,
$\mbox{E}(\upsilon_{x,t}\mid{\bf x})=0$. If we further take $a$ and $b$ close
to zero, which would correspond to a vague (non-informative) prior, then (3)
reduces to the frequentist non-parametric estimate of the hazard rate
$r_{x,t}/m_{x,t}$. Here “vague” is interpreted as a prior with large variance,
however, other non-informative priors include Jeffrey’s prior ($a=b=0.5$) or
the uniform prior ($a=b=1$) (e.g. Berger,, 1985). For any $c_{x,t}>0$, we know
that $0\leq\mbox{E}(\upsilon_{x,t}\mid{\bf x})\leq c_{x,t}$, so
$\sum_{(z,s)\in\partial_{x,t}}\mbox{E}(\upsilon_{z,s}\mid{\bf x})$ in (3) can
be seen as extra people dying and $\sum_{(z,s)\in\partial_{x,t}}c_{z,s}$ would
be additional people at risk, at age $x$ and time $t$. So ages-times $(x,t)$
more influential in their neighbours would have larger $\upsilon_{x,t}$
values.

##  4 Numerical studies

###  4.1 Simulation study

In this first study we test our model in a controlled scenario with the
presence of censoring. Let $Y_{t}\sim\mbox{Po}(\lambda_{t})$ be a poisson
random variable with mean $\lambda_{t}$. We define $Z_{t}=Y_{t}+1$ such that
$Z_{t}$ is a strictly positive random variable, and denote it as
$Z_{t}\sim\mbox{Po}^{+}(\lambda_{t})$. Let $C_{t}$ be a censoring age such
that $C_{t}\sim\mbox{Po}^{+}(\lambda_{C})$. We simulated data points $Z_{i,t}$
and censoring ages $C_{i,t}$ independently, and define observations
$X_{i,t}=\min(Z_{i,t},C_{i,t})$ and right-censoring indicators
$\delta_{i,t}=I(Z_{i,t}\leq C_{i,t})$, for $i=1,\ldots,n_{t}$ and
$t=1,\ldots,N_{T}$.

We took $\lambda_{t}=8+(t-1)/2$, which implies that the hazard rate $h_{t}(x)$
has an increasing trend in age $x$, for a fixed time $t$, and a decreasing
trend in time $t$, for a fixed age $x$. Figure 2 shows these hazard rates,
where the intensity of the colour increases for larger times $t$. Sample sizes
are $n_{t}=1,000$ for $t=1,\ldots,N_{T}$ and $N_{T}=15$. For the censoring
rate we took $\lambda_{C}=18$, and since this is fixed across time $t$, the
percentage of censoring values increases for larger $t$. Specifically, they go
from very light censoring ($2.89\%$) for $t=1$ to moderate censoring
($33.08\%$) for $t=15$. Given the possible simulated values, and to avoid
numerical problems we took a maximum age $N_{X}=18$ to define the hazards
$\pi_{x,t}$. To study the prediction properties of our model, we decided to
estimate the hazard rates for out of sample times $t=16,17$.

To specify the prior distribution for $\pi_{x,t}$ we took $a=b=0.001$ and
assumed $c_{x,t}=c$ with a range of values for $c\in\\{2,5,10,20\\}$ to
compare. For the order of dependence we took $p=1,\ldots,10$ and
$q=1,\ldots,10$ to explore. These values imply a total of $4\times 10\times
10=400$ different model specifications. For each of them we implemented a
Gibbs sampler with conditional distributions $(i)$, $(ii)$ and $(iii)$, and
ran it for $18,000$ iterations with a burn in of $6,000$ and keeping one of
every $3^{rd}$ iteration. Convergence of the chains was assessed informally by
looking at the trace plots, ergodic means and autocorrelation functions.
Figure 3 shows these convergence diagnostics for parameter $\omega$ when
$p=10$, $q=8$ and $c=5$.

To assess model fit we computed the L-measure which is a goodness of fit
statistic that summarises variance and mean square error (bias) of the
posterior distribution of each $\pi_{x,t}$. This is defined as (Laud and
Ibrahim,, 1995)

$L(\nu)=\frac{1}{N_{X}N_{T}}\sum_{x=1}^{N_{X}}\sum_{t=1}^{N_{T}}\mbox{Var}\left(\pi_{x,t}\mid{\bf
x}\right)+\frac{\nu}{N_{X}N_{T}}\sum_{x=1}^{N_{X}}\sum_{t=1}^{N_{T}}\left\\{\mbox{E}\left(\pi_{x,t}\mid{\bf
x}\right)-\pi_{x,t}^{0}\right\\}^{2},$ (4)

where $\pi_{x,t}^{0}$ denotes the true hazard from the positive poisson model
$\mbox{Po}^{+}(\lambda_{t})$, which does not have close analytical expression
but can be evaluated numerically, and $\nu\in[0,1]$ is a constant that
determines the importance of the bias in the overall measure. To assess
forecasting performance we computed the L-measure (4) but considered, in the
second summation, the range of values $t=N_{t}+1,\ldots,N_{T}+N$, which in
this case $N=2$. Note that $\pi_{x,t}^{0}$ is also available for this range of
values of $t$.

Figure 4 shows the values of the L-measure, with $\nu=1/2$, for the different
prior specifications and for the observed range of times (in sample) and
predictions (out of sample). In sample L-measure decreases for larger $p$ and
larger $q$, with little gain for $q\geq 8$. The measure also takes smaller
values for $c=5$. Therefore, we select our best fitting model that with
$p=10$, $q=8$ and $c=5$. To place our model in context, the L-measure for the
model that ignores dependence, obtained with $c_{x,t}=0$ for all $x$ and $t$,
is 0.0023, which is a lot higher than the values for most of our models. On
the other hand, the out of sample L-measure (bottom panel) prefers smaller
values of $p$, larger values of $q$ and larger values of $c$. In general there
is little gain for values $c\geq 10$. Therefore our best forecasting model is
that with $p=1$, $q=10$ and $c=10$.

We summarise posterior distributions of the hazard rates $\pi_{x,t}$ with the
mean, as a point estimate, together with $2.5\%$ and $97.5\%$ quantiles, to
define $95\%$ credible intervals. Figure 5 includes posterior estimates for
the first two times $t=1,2$ and for the out of sample times $t=16,17$. For
each time we show two estimates, one with independent priors obtained by
setting $c=0$ (left panel), and another with our best model (right panel). For
all times, $95\%$ credible intervals are a lot wider with independent priors
than with our best dependent model. Posterior estimates obtained with our best
models correct the lack of data by producing more precise estimates and are
able to accurately forecast the hazard rates for future times.

###  4.2 Real data analysis

The human mortality database (HMD,, 2021) is a project conducted by the
Department of Demography at the University of California in Berkeley, USA, and
by the Max Planck Institute for Demographic Research in Rostock, Germany. They
provide death counts and population sizes for different countries needed to
construct life tables. One of the most complete datasets is that of
Switzerland, which contains records from 1876 up to 2018. We decided to
constrain the years of study to the window from 1925 to 2014 and formed 18
quinquennial groups: $[1925,1929],[1930,1934],\ldots,[2010,2014]$. The ages of
death were also constrained from 0 to 99 and formed 20 quinquennial groups:
$[0,4],[5,9],\ldots,[95,99]$. In both types of groups the deaths and the
population were aggregated. The only consequence of doing this is that
estimated hazard rates for the quinquennial group would be an average of the
hazard rates for individual ages/years.

In notation of Section 3, available data are death counts at age $x$ and year
$t$, which correspond to $r_{x,t}$ in a scenario with no censoring, and
population sizes at age $x$ and year $t$, $Q_{x,t}$. We recover the number of
people at risk by $m_{x,t}=\sum_{y=x}^{N_{X}}Q_{y,t}$. Frequentist
nonparametric estimates of the hazard rates are
$\widehat{\pi}_{x,t}=r_{x,t}/m_{x,t}$, for $x=1,\ldots,N_{X}$ and
$t=1,\ldots,N_{T}$ with $N_{X}=20$ and $N_{T}=18$. These are shown in Figure
6, where the intensity of the colour increases for larger $t$ (more recent
dates). As we can see, the mortality rates are monotonically increasing along
age, and shown a slow but steady decreasing path along time. The latter is a
consequence of a larger life expectancy in developed countries like
Switzerland. The small gap that can be appreciated in the last ages occurs in
the 70’s, that is, between groups $[1970,1974]$ and $[1975,1979]$.

Having a closer look at the posterior conditional distribution (iii) in
Section 3, we note that observed data, summarised in sufficient statistics
$r_{x,t}$ and $m_{x,t}$, enter the expression additively to the latent
variables $\upsilon_{x,t}$ and $c_{x,t}-\upsilon_{x,t}$. Therefore, to define
the prior distributions, the values of $c_{x,t}$ have to be of the same order
of $r_{x,t}$. Moreover, to avoid numerical problems we decided to divide both,
the number of deaths and the population sizes, by a constant $k=10$, which has
no implications in the hazard rate estimates.

We fitted our bayesian nonparametric dependent model with the following prior
specifications: $a=b=0.001$ and assumed $c_{x,t}=c$ with a range of values for
$c\in\\{5,20,50,100\\}$ to compare. Due the point just raised, these chosen
values for $c$ are a lot higher than those considered in the simulation study.
For the order of dependence we took $p=1,2,3$ and $q=1,\ldots,10$ to explore.

We fitted our model for the Switzerland dataset for $t=1,\ldots,17$ and
decided to leave out the last quinquennial time group, $t=18$, for out of
sample comparison. We implemented a Gibbs sampler with $10,000$ iterations
with a burn in of $1,000$ and keeping one of every second iteration. Again,
convergence of the chain was assessed informally with good convergence
behaviour for all chains. As a goodness of fit for the different prior choices
we also used the L-measure but computed it for two cases: using in sample
estimations for $t=1,\ldots,17$; and out of sample estimations for $t=18$.
Since we do not know the true hazard rates $\pi_{x,t}^{0}$, needed in (4), we
use the frequentist nonparametric estimates $\widehat{\pi}_{x,t}$ instead.

Figure 7 summarises both L-measures with $\nu=1/2$. For both cases, in sample
and out of sample, smaller values of $p$ and larger values of $q$ are
preferred. On the other hand, there is an opposite preference in the parameter
$c$. For in sample estimation, smaller $c$ is preferred, whereas for out of
sample larger values of $c$ are preferred. Looking closer to the out of sample
L-measures, they are practically the same for $c$ values in $\\{20,50,100\\}$
for $p\geq 6$. Making a compromise between the two sets of measures, we take
as our best model that with $p=1$, $q=10$ and $c=20$.

Finally, in Figure 8 we show posterior estimates for some times. Again, we
take the posterior mean and quantiles $2.5\%$ and $97.5\%$ of $\pi_{x,t}$ to
define point and $95\%$ credible interval estimates. In the top row we show
estimates for $t=1$ and compare independence prior (left) with our best
dependent model fit (right). Little difference can be seen in both fits,
perhaps a smaller variance in the dependence model for ages $x\geq 19$. In the
bottom row we show out of sample estimates for $t=18$ and compare estimates
with $p=1$, $q=10$ and two values of $c$, $c=5$ (left) and $c=20$ (right). We
can appreciate the reduction in the length of the credible intervals when
increasing the value of $c$ from $5$ to $20$, however in both cases the out of
sample estimations are reasonably good.

To place our model in context, we also fitted Lee and Carter, (1992)’s model
(LC) described in Section 1. For the errors we assumed
$\epsilon_{x,t}\sim\mbox{N}(0,\tau_{\epsilon})$, that is a zero mean normal
distribution with precision $\tau_{\epsilon}$. We performed a bayesian
analysis and took independent prior distributions $a_{x}\sim\mbox{N}(0,0.1)$,
$b_{x}\sim\mbox{N}(0,0.1)$, and a dynamic linear prior
$k_{t}\sim\mbox{N}(k_{t-1},\tau_{k})$ for $t=2,\ldots,N_{T}$ with
$k_{1}\sim\mbox{N}(0,\tau_{k})$. Finally for the precisions we took
$\tau_{\epsilon}\sim\mbox{Ga}(0.1,0.1)$ and $\tau_{k}\sim\mbox{Ga}(0.1,0.1)$.
We also assessed forecasting performance of the models by considering the
following scenario: Fitted the model for $t=1,\ldots,N_{T}-N$ and estimated
$N$ future hazard rates for $t=N_{T}-N+1,\ldots,N_{T}$; we took values
$N=1,\ldots,10$. Posterior inference of both models was obtained by running a
Gibbs sampler with the same specifications as above. We did our own
implementation of LC model in jags (Plummer,, 2003).

The L-measures, in sample and out of sample, for our model with $p=1$, $q=10$
and $c=20$, and for the LC model are presented in Table 1. The in sample
L-measures are somehow stable in both models for the different values of $N$,
with an slight increase for our model and a slight decrease for the LC model,
for larger $N$. In general, the fitting of our model is better than LC model.
For comparing the forecasting performance (out of sample L-measures) between
the two models, we have reported the ratio of the two L-measures (Rout) in the
last column of Table 1. Here something interesting happens. LC model is better
than our model for $N\leq 4$, but for $N>4$ our model shows better forecasting
behaviour than the LC model.

In summary, our model has shown a better fitting than the LC model and our
model outperforms LC model in long term forecasting, for the particular
dataset analysed here.

##  5 Concluding Remarks

We have proposed a bayesian nonparametric model for the study of dynamic life
tables. Prior distributions for the hazard rates is based on a stochastic
process that allows for dependence in the hazards across ages and times. The
order of dependence is controlled by parameters $p$ and $q$ in each dimension,
respectively, and the strength of dependence is controlled by a set of
parameters ${\bf c}=\\{c_{x,t}\\}$. In contrast to other proposals, our model
admits the presence of right censored observations, which are not uncommon in
life tables estimation.

Posterior inference in our model relies in the implementation of a Gibbs
sampler, which for this paper was implemented in Fortran. Computational times
do not increase with sample size, they only depend on the number of ages,
$N_{X}$, and the number of times, $N_{T}$, under study. For the sizes
considered in the numerical examples and for $18,000$ iterations the code
takes less than 2 minutes to run.

Our model is unique in the sense that it can not only estimate future hazard
functions, but it can improve hazard rates estimation for observed ages and
times, specially in the ages where few deaths are observed or where censored
observations are present. Our model forecasting performance improves for
longer times ahead.

The inclusion of covariates to define sub-populations, like sex, states or
countries, might also be of interest. One way of dealing with this is to
consider a multiplicative hazard model, similar to Cox, (1972), but ensuring
that the effect of covariates still defines a proper discrete hazard rate. If
we denote by ${\bf Z}_{it}$ the covariate vector of individual $i$ and by
$\boldsymbol{\theta}$ a coefficients vector, then the hazard function for
individual $i$ could be $h_{it}(x)=\varphi({\bf
Z}_{it},\boldsymbol{\theta})h_{0t}(x)$, with $\varphi({\bf
Z}_{it},\boldsymbol{\theta})=\exp({\bf
Z}_{it}^{\prime}\boldsymbol{\theta})/\\{1+\exp({\bf
Z}_{it}^{\prime}\boldsymbol{\theta})\\}$ and $h_{0t}(x)$ a discrete baseline
hazard. Furthermore, the stochastic process $\mbox{BeP}_{p,q}(a,b,{\bf c})$,
as in (2), could be used as a nonparametric prior distribution for the
baseline hazard. Studying this and other alternative models to cope with
covariates is left to study for a future work.

## Acknowledgements

The author acknowledges support from Asociación Mexicana de Cultura, A.C. and
is grateful to valuable comments from the reviewers.

## References

  * Berger, (1985) Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer: New York.
  * Bernardo and Smith, (2000) Bernardo, J.M. and Smith, A.F.M. (2000). Bayesian Theory. Wiley: New York.
  * Currie et al., (2004) Currie, I.D., Durban, M. and Eilers, P.H.C. (2004). Smoothing and forecasting mortality rates. Statistical Modelling 4, 279–298.
  * Cox, (1972) Cox, D.R. (1972). Regression models and life tables. Journal of the Royal Statistical Society, Series B 34, 187–220.
  * Debón et al., (2008) Debón, A., Montes, F., Mateu, J., Porcu, E. and Bevilacqua, M. (2008). Modelling residuals dependence in dynamic life tables: A geostatistical approach. Computational Statistics and Data Analysis 52, 3128–3147.
  * Debón et al., (2010) Debón, A., Martínez-Ruiz, F. and Montes, F. (2010). A geostatistical approach for dynamic life tables: The effect of mortality on remaining lifetime and annuities. Insurance: Mathematics and Economics 47, 327–336.
  * HMD, (2021) Human Mortality Database. University of California, Berkeley (USA), and Max Planck Institute for Demographic Research (Germany). Available at www.mortality.org or www.humanmortality.de (data downloaded on February 11, 2021).
  * Laud and Ibrahim, (1995) Laud, P. and Ibrahim, J. (1995). Predictive model selection. Journal of the Royal Statistical Society, Series B 57, 247–262.
  * Jara et al., (2013) Jara, A., Nieto-Barajas, L.E. and Quintana, F. (2013). A time series model for responses on the unit interval. Bayesian Analysis 8, 723–740.
  * Klein and Moeschberger, (2003) Klein, J.P. and Moeschberger, M.L. (2003). Survival Analysis. Springer: New York.
  * Lee, (2000) Lee, R.D. (2000). The Lee-Carter method for forecasting mortality with various extensions and applications. North American Actuarial Journal 4, 80–91.
  * Lee and Carter, (1992) Lee, R.D. and Carter, L.R. (1992). Modeling and forecasting U.S. mortality. Journal of the American Statistical Association 87, 659–671.
  * Nieto-Barajas and Walker, (2002) Nieto-Barajas, L.E. and Walker, S.G. (2002). Markov beta and gamma process for modelling hazard rates. Scandinavian Journal of Statistics 29, 413–424.
  * Plummer, (2003) Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003), March 20–22, Vienna, Austria. ISSN 1609-395X, vol. 124, p. 125.
  * Renshaw and Haberman, (2003) Renshaw, A.E. and Haberman, S. (2003). Lee–Carter mortality forecasting with age-specific enhancement. Insurance: Mathematics and Economics 33, 255–272.
  * Renshaw and Haberman, (2006) Renshaw, A.E. and Haberman, S. (2006). A cohort-based extension to the Lee–Carter model for mortality reduction factors. Insurance: Mathematics and Economics 38, 556–570.
  * Richman and Wüthrich, (2019) Richman, R. and Wüthrich, M. (2019). A neural network extension of the Lee–Carter model to multiple populations. Annals of Actuarial Science, 1-21. doi:10.1017/S1748499519000071
  * Rossi et al., (2020) Rossi, R., Murari, A. Gaudio, P. and Gelfusa, M. (2020). Upgrading model selection criteria with goodness of fit tests for practical applications. Entropy 22, 447.
  * Smith and Roberts, (1993) Smith, A. and Roberts, G. (1993). Bayesian computations via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B 55, 3-–23.

| BeP | LC |
---|---|---|---
$N$ | $L_{\rm in}$ | $L_{\rm out}$ | $L_{\rm in}$ | $L_{\rm out}$ | Rout
$1$ | $0.000099$ | $0.000753$ | $0.000387$ | $0.000344$ | $2.18$
$2$ | $0.000102$ | $0.000879$ | $0.000417$ | $0.000499$ | $1.76$
$3$ | $0.000104$ | $0.001068$ | $0.000416$ | $0.000805$ | $1.32$
$4$ | $0.000105$ | $0.001266$ | $0.000445$ | $0.001189$ | $1.06$
$5$ | $0.000108$ | $0.001430$ | $0.000406$ | $0.001419$ | $1.00$
$6$ | $0.000119$ | $0.001593$ | $0.000346$ | $0.001990$ | $0.80$
$7$ | $0.000127$ | $0.001746$ | $0.000269$ | $0.184359$ | $0.01$
$8$ | $0.000153$ | $0.001864$ | $0.000184$ | $0.735184$ | $0.00$
$9$ | $0.000167$ | $0.002124$ | $0.000177$ | $1.214812$ | $0.00$
$10$ | $0.000169$ | $0.002413$ | $0.000160$ | $8.422003$ | $0.00$
Table 1: L-measures comparison at $\nu=1/2$ for $N=1,2,\ldots,10$ times left
out. In sample ($L_{\rm in}$) and out of sample ($L_{\rm out}$), for our best
fitting model $\mbox{BeP}(p=1,q=10,c=20)$ and LC model. Last column shows the
ratio between the out of sample L-measures.

Figure 2: True hazards $h_{t}(x)$ in simulated study for $t=1,\ldots,15$.
Darker colour corresponds to larger $t$.

Figure 3: MCMC convergence diagnostics for $\omega$ when $p=10$, $q=8$ and
$c=5$. Trace plot, ergodic means, autocorrelation function and probability
histogram.

Figure 4: L-measure gof statistics for different prior specifications. In
sample (top), out of sample (bottom). $p=1,\ldots,10$; $q=1,\ldots,10$ (within
each block); and $c=2,5,10,20$ in colour intensity from light to dark.

Figure 5: Posterior estimates of hazard rates $h_{t}(x)$ for $t=1,2,16,17$.
Independent prior with $c=0$ (left column), dependent prior (right column)
with $p=10$, $q=8$ and $c=5$ (in sample) and with $p=1$, $q=10$ and $c=10$
(out of sample).

Figure 6: Frequentist nonparametric estimates of hazard rates $h_{t}(x)$ for
$t=1,\ldots,18$. Darker colour corresponds to larger $t$.

Figure 7: L-measure gof statistics for different prior specifications. In
sample (top), out of sample (bottom). $p=1,\ldots,3$; $q=1,\ldots,10$ (within
each block); and $c=5,20,50,100$ (in colour intensity from light to dark).

Figure 8: Posterior estimates of hazard rates $h_{t}(x)$ for in sample (top
row) and out of sample (bottom row). For $t=1$ with $c=0$ (top left); for
$t=1$ with $p=1$, $q=10$ and $c=20$ (top right). For $t=18$ with $p=1$, $q=10$
and $c=5$ (bottom left); for $t=18$ with $p=1$, $q=10$ and $c=20$ (bottom
right).


11institutetext: Stony Brook University, Stony Brook NY 11790, USA

# Surgical Phase Recognition in Laparoscopic Cholecystectomy

Yunfan Li 11 Vinayak Shenoy 11 Prateek Prasanna 11 I.V. Ramakrishnan 11 Haibin
Ling 11 Himanshu Gupta 11

###### Abstract

Automatic recognition of surgical phases in surgical videos is a fundamental
task in surgical workflow analysis. In this report, we propose a Transformer-
based method that utilizes calibrated confidence scores for a 2-stage
inference pipeline, which dynamically switches between a baseline model and a
separately trained transition model depending on the calibrated confidence
level. Our method outperforms the baseline model on the Cholec80 dataset, and
can be applied to a variety of action segmentation methods.

###### Keywords:

Surgical Phase Recognition Deep Learning Robot-assisted surgery.

##  1 Introduction

Surgical workflow analysis is an important field in robot-assisted surgery
research, which has great potential in improving patient safety and achieving
better surgery outcomes[12]. In particular, automatic segmentation of surgical
steps is important for subsequent surgical training, intraoperative assistance
and workflow optimization[15, 6].

Early methods for surgical phase recognition utilized statistical models like
conditional random field [1, 17] and hidden Markov models(HMMs) [18, 16, 4].
However, these methods are limited in representation capacity and were unable
to model complicated long-term temporal relations. Later, long short-term
memory(LSTM) network [9] was introduced for long-term temporal dependency
modeling. SV-RCNet [10] combined ResNet [8] with LSTM in an end-to-end fashion
to model the spatio-temporal relations between video frames. Czempiel et al.
[2] proposed a multi-stage TCN model named TeCNO, which explores long-term
temporal relations from pre-computed spatial features.

The introduction of Transformer [19] models saw a major shift in the paradigm
of sequential modeling and significant improvement in various computer vision
related tasks, including surgical phase recognition. Gao et al. [5] proposed a
Transformer-based network which combines pre-computed spatial and temporal
embeddings within a fixed temporal window to predict surgical phase labels.
Czempiel et al. [3] introduced a novel attention regularization loss within
the Transformer framework to encourage the model to focus on high-quality
frames during training.

Laparoscopic Cholecystectomy(LC) procedures in most cases follow a series of
necessary steps [13]. Based on this observation, we first devised an inference
scheme which transitions between a set of 2-class classifiers based on
previous and current predictions. The intuition is that in surgeries where the
steps are strictly followed, there are only two possibilities as to which
phase a certain frame can belong to. Additionally, we found that 2-class
classifiers are better at distinguishing between two neighboring steps than a
full-class classifier. We will evaluate this methodology and discuss its
limitations.

Subsequently, we devised another inference scheme based on confidence levels
from the baseline model, the intuition being that when the baseline model is
less confident about the prediction, we can switch to a 2-class model with
higher accuracy. In safety critical applications such as surgical phase
recognition, classification models should not only be accurate, but also
indicate when they are likely to be wrong [14]. However, modern deep learning
models like the Transformers usually suffer from miscalibration issues [7],
therefore producing over-confident predictions, which limits the chance for
human intervention or other alternative methods. Therefore, we propose a
complementary scheme which addresses the problem of over-confident Transformer
models using temperature scaling, an effective confidence calibration method,
and achieves better results than the baseline model.

##  2 Methodology

###  2.1 Dataset

The Cholec80 dataset [18] includes 80 videos of laparoscopic cholecystectomy
procedures with a resolution of $1920\times 1080$ or $854\times 480$ pixels
recorded at 25 frames-per-second(fps). The dataset defines 7 surgical stages
for the procedure, and tool presence labels are provided for multi-task
learning. Following previous works, all videos are subsampled to 1 fps, and
all frames are resized to $250\times 250$ pixels. For fair comparison, we use
40 videls for training, 8 videos for validation, and 32 videos for testing
across all experiments.

###  2.2 Transformer model

Unlike a sequence-to-sequence model that processes input one at a time, a
transformer model relies on self-attention mechanism to attend to the entire
input sequence at once. The scoring functionality of the self-attention allows
the model to focus on only the relevant parts of the input.

The traditional transformer model (Fig. 1) is a stack of encoders and
decoders. Each encoder if broken into two sub-layers: a multi-head self-
attention mechanism and point-wise feed forward network. The output of an
encoder layer is the input to the subsequent encoder layer in the stack. The
decoder layer - in addition to the two sub-layers in the encoder - has another
multi-head attention layer that performs attention over encoder output.

Figure 1: Transformer Model

The attention function maps a query $Q$ with all the keys $K$. Its output is a
weighted sum of all its value vectors $V$, where the weights are softmax
scores computed using $Q$ and $K$. We can think of the query as the search bar
in youtube search that is mapped against a set of keys (video title, etc)
associated with candidate videos in their database, then present the best
matched values (videos). Each scaled dot-product attention unit in the multi-
head attention learns a set of weights, $W_{Q}$, $W_{K}$, and $W_{V}$. The
weights are multiplied with input, $x_{t}$ at timestep $t$ to produce query
vector ($q_{t}=x_{t}W_{Q}$ ), key vector ($k_{t}=x_{t}W_{K}$) and value vector
($v_{t}=x_{t}W_{V}$). Each unit computes the attention of $Q$ with $K$ using:

$Attention(Q,K,V)=softmax(\frac{QK}{\sqrt{d_{Q}}})V$

where $Q$, $K$ and $V$ are matrices where $t$th row are $q_{t}$, $k_{t}$ and
$v_{t}$ respectively. Each unit in the multi-head attention learns its own set
of weights. The outputs are concatenated and fed to the feed-forward layers.

###  2.3 Baseline model

We adopt Trans-SVNet [5] as our baseline model, which achieved state-of-the-
art results on the Cholec80 dataset. To be consistant with previous works, we
first preprocess the dataset as specified in TMRNet [11]. Following the
training pipeline of Trans-SVNet, we established a baseline result according
to our dataset split.

###  2.4 Transition models

Transition models are trained in the same way as the baseline model, except
that we only train 2-class classifiers on two neighboring phases at a time. In
total, this process yields 6 transition models. The intuition behind this
approach is that 2-class predictors can better distinguish between two
sequential phases than the 7-class model, as demonstrated in Table 1.

Table 1: 2-class Transformer performance compared to baseline model. Model | Accuracy (%)
---|---
Trans-SVNet(baseline) | 87.44
$Trans_{1-2}$ | 96.04
$Trans_{2-3}$ | 95.19
$Trans_{3-4}$ | 94.71
$Trans_{4-5}$ | 97.85
$Trans_{5-6}$ | 93.48
$Trans_{6-7}$ | 83.47

###  2.5 Transition-based inference

####  2.5.1 Inference strategy

Upon training the transition models, we designed our inference scheme as
follows: (1) Initialize a buffer $B_{N}$ which stores the latest $N$
predictions and fill it with 1; (2) At each timestep, we look at the majority
element in $B_{N}$, if the majority elements is $i~{}(i<7)$, we use
$Trans_{i\\_(i+1)}$ for predicting the current label; if $i$ is 7, we use
$Trans_{6\\_7}$; (3) Append the current prediction to $B_{N}$ and pop out the
oldest prediction from $B_{N}$. The buffer size $N$ is set to 100.

####  2.5.2 Results and discussion

Table 2 shows the results from transition-based inference strategy. As we can
see, it does not show improvement upon the baseline model. This strategy is
good at avoiding noisy predictions from the baseline model, in which case the
predicted phase labels tend to jump back and forth during transition periods.
However, it also suffers from cascading effect as shown in Fig. 2, which means
a streak of incorrect predictions can lead to using the wrong transition model
much further in the video, which causes bad overall results.

Figure 2: Example of cascading effect. Table 2: Results on different inference strategies. Model | Accuracy (%)
---|---
Trans-SVNet(baseline) | 87.44
Transition-based | 86.92
Confidence-based w/o calibration | 65.64
Confidence-based w/ calibration | 88.02

###  2.6 Confidence-based inference

Expanding on the previous strategy, we devised another strategy which relies
on the confidence level output from the baseline model. Intuitively, when the
output logits have a low value, it is less confident in its prediction than an
otherwise high value. Based on this relative confidence level, we can set a
threshold and switch to a transition model when the confidence of the model is
below a desired level.

####  2.6.1 Inference strategy

We define our confidence-based inference strategy as follows: (1) Define a
threshold $t_{conf}$ for the confidence score and the latest prediction
$p_{last}=1$; (2) For each timestamp, use the baseline model to get a
prediction $p_{base}$ and the corresponding confidence level $c_{base}$; if
$c_{base}>t_{conf}$, update $p_{last}$ to $p_{base}$; if $c_{base}\leq
t_{conf}$, we look at the value of $p_{last}=i~{}(i<7)$, and use
$Trans_{i\\_(i+1)}$ to generate a substitute prediction $p_{s}$, then update
$p_{last}$ to $p_{s}$; when $p_{last}$ is 7, we simply use $Trans_{6\\_7}$.

####  2.6.2 Confidence calibration

While modern deep leaning models have dramatically improved neural network
accuracy, they are also more prone to miscalibration(cite). This problem
persists when applying our confidence-based inference strategy, as it heavily
relies on reasonable confidence scores from the model. To address this issue,
we apply temperature scaling(cite) after training to achieve a better
calibrated model. Temperature scaling uses a single scalar parameter $T>0$ for
calibration. Given the logit vector z, the new confidence score is

$\hat{q}=\max_{k}\sigma_{SM}(\textbf{z}/T)^{(k)}$ (1)

$\sigma_{SM}$ is the Softmax(cite) function, whereas $k=\\{1\ldots K\\}$ is
one of the $K$ classes. We use negative log-liklihood (NLL) [7] and expected
calibration error (ECE) [7] as metrics for measuring the level of calibration.
For both metrics, smaller values correspond to better calibration. The
calibration results are shown in Table 3.

Table 3: Confidence calibration results. Model | NLL | ECE
---|---|---
baseline | 0.576 | 0.215
calibrated | 0.402 | 0.031

####  2.6.3 Results and discussion

Table 2 shows the results of the confidence-based inference strategy. As we
can see, without calibration, the strategy performs poorly due to over-
confident logits from the baseline model. With temperature scaling, our
strategy was able to outperform the baseline model, demonstrating the
effectiveness of confidence calibration.

##  3 Conclusion

In this technical report, we explored incorporating transitional constraints
into surgical phase recognition with Transformer models. Through experiments,
we discovered that applying local transitional constraints can lead to
catastrophic cascading effects, while in the meantime avoids noisy boundaries.
Our proposed confidence-based inference strategy was able to leverage the
superior performance of 2-class models and achieved better overall accuracy.

## References

  * [1] Katia Charrière, Gwénolé Quellec, Mathieu Lamard, David Martiano, Guy Cazuguel, Gouenou Coatrieux, and Béatrice Cochener.  Real-time analysis of cataract surgery videos using statistical models.  Multimedia Tools and Applications, 76(21):22473–22491, 2017.
  * [2] Tobias Czempiel, Magdalini Paschali, Matthias Keicher, Walter Simson, Hubertus Feussner, Seong Tae Kim, and Nassir Navab.  Tecno: Surgical phase recognition with multi-stage temporal convolutional networks.  In International conference on medical image computing and computer-assisted intervention, pages 343–352. Springer, 2020.
  * [3] Tobias Czempiel, Magdalini Paschali, Daniel Ostler, Seong Tae Kim, Benjamin Busam, and Nassir Navab.  Opera: Attention-regularized transformers for surgical phase recognition.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 604–614. Springer, 2021.
  * [4] Olga Dergachyova, David Bouget, Arnaud Huaulmé, Xavier Morandi, and Pierre Jannin.  Automatic data-driven real-time segmentation and recognition of surgical workflow.  International journal of computer assisted radiology and surgery, 11(6):1081–1089, 2016.
  * [5] Xiaojie Gao, Yueming Jin, Yonghao Long, Qi Dou, and Pheng-Ann Heng.  Trans-svnet: accurate phase recognition from surgical videos via hybrid embedding aggregation transformer.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 593–603. Springer, 2021.
  * [6] Carly R Garrow, Karl-Friedrich Kowalewski, Linhong Li, Martin Wagner, Mona W Schmidt, Sandy Engelhardt, Daniel A Hashimoto, Hannes G Kenngott, Sebastian Bodenstedt, Stefanie Speidel, et al.  Machine learning for surgical phase recognition: a systematic review.  Annals of surgery, 273(4):684–693, 2021.
  * [7] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  In International Conference on Machine Learning, pages 1321–1330. PMLR, 2017.
  * [8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [9] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [10] Yueming Jin, Qi Dou, Hao Chen, Lequan Yu, Jing Qin, Chi-Wing Fu, and Pheng-Ann Heng.  Sv-rcnet: workflow recognition from surgical videos using recurrent convolutional network.  IEEE transactions on medical imaging, 37(5):1114–1126, 2017.
  * [11] Yueming Jin, Yonghao Long, Cheng Chen, Zixu Zhao, Qi Dou, and Pheng-Ann Heng.  Temporal memory relation network for workflow recognition from surgical video.  IEEE Transactions on Medical Imaging, 40(7):1911–1923, 2021.
  * [12] Lena Maier-Hein, Swaroop S Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, et al.  Surgical data science for next-generation interventions.  Nature Biomedical Engineering, 1(9):691–696, 2017.
  * [13] Arnab Majumder, Maria S. Altieri, and L. Michael Brunt.  How do i do it: laparoscopic cholecystectomy.  Annals of Laparoscopic and Endoscopic Surgery, 5(0), 2020.
  * [14] Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang.  Confidence-aware learning for deep neural networks.  In international conference on machine learning, pages 7034–7044. PMLR, 2020.
  * [15] Nicolas Padoy.  Machine and deep learning for workflow recognition during surgery.  Minimally Invasive Therapy & Allied Technologies, 28(2):82–90, 2019.
  * [16] Nicolas Padoy, Tobias Blum, Hubertus Feussner, Marie-Odile Berger, and Nassir Navab.  On-line recognition of surgical activity for monitoring in the operating room.  In AAAI, pages 1718–1724, 2008.
  * [17] Gwénolé Quellec, Mathieu Lamard, Béatrice Cochener, and Guy Cazuguel.  Real-time segmentation and recognition of surgical tasks in cataract surgery videos.  IEEE transactions on medical imaging, 33(12):2352–2360, 2014.
  * [18] Andru P Twinanda, Sherif Shehata, Didier Mutter, Jacques Marescaux, Michel De Mathelin, and Nicolas Padoy.  Endonet: a deep architecture for recognition tasks on laparoscopic videos.  IEEE transactions on medical imaging, 36(1):86–97, 2016.
  * [19] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.


# The Cognition of Counterexample in Mathematics Students

Shannon Ezzat  Department of Mathematics and Statistics
University of Winnipeg
515 Portage Avenue,
Winnipeg, Manitoba, Canada R3B 2E9<EMAIL_ADDRESS>(Shannon Ezzat) and
Scott Rodney  Department of Mathematics, Physics, and Geology
Cape Breton University
Box 5300, 1250 Grand Lake Road
Sydney, Nova Scotia, Canada B1P 6L2<EMAIL_ADDRESS>(Scott Rodney)

###### Abstract.

Studying Mathematics requires a synthesis of skills from a multitude of
academic disciplines; logical reasoning being chief among them. This paper
explores mathematical logical preparedness of students entering first year
university mathematics courses and also the effectiveness of using logical
facility to predict successful course outcomes. We analyze data collected from
students enrolled at the University of Winnipeg in a pre-service course for
high school teachers. We do find that, being able to successfully answer
logical questions, both before and after intervention, are significant in
relation to improved student outcomes.

Keywords. Counterexample, mathematical thinking, logical inference, student
success

##  1\. Introduction

This paper explores mathematical logic preparedness and its effect on success
in the first year university mathematics classroom. It is well known that
skill sets among students entering courses in university mathematics vary
widely from student to student and thus an understanding of the abilities and
effects of logical aptitude is a worthwhile endeavour to help improve student
success in mathematics.

This work focuses on two themes: student facility in reasoning about
counterexample, and how well assessment of this logical skill can predict
successful course outcomes in introductory mathematics courses.

To address these questions, a 3-month study of university students was
initiated in MATH-2903, Math for Early/Middle Year Teachers 111University of
Winnipeg (UW), Manitoba, Canada during the winter semester of the 2017-18
academic year. As indicated by its name, this class is designed for pre-
service elementary- and middle-school teachers. The prerequisite for the
course is any university-level math class from Grade 12 in Manitoba (with a
grade above 65% for applied math or any passing grade in precalculus) and does
not count towards any math major course, or as a required math course for
Bachelor of Science students. The students in this course predominantly have
poor to adequate arithmetic and algebraic skills and a substantially negative
attitude towards mathematics as a subject.

Our methods are inspired by several works on student ability with logic and
counterexample including [BF] which discusses undergraduate students
understanding of various concepts in calculus using the tool of diagnostic
tests, intervention/primers, and quizzes; [L], which studies the cognition of
counterexample of high school students in a similar fashion; and [IS], which
studies undergraduate (and staff) success in the Wason Selection Task. We use
these tools to measure students understanding of logical statements, and
specifically counterexamples to these statements.

A careful description of our study is found in Section 2. Section 3 contains
statistical analysis discussing evidence of predictive assessment question
types. A summary of our results together with concluding remarks is found in
Section 4. Note that in a future note the authors will report results for a
similar study from MATH-1105, Differential and Integral Calculus I 222Cape
Breton University (CBU), Nova Scotia, Canada.

Reasoning logically using counterexample is linked to the idea of conditional
statements. We begin with a discussion about conditional statements and the
notion of counterexample itself and its importance in mathematical pedagogy,
including a particular instance of a question type designed by Peter Wason [W]
called the Wason Selection Task. We refer to this as the Wason question in our
tests.

###  1.1. Conditional Statements

A conditional statement consists of two parts, an antecedent (if-part) and a
consequent (then-part). In general, these are of the form _“If P then Q”_ ,
for some statements P and Q. An example would be the following:

_If an integer is even then the rightmost digit of the number is either a
$2$,$4$,$6$ or $8$_.

An important component of logic and mathematics in general, is determining the
truth value (True(T) or False(F)) of a given conditional statement. To that
end it is sometimes helpful to reword conditionals using a universal
quantifier. In general, _“If P then Q”_ becomes _“All P are Q”_ and specific
to the example above:

_All even integers have the rightmost digit being $2$,$4$,$6$ or $8$_.

To falsify such a universally quantified statement is to demonstrate the
existence of an object that satisfies statement P but does not satisfy
statement Q; in our example, $10$ is such an object. We call such an object a
_counterexample_ to the conditional statement. In fact, a conditional
statement is false exactly when such an object exists. For more details, the
reader is encouraged to refer to the introductory chapters of most textbooks
in logic or discrete mathematics. It is important to see that the idea of
counterexample is not necessarily intuitive. We note one more example from our
diagnostic study to illustrate this point:

> If two shapes have the same perimeter then they have the same area.

On the diagnostic test, 31 of 62 responded correctly indicating that the
statement is false while only 8 of those justified their conclusion using a
counterexample. Counterexamples provided by correct respondents most often
used rectangles with few referring to circles or triangles.

###  1.2. The Importance of Counterexample

In mathematical reasoning, the notion of counterexample is fundamental.
Counterexamples allow us to understand the limits of mathematical results and
why each condition in a mathematical logical statement is needed to justify
the conclusion. In [K], Klymchuck writes that, for undergraduate students,
understanding by way of counterexamples “can improve students’ conceptual
understanding in mathematics, reduce their common misconceptions, provide a
broader view on the subject and enhance students’ critical thinking skills”.
For students that plan on a career in education Vinsonhaler and Lynch [VL]
believe that primary and secondary school teachers “ support student reasoning
and thinking and promote productive struggle by incorporating counterexamples
into the classroom.” Zazkis and Chernoff [ZC] suggest that counterexamples can
be particularly convincing when they are “in accord with individuals’ example
spaces”.

###  1.3. The Wason Question

The card selection question was originally reported by Wason [W]. In his
experiment, there are cards on a table and the participant is told that there
is a number on one side and a letter on the other. The participants are told
of a rule, for example _“If a card has a D on one side then it has a 3 on the
other side”_ or, reworded as a universally quantified statement _“All cards
with a D on one side have a 3 on the other side”_. Four cards are presented
that correspond to each logical case (P true, P false, Q true, Q false) and
the participant is asked _“to select all those cards, but only those cards,
which you would have to turn over in order to discover whether or not the rule
has been violated.”_
To answer the question, the participant is required to assess the conditional
statement four times in relation to each card presented. The correct answer is
to select the cards where either P is true or Q is false since these
conditions are necessary (but not sufficient) for the card to be a
counterexample or exception to the rule. Interestingly, most of the population
does not make this connection. Indeed, Inglis and Simpson in [IS] found from
an online, self-selected sample at the University of Warwick, only 29 percent
of math students, 8 percent of history students and 49 percent of mathematics
staff responded correctly. During our study described below, only 2 students
answered this style of question correctly on the pre-test; see §2.

###  1.4. Proving a conditional true

More difficult perhaps is the skill of reasoning the justification of a true
conditional. Consider the following sentence:

> If the sum of two positive whole numbers is odd then their product is even.

To justify the truth of this statement the participant must covey that, since
the sum of the numbers is odd, one number must be even while the other odd.
This requires more algebraic ability, and usually a considerable amount of
lecturing to convey; indeed, many universities have a course in the first or
second year of the program (usually Discrete Mathematics or a standalone
Introduction to Proofs course) that spend considerable time teaching the
concept of correct proof writing. We note that the diagnostic test did not
include any questions of this type, however as part of the course material,
some lecture time was spent teaching how to prove a conditional statement for
simple examples; e.g. “If $a,b,c$ are three consecutive whole numbers then
their sum has a factor of 3”.

As we will discuss in more detail, the ability to understand and construct
counterexamples to a conditional statement is not inherent in most students
entering Math 2903. These results are not surprising in the context of similar
statistical results found in [IS] and [L]. However, perhaps more interesting
is that on average, students who are able to respond well to these problems
have better course outcomes. We also observe that students who learn these
skills through in-course intervention experience better course outcomes on
average. As such, our study suggests that logical skill assessment can be used
as a tool to help improve course outcomes for students. Such assessment is
also effective in identifying struggling students early on so that they can
find effective help. Lastly, it is also quite interesting that a simple
intervention can on average have a measurable impact on student success.

##  2\. The Study

The data used for this study comes from Math 2903, Math for Early/Middle Year
Teachers at the University of Winnipeg. We chose students in Math 2903 since
it was, most likely, the students’ first university mathematics class. Indeed,
the likelihood that our student populations in the class had been exposed to
logical reasoning on an academic level (for example, a course on critical
thinking) was quite low. This is borne out in the results.

The study was conducted in four phases. In the first week of classes (Phase
I), students were given a diagnostic test to benchmark their understanding of
logical thinking. In the third week (Phase II), students were given an
intervention (75-minute lab/lecture on the idea of conditional statements and
counterexamples) complete with a worksheet for practice. During the 6th week
of classes (Phase III), a post test was given as part of the in-class midterm
exam. Final grades were tallied (Phase IV) and small sample size statistical
methods were used to analyze aggregate data collected. In what follows, we
give detailed descriptions of each phase of the study but leave Phase IV to
§3.

###  2.1. I - The Diagnostic Test

The Diagnostic test was designed to gauge incoming student ability to reason
by counterexample and was given during the first week of class. The problems
appearing on the test were selected from 2 categories similar to those found
in [L]. We describe them now. We also note that specific questions can be
found in §3.

####  2.1.1. Type I: The Wason selection task

As described in the introduction, this exercise concerns rule verification for
a finite set of objects. A complete description of the Wason selection task
can be found in both of [L] and [IS]. For this study, we depicted an array of
6 cards each with a number on one side and a letter on the other, see (2).

(2)

The student was told: “Each card has a number and letter on opposite sides.
Every card with a vowel on one side has an even number on the other. Of the
six cards below (it can be more than one) which must you flip over in order to
determine whether or not the rule has been violated? Try your best to explain
your reasoning.” Note that the correct answer to this problem is to flip the
A, 9, and 3, as these could be possible counterexamples; for example, the A
could have a 3 on the other side, and the 3 or 9 could have an A on the other
side, and these cards would violate the rule).

####  2.1.2. Type II: Provide a counterexample to a given conditional
statement

Here the student is asked to falsify a conditional statement by constructing a
counterexample. For example,

> “If two shapes have the same perimeter then they have the same area”

is seen to be false by considering two rectangles with dimensions $2\times 4$
and $5\times 1$. This forms the required counterexample.

###  2.2. II - The Intervention

The intervention consisted of a 75 minute primer on conditional statements
that occurred during the second week of classes. During the intervention,
students were introduced to the concept of a conditional statement, initially
via a truth table, and the relationship between universally quantified
statements and conditionals. Students were then shown an applied example of a
liquor inspector enforcing the rule described by the conditional statement “If
you are drinking alcohol then you must be 18 years of age or older” to
reinforce the idea of when a conditional statement is false. This was then
connected to the notion of counterexample. The intervention lecture closed
with a discussion of the Wason selection task where students were shown the
relationship between the truth values of the conditional sentence defining the
rule and the values on each of the cards. This was followed by a group work
session where students reinforced these ideas by completing a practice
worksheet with their peers.

In addition to this 75 minute primer, as part of the course work students were
given one lesson on verifying a conditional statement via an algebraic proof.
The students were also given a small number of beginner practice problems, and
one problem on an assignment to complete. Note that this type of question was
not included on the original diagnostic test as algebraic verification is a
technique that is more specialized and the instructor of the course (the first
author) believed that most students would not have been able to answer the
question appropriately.

###  2.3. III - The Post Test

The post test was given as part of the MATH-2903 class midterm test. The
students were given one Wason Selection Task question, as well as one question
where the students would have to identify which of two conditional statements
had a counterexample, provide a counterexample for that statement, and
algebraically justify the other conditional statement.

##  3\. Data and Results

The aim of the data analysis found in this section is to compile and compare
results of the diagnostic test (DT) and post test (PT). Our results, presented
in tables 1-6, analyze relationships between success/retention/difficulty with
specific problems on our tests with mean final course grades. In order to
present this efficiently, we group and name specific problems of interest
together with our raw data. Our data consists of evaluated student performance
in terms of the following categories.

  * (i)

CR: _Correct answer and reasoning._ This evaluation is given if the student
provides a clear, complete solution to the problem providing a counterexample
if necessary.

  * (ii)

C: _Correct answer with incorrect reasoning._ The student was able to answer
the problem correctly but lacked sufficient justification such as, for
example, an erroneous or missing counterexample.

  * (iii)

I: _Incorrect answer._ The student has answered incorrectly with or without
justification.

  * (iv)

B: _The student left the problem blank._

###  3.1. The Diagnostic Test

The diagnostic test consisted of 5 questions of varying type that we state now
followed by student response data.

  * •

DT:1 If a number $n$ is less than 1, then the expression $n^{2}-1$ is
negative.

  * •

DT:2 If two shapes have the same perimeter, then they have the same area.

  * •

DT:3 If the sum of two numbers is odd, then their product is even.

  * •

DT:4 If two numbers $x,y$ have positive sum, then both $x$ and $y$ are
positive.

  * •

DT:5 This is the Wason selection task problem exactly as given in section
§2.1.1. with cards as presented in figure (2).

Table 1. Summary of student answers to counterexample questions on diagnostic test Question |  CR |  C |  I |  B
---|---|---|---|---
DT:1 |  19 |  14 |  19 |  10
DT:2 |  8 |  23 |  20 |  11
DT:3 |  18 |  18 |  9 |  17
DT:4 |  26 |  26 |  6 |  4

For the Wason selection task problem DT:5, we analyze student responses more
finely using common observed card selections. Category “A93” refers to the
student selecting the correct cards, the ace, nine, and three. The other
categories, “A”, “A4”, “All”, “Blank” and “Other” are self evident.

Table 2. Summary of student answers to Wason Selection Task question on diagnostic test Answers | A93 | A | A4 | All | Blank | Other
---|---|---|---|---|---|---
Question 2 | 3 | 2 | 26 | 12 | 10 | 9

######  Remark 3.1.

We note that very few students correctly answered the question (A93) on the
diagnostic test. Furthermore the most common answer was to select the A and
four, (A4). This choice is the common fallacy that the conditional “if $P$
then $Q$” is equivalent to “$P$ and $Q$”. This result aligns with previous
work by Inglis and Simpson [IS], and Wason [W].

###  3.2. The Post Test:

The post test consisted of three questions PT:1, PT:2, PT:3 on the first
midterm of Math 2903. Question PT:1 was the Wason selection task problem as in
§2.1.1, see (2), with the altered rule

“if a card has an odd number on one side then it has a vowel on the other”.

The correct answer is to select the nine, three, K and Q (93KQ) as in (2).
Questions PT:2 and PT:3 appeared as two parts of a single problem that we
state now.

_One of the following conditional statements is always true and the other has
a counterexample. ***Circle (a) or (b)*** of the the true statement and
justify that it is true (using the direct method). Find a counterexample for
the other statement (and explain why it is a counterexample)._

  * •

PT:2 _(a) If_ $m$ and $n$ are whole numbers and $m\times n$ is even, then both
$m$ and $n$ are even.

  * •

PT:3 _(b) If_ $n$ is an even number then $n^{2}$ is a multiple of $4$.

The results of the post-test questions are shown in the table below. Students
are grouped into two categories: correct, and otherwise.

Table 3. Number of students answering post-test questions correctly Question | Correct Answer | Correct | Incorrect
---|---|---|---
PT:1 | Selected (93KQ) | 21 | 38
PT:2 | Provided a counterexample | 41 | 18
PT:3 | Provided a correct proof | 12 | 47

######  Remark 3.2.

In contrast with Remark 3.1 and noting that this specific question type was
covered in the stage II intervention, it is very encouraging that a far larger
proportion of students answered correctly. However, checking the truth of “$P$
and $Q$” is still frequent.

###  3.3. Relationship between counterexample knowledge and final grade

Below are tables with mean final grades in Math 2903 grouped by their results
on the diagnostic and post-tests, respectively. We used independent two-sample
t-tests (equal variances not assumed) to determine if these groups means
differ significantly. We note that this analysis is moot for problem PT:5 as
most students answered incorrectly.

Table 4. Comparison of mean final grade (MFG) of students by diagnostic question success Question | MFG (Correct/Justified) | MFG (Otherwise) | p-value
---|---|---|---
DT:1 | 73.9 | 66.5 | 0.072
DT:2 | 81.2 | 66.7 | 0.019
DT:3 | 75.4 | 66.1 | 0.029
DT:4 | 70.0 | 67.9 | 0.582
DT:5 | * | * | *
Table 5. Comparison of mean final grade (MFG) of students by post-test question success Question | MFG (Correct/Justified) | MFG (Otherwise) | p-value
---|---|---|---
PT:1 | 78.0 | 64.2 |  $<$0.001
PT:2 | 70.0 | 67.1 | 0.471
PT:3 | 82.3 | 65.8 |  $<$0.001

To aid in answering the question of whether learning logic and counterexample
affects student success, for each of the parts of DT:1-DT:4 on the Math 2903
diagnostic test we looked at the mean final grades for the students who did
not answer the question completely correctly (not both correct and justified).
We then compared the mean final grades of those answering the PT:2 on the
post-test question correctly (CR: correct and justified) and those that did
not. We again performed an independent two-sample t-test (equal variances not
assumed) to determine if these groups means differ significantly. The results
are in the table below.

Table 6. Comparison of mean final grade (MFG) of students with incorrect diagnostic test answers by post-test question success Incorrect | MFG (PT:2 CR) | MFG (Otherwise) | p-value
---|---|---|---
DT:1 | 69.1 | 61.6 | 0.104
DT:2 | 67.2 | 65.9 | 0.76
DT:3 | 68.7 | 60.2 | 0.053
DT:4 | 68.9 | 66.1 | 0.610

##  4\. Conclusions

We set out to study the effect of minimal logical training on student
outcomes. Our data shows interesting trends associated to the effect of
understanding the idea of counterexamples in logic. Through our investigations
we found that certain problems types can be used as checks for students less
likely to find success in their mathematics course. We now examine these ideas
specifically in relation to our data and then once again in the context of our
perspective on learning mathematics in general.

From the data above, we can reasonably conclude two important things: first,
that very few students have a strong idea of counterexample entering Math
2903, though many do have some intuition about whether a statement is true or
false, without the skills to definitively show why. Indeed, from Table 2 it
can be seen that only 2 out of 60 students chose the correct answer, and the
plurality of students checking both P true and Q true. This is in line with
the findings presented in [IS]. Additionally, only 8 of 62 students answered
DT:2 correctly.

From the midterm data, it is worth noting that these pre-service teachers can
learn logical reasoning and counterexample. Given the data for the post-test,
it is clear that students that do successfully learn the idea of logic and
counterexamples have more success in the course overall than students that do
not. From Table 5 we note that while the Wason question (PT:1) was worth
approximately 1 per cent of the students’ final grade, students that do answer
the question correctly do, on average, 13.8 percentage points better in the
course than those that do not. We emphasize that this question tests exactly
one logical idea, without any computational or algebraic skills needed.
Indeed, the Wason Selection Task seems to be a highly discriminant question.
More research is needed to determine whether this logical knowledge is a
driver of higher course grades, or whether stronger students simply learn
these ideas more quickly than other students.

Question PT2, where the student needed to supply a counterexample, was not
very predictive. We believe this is the case since the question may have been
too intuitive, though students who did not understand counterexamples in the
beginning of the course and did indeed answer this question correctly seemed
to have a significant, or near-significant, difference in mean grades as
opposed to students who did not learn the idea of counterexample. We
conjecture that it is important to understand the idea of counterexample at
its core and not just intuitively for easy cases, as a central idea of
mathematics is using logical and mathematical techniques to understand non-
intuitive problems. Indeed, if we chose a slightly more challenging
counterexample problem it is possible that we would have seen more significant
differences in Table 6.

Question PT3 was highly predictive. We are not surprised by this data; the
skills to write a correct proof mean that you understand the statement and the
associated objects it deals with in generality, and have the computational and
algebraic skills necessary to complete this proof.

In our comparison of mean final grades of students that did not answer the
first four diagnostic questions correctly, we find that, while the mean final
grades of the students who did answer PT:2 correctly was higher for all
diagnostic test questions, we find that at $\alpha=0.05$ none of these means
were significantly higher than the groups of students that answered PT:2
incorrectly. However, we would conjecture that we may have found a significant
difference in mean final grade if PT:2 was a more difficult question.

Putting this all together, we believe that the data supports the current
research that counterexample plays a very strong role in mathematical
understanding in early undergraduate mathematics. Indeed, a student gaining
understanding of the idea significantly improves student success in their
respective course.

## References

  * [BF] C. Bardelle and P. L. Ferrari, Definitions and examples in elementary calculus: the case of monotonicity of functions, ZDM Mathematics Education 2011 43:233-246, DOI 10.1007/s11858-101-0303-4
  * [W] P. C. Wason, Reasoning about a rule, Quarterly Journal of Experimental Psychology 1968 Vol. 20 Is. 3:273-281
  * [K] S. Klymchuk, Using Counter-Examples to Enhance Learners’ Understanding of Undergraduate Mathematics 2009, Good Practice Publication Grants New Zealand: Ako Aotearoa National Centre on Tertiary Teaching Excellence (Online)
  * [VL] R. Vinsonhaler and A. G. Lynch, Students’ Understanding of Counterexamples 2020, Mathematics Teacher: Learning and Teaching PK-12 MTLT, Vol. 113 Iss. 3:223-228
  * [DO] B. Divjak and D. Oreški, Prediction of Academic Performance Using Discriminant Analysis, Proceedings of the ITI 2009 31st Int. Conf. on Information Technology Interfaces, June 22-25, 2009, Cavtat, Croatia
  * [IS] A. Inglis and M. Simpson, Mathematics and the selection task, Proceedings of the 28th Conference of the International Group for the Psychology of Mathematics Education, 2004
  * [KBT] F. Bagheri, O. Kara, and T. Tolin, Factors Affecting Students’ Grades In Principles Of Economics, Amer. J. Bus. Ed. 2009 Vol. 2 No.7:25-34
  * [L] K. Lee, Student’s logical reasoning and mathematical proving of implications, PhD thesis, Michigan State University, 2011
  * [ZC] R. Zazkis, E.J. Chernoff, What makes a counterexample exemplary?, Educational Studies in Mathematics, Volume 68:195-208


# On sequences of integers with small prime factors

C.L. Stewart  Department of Pure Mathematics, University of Waterloo
Waterloo, Ontario, Canada N2L 3G1<EMAIL_ADDRESS>For Professor Henryk
Iwaniec on the occasion of his seventy-fifth birthday

###### Abstract.

We show that the difference between consecutive terms in sequences of integers
whose greatest prime factor grows slowly tends to infinity.

###### Key words and phrases:

small prime factors, linear forms in logarithms

###### 2020 Mathematics Subject Classification:

Primary 11N25; Secondary 11J86

##  1\. Introduction

Let $y$ be a real number with $y\geq 3$ and let $1=n_{1}<n_{2}<n_{3}<...$ be
the increasing sequence of positive integers composed of primes of size at
most $y$. In 1908 Thue [15] proved that

(1) $\lim_{i\to\infty}n_{i+1}-n_{i}=\infty,$

see also Pólya [12] and Erdős [5]. Thue’s result was ineffective. In
particular his proof does not allow one to determine, for every positive
integer $m$, an integer $i(m)$ such that $n_{i+1}-n_{i}$ exceeds $m$ whenever
$i$ is larger than $i(m)$. Cassels [3] showed how (1) can be made effective by
means of estimates due to Gelfond [6] for linear forms in two logarithms of
algebraic numbers. In 1973 Tijdeman [16] proved, by appealing to work of Baker
[2] on estimates for linear forms in the logarithms of algebraic numbers, that
there is a positive number $c$, which is effectively computable in terms of
$y$, such that

(2) $n_{i+1}-n_{i}>n_{i}/(\log n_{i})^{c}$

for $n_{i}\geq 3$. In addition Tijdeman showed that there are arbitrarily
large integers $n_{i}$ for which (2) fails to hold when $c$ is less than
$\pi(y)-1$; here $\pi(x)$ denotes the counting function for the primes up to
$x$.

Now let $y=y(x)$ denote a non-decreasing function from the positive real
numbers to the real numbers of size at least $3$. For any integer $n$ let
$P(n)$ denote the greatest prime factor of $n$ with the convention that
$P(0)=P(\pm 1)=1$. Let $(n_{i})^{\infty}_{i=1}$ be the increasing sequence of
positive integers $n_{i}$ for which

(3) $P(n_{i})\leq y(n_{i}).$

For any integer $k$ with $k\geq 2$ let $\log_{k}$ denote the $k$-th iterate of
the function $x\rightarrow\max(1,\log x)$ for $x>0$. We shall prove that (1)
holds provided that

(4) $y(n)=o(\frac{\log_{2}n\log_{3}n}{\log_{4}n}).$

Furthermore if we assume the abc conjecture, see §2, then we can prove that
(1) holds provided that

(5) $y(n)=o(\log n).$

For any real number $x\geq 2$ put

$\delta(x)=\exp(\frac{x\log_{2}x}{\log x}).$

We shall deduce (4) from the following result.

######  Theorem 1.

Let $y=y(x)$ be a non-decreasing function from the positive real numbers to
the real numbers of size at least $3$. Let $(n_{1},n_{2},...)$ be the
increasing sequence of positive integers $n_{i}$ for which (3) holds. There is
an effectively computable positive number $c$ such that for $i\geq 3,$

(6) $n_{i+1}-n_{i}>n_{i}/(\log n_{i})^{\delta(cy(n_{i+1}))}.$

Furthermore there is an effectively computable positive number $c_{1}$ such
that for infinitely many positive integers $i$

(7) $n_{i+1}-n_{i}<n_{i}\exp(c_{1}y(n_{i}))/(\log n_{i})^{r-1},$

where $r=\pi(y(\sqrt{n_{i}})).$

Observe that we obtain (1) from (6) when (4) holds on noting that in this case
$n_{i+1}\leq 2n_{i}$ and

$(\log n)^{\delta(cy(n))}=o(n).$

In order to establish (6) we shall appeal to an estimate for linear forms in
the logarithms of rational numbers due to Matveev [9], [10]. The upper bound
(7) follows from an averaging argument based on a result of Ennola [4].

We are able to refine the lower bound (6) provided that the abc conjecture is
true.

######  Theorem 2.

Let $y=y(x)$ be a non-decreasing function from the positive real numbers to
the real numbers of size at least $3$. Let $(n_{1},n_{2},...)$ be the
increasing sequence of positive integers $n_{i}$ for which (3) holds and let
$\varepsilon$ be a positive real number. If the abc conjecture is true then
there exists a positive number $c_{1}=c_{1}(\varepsilon)$, which depends on
$\varepsilon$, and a positive number $c_{2}$ such that for $i\geq 1$,

(8)
$n_{i+1}-n_{i}>c_{1}(\varepsilon)n_{i}^{1-\varepsilon}/\exp{(c_{2}y(n_{i+1}))}.$

We obtain (1) from (8) when (5) holds since in this case

$\exp{(c_{2}y(n))}=o(n).$

##  2\. Preliminary lemmas

For any non-zero rational number $\alpha$ we may write $\alpha=a/b$ with $a$
and $b$ coprime integers and with $b$ positive. We define $H(\alpha)$, the
height of $\alpha$, by

$H(\alpha)=\max(|a|,|b|).$

Let $n$ be a positive integer and let $\alpha_{1},...,\alpha_{n}$ be positive
rational numbers with heights at most $A_{1},...,A_{n}$ respectively. Suppose
that $A_{i}\geq 3$ for $i=1,...,n$ and that
$\log\alpha_{1},...,\log\alpha_{n}$ are linearly independent over the
rationals where $\log$ denotes the principal value of the logarithm. Let
$b_{1},...,b_{n}$ be non-zero integers of absolute value at most $B$ with
$B\geq 3$ and put

$\Lambda=b_{1}\log\alpha_{1}+...+b_{n}\log\alpha_{n}.$

######  Lemma 3.

There exists an effectively computable positive number $c_{0}$ such that

$\log|\Lambda|>-c_{0}^{n}\log A_{1}...\log A_{n}\log B.$

###### Proof.

This follows from Theorem 2.2 of Nesterenko [11], which is a special case of
the work of Matveev [9], [10]. ∎

Let $x$ and $y$ be positive real numbers with $y\geq 2$ and let $\Psi(x,y)$
denote the number of positive integers of size at most $x$ all of whose prime
factors are of size at most $y$. Let $r$ denote the number of primes of size
at most $y$ so that $r=\pi(y).$

######  Lemma 4.

For $2\leq y\leq(\log x)^{1/2}$ we have

$\Psi(x,y)=\frac{(\log x)^{r}}{\prod^{r}_{i=1}(i\log p_{i})}(1+O(y^{2}(\log
x)^{-1}(\log y)^{-1})).$

###### Proof.

This is Theorem 1 of [4]. ∎

We also recall the abc conjecture of Oesterlé and Masser [7], [8], [14]. Let
$x,y$ and $z$ be positive integers. Denote the greatest square-free factor of
$xyz$ by $G=G(x,y,z)$ so

$G=\prod_{{\begin{subarray}{c}p|xyz\\\ p,prime\end{subarray}}}p.$

######  Conjecture 5.

(abc conjecture) For each positive real number $\varepsilon$ there is a
positive number $c(\varepsilon)$, which depends on $\varepsilon$ only, such
that for all pairwise coprime positive integers $x,y$ and $z$ with

$x+y=z$

we have

$z<c(\varepsilon)G^{1+\epsilon}.$

For a refinement of the abc conjecture see [13].

##  3\. Proof of Theorem 1

Let $c_{1},c_{2},...$ denote effectively computable positive numbers.
Following [16], for $i\geq 3$ we have $n_{i}\geq 3,$

(9) $n_{i+1}-n_{i}=n_{i}(\frac{n_{i+1}}{n_{i}}-1)$

and, since $e^{z}-1>z$ for $z$ positive,

(10) $\frac{n_{i+1}}{n_{i}}-1>\log\frac{n_{i+1}}{n_{i}}.$

Let $p_{1},...,p_{r}$ be the primes of size at most $y(n_{i+1})$. Notice that
$r\geq 2$ since $y(n_{i+1})\geq 3$. Then
$\frac{n_{i+1}}{n_{i}}=p_{1}^{l_{1}}...p_{r}^{l_{r}}$ with $l_{1},...,l_{r}$
integers of absolute value at most $c_{1}\log n_{i+1}$ and, since $n_{i+1}\leq
2n_{i},$

(11) $\max(|l_{1}|,...,|l_{r}|)\leq c_{2}\log n_{i}.$

Since

$\log\frac{n_{i+1}}{n_{i}}=l_{1}\log p_{1}+\cdots+l_{r}\log p_{r}$

it follows from (11) and Lemma 3 that

(12) $\log\frac{n_{i+1}}{n_{i}}>(\log n_{i})^{-c_{3}^{r}\log p_{1}...\log
p_{r}}.$

By the arithmetic-geometric mean inequality

(13) $\prod^{r}_{i=1}\log p_{i}\leq(\frac{1}{r}\sum^{r}_{i=1}\log p_{i})^{r}$

and by the prime number theorem

(14) $\sum^{r}_{i=1}\log p_{i}<c_{4}r\log r.$

Thus, from (12), (13) and (14),

(15) $\log\frac{n_{i+1}}{n_{i}}>(\log n_{i})^{-(c_{5}\log r)^{r}}.$

Observe that $r\geq 2$ and so

(16) $(c_{5}\log r)^{r}<e^{c_{6}r\log_{2}r}.$

Further

$3\leq p_{r}\leq y(n_{i+1})$

and so

(17) $r\leq c_{7}y(n_{i+1})/\log y(n_{i+1}).$

Thus, by (16) and (17),

(18) $(c_{5}\log r)^{r}<\delta(c_{8}y(n_{i+1}))$

and (6) follows from (9), (10), (15) and (18).

We shall now establish (7). Observe that if $n_{i}$ satisfies (3) then since
$y(t)\geq 3$ for all positive real numbers $t$, $P(2n_{i})\leq y(n_{i})\leq
y(2n_{i})$ and so $2n_{i}=n_{j}$ for some integer $j$ with $j>n$. In
particular $n_{i+1}\leq 2n_{i}$ hence $n_{i+1}-n_{i}\leq n_{i}$ so

(19) $n_{i+1}-n_{i}<2n_{i}.$

Suppose that $X$ is a real number with $X\geq 9$ and that $i$ is a positive
integer with $n_{i+1}$ and $n_{i}$ in the interval $(\sqrt{X},X]$. If, in
addition,

(20) $y(\sqrt{X})>(\log X)^{\frac{1}{4}}$

then, since $\sqrt{X}<n_{i}\leq X,$

(21) $y(n_{i})>(\log n_{i})^{\frac{1}{4}}.$

Since $y$ is non-decreasing

(22) $\pi(y(\sqrt{n}_{i}))-1\leq\pi(y(n_{i}))$

and by the prime number theorem

$\pi(y(n_{i}))<c_{9}\frac{y(n_{i})}{\log y(n_{i})}.$

By (21),

(23) $\pi(y(n_{i}))<c_{10}\frac{y(n_{i})}{\log_{2}n_{i}}.$

Thus by (22) and (23),

(24) $(\log n_{i})^{\pi(y(\sqrt{n}_{i}))-1}<e^{c_{10}y(n_{i})}.$

We may suppose that $c_{1}$ exceeds $1+c_{10}$ and in this case, by (24),

$\exp(c_{1}y(n_{i}))/(\log
n_{i})^{\pi(y(\sqrt{n}_{i}))-1}\geq\exp(y(n_{i}))\geq\exp(3)\geq 2,$

and therefore (7) follows from (19).

We shall now show that there is a positive number $c_{11}$ such that if $X$ is
a real number with $X>c_{11}$ then there is a positive integer $i$ for which
$n_{i+1}$ and $n_{i}$ are in $(\sqrt{X},X]$ and satisfy (7). Accordingly let
$X$ be a real number with $X\geq 9$ and put

$r=\pi(y(\sqrt{X})).$

Notice that $r\geq 2$ since $y(t)\geq 3$ for all positive real numbers $t$. By
the preceding paragraph we may suppose that

$y(\sqrt{X})\leq(\log X)^{\frac{1}{4}}.$

Let $A(X)$ be the set of integers $n$ with

(25) $\sqrt{X}<n\leq X$

for which

(26) $P(n)\leq y(\sqrt{X}).$

Note that the members of $A(X)$ occur as terms in the sequence
$(n_{1},n_{2},...)$. The cardinality of $A(X)$ is

$\Psi(X,y(\sqrt{X}))-\Psi(\sqrt{X},y(\sqrt{X}))$

and so for $X>c_{12}$ is, by Lemma 4, at least

(27) $\frac{(\log X)^{r}}{2\prod^{r}_{i=1}i\log p_{i}}.$

Let $j$ be the positive integer for which

$\frac{X}{2^{j}}<\sqrt{X}\leq\frac{X}{2^{j-1}}$

and consider the intervals $(\frac{X}{2^{k}},\frac{X}{2^{k-1}}]$ for
$k=1,...,j.$ Then $j\leq 1+\frac{\log X}{2\log 2}$ and so, for $X>c_{13}$,

(28) $j\leq\log X.$

Thus, by (27) and (28), there is an integer $h$ with $1\leq h\leq j$ for which
the interval $(\frac{X}{2^{h}},\frac{X}{2^{h-1}}]$ contains at least

$\frac{(\log X)^{r-1}}{2\prod^{r}_{i=1}i\log p_{i}}$

integers from $A(X)$. Notice that

$\prod^{r}_{i=1}i\log p_{i}\leq(r\log y(\sqrt{X}))^{r}.$

Thus, since $y(\sqrt{X})\leq(\log X)^{\frac{1}{4}}$ and, since $r\geq 2$,
$r-1\geq\frac{r}{2}$ we see that for $X>c_{14},$ the interval
$(\frac{X}{2^{h}},\frac{X}{2^{h-1}}]$ contains at least

$\frac{(\log X)^{r-1}}{3(r\log y(\sqrt{X}))^{r}}+1$

terms from $A(X)$ hence two of them, say $n_{i+1}$ and $n_{i}$, satisfy

$n_{i+1}-n_{i}<\frac{X}{2^{h}(\log X)^{r-1}}3(r\log y(\sqrt{X}))^{r}.$

Since $n_{i}>\frac{X}{2^{h}}$ it follows that

$n_{i+1}-n_{i}<3\frac{n_{i}}{(\log n_{i})^{r-1}}(r\log y(\sqrt{X}))^{r}.$

By (25), $\sqrt{n}_{i}\leq\sqrt{X}\leq n_{i}$ hence, since $y$ is non-
decreasing, $y(\sqrt{n}_{i})\leq y(\sqrt{X})\leq y(n_{i})$. Thus

$n_{i+1}-n_{i}<3\frac{n_{i}}{(\log n_{i})^{r-1}}(r\log y(n_{i}))^{r}$

and so

(29) $n_{i+1}-n_{i}<3\frac{n_{i}}{(\log n_{i})^{r^{\prime}-1}}(s\log
y(n_{i}))^{s}$

where $r^{\prime}=\pi(y(\sqrt{n}_{i}))$ and $s=\pi(y(n_{i}))$. By the prime
number theorem there is a positive number $c_{15}$ such that

(30) $3(s\log y(n_{i}))^{s}<e^{c_{15}y(n_{i})}.$

Estimate (7) now follows from (29) and (30). On letting $X$ tend to infinity
we find infinitely many pairs of integers $n_{i+1}$ and $n_{i}$ which satisfy
(7).

##  4\. Proof of Theorem 2

Let $i\geq 1$ and put

(31) $n_{i+1}-n_{i}=t.$

Let $g$ be the greatest common divisor of $n_{i+1}$ and $n_{i}$. Then

$\frac{n_{i+1}}{g}-\frac{n_{i}}{g}=\frac{t}{g}.$

Let $\varepsilon>0$. By the abc conjecture there is a positive number
$c(\varepsilon)$ such that

$\frac{n_{i}}{g}<c(\varepsilon)(\frac{t}{g}\prod_{p\leq
y(n_{i+1})}p)^{1+\varepsilon}$

hence

(32) $(\frac{n_{i}}{c(\varepsilon)})^{\frac{1}{1+\varepsilon}}<t\prod_{p\leq
y(n_{i+1})}p.$

By the prime number theorem, since $y(n_{i+1})\geq 3,$ there exists a positive
number $c_{2}$ such that

(33) $\prod_{p\leq y(n_{i+1})}p<e^{c_{2}y(n_{i+1})}.$

The result follows from (31), (32) and (33).

##  5\. Acknowledgements

This research was supported in part by the Canada Research Chairs Program and
by grant A3528 from the Natural Sciences and Engineering Research Council of
Canada.

## References

  * [1]
  * [2] A. Baker, A sharpening of the bounds for linear forms in logarithms, Acta Arith. 21 (1972), 117–129.
  * [3] J.W.S. Cassels, On a class of exponential equations, Arkiv f. Mat. 4 (1960), 231-233.
  * [4] V. Ennola, On numbers with small prime divisors, Ann. Acad. Sci. Fennicae (Series AI) 440 (1969), 1-16.
  * [5] P. Erdős, Some recent advances and current problems in number theory, Lectures on Modern Mathematics, Vol. III, 196-244, Wiley, New York, 1965.
  * [6] A.O. Gelfond, Transcendental and Algebraic Numbers, (Moscow, 1952; Dover, New York, 1960).
  * [7] D.W. Masser, Open problems, Proc. Symp. Analytic Number Theory (W.W.L. Chen, ed.), Imperial College, London, 1985.
  * [8] D.W. Masser, Abcological anecdotes, Mathematika 63 (2017), 713-714.
  * [9] E.M. Matveev, An explicit lower bound for a homogeneous rational linear form in logarithms of algebraic numbers, (Russian) Izv. Ross. Akad. Nauk Ser. Mat. 62 (1998), 81–136, (English) Izv. Math. 62 (1998), 723–772.
  * [10] E.M. Matveev, An explicit lower bound for a homogeneous rational linear form in logarithms of algebraic numbers. II, (Russian) Izv. Ross. Akad. Nauk Ser. Mat. 64 (2000), 125–180, (English) Izv. Math. 64 (2000), 1217–1269.
  * [11] Y. Nesterenko, Linear forms in logarithms of rational numbers, Diophantine approximation (Cetraro 2000), Lecture Notes in Math., vol. 1819, 53-106, Springer, Berlin, 2003.
  * [12] G. Pólya, Zur arithmetischen Untersuchung der Polynome, Math. Z. 1 (1918), 143-148.
  * [13] O. Robert, C.L. Stewart and G.Tenenbaum, A refinement of the abc conjecture, Bull. London Math. Soc. 46 (2014), 1156-1166.
  * [14] C.L. Stewart and Kunrui Yu, On the abc conjecture, II, Duke Math. Journal 108 (2001), 169-181.
  * [15] A. Thue, Bermerkungen über gewisse Näherungsbrüche algebraischer Zahlen, Christiania Vidensk. Selsk. Skr. (1908), Nr.3.
  * [16] R. Tijdeman, On integers with many small prime factors, Compositio Math. 26 (1973), 319–330.