latexText page

# Defining and comparing SICR-events for classifying impaired loans under IFRS
9

Arno Botha  ORC iD: 0000-0002-1708-0153; Corresponding author:
<EMAIL_ADDRESS>Centre for Business Mathematics and Informatics,
North-West University, Private Bag X6001, Potchefstroom, 2520, South Africa
Esmerelda Oberholzer  ORC iD: 0000-0003-1985-7314;
<EMAIL_ADDRESS>Centre for Business Mathematics and Informatics,
North-West University, Private Bag X6001, Potchefstroom, 2520, South Africa
Janette Larney  ORC iD: 0000-0003-0091-9917<EMAIL_ADDRESS>Centre
for Business Mathematics and Informatics, North-West University, Private Bag
X6001, Potchefstroom, 2520, South Africa Riaan de Jongh  ORC iD:
0000-0002-7979-256X<EMAIL_ADDRESS>Centre for Business Mathematics
and Informatics, North-West University, Private Bag X6001, Potchefstroom,
2520, South Africa

###### Abstract

The IFRS 9 accounting standard requires the prediction of credit deterioration
in financial instruments, i.e., significant increases in credit risk (SICR).
However, the definition of such a SICR-event is inherently ambiguous, given
its reliance on comparing two subsequent estimates of default risk against
some arbitrary threshold. We examine the shortcomings of this approach and
propose an alternative framework for generating SICR-definitions, based on
three parameters: delinquency, stickiness, and the outcome period. Having
varied these parameters, we obtain 27 unique SICR-definitions and fit logistic
regression models accordingly using rich South African mortgage data; itself
containing various macroeconomic and obligor-specific input variables. This
new SICR-modelling approach is demonstrated by analysing the resulting
portfolio-level SICR-rates (of each SICR-definition) on their stability over
time and their responsiveness to economic downturns. At the account-level, we
compare both the accuracy and flexibility of the SICR-predictions across all
SICR-definitions, and discover several interesting trends during this process.
These trends form a rudimentary expert system for selecting the three
parameters optimally, as demonstrated in our recommendations for defining
SICR-events. In summary, our work can guide the formulation, testing, and
modelling of any SICR-definition, thereby promoting the timeous recognition of
credit losses; the main imperative of IFRS 9.

Keywords—  IFRS 9; Credit risk modelling; Classification systems; SICR
definitions

JEL:  C31, C44, G21.

Word count (excluding front matter and appendix): 9077

Figure count: 10

## 1 Introduction

It is no easy task to define a so-called SICR-event, or a significant increase
in credit risk (SICR), which is essentially a binary event. One common
approach relies on estimating a loan’s default risk, also known as its
probability of default (PD) where ‘default’ is another type of binary event.
Let this PD be denoted by $p_{D}(x,t)$ given risk information $x$ observed at
time $t$ for a specific loan account. A SICR-event can then be defined by
comparing $p_{D}(x,t_{r})$ with $p_{D}(x,t_{0})$ between reporting time
$t_{r}$ and account origination time $t_{0}$, which reflects §5.5.9 in the
global accounting standard [29]. Should the magnitude
$p_{D}(x,t_{r})-p_{D}(x,t_{0})$ exceed some arbitrarily chosen threshold, then
a SICR-event is said to have occurred. This approach immediately highlights at
least two challenges in establishing whether credit quality has deteriorated
significantly. Selecting an appropriate threshold for the magnitude is non-
trivial and highly subjective, which is exacerbated by IFRS 9 being principled
instead of overly prescriptive. Secondly, any reliance on the point estimate
$p_{D}(x,t)$ tacitly requires a certain degree of accuracy, lest the
subsequent comparison become meaningless. However, attaining sufficient
accuracy can itself become challenging given the stochastic nature of default
risk, especially when considering an ever-changing macroeconomic environment.

We explore an alternative way of identifying SICR-events using predictive
modelling instead of a PD-comparison, without diverging from the principles of
IFRS 9 (discussed later). In particular, a predictive model (or supervised
classifier) can incorporate both forward-looking and past-due information in
predicting a SICR-event. Training any supervised classifier, however, first
requires defining the target event, which can itself be challenging. In this
regard, we formulate a concise SICR-framework from which various SICR-
definitions can be generated, before training classifiers. By varying the
framework’s three parameters, we obtain a small list of viable SICR-
definitions. Each resulting definition is then used as the target definition
in training a specific classifier from the same input data. Accordingly, SICR-
definitions can be implicitly evaluated by comparing the performance of these
classifiers against one another. By doing so, we demonstrate the inherent
trade-offs amongst the various SICR-definitions themselves. These trade-offs
and broad relationships can be encoded into an informal expert system, thereby
helping banks select a suitable SICR-definition given their unique contexts.
In closing, our approach relies fundamentally on finding a suitable SICR-
definition, followed by building a bespoke SICR-model and classifying loans
accordingly.

The present study is closest in design to the work of [23], [24], [8], and
[9]. In particular, [23] proposed an algorithm (using random forests with data
from Barbados) that yields the ‘best’ default definition based on maximising
prediction accuracy. When measured in days past due (DPD), these definitions
included: 30, 60, and 90 days. This work was later extended in [24] using
Support Vector Machines (SVMs) and included 120 and 150 DPD as additional
definitions. In both studies, the author demonstrated that the overall
prediction accuracy is significantly affected by the chosen definition of
default. In [8] and [9], a procedure was devised wherein a delinquency
threshold is found at which loan recovery (including legal action) is loss-
optimal, thereby informing the default definition. Our work differs
contextually in that we explore various SICR-definitions (and its underlying
parameters) instead of default definitions.

The notion of SICR-events under IFRS 9 is critically reviewed in section 2,
which includes an in-depth examination of the PD-comparison approach and its
aforementioned challenges in defining SICR-events. Literature on alternative
approaches is then surveyed, followed by examining the support in IFRS 9 for
such alternatives. In section 3, we present a simple three-parameter SICR-
framework for generating SICR-definitions by sensibly varying its parameters,
as illustrated with a few examples. These SICR-definitions are then used in
building various supervised classifiers using logistic regression; itself
reviewed in the appendix. The subsequent modelling results are discussed and
compared across SICR-definitions in section 4, having used residential
mortgage data from a large South African bank. We demonstrate various
relationships amongst SICR-definitions across a variety of aspects; all of
which forms a reusable analytical framework in guiding the selection of a
SICR-definition. Finally, we conclude the study in section 5 with
recommendations and outline avenues of future research. The source code
accompanying this study is published in [10].

## 2 Towards identifying SICR-events: A critical re-evaluation under IFRS 9

The recent introduction of IFRS 9 prompted a paradigm shift in the modelling
of credit risk. Generally, the value of a financial asset should be
comprehensively adjusted over time in line with a bank’s (evolving)
expectation of credit risk. The principle is to forfeit a portion of income
today into a loss provision that ideally offsets amounts that may be written-
off tomorrow. Doing so helps to smooth overall earnings volatility, which is
itself a central tenet of risk management, as explained in [46, pp. 38–44 ].
IFRS 9 requires that this loss provision be regularly updated based on a
statistical model, i.e., the asset’s Expected Credit Loss (ECL). Given a new
ECL-value, a bank adjusts its loss provision either by raising more from
earnings or releasing a portion thereof back into the income statement. This
ECL-model represents the probability-weighted sum of cash shortfalls that a
bank expects to lose over a certain horizon; see [29, §5.5.17–18, §B5.5.28–35,
§B5.5.44–48 ], as well as [48].

Regarding the ECL’s calculation, IFRS 9 adopts a staged approach in §5.5.3 and
§5.5.5 that is based on the extent of the perceived deterioration in the
underlying risk. In principle, each of the three stages requires a
progressively more severe ECL-estimate, as illustrated in Fig. 1. Stage 1
typically includes most loan assets, provided they either have low credit risk
or have not experienced an SICR-event since origination. Stage 2 includes
those assets that have deteriorated quite significantly in their credit
quality (regardless of measure or SICR-definition), but do not yet qualify as
fully ‘credit-impaired’ (i.e., default); a middle ground of sorts. Lastly,
Stage 3 includes those assets with objective evidence of credit impairment,
i.e., their future cash flows are likely compromised, e.g., defaulted
accounts. These stages can be differentiated from one another by the time
horizon of the eventual ECL-estimate: 12 months for Stage 1 and lifetime for
Stages 2-3. In particular, a first-stage loss is the portion of lifetime ECL
that may occur over the next 12 months, whereas all possible loss-inducing
events over the entirety of the asset’s remaining life are considered for a
second-stage (or third-stage) loss. Together, these stages ought to reflect a
more general pattern of deterioration (or improvement) in credit quality over
time, which allows for recognising credit losses more timeously; see §B5.5.2
of IFRS 9, [16], and [39].

Figure 1: Illustrating the one-period evolution of credit risk within the IFRS
9 staged impairment framework. Each subsequent stage implies a greater ECL-
estimate to reflect deeper credit deterioration. Arrows indicate possible
migrations, subject to meeting certain qualitative criteria. The exception is
the probabilistic SICR-component (shaded in dark green), which can include
various factors that may predict a SICR-event. From [7].

Migration between Stage 1 and 2 requires a SICR-component, as conceptualised
in Fig. 1. A loan’s loss estimate will generally attract a greater provision
charge (or coverage rate) when in Stage 2 than in Stage 1. IFRS 9 primarily
defines a SICR-event (thus Stage 2) by comparing $p_{D}(x,t_{r})$ against
$p_{D}(x,t_{0})$ at two different points in time $t_{r}>t_{0}$, whereupon the
difference $p_{D}(x,t_{r})-p_{D}(x,t_{0})=m(x,t_{r})$ is evaluated against a
chosen threshold $u>0$. If $m(x,t_{r})>u$, then the loan is migrated to Stage
2, otherwise it remains in Stage 1. The converse is presumably true as well: a
Stage 2 loan is migrated back to Stage 1 once its risk has improved, i.e., if
$m(x,t^{\prime}_{r})\leq u$ at some future time $t^{\prime}_{r}>t_{r}$. Doing
so would be cost-efficient, particularly since overzealous Stage 2
classification can become prohibitively costly, even if risk-prudent.

However, this so-called PD-comparison approach suffers from at least two
challenges in identifying SICR-events. Firstly, the approach presumes that the
estimation of PD is indeed accurate; a presumption challenged by [13] and
[12]. In particular, severe model risk is introduced when selecting an
inappropriate modelling technique or when failing to capture the time-dynamic
nature of lifetime PD. Moreover, the era of big data and associated high-
dimensional input spaces are exceptionally challenging when selecting
predictive variables; see [25, §2.5 ]. Furthermore, issues concerning data
quality (and data preparation) still persist in practice, which means the
accuracy of estimation remains questionable. Notwithstanding quality, the
paucity of data is another problem when calibrating any technique, perhaps
even more so for low default portfolios, as discussed in [1, §8 ]. These
issues clearly demonstrate the challenges of producing a single PD-estimate,
let alone two.

Secondly, the choice of an appropriate threshold $u$ against which $m(x,t)$
should be evaluated is ambiguous and contentious. Neither IFRS 9 nor most
regulators offer any firm guidance on the choice of $u$. Conversely, the [14]
defines $u=200\%$, yet provides no explanation for this seemingly arbitrary
value. In fact, the [38] observed multiple threshold-values that were in use
across UK banks and even across different portfolios; all of which attests to
further arbitrariness. In fact, a single loan portfolio can theoretically even
use multiple $u$-values in rendering overall SICR-classification within that
portfolio as more risk-sensitive. The UK-regulator is admittedly unsurprised
by these differences, presumably due to the underlying differences across
banks in their risk appetites, strategies, and portfolio compositions. It is
feasible that one bank’s SICR-approach can react differently to the same
macroeconomic reality, compared to a competing bank’s SICR-approach. Given
this complexity, one cannot fault the UK-regulator for expecting greater
consistency in the design of SICR-approaches over the longer term, without
necessarily ignoring the idiosyncrasies amongst banks or their portfolios.

However, IFRS 9 was always intended to be "principles-based and less complex"
(see §IN2 in IFRS 9); the lack of firm guidance on the choice of $u$ is
therefore unsurprising. Instead of detailed prescriptions, the emphasis is on
the purpose behind the rule, which in turn will likely encourage better
substantive compliance; see [4]. Accordingly, the lack of prescription
regarding SICR-classification seems particularly appropriate, given that it
promotes careful evaluation of relevant factors that may influence the
individual bank’s SICR-classification. Furthermore, the literature is
relatively scant regarding the choice of $u$ and is largely limited to
corporate lending. In particular, [12] introduced three metrics that translate
a portfolio’s PD term-structure into a measure of spread, which is then used
in measuring credit deterioration since origination for SICR-classification.
However, not only do these measures depend on observable market prices, but
their application still requires appropriate thresholds, with little guidance
offered by the authors. In contrast, [15] proposed that a threshold be found
based on the trade-off between income volatility and early default
recognition, formulated within a Merton-type framework. That said, the
method’s success still relies on subjective parameter choices and the
availability of market prices. Lastly, [11] suggested an approach for
verifying Stage 2 classification by using an underlying PD-model and its
accuracy ratio. This approach is centred on maximising the so-called Stage 2
"hit rate", or proportion of SICR-flagged accounts that eventually defaulted.
However, the premise hereof is perhaps a bit myopic and even cynical: that all
SICR-flagged loans are destined to default. In contrast, a loan’s risk profile
may very well improve after Stage 2 classification, whereupon it should
rightfully cure back to Stage 1. The dynamicity of credit risk and its
evolution over time is therefore completely ignored when simply maximising the
Stage 2 "hit rate".

Notwithstanding the previous challenges, §B.5.5.12 in IFRS 9 provides a
reprieve. It is not strictly necessary to compare explicit PD-estimates at two
points, provided that the evolution of default risk over time is incorporated
in some other way. In principle, a SICR-event should reasonably preempt a
default event in most cases, which suggests using loan delinquency (and its
evolution) directly in defining a SICR-event itself. Finding a statistical
relationship between future SICR-events and present inputs becomes the basis
of so-called ‘SICR-modelling’. Such a binary classification task can assist
greatly in predicting SICR-events quite accurately, perhaps using a rich set
of macroeconomic and obligor-specific inputs. In fact, IFRS 9 already requires
the use of "all reasonable and supportable information" to identify a SICR-
event (cf. §5.5.4, §5.5.9, §5.5.11, §5.5.17), which further supports
statistical modelling. Moreover, the PD-comparison approach requires both an
accurate PD-model and a suitable threshold $u$, all of which is a relatively
‘indirect’ way of trying to identify a SICR-event. Instead, SICR-modelling is
arguably a more direct approach of classifying future impaired loans into
Stage 2, given "all reasonable and supportable information" that is observed
today and subsequently used as predictive inputs. A resulting SICR-model is
likely to be more parsimonious than a PD-model since the inputs of the latter
predominantly relate to default risk and not necessarily to the increase in
credit risk.

Regarding macroeconomic factors, §B5.5.4 in IFRS 9 already mandates their use
in identifying SICR-events, which is also implicitly required in §B5.5.14. In
addition, many authors have found that macroeconomic information can
significantly improve PD-prediction; see [42], [2], [6], and [13]. The work of
[33] explored default survival models that were trained before and after the
2008 Global Financial Crisis (GFC), which yielded markedly different
parametrisations. In turn, the authors explicitly show the dynamic effect (and
value) of using macroeconomic information explicitly within PD-prediction. A
study by [18] further showed that SICR-classification is highly pro-cyclical
and sensitive to economic downturns, at least within the Irish market. The
authors followed the PD-comparison approach with $u=200\%$ for SICR-
classification, having used Irish residential mortgage data from 2008 to 2015.
These previous studies, together with the IFRS 9 prescription, should bode
well for building bespoke SICR-models wherein macroeconomic covariates are
explicitly used.

## 3 A concise three-parameter SICR-framework for generating SICR-definitions

A useful starting point for defining a SICR-event is that of a delinquency
measure, which should quantify the gradual erosion of trust between bank and
borrower in honouring the credit agreement. The $g_{0}$-measure (or the
unweighted number of payments in arrears) is selected from [8] for its
intuitive appeal and industry-wide ubiquity. Now consider an account’s
$g_{0}$-measured delinquency over its lifetime $T$, as measured in monthly
cohorts $t=1,\dots,T$. In defining a SICR-event, one can compare $g_{0}(t)$ at
time $t$ against a specifiable threshold $d\geq 0$, i.e., $g_{0}(t)\geq d$. In
fact, delinquency can be tested over multiple consecutive months, thereby
ensuring that a ‘true’ SICR-event is identified at $t$. More formally, a SICR-
event is said to have occurred at time $t$ if $g_{0}(v)\geq d$ holds true
across a fixed time span $v\in[t-(s-1),t]$. The specifiable parameter $s\geq
1$ is the number of consecutive months for which delinquency is tested; put
differently, $s$ is the so-called ‘stickiness’ of the aforementioned
delinquency test. These ideas are formalised within the Boolean-valued
decision function $\mathcal{G}(d,s,t)$ that yields a binary SICR-status at an
end-point $t$, defined as

$\mathcal{G}(d,s,t)=\left[\left(\sum_{v=t-(s-1)}^{t}{\left[g_{0}(v)\geq
d\right]}\right)=s\right]\ \text{for}\ t\geq s\,,$ (1)

where $[a]$ are Iverson brackets that outputs 1 if the enclosed statement $a$
is true and 0 otherwise. We illustrate Eq. 1 for $s=1$ and $s=2$ in Table 1
using a hypothetical loan with monthly delinquency observations. For $s=1$,
the SICR-status relies on testing $g_{0}(t)\geq d$ at a single period $t$,
which is akin to having no $s$-parameter. For $s=2$, $g_{0}(t)\geq d$ is
tested twice at two consecutive periods $t-1$ and $t$. If both delinquency
tests are true, then the resulting sum of the two Iverson statements will
equal $s$, thereby signalling a SICR-event at time $t$. The $s$-parameter
smooths away rapid 0/1-fluctuations in the SICR-status over time, i.e., the
SICR-status becomes ‘stickier’ as $s$ increases.

Table 1: Illustrating two formulations of the SICR-decision function $\mathcal{G}$ from Eq. 1 for $(d=1,s=1)$ and $(d=1,s=2)$. Accordingly, SICR-statuses are created using $\mathcal{G}$ for a hypothetical loan and its $g_{0}$-measured delinquency over time $t$. The $\mathcal{Z}_{t}(d,s,k)$-process from Eq. 2 then lags each SICR-status back $k$ periods in creating SICR-outcomes, e.g., $\mathcal{Z}_{t}(1,1,3)$ at $t=4$ will equate to $\mathcal{G}(1,1,4+3)=1$ three months later in the ‘future’. Time $t$ |  Delinquency $g_{0}(t)$ |  SICR-status $\mathcal{G}(1,1,t)$ |  SICR-Outcome $\mathcal{Z}_{t}(1,1,3)$ |  SICR-status $\mathcal{G}(1,2,t)$ |  SICR-Outcome $\mathcal{Z}_{t}(1,2,3)$ |  Default $g_{0}(t)\geq 3$
---|---|---|---|---|---|---
3 |  0 |  0 |  0 |  |  0 |  0
4 |  0 |  0 |  1 |  0 |  0 |  0
5 |  1 |  1 |  1 |  0 |  1 |  0
6 |  0 |  0 |  1 |  0 |  1 |  0
7 |  1 |  1 |  |  0 |  |  0
8 |  2 |  1 |  |  1 |  |  0
9 |  3 |  1 |  |  1 |  |  1

Eq. 1 relies on two specifiable parameters $d$ and $s$ in classifying a loan’s
accrued delinquency over time $t$. The loan’s resulting binary-valued SICR-
statuses, i.e., its $\mathcal{G}(d,s,t)$-values, can now be used within a
typical cross-sectional modelling setup for predicting future SICR-events. In
preparing the modelling dataset, we observe all predictive information of loan
$i$ at a particular time $t$. Then, the loan’s future SICR-status at time
$t+k$ is duly merged, thereby taking a ‘snapshot’ at two points in time, or a
cross-section. However, the chosen value for this third parameter $k\geq 0$
(or outcome period) can significantly affect modelling results. In particular,
both [32] and [37] examined the outcome period’s effect in predicting default
risk, using Irish and South African credit data respectively. Too short a
horizon yielded overly volatile results, largely due to risk immaturity and/or
seasonal effects. Too long a window led to increasingly inaccurate models, in
addition to greater asynchronism with market conditions or even the
portfolio’s risk composition. Since a SICR-event should ideally preempt a
default event in reality, our (cross-sectional) study also contends with
various parameter choices for $k$. More formally, a process
$\mathcal{Z}_{t}(d,s,k)$ prepares a given loan’s monthly performance history
by evaluating Eq. 1 at ‘future’ time $t+k$, though assigns the result to time
$t$; see Table 1 as an example of using $k=3$. Accordingly, the binary-valued
SICR-outcome variable $Y_{t}$ at time $t=1,\dots,T-k$ is created as

$\mathcal{Z}_{t}(d,s,k)\ :\quad Y_{t}=\mathcal{G}(d,s,t+k)\,.$ (2)

Various SICR-definitions are generated using the
$\mathcal{Z}_{t}(d,s,k)$-process from Eq. 2 (or ‘SICR-framework’), simply by
systematically varying its parameters $(d,s,k)$. For this study, the parameter
space includes: 1) the threshold $d\in\\{1,2\\}$ of $g_{0}$-measured
delinquency beyond which SICR is triggered; 2) the level of stickiness
$s\in\\{1,2,3\\}$ within the delinquency test; and 3) the choice of outcome
period $k\in\left\\{3,6,9,12\right\\}$ when modelling SICR-events. While the
parameter spaces of $d$ and $s$ are appreciatively small, the same luxury does
not hold for the outcome period $k$, which can indeed assume many values. Its
enumeration is ultimately guided by experimentation and expert judgement in
balancing rigour against practicality. That said, more extreme periods of
$k>12$ are investigated later in subsection 4.2, though having restricted $d$
and $s$. Regardless, the combined parameter space yields 24 different
combinations of the triple $(d,s,k)$, as enumerated in Table 2. Each
combination serves as a particular target definition in building a
corresponding SICR-model. In this regard, the chosen modelling technique is
binary logistic regression, given its ubiquity in credit risk modelling, as
reviewed in the appendix. The resulting logit-models yield probability scores
$h\left(\boldsymbol{x}_{it}\right)\in[0,1]$ for realised inputs
$\boldsymbol{x}_{it}$ of a particular account $i$ at time $t$.

Table 2: Numbered SICR-definitions, generated by varying the parameters within
the $\mathcal{Z}_{t}(d,s,k)$-process. Definitions are grouped into six classes
and shaded accordingly.

# |  Definition |  Delinquency threshold |  Stickiness |  Outcome period
---|---|---|---|---
1 |  1a(i) |  $d\geq 1$ |  $s=1$ |  $k=3$
2 |  1a(ii) |  $d\geq 1$ |  $s=1$ |  $k=6$
3 |  1a(iii) |  $d\geq 1$ |  $s=1$ |  $k=9$
4 |  1a(iv) |  $d\geq 1$ |  $s=1$ |  $k=12$
5 |  1b(i) |  $d\geq 1$ |  $s=2$ |  $k=3$
6 |  1b(ii) |  $d\geq 1$ |  $s=2$ |  $k=6$
7 |  1b(iii) |  $d\geq 1$ |  $s=2$ |  $k=9$
8 |  1b(iv) |  $d\geq 1$ |  $s=2$ |  $k=12$
9 |  1c(i) |  $d\geq 1$ |  $s=3$ |  $k=3$
10 |  1c(ii) |  $d\geq 1$ |  $s=3$ |  $k=6$
11 |  1c(iii) |  $d\geq 1$ |  $s=3$ |  $k=9$
12 |  1c(iv) |  $d\geq 1$ |  $s=3$ |  $k=12$

# |  Definition |  Delinquency threshold |  Stickiness |  Outcome period
---|---|---|---|---
13 |  2a(i) |  $d\geq 2$ |  $s=1$ |  $k=3$
14 |  2a(ii) |  $d\geq 2$ |  $s=1$ |  $k=6$
15 |  2a(iii) |  $d\geq 2$ |  $s=1$ |  $k=9$
16 |  2a(iv) |  $d\geq 2$ |  $s=1$ |  $k=12$
17 |  2b(i) |  $d\geq 2$ |  $s=2$ |  $k=3$
18 |  2b(ii) |  $d\geq 2$ |  $s=2$ |  $k=6$
19 |  2b(iii) |  $d\geq 2$ |  $s=2$ |  $k=9$
20 |  2b(iv) |  $d\geq 2$ |  $s=2$ |  $k=12$
21 |  2c(i) |  $d\geq 2$ |  $s=3$ |  $k=3$
22 |  2c(ii) |  $d\geq 2$ |  $s=3$ |  $k=6$
23 |  2c(iii) |  $d\geq 2$ |  $s=3$ |  $k=9$
24 |  2c(iv) |  $d\geq 2$ |  $s=3$ |  $k=12$

## 4 Comparing SICR-definitions using South African mortgage data

The SICR-modelling results are structured as follows. First, the resampling
scheme (and underlying data) is explained and verified in subsection 4.1,
followed by broadly describing the selection process of input variables and
the dichotomisation of probability scores into IFRS 9 staging decisions. In
subsection 4.2, we examine the effect of the outcome period $k$ within the
1a-definition class in Table 2 (light blue), having included additional
outcome periods beyond the 12-month boundary. Thereafter, the stickiness
parameter $s$ is investigated in subsection 4.3 for $d=1$ across all $k$ and
$s$, i.e., classes 1a-c in Table 2 (lighter shades). Lastly, we demonstrate in
subsection 4.4 the futility of using $d=2$ across all $k$ and $s$, having
analysed the remaining classes 2a-c in Table 2 (darker shades).

### 4.1. Data calibration: resampling scheme, feature selection, and
dichotomisation

SICR-models are trained and validated using a data-rich portfolio of mortgages
that was provided by a large South African bank. After applying the
$\mathcal{Z}_{t}(d,s,k)$-process from Eq. 2, the resulting credit dataset is
structured as $\mathcal{D}=\left\\{i,t,Y_{it},\boldsymbol{X}\right\\}$ for
each SICR-definition in Table 2 using a portfolio of $N$ loans, indexed by
$i=1,\dots,N$. In this respect, $Y_{it}\in\\{0,1\\}$ indicates at time $t$
whether account $i$ experienced a SICR-outcome $k$ periods later, given a
particular SICR-definition. In predicting $Y_{it}$, consider
$\boldsymbol{X}=\left\\{\boldsymbol{X}_{i},\boldsymbol{X}_{t},\boldsymbol{X}_{it}\right\\}$
as a random vector of input variables that are thematically grouped as
follows: 1) account-level information $\boldsymbol{X}_{i}$ for loan $i$, e.g.,
repayment type (debit order, cash); 2) macroeconomic information
$\boldsymbol{X}_{t}$ at time $t$, e.g., the prevailing inflation rate; 3)
time-dependent behavioural information $\boldsymbol{X}_{it}$, e.g., time in
performing spell.

In circumventing computing constraints and for confidentiality purposes, this
mortgage portfolio is sub-sampled using two-way stratified sampling across a
wide sampling window of January-2007 up to November-2019. For every SICR-
definition, the resulting $\mathcal{D}$ is grouped by the binary-valued SICR-
outcomes $Y_{it}$ within each monthly cohort $t$, thereby resulting in about
310 strata. Observations are then sampled randomly within each stratum in
creating the sub-sampled dataset $\mathcal{D}_{S}$. The sampling proportion is
dynamically set for each SICR-definition such that $\mathcal{D}_{S}$ will be
of fixed size, i.e., about 250,000 monthly observations in total. Finally, a
simple cross-validation resampling scheme is used (with a 70%-30% ratio) to
partition the data $\mathcal{D}_{S}$ into two non-overlapping sets: a training
set $\mathcal{D}_{T}$ and a validation set $\mathcal{D}_{V}$; see [25, pp.
249–254 ]. In verifying sampling representativeness, one can compare each
sample’s event rate over time, i.e., the so-called SICR-rate per context. More
formally, the SICR-rate (or incidence rate) at $t$ is the conditional
probability $\mathbb{P}\left(Y_{t+k}=1|\,Y_{t}=0\right)$ across all Stage 1
accounts $i$ at $t$ that became SICR-flagged at $t+k$. This SICR-rate is
estimated by $\sum_{i}{\left[Y_{it}=1\right]/n_{t}}$, where $[\cdot]$ are
Iverson brackets and $n_{t}$ denotes the number of at-risk Stage 1 accounts at
$t$. Evidently, the line graphs in Fig. 2 for the 1a(i)-definition from Table
2 are reasonably close to one another across all samples. Further, the Mean
Absolute Error (MAE) of the SICR-rates between $\mathcal{D}$ and each
respective sample is calculated for the same 1a(i)-definition as
$\mathcal{D}_{T}$: 0.28% and $\mathcal{D}_{V}$: 0.43%; both of which are
deemed as reasonably low. Similar results hold for all other SICR-definitions,
which suggests that the resampling scheme is indeed representative of the
population at large. This bodes well for deriving SICR-models later that can
generalise beyond training data.

Figure 2: Comparing observed conditional SICR-rates (given Stage 1) over
monthly periods across different samples, using the 1a(i)-definition. The Mean
Absolute Error (MAE) between each sample and the full set $\mathcal{D}$ is
overlaid in summarising the line graph discrepancies over time.

Feature selection is mainly conducted using repeated logistic regressions on a
bigger sub-sampled dataset $\mathcal{D}_{S}$ of 1 million observations
(instead of the previous 250,000), before resampling into $\mathcal{D}_{T}$
and $\mathcal{D}_{V}$ as before. The selection process is itself interactive
and guided by expert judgement, model parsimony, statistical significance,
macroeconomic theory, and predictive performance on $\mathcal{D}_{V}$. In
particular, predictive performance is extensively evaluated using classical
ROC-analysis and the resulting AUC-measure; see [17]. Highlights of this
interactive selection process are given in the appendix, along with the input
space of each SICR-definition. Earlier modelling attempts experimented with
both a best subset approach (stepwise regression) and the LASSO shrinkage
method in selecting inputs, as discussed in [30, §6 ]. However, the necessary
computation times proved to be excessive (especially for the stepwise method)
and even unstable, whilst yielding negligible predictive performance and
overly small models. Moreover, the declaration of [26] – "the data analyst
knows more than the computer" – seems apt, cautioning against potential data
dredging when trying to automate feature selection, devoid of human expertise.

In this work, we are examining the effect of a SICR-definition within a
broader multi-definition setup. Therefore and as a last step, selected
features are ‘standardised’ within each definition class in Table 2 such that
all SICR-models have the same input space per $(d,s)$-tuple across all
$k$-values. By standardising the input space, one can therefore ascribe
observable patterns in model performance only to variations in the SICR-
definition itself, without contending too much with changes in the input
space. Furthermore, large sample sizes are known to affect $p$-values when
testing the statistical significance of regression coefficients, as
demonstrated in [34]. The $p$-values can easily approach zero as the sample
size increases, notwithstanding the greater statistical power availed by
larger sizes. This phenomenon overlaps with the so-called Hughes principle
from [28]: a model’s predictive power will generally increase for every
additional input, but decrease again after reaching some point, provided that
the sample size stays constant. Accordingly, the input space in our study is
retested for statistical significance after deliberately decreasing the sample
size of $\mathcal{D}_{S}$ from 1 million to 250,000 observations. The vast
majority of the inputs remain statistically significant across all $k$-values
within each definition class, which further reassures our standardisation
process as robust.

The logit-models will need to be dichotomised and duly transformed into
probabilistic classifiers to render 0/1 staging decisions under IFRS 9; see
the appendix for details. In particular, an appropriate cut-off
$c_{dsk}$-value is required for the probability scores
$h\left(\boldsymbol{x}_{it}\right)$ resulting from each SICR-model, given an
underlying $(d,s,k)$-based SICR-definition. However, SICR-events are
relatively rare outcomes and the consequences of misclassifying positives vs.
negatives are reasonably unequal. Under IFRS 9, false negatives $F^{-}$ should
be costlier than false positives $F^{+}$ in that the former implies the bank
has failed to increase its loss provision for those accounts with increasing
credit risk, i.e., actual SICR-events. Accordingly, misclassification costs
are assigned as $c_{F^{-}}=6$ for false negatives and $c_{F^{+}}=1$ for false
positives. These costs are deduced using expert judgement and experimentation,
though can certainly be refined or better estimated in future work. At
present, these costs imply a ratio of $a=6/1$, which is deemed intuitive and
risk-prudent. Given $a$, each $c_{dsk}$-value is then found using the
Generalised Youden Index $J_{a}$ (see appendix), as implemented within the
`R`-function `optimal.cutpoints()` from [35].

### 4.2. The effect of the outcome period $k$ when defining and predicting
SICR-events

In general, SICR-classification should react dynamically to changes in credit
risk and its evolution over time. Shorter outcome periods $k$ are therefore
more sensible than longer periods in achieving this dynamicity. However, the
‘optimal’ choice of outcome period is yet unclear, as is the very idea of
‘optimality’ within this SICR-modelling context. To help fill this gap, we
deliberately vary $k$ within this particular subsection from 3 months up to an
extreme of 36 months when training SICR-models. The other parameters are kept
constant at $d=1$ and $s=1$, i.e., definition class 1a within Table 2. These
two values are relatively benign for the following two reasons. Firstly, the
underlying SICR-test $g_{0}(t)\geq d$ from Eq. 1 suggests that $d=2$ will
yield a subset of SICR-cases that are already selected by $d=1$; the latter
choice therefore leads to a ‘broader’ SICR-definition. Secondly, $s=1$ implies
zero ‘stickiness’ and simplifies the resulting SICR-definition. Both choices
of $d$ and $s$ should therefore have minimal interference when studying the
effect of $k$, as intended in this subsection.

Figure 3: Comparing actual SICR-rates over time and across outcome periods
$k\in\\{3,6,9,12,18,24,36\\}$ within $\mathcal{D}_{S}$ for SICR-definition
class 1a from Table 2. The mean and standard deviation of each resulting time
series are summarised within the inset graphs. Encircled points denote the
maximum of each series over time.

The observed SICR-rates are shown in Fig. 3 over time and across all chosen
$k$-values. Evidently, each time graph has a different but increasing mean-
level as $k$ increases, especially when examined after the anomalous 2008
Global Financial Crisis (GFC). However, a longer outcome period invariably
allows greater opportunity for a Stage 1 account to develop sufficient
delinquency; enough to enter – and remain in – default. Since $g_{0}(t+k)\geq
3>d$ from Eq. 1 will hold for both default and SICR-events respectively,
larger $k$-values will therefore increasingly capture a greater proportion of
defaulting accounts. This phenomenon explains the increasing mean SICR-rates
that are correlated with increasing $k$-values; a relationship that becomes
almost linear when removing the 2008-GFC period during mean-calculation. In
fact, Fig. 3 shows that overall mean-levels seemingly reach a plateau near
$4\%$ for $k\geq 12$, having rapidly increased for smaller $k$-values. At the
very least, this may suggest that choosing $k\geq 18$ has a negligible
contribution to the overall SICR-mean, which may already have stabilised at
$k=12$. At worst, choosing $k\geq 18$ will increasingly select default-
instances into the sample, thereby ‘contaminating’ the SICR-mean. Doing so can
detract from the very idea of SICR-staging, which should ideally act as a pro-
cyclical "early warning system" for impending credit risk; see §B.5.5.21 in
[29] and [18].

Apart from differing SICR-means, Fig. 3 shows that each $k$-value results in a
time graph with a different volatility pattern. Evidently, the SICR-rates of
extreme $k$-values are more stable over time relative to other $k$-values. In
particular, the series resulting from $k\leq 3$ and $k>24$ exhibit lower
standard deviation than their peers. However, stable SICR-rates may not
necessarily be a useful pursuit, especially not during an unfolding
macroeconomic crisis and its subsequent effect on default rates. In this
respect, the SICR-rates associated with extreme $k$-values are relatively
stable, though also failed to track increasing default rates (not shown)
during the volatile 2008-GFC period. Accordingly, when defining a SICR-event,
the resulting SICR-rates should reasonably exceed default rates (or some
lagged variant thereof) since SICR-staging should ideally preempt default.
This principle avails a useful heuristic in disqualifying both $k\leq 3$ and
$k>24$, given 12-month default rates of about $6\%$ that prevailed at the
height of the 2008-GFC.

Figure 4: Various summary statistics of the actual SICR-rates from Fig. 3
across chosen outcome periods $k$ for SICR-definition class 1a from Table 2.
Summaries include the earliest, maximum, and mean after Dec-2009 (‘post-GFC’),
as well as differences amongst these summaries, i.e., the early-warning degree
and the post-GFC recovery degree. Desirable $k$-values are encircled and
discussed.

For each of the remaining $k$-values, the earliest SICR-rate $a(k)$ (at
January-2007) can be compared to the maximum SICR-rate $b(k)$, which typically
occurs during the 2008-GFC. Effectively, this comparison constitutes the
degree to which a SICR-definition can respond to unfolding calamities. The so-
called early-warning degree $b(k)-a(k)$ is graphed in Fig. 4 in pink; larger
values are deemed as better. Clearly, $k\in[6,12]$ yield SICR-rates that
reassuringly increased by about 2%-3% points from January-2007 leading into
the crisis, thereby demonstrating considerable responsiveness to externalities
like the 2008-GFC. Furthermore, the maxima $b(k)$ can be compared to the post-
crisis SICR-means $c(k)$ in measuring the magnitude by which the resulting
SICR-rate can normalise. The so-called recovery degree $b(k)-c(k)$ is shown in
Fig. 4 in green; larger values are again deemed as better. The SICR-rates
resulting from $k\in[6,12]$ recovered substantially by about 5%-7% points back
to their post-GFC SICR-means, which suggests appropriate resiliency within the
underlying definitions. While $k\in[18,24]$ had similar recovery degrees, the
resulting SICR-rates are already high even before the 2008-GFC. Accordingly,
longer outcome periods cannot timeously signal impending distress and would
likely result in SICR-models producing overly punitive and ‘paranoid’
predictions. In the extreme case, very long outcome periods, e.g., $k\geq 36$,
can miss the entirety of a crisis period, as evidenced by low-values for both
the early-warning and post-GFC recovery degrees. In contrast, SICR-definitions
with shorter outcome periods $k\leq 12$ can react more flexibly during market
failures without becoming punitive ahead of time; a desirable quality.

In assessing the resulting SICR-model built from each SICR-definition in class
1a, various performance measures are calculated, shown in Table 3. Considering
the expected probabilities from each logit-model, the resulting AUC-values
suggest that smaller outcome periods yield more accurate SICR-models than
longer periods; see Fig. 5(a). This trend is also reflected in the widening
confidence intervals of these AUC-values as $k$ increases. Reassuringly, this
finding corroborates the work of [32] and [37] wherein the outcome period was
similarly varied in PD-modelling – an older ‘cousin’ of SICR-modelling – which
resulted in a similar AUC-trend across $k$-values. Moreover, the nonlinear
reduction in successive AUC-values seems to subside after $k\geq 24$, which
suggests yet again that examining smaller $k\leq 18$ values is more worthwhile
when defining SICR-outcomes. When dichotomising the logit-models into discrete
classifiers using fixed $c_{dsk}$-values, the resulting AUC-values follow a
similar (though even more pronounced) downwards trend as $k$ increases; see
Fig. 5(b). Longer outcome periods also result in fewer observed SICR-events,
as evidenced by a decreasing prevalence rate $\phi_{dsk}$. A rarer SICR-event
partly explains why the cut-off $c_{dsk}$ also decreases in tandem with
greater $k$-values, given the presence of $\phi_{dsk}$ in Eq. 5 when
calculating the Generalised Youden Index $J_{a}$; see appendix. Fewer SICR-
events can generally exacerbate the task of finding a statistical relationship
using logistic regression, hence the lower AUC-values of logit-models as $k$
increases.

Table 3: Various performance measures of SICR-models across different $k$-values for definition class 1a ($d=1,s=1$) from Table 2. The SICR-prevalence $\phi_{dsk}$ estimates $\mathbb{P}\left(\mathcal{C}_{1}\right)$ per definition. Prediction flexibility $\omega_{dsk}$ is measured by calculating the sample mean across all $\sigma_{i}$ within $\mathcal{D_{V}}$, where $\sigma_{i}$ is the standard deviation of the predicted SICR-probabilities $h\left(\boldsymbol{x}_{it}\right)$ for account $i$ over time $t$. SICR-rate instability $\sigma_{dsk}$ is measured by taking the standard deviation of actual SICR-rates in Fig. 3. In dichotomising $h\left(\boldsymbol{x}_{it}\right)$, cut-offs $c_{dsk}$ are found using the Generalised Youden Index $J_{a}$ with a misclassification cost ratio of $a=6/1$. AUC-values are given with bootstrapped 95% confidence intervals using $\mathcal{D}_{V}$ for both probabilistic and discrete SICR-classifiers. Definition |  Outcome period $k$ |  Prevalence $\phi_{dsk}$ |  AUC-Probabilistic |  Flexibility $\omega_{dsk}$ |  Instability $\sigma_{dsk}$ |  Cut-off $c_{dsk}$ |  AUC-Discrete
---|---|---|---|---|---|---|---
1a(i) |  $k=3$ |  6.16% |  91.3% $\pm$ 0.47% |  4.2% |  1.00% |  12.1% |  82.4% $\pm$ 0.65%
1a(ii) |  $k=6$ |  6.13% |  88.5% $\pm$ 0.53% |  3.6% |  1.43% |  10.9% |  78.1% $\pm$ 0.69%
1a(iii) |  $k=9$ |  6.07% |  86.5% $\pm$ 0.55% |  3.4% |  1.64% |  11.5% |  75.0% $\pm$ 0.70%
1a(iv) |  $k=12$ |  5.99% |  84.8% $\pm$ 0.60% |  3.3% |  1.81% |  11.6% |  73.4% $\pm$ 0.72%
1a(v) |  $k=18$ |  5.73% |  82.2% $\pm$ 0.66% |  2.7% |  1.59% |  11.5% |  70.3% $\pm$ 0.76%
1a(vi) |  $k=24$ |  5.46% |  80.6% $\pm$ 0.71% |  2.4% |  1.27% |  11.5% |  68.6% $\pm$ 0.75%
1a(vii) |  $k=36$ |  5.19% |  79.0% $\pm$ 0.75% |  1.9% |  0.65% |  9.4% |  68.2% $\pm$ 0.76%

(a) Logistic Regression (Probabilistic)

(b) Logistic Regression (Discrete)

Figure 5: ROC-graphs using logistic regression as classifier with SICR-
definition class 1a ($d=1,s=1$), having varied the outcome period $k$. In (a),
the probability scores are evaluated for each resulting logit model. In (b),
the probability scores are discretised by imposing a cost-sensitive cut-off
score (Generalised Youden Index), whereupon new ROC-graphs are obtained. In
both (a) and (b), the AUC-values are reported for each SICR-definition using
$\mathcal{D}_{V}$, together with bootstrapped 95% confidence intervals and
corresponding Gini-values.

Pursuing greater AUC-values may at first seem worthwhile when selecting a
SICR-definition; however, there are other considerations. In particular, one
can measure the degree to which a SICR-model’s predictions vary over the
lifetime of an average account, i.e., the so-called prediction flexibility
$\omega_{dsk}$ in Table 3. Evidently, SICR-models built from shorter outcome
periods yield probability scores $h\left(\boldsymbol{x}_{it}\right)$ that vary
more over time $t$, relative to longer outcome periods (or larger $k$-values).
Like AUC-values, $\omega_{dsk}$ appears to be a monotonically decreasing
function of $k$, which invariably couples greater accuracy with greater
prediction flexibility (or variance). From Fig. 3, it is clear that the
underlying SICR-process is itself dynamic and stochastic upon its aggregation,
i.e., the portfolio-level SICR-rate. It is therefore unsurprising that more
accurate SICR-models (lower $k$-values) also exhibit greater variance in their
predictions as a necessary consequence. Barring the extremes ($k\leq 3$), this
result is corroborated by Fig. 4 wherein shorter outcome periods also yield
more responsive SICR-rates leading up to the 2008-GFC, despite the ‘unstable’
account-level SICR-predictions. Furthermore, this relationship between
accuracy and variance is reminiscent of a well-known phenomenon within all
statistical learning, called the bias-variance trade-off; see [25, §2.9, §7 ]
and [30, §2 ].

On the other hand, the supposed ‘flexibility’ in account-level SICR-
predictions, as measured by $\omega_{dsk}$ when varying $k$, may not
necessarily produce the same dynamicity at the portfolio-level. In particular,
defining SICR-outcomes using extremely short outcome periods $k\leq 3$ may
demonstrably lead to highly flexible (and accurate) SICR-predictions at the
account-level. However, the same volatile SICR-predictions ‘stabilise’
somewhat upon their aggregation into the portfolio-level expected SICR-rate,
as shown in Fig. 6(a) in orange. It is therefore useful to compare the
account-level $\omega_{dsk}$ against the standard deviation of a SICR-rate’s
time graph, i.e., the so-called instability $\sigma_{dsk}$ in Table 3; itself
graphed in Fig. 3. Evidently, extremely flexible SICR-predictions, as produced
with $k\leq 3$, can lead to rapid oscillations in moving an account between
Stages 1 and 2 over time. This oscillatory effect dampens the overall
transition into Stage 2 when aggregating across accounts, hence the less
responsive SICR-rate in Fig. 4 (in pink) and the lower $\sigma_{dsk}$-values
when selecting $k\leq 3$. It is therefore questionable to adopt such extremely
short outcome periods when, despite their greater prediction accuracy, the
associated volatility of the resulting SICR-predictions do not meaningfully
translate into more dynamic SICR-rates, as expected at the portfolio-level.

Having disqualified $k\leq 3$, we can similarly disregard $k\geq 18$ given the
worsening accuracy (AUC) of SICR-predictions. Larger $k$-values also result in
less flexible SICR-predictions over time, as measured by decreasing
$\omega_{dsk}$-values. Simultaneously, the resulting SICR-rates from these
larger $k$-values are less responsive to externalities such as the 2008-GFC,
as supported by decreases in both the early-warning degree and in
$\sigma_{dsk}$. Regarding the remaining $k$-choices, we note that the rate of
change in $\omega_{dsk}$ from Table 3 slows down significantly for
$k\in[6,12]$. This plateauing effect suggests that selecting slightly larger
$k$-values within this range will not overly erode the flexibility of account-
level SICR-predictions. At the same time, the increasingly higher
$\sigma_{dsk}$-values, as associated respectively with $k\in\\{6,9,12\\}$,
implies greater dynamicity in the overall SICR-rate of the portfolio,
especially when subjected to an unfolding macroeconomic crisis. The midpoint
hereof is $k=9$, which seems ‘optimal’ when considering the trade-offs amongst
the AUC-values, $\omega_{dsk}$, $\sigma_{dsk}$, and the responsiveness of the
resulting SICR-rate to externalities.

(a) 1a(i): $k=3$

(b) 1a(ii): $k=6$

(c) 1a(iii): $k=9$

(d) 1a(iv): $k=12$

Figure 6: Comparing actual versus expected SICR-rates over time within
$\mathcal{D}_{S}$ for SICR-definition class 1a ($d=1,s=1$) across shorter
outcome periods $k\in[3,6,9,12]$. Expected SICR-rates are the mean probability
scores obtained from the corresponding SICR-model. The discretised variety
result from imposing the corresponding cut-off $c_{dsk}$ on these probability
scores, followed by taking the mean SICR-rate. The MAE between the actual and
expected SICR-rates are overlaid in summarising the line graph discrepancies
over time.

The time graphs of actual versus expected SICR-rates are shown in Fig. 6 for
each $k\in\\{3,6,9,12\\}$, having aggregated actual SICR-statuses and
associated SICR-predictions respectively into the series $A_{t}$ and $B_{t}$
over time $t$. Ideally, both time graphs (green and orange) should closely
overlap each other, which would indicate aggregated predictions agreeing with
reality; hence an excellent SICR-model. Accordingly, we measure the
discrepancy between $A_{t}$ and $B_{t}$ using the MAE across all $k$-values
(including $k\geq 18$, even if not shown), denoted by $m_{1}$ and given in
Fig. 6. Barring the extreme case of $k\geq 36$, all $m_{1}$-values are fairly
similar with a mean error of 0.44% across $k$, which is reassuringly low and
corroborates the relatively high AUC-values in Table 3. Evidently, the
underlying SICR-models can accurately predict SICR-events regardless of $k$,
despite the aggregated SICR-rates becoming less responsive to externalities as
$k$ increases.

Another type of SICR-rate emerges in Fig. 6 when first dichotomising the SICR-
models’ probability scores $h\left(\boldsymbol{x}_{it}\right)$ using
$c_{dsk}$. The resulting ‘discrete’ expected SICR-rate $C_{t}$ (purple) is
similarly compared to $A_{t}$ by calculating the associated MAE; itself
denoted as $m_{2}$ and printed in Fig. 6. Clearly, there are some large
discrepancies between $A_{t}$ and $C_{t}$ during some periods, particularly
during the 2008-GFC, with a mean $m_{2}$-value of 1.12% across
$k\in\\{3,6,9,12,18,24,36\\}$; almost three times larger than the mean
$m_{1}$-value. Nonetheless, the smallest $m_{2}$-value occurred at $k=9$,
which further supports its choice as the ‘optimal’ outcome period.
Experimentation showed however that $C_{t}$ is highly sensitive to the choice
of the $c_{dsk}$-value, which can certainly be tweaked in future work. In this
regard, when estimating the associated Generalised Youden Index $J_{a}$, the
misclassification cost ratio $a$ can be further increased in minimising false
negatives. Doing so should at least increase the prevalence of $C_{t}\geq
A_{t}$ over $t$, i.e., rather the bank be over-provided than under-provided in
its loss provision; a risk-prudent outcome under IFRS 9. Moreover, the
$a$-value can itself be attenuated to a chosen SICR-definition and the
resulting SICR-model, instead of fixing $a$ to a single value across all SICR-
definitions, as in this study.

### 4.3. Varying the level of stickiness $s$ within SICR-definitions

Recall that the $s$-parameter in Eq. 1 controls the number of account-level
delinquency tests that are conducted over time in sequence. The premise of
larger $s$-values is to filter out fickle $\mathcal{G}(d,s,t)$-values (or
SICR-statuses) that fluctuate between 0 and 1 over time $t$ for a given
account. As $s$ increases, the resulting $\mathcal{Z}_{t}(d,s,k)$-values (or
SICR-outcomes) will increasingly equal 1 for more persistent bouts of
delinquency. Put differently, overall SICR-classification becomes more
strenuous and "less paranoid" for larger $s$-values, which results in SICR-
events becoming rarer. In turn, scarcer SICR-events at the account-level imply
slumping SICR-rates when aggregating to the portfolio-level, as shown in Fig.
7.

Figure 7: Comparing actual SICR-rates over time and across outcome periods
$k\in\\{3,6,9,12\\}$ within $\mathcal{D}_{S}$ for SICR-definition classes 1b
and 1c from Table 2. Graph design follows that of Fig. 3.

Consider any actual SICR-rate $A_{t}(k,s)$ from either Fig. 3 or Fig. 7
resulting from a particular $(k,s)$-tuple, and compare with another series
$A_{t}(k,s+1)$. Evidently, the latter SICR-rate is generally smaller than the
former over time, i.e., $A_{t}(k,s)>A_{t}(k,s+1)$ is true at most $t$ for
$s\in\\{1,2\\}$. Moreover, both the mean values and standard deviation of the
respective SICR-rates decrease as $s$ increases when keeping $k$ constant at
any value, as shown in the inset graphs of Fig. 3 and Fig. 7. These results
attest to the stabilising yet strenuous effect of larger $s$-values on overall
SICR-classification, thereby availing $s$ as a useful lever in defining SICR-
events. Notwithstanding, Fig. 7 shows that larger $k$-values still yield
increasingly more unstable SICR-rates across $s\geq 2$ with progressively
greater SICR-means, which is similar to subsection 4.2 for $s=1$. Accordingly,
the $s$-parameter does not seem to override the ‘base’ destabilising effects
of the $k$-parameter, despite the former’s generally stabilising yet
debilitating effect on SICR-rates and its prevalence. Furthermore, if SICR-
staging should ideally preempt default, then the resulting SICR-rates should
reasonably exceed default rates; a principle already used in partially
disqualifying certain $k$-values in subsection 4.2. As such, Fig. 7 shows that
both $A_{t}(k=3,s=2)$ and $A_{t}(k\leq 6,s=3)$ do not exceed the prevailing 6%
default rates during the 2008-GFC, which suggests discarding the associated
SICR-definitions.

As in subsection 4.2, we calculate the early-warning degree $b(k,s)-a(k,s)$
for $s\geq 2$, graphed in Fig. 8 in pink. Clearly, $k\in\\{6,9\\}$ produce
SICR-rates that react strongly to the 2008-GFC by increasing quite
reassuringly by 2%-3% points, which are similar to the SICR-rates in Fig. 4
for $s=1$. However, the SICR-rates yielded by $k=12$ and $s\geq 2$ had a
lacklustre response to unfolding crises with $b(k,s)-a(k,s)\leq 1\%$ points,
at least relative to their sibling series in Fig. 4 for $(k=12,s=1)$. That
said, the affected SICR-rate series in Fig. 7 is already high even before the
2008-GFC, which explains the lacklustre response. This result is reminiscent
to the SICR-rates in Fig. 4 for longer outcome periods $k\geq 18$ and $s=1$,
where the unresponsiveness partially disqualified the associated SICR-
definitions. Nevertheless, we also calculate the recovery degree
$b(k,s)-c(k,s)$ for $s\geq 2$, shown in Fig. 8 in green. Larger $k$-values
imply SICR-rates that recover slightly faster (by 5%-6% points) back to their
post-GFC SICR-means, thereby affirming appropriate resiliency within the
underlying SICR-definitions; similar to Fig. 4 for $k\leq 12$ and $s=1$. On
the other hand, the $s$-parameter does seem to dampen the resiliency-level
itself as $s$ increases, when keeping $k$ constant at any value. In
particular, $b(k,s+1)-c(k,s+1)$ is universally less than $b(k,s)-c(k,s)$ for
$s\leq 2$ at every investigated $k$-value. This result again suggests that,
although increasingly muted by the $s$-parameter, the ‘base’ effects of the
$k$-parameter remain within the resulting SICR-rates.

Figure 8: Various summary statistics of the actual SICR-rates from Fig. 7
across chosen outcome periods $k$ for SICR-definition classes 1b and 1c from
Table 2. Graph design follows that of Fig. 4.

The same performance measures from Table 3 are repeated in Table 4 for
evaluating the SICR-models developed using SICR-definition classes 1b and 1c.
Evidently, larger $k$-values lead to increasingly scarcer SICR-events
(progressively lower prevalence $\phi_{dsk}$-values) across all $s$-values.
This scarcity is further exacerbated by larger $s$-values, which further
scuppers SICR-prevalence $\phi_{dsk}$ when fixing $k$ to any value.
Furthermore, the same trend in AUC-estimates (and associated 95% confidence
intervals) remains intact such that the discriminatory power wanes as the
outcome period lengthens. However, stickier SICR-definitions (or larger
$s$-values) seemingly produce increasingly more accurate SICR-models, when
keeping $k$ constant at any value. This result suggests that the $s$-parameter
succeeds in filtering out fickle SICR-statuses and retains only the more
persistent cases of delinquency, as described earlier. Amongst other effects,
larger $s$-values therefore equate to "purifying the intrinsic SICR-signal" in
finding a more accurate statistical relationship amongst input variables,
thereby explaining larger AUC-values.

Table 4: Various performance measures of SICR-models across different $k$-values for SICR-definition classes 1b ($d=1,s=2$) and 1c ($d=1,s=3$) from Table 2. Table design follows that of Table 3. Definition |  Outcome period $k$ |  Prevalence $\phi_{dsk}$ |  AUC-Probabilistic |  Flexibility $\omega_{dsk}$ |  Instability $\sigma_{dsk}$ |  Cut-off $c_{dsk}$ |  AUC-Discrete
---|---|---|---|---|---|---|---
1b(i) |  $k=3$ |  4.74% |  93.8% $\pm$ 0.41% |  3.7% |  1.07% |  10.8% |  85.4% $\pm$ 0.70%
1b(ii) |  $k=6$ |  4.72% |  89.4% $\pm$ 0.60% |  3.0% |  1.38% |  11.3% |  78.4% $\pm$ 0.80%
1b(iii) |  $k=9$ |  4.68% |  88.0% $\pm$ 0.62% |  2.9% |  1.54% |  8.6% |  76.8% $\pm$ 0.83%
1b(iv) |  $k=12$ |  4.61% |  86.5% $\pm$ 0.60% |  2.6% |  1.71% |  10.3% |  73.8% $\pm$ 0.79%
1c(i) |  $k=3$ |  3.82% |  95.7% $\pm$ 0.37% |  3.1% |  0.97% |  6.8% |  90.3% $\pm$ 0.69%
1c(ii) |  $k=6$ |  3.81% |  91.5% $\pm$ 0.57% |  2.8% |  1.28% |  9.2% |  81.8% $\pm$ 0.86%
1c(iii) |  $k=9$ |  3.78% |  88.9% $\pm$ 0.64% |  2.5% |  1.44% |  10.8% |  76.0% $\pm$ 0.91%
1c(iv) |  $k=12$ |  3.73% |  86.7% $\pm$ 0.68% |  2.2% |  1.57% |  8.6% |  74.0% $\pm$ 0.89%

Larger $s$-values also produce SICR-predictions
$h\left(\boldsymbol{x}_{it}\right)$ that are progressively less flexible over
$t$ for an average account $i$. This decreased flexibility is evident in the
smaller $\omega_{dsk}$-values when fixing $k$ to any value in either Table 3
or Table 4. Furthermore, and similar to AUC-values, $\omega_{dsk}$ remains a
monotonically decreasing function of $k$, as shown previously in subsection
4.2, regardless of the $s$-parameter. Consequentially, SICR-models with
shorter outcome periods and stickier SICR-definitions will yield more accurate
predictions $h\left(\boldsymbol{x}_{it}\right)$, albeit with greater variance
as $k$ decreases and lower variance as $s$ increases. However, upon
aggregating the highly flexible/accurate SICR-predictions from models built
with $k=3$, we again observe less dynamic SICR-rates in Fig. 7 across all
$s$-values; itself further corroborated by lower instability
$\sigma_{dsk}$-values for $k=3$. As in subsection 4.2, using extremely short
outcome periods $k=3$ are seemingly futile since the resulting SICR-rates are
surprisingly stagnant and insensitive to externalities, despite the greater
volatility of account-level SICR-predictions. Having disqualified $k=3$ for
all $s$-values, we similarly discard $k=12$ for $s\geq 2$, given: 1) the
lacklustre response of the associated SICR-rates to the 2008-GFC in Fig. 8;
and 2) the underlying SICR-predictions that are relatively inaccurate and
inflexible, given their AUC-values and $\omega_{dsk}$-values. Notwithstanding
the remaining $k$-values, the average change in $\omega_{dsk}$-values across
$s$ is also the smallest for $k\in\\{6,9\\}$, respectively -0.4% and -0.45%.
This plateauing effect suggests that the loss in prediction flexibility of
slightly larger $k$-values is not too onerous.

(a) 1b(i): $k=3$

(b) 1b(ii): $k=6$

(c) 1b(iii): $k=9$

(d) 1b(iv): $k=12$

Figure 9: Comparing actual versus expected SICR-rates over time within
$\mathcal{D}_{S}$ for SICR-definition class 1b ($d=1,s=2$) across outcome
periods $k\in[3,6,9,12]$. Graph design follows that of Fig. 6.

As in Fig. 6, we compare the time graphs of actual ($A_{t}$) versus expected
($B_{t}$) SICR-rates across both $k\in\\{3,6,9,12\\}$ and $s\in\\{1,2,3\\}$;
see Figs. 9–10. Having calculated the MAE $m_{1}(k,s)$ between $A_{t}(k,s)$
and $B_{t}(k,s)$ for a given $(k,s)$-tuple, the discrepancy is mostly similar
across $k$-values when keeping $s$ constant. By implication, the aggregated
predictions from SICR-models agree closely with observed reality for any $k$,
notwithstanding that both $A_{t}$ and $B_{t}$ become less dynamic as $k$
increases. A decreasing trend appears in the mean errors across $k$ as $s$
increases, i.e., $\\{0.44\%,0.37\%,0.32\%\\}$ respectively for
$s\in\\{1,2,3\\}$, which corroborates the associated increases in AUC-values
within Tables 3–4. Lastly, the discrete expected SICR-rates $C_{i}$ (purple)
can be similarly analysed by computing the MAE $m_{2}(k,s)$ between
$A_{t}(k,s)$ and $C_{t}(k,s)$ for any given $(k,s)$-tuple. The mean errors
$\\{1.03\%,1.07\%,1.22\%\\}$ across $k$ remain 2-4 times greater than the mean
$m_{1}$-values, respective to each $s$-value. However, every $C_{t}$ series
remains highly sensitive to the chosen cut-off $c_{dsk}$-value across both $k$
and $s$, which impedes meaningful analysis. Larger $s$-values can however
achieve a greater prevalence of $C_{t}\geq A_{t}$ without changing the
misclassification cost ratio $a$, which is reassuringly risk-prudent under
IFRS 9.

(a) 1c(i): $k=3$

(b) 1c(ii): $k=6$

(c) 1c(iii): $k=9$

(d) 1c(iv): $k=12$

Figure 10: Comparing actual versus expected SICR-rates over time within
$\mathcal{D}_{S}$ for SICR-definition class 1c ($d=1,s=3$) across outcome
periods $k\in[3,6,9,12]$. Graph design follows that of Fig. 6.

These results suggest the following ‘optima’ in defining SICR-events for
$d=1$, given the trade-offs amongst AUC-values, flexibility $\omega_{dsk}$,
instability $\sigma_{dsk}$, and the responsiveness/resiliency of resulting
SICR-rates amidst macroeconomic malaise. For no stickiness $s=1$, choose
$k\in[6,12]$; for some stickiness $s=2$, choose $k\in[6,9]$; for lots of
stickiness $s=3$, choose $k=9$. Smaller $k$-values within these ranges will
yield more accurate and flexible SICR-predictions, whereupon the resulting
SICR-rates become less dynamic, have lower means, and are less responsive to
externalities. Larger $s$-values will also produce more accurate but less
flexible SICR-predictions, though the resulting SICR-rates have markedly lower
means and are increasingly insensitive to externalities. These trade-offs are
intuitively balanced at $k=9$ across $s$ as well as at $s=2$ across
$k\in\\{6,9\\}$, though future studies can certainly investigate further.

### 4.4. The negative impact of greater delinquency $d=2$ within SICR-
definitions

In the interest of prudence, a rebuttable presumption is articulated in
§5.5.11 and §B5.5.20 within [29], such that a SICR-event occurs once the
arrears has reached 30 days past due, i.e., $g_{0}(t)\geq d$ where $d=1$.
However, this presumption (or ‘backstop’) can be rebutted if there is evidence
against the supposed deterioration of credit quality, despite the delinquency
accruing to $g_{0}(t)=2$. Examples hereof include simple administrative errors
that wrongfully advanced delinquency; or, perhaps more fundamentally, when the
increased delinquency proved to be temporary. We examine the latter case by
adjusting $d=2$ within our SICR-framework, thereby resulting in the remaining
twelve SICR-definitions and associated SICR-models, i.e., classes 2a-2c in
Table 2 (darker shades). Reassuringly, 5(b) shows that the high-level trends
in most performance measures remain largely intact (albeit greatly muted)
across $k$ and $s$ for $d=2$, compared to the results of $d=1$ from Sections
4.2–4.3. The exception is the AUC-measure, whose values are similar to those
AUC-values from $d=1$, particularly for $s\leq 2$. By implication, the
underlying SICR-models can still produce reasonably accurate SICR-predictions
when setting $d=2$. However, the confidence intervals of these AUC-estimates
expand quite noticeably as either $k$ or $s$ increases. This loss of
confidence is likely due to the growing scarcity of SICR-events when fixing
$d=2$, as reflected in the extremely low prevalence rates $\phi_{2sk}$.

In isolation, the extremely low $\phi_{2sk}$-rates in 5(b) imply that the
resulting Stage 2 provisions under IFRS 9 would be similarly small, which is
certainly unintuitive given the greater underlying delinquency of using $d=2$.
In fact, the mean $\phi_{2sk}$-rates over $k$ respective to $s=\\{1,2,3\\}$
are about $\\{9.5,23.4,52.4\\}$ times smaller than the corresponding mean
$\phi_{1sk}$-rates for $d=1$. Since the vast majority of delinquent cases
$g_{0}(t)\geq d$ clearly coincide with $d=1$, it is therefore questionable to
build bespoke SICR-models with $d=2$, particularly since the former already
includes the latter cases by definition. Furthermore, the resulting actual
SICR-rates implied by $d=2$ are significantly lower than those of $d=1$, i.e.,
$A_{t}(d=2,s,k)<A_{t}(d=1,s,k)$ over time $t$ and across all
$(s,k)$-combinations. Accordingly, the default rates of the 2008-GFC exceed
these $A_{t}(2,s,k)$-rates considerably, which contradicts IFRS 9 in providing
timeously for credit losses. Moreover, these $A_{t}(2,s,k)$-rates are not as
dynamic as their $A_{t}(1,s,k)$-counterparts in responding to macroeconomic
crises, as partially measured by $\sigma_{dsk}$ in 5(b). Given these results,
we recommend against using $d=2$ and therefore support using the backstop, as
implicitly included within our SICR-framework when using $d=1$.

Table 5: Selected performance measures of SICR-models across all SICR-
definitions from Table 2. Table design follows that of Table 3 and Table 4.

(a) Delinquency threshold $d=1$ Definition |  Prevalence $\phi_{dsk}$ |  Instability $\sigma_{dsk}$ |  AUC-Probabilistic
---|---|---|---
1a(i) |  6.16% |  1.00% |  91.3% $\pm$ 0.47%
1a(ii) |  6.13% |  1.43% |  88.5% $\pm$ 0.53%
1a(iii) |  6.07% |  1.64% |  86.5% $\pm$ 0.55%
1a(iv) |  5.99% |  1.81% |  84.8% $\pm$ 0.60%
1b(i) |  4.74% |  1.07% |  93.8% $\pm$ 0.41%
1b(ii) |  4.72% |  1.38% |  89.4% $\pm$ 0.60%
1b(iii) |  4.68% |  1.54% |  88.0% $\pm$ 0.62%
1b(iv) |  4.61% |  1.71% |  86.5% $\pm$ 0.60%
1c(i) |  3.82% |  0.97% |  95.7% $\pm$ 0.37%
1c(ii) |  3.81% |  1.28% |  91.5% $\pm$ 0.57%
1c(iii) |  3.78% |  1.44% |  88.9% $\pm$ 0.64%
1c(iv) |  3.73% |  1.57% |  86.7% $\pm$ 0.68%

(b) Delinquency threshold $d=2$ Definition |  Prevalence $\phi_{dsk}$ |  Instability $\sigma_{dsk}$ |  AUC-Probabilistic
---|---|---|---
2a(i) |  0.64% |  0.23% |  91.8% $\pm$ 1.18%
2a(ii) |  0.64% |  0.26% |  86.5% $\pm$ 1.56%
2a(iii) |  0.64% |  0.26% |  84.6% $\pm$ 1.66%
2a(iv) |  0.63% |  0.26% |  81.7% $\pm$ 1.95%
2b(i) |  0.20% |  0.10% |  97.1% $\pm$ 1.09%
2b(ii) |  0.20% |  0.11% |  90.6% $\pm$ 2.29%
2b(iii) |  0.20% |  0.11% |  90.3% $\pm$ 2.16%
2b(iv) |  0.20% |  0.11% |  84.1% $\pm$ 2.87%
2c(i) |  0.07% |  0.06% |  76.3% $\pm$ 4.20%
2c(ii) |  0.07% |  0.06% |  81.6% $\pm$ 3.40%
2c(iii) |  0.07% |  0.06% |  74.7% $\pm$ 3.99%
2c(iv) |  0.08% |  0.06% |  74.5% $\pm$ 3.62%

## 5 Conclusion

The meaning of a SICR-event has become needlessly nebulous when modelling loan
impairments under IFRS 9. The resulting complexity is arguably a consequence
of using an approach based on drawing arbitrary PD-comparisons; an approach
with at least two prominent challenges. Firstly, it requires PD-estimates that
are reasonably accurate at any two time points, which is itself challenging.
Secondly, the approach requires evaluating the difference between any two PD-
estimates against a subjectively-chosen threshold (or ‘magnitude’), whose
selection can be ambiguous and contentious. Intuitively, too small a threshold
can trigger the mass migration of loans into Stage 2, which can become
prohibitively cost-inefficient and even ‘paranoid’. On the other hand, too
large a threshold may never be materially breached, thereby keeping loans
‘naively’ in Stage 1 and leaving a bank grossly under-provided. At the moment,
choosing any threshold is non-trivial given the lack of an overarching
optimisation framework. Practitioners and regulators alike have little choice
but to rely on subjective discretion and/or regulatory prescription; both of
which can be sub-optimal. More generally, these two challenges of the PD-
comparison approach can counteract the main imperative of IFRS 9, i.e.,
recognising credit losses timeously.

As an alternative, we contribute a concise and simple SICR-framework from
which SICR-definitions may be generated and tested. Any such (target)
definition can then be used in building a statistical SICR-model (or
supervised binary classifier); itself another contribution. This SICR-model
can probabilistically classify a loan into either Stage 1 or 2, using a rich
and dynamic set of macroeconomic and obligor-specific input variables. As
supported by §B.5.5.12 in IFRS 9, our SICR-modelling approach does not rely on
PD-comparisons and therefore requires neither underlying PD-models nor
selecting any magnitude threshold. A SICR-modelling approach is relatively
more parsimonious since the inputs can relate more directly to the change in
credit risk, instead of just default risk alone. Moreover, a SICR-modelling
approach prevents any pre-existing issues within a PD-model from bleeding into
staged impairment classification under IFRS 9, which can be a practical
benefit.

In generating SICR-definitions, our framework avails three useful parameters:
1) the delinquency threshold $d$ in testing accrued delinquency at any point;
2) the level of stickiness $s$ when testing delinquency over consecutive
periods; and 3) the outcome period $k$ over which to predict SICR-statuses. In
varying these parameters, we effectively produced 27 different SICR-
definitions as unique combinations of the triple $(d,s,k)$. Each SICR-
definition is applied on the same South African mortgage data from 2007-2019,
whereupon an account-level SICR-model is estimated using binary logistic
regression per definition. We demonstrate that shorter outcome periods can
yield SICR-predictions that are increasingly more accurate and flexible over
loan life, at least for $k\geq 6$ months. However, upon aggregating these
account-level predictions to the portfolio-level, the resulting SICR-rate
appears less dynamic over time for smaller $k$-values, have progressively
lower means, and are increasingly insensitive to unfolding economic crises
like the 2008-GFC. Some of these relationships are not necessarily linear:
overly long outcome periods ($k\geq 18$) yield SICR-rates that are similarly
unresponsive to market failures, in addition to the degrading prediction
accuracy.

The $s$-parameter has a stabilising yet strenuous effect on SICR-
classification, wherein SICR-events become scarcer as $s$ increases. Greater
stickiness yield account-level SICR-predictions that are more accurate but
also less flexible over loan life. From these stickier SICR-definitions, the
resulting portfolio-level SICR-rates become less dynamic over time, have lower
means, and are increasingly insensitive to the 2008-GFC. Furthermore, both $k$
and $s$ parameters interact with each other in that SICR-predictions become
more accurate as $k$ decreases and $s$ increases. However, the SICR-
predictions’ variance over time (or flexibility) decreases for larger $s$ but
increases again for smaller $k$. Lastly, choosing $d=2$ yields extremely
scarce SICR-events across all values of $s$ and $k$ that would compromise the
resulting Stage 2 provision-levels; all of which supports the ‘backstop’
($d=1$) of IFRS 9. In conclusion, our work forms a reusable analytical
framework in which any SICR-definition can be examined on the accuracy and
flexibility of the resulting SICR-predictions, the instability of implied
SICR-rates, and its responsiveness to economic downturns. A reasonable trade-
off exists amongst these factors when choosing $k=9$ across any $s$-value, as
well as when selecting $s=2$ across $k\in\\{6,9\\}$. These parameter choices
should yield SICR-models whose predictions are both highly accurate and
reasonably flexible over time, while the resulting SICR-rates remain dynamic
and reassuringly sensitive to externalities.

Future research can examine SICR-modelling using data from other loan
portfolios and across other credit markets; both of which may affect the
resulting choices of $(k,s)$. In this regard, future studies can explore an
even finer-grained list of $k$-values, particularly for $k\in[4,12]$. Doing so
can refine the relationships that we have found, which can help in devising a
more mature expert system for selecting parameters optimally. Furthermore, the
misclassification cost ratio can be tweaked towards improving the discrete
output of a specific SICR-model, instead of keeping the ratio constant across
all SICR-definitions, as we have done. In minimising false negatives, the
misclassification cost can itself be embedded when training a SICR-model,
instead of imposing such costs exogenously afterwards. As for modelling
techniques, future researchers can certainly expand our study by experimenting
with more advanced binary classifiers than logistic regression, e.g., Support
Vector Machines as in [24]. Lastly, a future study can focus on stress-testing
relevant input variables (e.g., macroeconomic covariates) within a SICR-model,
perhaps towards forecasting overall SICR-rates given a particular
macroeconomic scenario.

### Acknowledgements

This work is not financially supported by any institution or grant, with no
known conflicts of interest that may have influenced the outcome of this work.
The authors would like to thank Prof. Dirk Tasche for reviewing our work, as
well as all anonymous referees and editors for their valuable contributions.

##  Appendix A Appendix

In subsection A.1, we discuss the fundamentals of a statistical technique
called logistic regression, its use in quantitative finance, as well as the
Generalised Youden Index $J_{a}$ in dichotomising a logit-model. Thereafter,
the input spaces are summarised in subsection A.2 across all SICR-models,
whereupon we discuss some interesting patterns found during feature selection.

### A.1. Logistic regression as supervised classifier technique

Like many supervised techniques, logistic regression aims to find a
statistical relationship between a set of random input variables
$\boldsymbol{X}$ and the binary-valued outcome (or Bernoulli random variable)
$Y\in\left\\{\mathcal{C}_{0}:0\ ;\ \mathcal{C}_{1}:1\right\\}$. This $Y$
records either a non-event $\left(\mathcal{C}_{0}\right)$ or a SICR-event
$\left(\mathcal{C}_{1}\right)$, both of which are also called a ‘negative’ or
‘positive’ respectively. From [27, pp. 1–10 ], [3, §4 ], [25, §4 ], and [30,
§4.3 ], the focus of logistic regression is on estimating the conditional mean
$\mathbb{E}\left[Y|\boldsymbol{X}\right]$ as the probability of class $Y$
given $\boldsymbol{X}$. Let $\pi(\boldsymbol{X})\in\left[0,1\right]$ represent
the posterior class probability of $Y=\mathcal{C}_{1}$ given a $p$-dimensional
random vector $\boldsymbol{X}$. The conditional mean
$\mathbb{E}\left[Y|\boldsymbol{X}\right]$ for either class $\mathcal{C}_{0}$
or $\mathcal{C}_{1}$ is commonly estimated using the standard logistic
function $\sigma:\mathbb{R}\rightarrow[0,1]$, itself defined as
$\sigma(w)=e^{w}/(1+e^{w})=(1+e^{-w})^{-1}$. Then, $w$ is replaced with a
linear combination $\boldsymbol{\beta}^{T}\boldsymbol{x}$ of the realised
input variables $\boldsymbol{x}=\left\\{x_{1},\dots,x_{p}\right\\}$ and its
associated coefficient vector
$\boldsymbol{\beta}=\left\\{\beta_{1},\dots,\beta_{p}\right\\}$ with intercept
$\beta_{0}$. Accordingly, $\mathbb{E}\left[Y|\boldsymbol{X}\right]$ for either
class $\mathcal{C}_{0}$ or $\mathcal{C}_{1}$ is expressed respectively as

$\displaystyle\mathbb{P}\left[Y=\mathcal{C}_{1}|\boldsymbol{X}=\boldsymbol{x}\right]$
$\displaystyle=$ $\displaystyle\pi(\boldsymbol{x})$ $\displaystyle=$
$\displaystyle\frac{\exp\left(\boldsymbol{\beta}^{T}\boldsymbol{x}+\beta_{0}\right)}{1+\exp\left(\boldsymbol{\beta}^{T}\boldsymbol{x}+\beta_{0}\right)}$
$\displaystyle=$
$\displaystyle\big{(}1+e^{-\boldsymbol{w}}\big{)}^{-1}\quad\text{and}$
$\displaystyle\mathbb{P}\left[Y=\mathcal{C}_{0}|\boldsymbol{X}=\boldsymbol{x}\right]$
$\displaystyle=$ $\displaystyle 1-\pi(\boldsymbol{x})$ $\displaystyle=$
$\displaystyle\frac{1}{1+\exp\left(\boldsymbol{\beta}^{T}\boldsymbol{x}+\beta_{0}\right)}$
$\displaystyle=$ $\displaystyle\big{(}1+e^{\boldsymbol{w}}\big{)}^{-1}\,.$ (3)

Dividing $\pi(\boldsymbol{x})$ in subsection A.1 by $1-\pi(\boldsymbol{x})$
yields the odds in favour of class $\mathcal{C}_{1}$ occurring. Taking the
natural log hereof then transforms the odds into a (more appealing) symmetric
quantity. E.g., the natural log of an odds (in favour) of 4:1 versus its
opposite of 1:4 is 0.602 and -0.602 respectively. More formally, the function
$g(\boldsymbol{x})=\ln{\left[\frac{\pi(\boldsymbol{x})}{1-\pi(\boldsymbol{x})}\right]}$
effectively maps the linear combination
$\boldsymbol{\beta}^{T}\boldsymbol{x}+\beta_{0}$ to the so-called log-odds or
logit. This logit $g(\boldsymbol{x})$ avails many of the desirable properties
of a linear regression model, e.g., the output of $g(\boldsymbol{x})$ may be
continuous, can range from $-\infty$ to $\infty$, and is linear in its
parameters. Lastly, the unknown regression coefficients $\boldsymbol{\beta}$
in subsection A.1 are found using the classical maximum likelihood estimation
procedure from statistical literature. That is, $\boldsymbol{\beta}$-values
are found such that the predicted probability
$h(\boldsymbol{x}_{i})=\hat{\pi}(\boldsymbol{x}_{i})$ of each observation $i$
approximates the observed 0/1-encoded $y_{i}$-value as closely as possible.

For each trained logit-model, the probability scores $h(\boldsymbol{x})$ that
estimate $\mathbb{P}\left[\mathcal{C}_{1}|\boldsymbol{x}\right]$ will need to
be dichotomised (or discretised) in yielding binary 0/1-decisions. This
dichotomisation implies choosing a cut-off $c\in[0,1]$ such that the
discretised classifier $h^{\prime}(\boldsymbol{x})=1$ if $h(\boldsymbol{x})>c$
and $h^{\prime}(\boldsymbol{x})=0$ if otherwise. For every possible $c$-value,
the probability of a true positive (or SICR-event correctly predicted as such)
is
$q(c)=\mathbb{P}\left(h\left(\boldsymbol{x}\right)>c\,|\,\mathcal{C}_{1}\right)$,
also known as sensitivity. Likewise, the probability of a true negative (or
non-event correctly predicted as such) is
$p(c)=\mathbb{P}\left(h\left(\boldsymbol{x}\right)\leq
c\,|\,\mathcal{C}_{0}\right)$, also called specificity. In finding the optimal
cut-off $c^{*}$ that incorporates both sensitivity and specificity, consider
the Youden Index $J$ that is widely used in the biostatistical literature; see
[49], [21], and [40]. This index $J$ is defined as the maximisation problem

$J=\max_{c}{\left\\{q(c)+p(c)-1\right\\}}\,.$ (4)

Clearly, the classical $J$ assigns equal weight to both sensitivity and
specificity, which inappropriately equates the misclassification cost of a
false negative to that of a false positive. However, the Generalised Youden
Index $J_{a}$ improves upon $J$ by rendering $c$ sensitive to both types of
misclassification costs; see [19], [31], and [40]. In particular, let $a>0$ be
a cost multiple (or ratio) of a false negative relative to a false positive.
If $\phi$ is the estimated prevalence of the $\mathcal{C}_{1}$-event, i.e.,
the prior $\mathbb{P}\left(\mathcal{C}_{1}\right)$, then $J_{a}$ is expressed
for a given $c$ as

$J_{a}(c)=q(c)+\frac{1-\phi}{a\phi}\cdot p(c)-1\,,$ (5)

whereupon $c^{*}$ is given by

$c^{*}=\arg\max_{c}{J_{a}(c)}\,.$ (6)

The use of logistic regression in binary classification is ubiquitous,
particularly in the field of application credit scoring, as was first
demonstrated in [47]. In this regard, logistic regression is considered by
many authors to be the most successful regression technique in quantitative
finance; see [22], [45, §4.5, §10–11 ], [41], [44, §1 ], and [5]. Beyond
application credit scoring, this technique is also typically used in pre-
screening loan offers, detecting fraud cases, scoring collection success,
informing direct marketing offers, and in risk-based pricing. As such, the
ubiquity of the logistic regression technique suggests its use in the present
study. At the very least, this technique and its results can serve as a
benchmark when using more advanced classification techniques in future.

### A.2. Feature selection: constructing the input space of each SICR-model

Several input variables from multiple themes are considered in constructing
the ’standardised’ input space for each SICR-definition class, as explained in
subsection 4.1. In Table 6, we summarise and briefly describe the final set of
input variables per associated definition class. Given their widespread
prevalence, macroeconomic variables (and their lagged variants) have a notable
impact on SICR-events irrespective of definition, which supports the forward-
looking information requirement of [29]. This impact is generally greater for
SICR-definition class 1 ($d=1$) than for class 2 ($d=2$), while the exact set
of macroeconomic variables (MVs) depends on the choice of both $d$ and $s$. By
implication, some MVs are more pertinent than others in predicting greater
arrears-levels ($d=2$), and vice versa for predicting smaller arrears-levels
($d=1$). Given the floating interest rates within a mortgage portfolio, it is
intuitive that a change in the central bank rate (`Repo_Rate_0mo`) immediately
affects a borrower’s affordability and associated SICR-risk; particularly for
$d=1$. Furthermore, we purposefully included both repo rate and inflation
growth despite the former’s role in controlling the latter, which implies high
collinearity. In extreme market conditions, the supposed structural
relationship between these two variables can change due to government
intervention, such as during the COVID-19 pandemic. Nonetheless, both
variables remain statistically significant across most SICR-models,
particularly for $d=1$.

Table 6: Describing the selected features across the different SICR-models within the various SICR-definition classes from Table 2. Variable |  Description |  Definitions |  Theme
---|---|---|---
`ArrearsTrend_3mo` |  The 3-month arrears trend, obtained qualitatively by comparing the current arrears-level to that of 3 months ago |  1a, 1b, 1c, 2a, 2b, 2c |  Delinquency
`BalanceLog` |  Log-transformed outstanding balance at month-end |  1a, 1c, 2a, 2c |  Account-level
`BalanceToTerm` |  Outstanding balance divided by the contractual term of the loan |  1b, 2b |  Account-level
`DTI_Level_0mo` |  Debt-to-Income: Average household debt expressed as a percentage of household income per quarter, interpolated monthly |  1a, 1b, 1c, 2a, 2b |  Macroeconomic
`DTI_Level_12mo` |  Debt-to-Income: 12-month lagged version of `DTI_Level_0mo` |  1a, 1b, 1c, 2a, 2b |  Macroeconomic
`Employment_Growth_0mo` |  Year-on-year growth rate in the 4-quarter moving average of employment per quarter, interpolated monthly |  2b, 2c |  Macroeconomic
`Employment_Growth_12mo` |  12-month lagged version of `Employment_Growth_0mo` |  2b, 2c |  Macroeconomic
`g0_Delinq` |  Delinquency measure: number of payments in arrears; see [8] |  1a, 1b, 1c, 2a, 2b, 2c |  Delinquency
`Inflation_Growth_0mo` |  Year-on-year growth rate in inflation index (CPI) per month |  1a, 1b, 1c, 2a, 2b, 2c |  Macroeconomic
`InterestRate_Margin` |  Margin between an account’s nominal interest rate and the current prime lending rate; proxy for embedding risk-based pricing principles |  1a, 1b, 1c, 2a, 2b, 2c |  Account-level
`Num_ArrearsEver_24mo` |  Duration (in months) of account delinquency within the last 24 months (excluding current point) |  1a, 1b, 1c, 2a, 2b, 2c |  Delinquency
`PayMethod` |  Binned instalment payment methods, e.g., cash, debit order, payroll |  1a, 1b, 1c, 2a, 2b, 2c |  Behavioural
`PerfSpell_Num` |  Current performing spell number in tracking previous default spells |  1a, 1b, 1c, 2a |  Delinquency
`Prelim_Perc` |  Undrawn/prepaid proportion of available credit limit |  1a, 1b, 1c, 2a, 2b, 2c |  Behavioural
`RealGDP_Growth_0mo` |  Year-on-year growth rate in the 4-quarter moving average of real GDP per quarter, interpolated monthly |  1a |  Macroeconomic
`RealIncome_Growth_0mo` |  Year-on-year growth rate in the 4-quarter moving average of real income per quarter, interpolated monthly |  1b, 1c, 2c |  Macroeconomic
`RealIncome_Growth_12mo` |  12-month lagged version of `RealIncome_Growth_0mo` |  1b, 1c, 2c |  Macroeconomic
`Receipt_InfLog` |  Log-transformed inferred customer receipts (or cash inflows) at month-end |  2b |  Account-level
`Repo_Rate_0mo` |  Prevailing repurchase rate set by the South African Reserve Bank |  1a, 1b, 1c, 2a, 2b, 2c |  Macroeconomic
`Term` |  Contractual term of the loan |  1a, 1b, 1c |  Account-level
`TimeInPerfSpell` |  Duration (in months) of current performing spell before default or competing risk |  1a, 1b, 1c, 2a, 2b |  Delinquency

In measuring the relative importance of input variables, we use a technique
from coalitional game theory called Shapley-values; see [36, §9.5 ]. Given an
instance $\boldsymbol{x}_{it}=\left\\{x_{it1},\dots,x_{itp}\right\\}$ observed
from a $p$-dimensional input space for account $i$ at time $t$, the
contribution (or Shapley-value) of the $j^{\text{th}}$ input variable to the
SICR-prediction $h(\boldsymbol{x}_{it})$ is calculated as

$S_{itj}={\beta}_{j}{x}_{itj}-\bar{\mu}_{j}\,,$ (7)

where $\bar{\mu}_{j}=\mathbb{E}_{it}\left(\beta_{j}\boldsymbol{X}_{j}\right)$
is the average predictor value of input $j$, estimated across all
$(i,t)$-cases. However, this process quickly becomes time-consuming as
dimensionality increases, which is why [43] proposed an efficient estimator
$\psi$ for Eq. 7 using Monte Carlo sampling, i.e., $\psi_{itj}\approx
S_{itj}$. Using the `fastshap` R-package from [20], the average absolute
Shapley-value of the $j^{\text{th}}$ input
$\bar{\psi}_{j}=\mathbb{E}_{it}\left(\rvert\psi_{itj}\rvert\right)$ is duly
estimated across all $(i,t)$-cases for SICR-definitions 1a(i)-(iv).

Our results show that the most important input variable across the
investigated SICR-models is `Prelim_Perc` (undrawn/prepaid proportion), where
large values indicate an intuitive buffer from which borrowers can draw during
distressed times. Since the delinquency-level (`g0_Delinq`) helps define SICR-
events, its high $\bar{\psi}$-value reassuringly suggests great importance;
likewise for another delinquency-themed input, `Num_ArrearsEver_24mo`. In
distressed times, borrowers tend to switch their payment method to more
flexible forms (e.g., from debit order to cash), which explains the
$\bar{\psi}$-value of `PayMethod` being the fourth largest. Regarding
macroeconomic variables, `DTI_Level_0mo` contributes at first meaningfully to
SICR-prediction for lower $k$-values, though its lagged sibling
`DTI_Level_12mo` overtakes for $k\geq 9$; thereby underscoring the importance
of testing lags in SICR-modelling. Lastly, the $\bar{\psi}$-value of
`InterestRate_Margin` remained consistently within the top 7 across $k$, which
corroborates the use of risk-based pricing information within SICR-modelling.

## References

  * [1] Bart Baesens, Daniel Rösch and Harald Scheule “Credit risk analytics: Measurement techniques, applications, and examples in SAS” Hoboken, New Jersey: John Wiley & Sons, 2016
  * [2] Toni Bellotti and Jonathan Crook “Credit scoring with macroeconomic variables using survival analysis” In _Journal of the Operational Research Society_ 60.12 Taylor & Francis, 2009, pp. 1699–1707 DOI: 10.1057/jors.2008.130
  * [3] Christopher M Bishop “Pattern recognition and machine learning” New York: Springer, 2006
  * [4] Julia Black, Martyn Hopper and Christa Band “Making a Success of Principles-Based Regulation” In _Law and Financial Markets Review_ 1, 2007 DOI: 10.1080/17521440.2007.11427879
  * [5] Christine Bolton “Logistic regression and its application in credit scoring”, 2010 URL: http://hdl.handle.net/2263/27333
  * [6] Diana Bonfim “Credit risk drivers: Evaluating the contribution of firm level information and of macroeconomic dynamics” In _Journal of Banking & Finance_ 33.2 Elsevier, 2009, pp. 281–299 DOI: 10.1016/j.jbankfin.2008.08.006
  * [7] Arno Botha “A procedure for loss-optimising the timing of loan recovery under uncertainty”, 2021 DOI: 10.13140/RG.2.2.12015.30888/2
  * [8] Arno Botha, Conrad Beyers and Pieter De Villiers “Simulation-based optimisation of the timing of loan recovery across different portfolios” In _Expert Systems with Applications_ 177, 2021 DOI: 10.1016/j.eswa.2021.114878
  * [9] Arno Botha, Conrad Beyers and Pieter De Villiers “The loss optimisation of loan recovery decision times using forecast cash flows” In _Journal of Credit Risk_ 18.1, 2022 DOI: 10.21314/JCR.2020.275
  * [10] Arno Botha and Esmerelda Oberholzer “Defining and Comparing SICR-events for classifying impaired loans under IFRS 9: Introducing SICR-modelling using logistic regression [Source Code]” Zenodo, 2023 DOI: 10.5281/zenodo.7641624
  * [11] V. Brunel “Loan classification under IFRS 9” In _Risk.net_ , 2016 URL: https://www.risk.net/risk-management/credit-risk/2456385/loan-classification-under-ifrs-9
  * [12] Gaurav Chawla, Lawrence Jr Forest and Scott Aguais “Some options for evaluating significant deterioration under IFRS 9” In _Journal of Risk Model Validation_ 10.3, 2016 DOI: 10.21314/JRMV.2016.158
  * [13] Jonathan Crook and Anthony Bellotti “Time Varying and Dynamic Models for Default Risk in Consumer Loans” In _Journal of the Royal Statistical Society Series A_ 173, 2010, pp. 283–305 DOI: 10.1111/j.1467-985X.2009.00617.x
  * [14] European Banking Authority “2018 EU-wide stress tests: results” European Banking Authority, 2018 URL: https://www.eba.europa.eu/risk-analysis-and-data/eu-wide-stress-testing/2018
  * [15] Logan Ewanchuk and Christoph Frei “Recent Regulation in Credit Risk Management: A Statistical Framework” In _Risks_ 7.2, 2019 DOI: 10.3390/risks7020040
  * [16] EY “Impairment of Financial Instruments under IFRS 9”, 2014 URL: https://www.ey.com/en_gl/ifrs-technical-resources/impairment-of-financial-instruments-under-ifrs-9
  * [17] Tom Fawcett “An introduction to ROC analysis” In _Pattern recognition letters_ 27.8 Elsevier, 2006, pp. 861–874 DOI: 10.1016/j.patrec.2005.10.010
  * [18] Edward Gaffney and Fergal McCann “The cyclicality in SICR: mortgage modelling under IFRS 9” In _ESRB: Working Paper Series No. 2019/92_ , Availaible at SSRN:, 2019 DOI: 10.2139/ssrn.3723448
  * [19] Seymour Geisser “Comparing two tests used for diagnostic or screening purposes” In _Statistics & Probability Letters_ 40.2 Elsevier, 1998, pp. 113–119 DOI: 10.1016/S0167-7152(98)00067-4
  * [20] Brandon M. Greenwell “Scalable Shapley Explanations in R [‘fastshap’ R-package]” GitHub, 2021 URL: https://bgreenwell.github.io/intro-fastshap/slides.html#1
  * [21] Markus Greiner, Dirk Pfeiffer and RD Smith “Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests” In _Preventive veterinary medicine_ 45.1-2 Elsevier, 2000, pp. 23–41 DOI: 10.1016/S0167-5877(00)00115-X
  * [22] David J Hand and William E Henley “Statistical classification methods in consumer credit scoring: a review” In _Journal of the Royal Statistical Society: Series A (Statistics in Society)_ 160.3 Wiley Online Library, 1997, pp. 523–541 DOI: 10.1111/j.1467-985X.1997.00078.x
  * [23] Terry Harris “Default definition selection for credit scoring” In _Artificial Intelligence Research_ 2.4 Sciedu Press, 2013, pp. 49–62 DOI: 10.5430/air.v2n4p49
  * [24] Terry Harris “Quantitative credit risk assessment using support vector machines: Broad versus Narrow default definitions” In _Expert Systems with Applications_ 40.11 Elsevier, 2013, pp. 4404–4413 DOI: 10.1016/j.eswa.2013.01.044
  * [25] Trevor Hastie, Robert Tibshirani and Jerome Friedman “The elements of statistical learning: Data mining, inference, and prediction” New York: Springer, 2009 DOI: 10.1007/978-0-387-84858-7
  * [26] Harold V Henderson and Paul F Velleman “Building multiple regression models interactively” In _Biometrics_ 37.2 JSTOR, 1981, pp. 391–411 DOI: 10.2307/2530428
  * [27] David W Hosmer and Stanley Lemeshow “Applied Logistic Regression” New York, USA: John Wiley & Sons, Inc., 2000 DOI: 10.1002/0471722146
  * [28] Gordon Hughes “On the mean accuracy of statistical pattern recognizers” In _IEEE Transactions on Information Theory_ 14.1 IEEE, 1968, pp. 55–63 DOI: 10.1109/TIT.1968.1054102
  * [29] IFRS 9 “International Financial Reporting Standard (IFRS) 9: Financial Instruments” London: IFRS Foundation, 2014 URL: https://www.ifrs.org/issued-standards/list-of-standards/ifrs-9-financial-instruments/
  * [30] Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani “An introduction to statistical learning: with applications in R” New York: Springer, 2013 DOI: 10.1007/978-1-4614-7138-7
  * [31] Kim Kaivanto “Maximization of the sum of sensitivity and specificity as a diagnostic cutpoint criterion” In _Journal of Clinical Epidemiology_ 61.5 Elsevier, 2008, pp. 517–518 DOI: 10.1016/j.jclinepi.2007.10.011
  * [32] Kenneth Kennedy et al. “A window of opportunity: Assessing behavioural scoring” In _Expert Systems with Applications_ 40.4 Elsevier, 2013, pp. 1372–1380 DOI: 10.1016/j.eswa.2012.08.052
  * [33] Mindy Leow and Jonathan Crook “The stability of survival model parameter estimates for predicting the probability of default: Empirical evidence over the credit crisis” In _European Journal of Operational Research_ 249.2 Elsevier, 2016, pp. 457–464 DOI: 10.1016/j.ejor.2014.09.005
  * [34] Mingfeng Lin, Henry C Lucas Jr and Galit Shmueli “Too big to fail: large samples and the p-value problem” In _Information Systems Research_ 24.4 INFORMS, 2013, pp. 906–917 DOI: 10.1287/isre.2013.0480
  * [35] Monica Lopez-Raton and Maria Xose Rodriguez-Alvarez “Computing Optimal Cutpoints in Diagnostic Tests [‘OptimalCutpoints’ R-package]” CRAN, 2021 URL: https://cran.r-project.org/web/packages/OptimalCutpoints/OptimalCutpoints.pdf
  * [36] Christoph Molnar “Interpretable Machine Learning”, 2022 URL: https://christophm.github.io/interpretable-ml-book/
  * [37] Jonah Mushava and Michael Murray “An experimental comparison of classification techniques in debt recoveries scoring: Evidence from South Africa’s unsecured lending market” In _Expert Systems with Applications_ 111 Elsevier, 2018, pp. 35–50 DOI: 10.1016/j.eswa.2018.02.030
  * [38] PRA “Main thematic findings for the implementation of ECL”, 2019 URL: https://www.bankofengland.co.uk/prudential-regulation/letter/2019/written-auditor-reporting-thematic-feedback-from-the-2018-2019-reporting-period
  * [39] PWC “IFRS 9: Expected Credit Losses”, 2014 URL: https://www.pwc.com/gx/en/audit-services/ifrs/publications/ifrs-9/ifrs-in-depth-expected-credit-losses.pdf
  * [40] Enrique F Schisterman, David Faraggi, Benjamin Reiser and Jessica Hu “Youden Index and the optimal threshold for markers with mass at zero” In _Statistics in Medicine_ 27.2 Wiley Online Library, 2008, pp. 297–315 DOI: 10.1002/sim.2993
  * [41] Naeem Siddiqi “Credit Risk Scorecards: Developing and Implementing Intelligent Credit Scoring” Hoboken, New Jersey: John Wiley & Sons, 2005
  * [42] Dietske Simons and Ferdinand Rolwes “Macroeconomic Default Modeling and Stress Testing” In _International Journal of Central Banking_ 5.3, 2009, pp. 177–204 URL: https://www.ijcb.org/journal/ijcb09q3a6.pdf
  * [43] Erik Štrumbelj and Igor Kononenko “Explaining prediction models and individual predictions with feature contributions” In _Knowledge and Information Systems_ 41, 2014, pp. 647–665 DOI: 10.1007/s10115-013-0679-x
  * [44] Lyn C Thomas “Consumer Credit Models: Pricing, Profit and Portfolios” New York: Oxford University Press, 2009 DOI: 10.1093/acprof:oso/9780199232130.001.1
  * [45] Lyn C Thomas, David B Edelman and Jonathan N Crook “Credit scoring and its applications” Philadelphia: Society for Industrial & Applied Mathematics, 2002 DOI: 10.1137/1.9780898718317
  * [46] Tony Van Gestel and Bart Baesens “Credit Risk Management: Basic Concepts” Oxford University Press, 2009 DOI: 10.1093/acprof:oso/9780199545117.001.0001
  * [47] John C. Wiginton “A Note on the Comparison of Logit and Discriminant Models of Consumer Credit Behavior” In _Journal of Financial and Quantitative Analysis_ 15.3, 1980, pp. 757–770 DOI: 10.2307/2330408
  * [48] Xin Xu “Estimating Lifetime Expected Credit Losses Under IFRS 9” In _Unisys Machine Learning and Advanced Analytics Services_ , Availaible at SSRN:, 2016 DOI: 10.2139/ssrn.2758513
  * [49] William J Youden “Index for rating diagnostic tests” In _Cancer_ 3.1 Wiley Online Library, 1950, pp. 32–35 DOI: 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3