An interpretable neural network-based non-proportional odds model for
  ordinal regression
This study proposes an interpretable neural network-based non-proportional
odds model (N$^3$POM) for ordinal regression. N$^3$POM is different from
conventional approaches to ordinal regression with non-proportional models in
several ways: (1) N$^3$POM is defined for both continuous and discrete
responses, whereas standard methods typically treat the ordered continuous
variables as if they are discrete, (2) instead of estimating response-dependent
finite-dimensional coefficients of linear models from discrete responses as is
done in conventional approaches, we train a non-linear neural network to serve
as a coefficient function. Thanks to the neural network, N$^3$POM offers
flexibility while preserving the interpretability of conventional ordinal
regression. We establish a sufficient condition under which the predicted
conditional cumulative probability locally satisfies the monotonicity
constraint over a user-specified region in the covariate space. Additionally,
we provide a monotonicity-preserving stochastic (MPS) algorithm for effectively
training the neural network. We apply N$^3$POM to several real-world datasets.
Akifumi Okuno, Kazuharu Harada
# An interpretable neural network-based

###### Abstract

This study proposes an interpretable neural network-based non-proportional odds model (N\({}^{3}\)POM) for ordinal regression. In the model, the response variable can take continuous values, and the regression coefficients vary depending on the predicting ordinal response. Contrary to conventional approaches, where the linear coefficients of regression are directly estimated from the discrete response, we train a non-linear neural network that outputs the linear coefficients by taking the response as its input. By virtue of the neural network, N\({}^{3}\)POM may have flexibility while preserving the interpretability of the conventional ordinal regression. We show a sufficient condition under which the predicted conditional cumulative probability (CCP) locally satisfies the monotonicity constraint over a user-specified region in the covariate space. We also provide a monotonicity-preserving stochastic (MPS) algorithm for adequately training the neural network.

**Keywords:** Continuous ordinal regression, Non-proportional odds model, Neural network

## 1 Introduction

Ordinal regression modeling treats response as an ordinal scale and aims to understand the relationship between the response order and covariates (Agresti, 2010). In the context of ordinal regression modeling, response variables are typically assumed to be ordinal and discrete (e.g., stage of cancer, scores of wine quality). While standard regression-based approaches are mainly interested in the actual value of the response, this study focuses on thresholds of the responses; that is, the probability of the response being less than or equal to a specific threshold as a function of the covariates.

Let \(d,J\in\mathbb{N}\) and consider \((G,X)\), a pair of such discrete ordinal response variables \(G\in\{1,2,\ldots,J\}\) and their covariate \(X\in\mathbb{R}^{d}\). A standard model for analyzing such a threshold is the proportional odds model (POM):

\[\text{logit}(\mathbb{P}_{\text{POM}}(G\leq j\mid X=\mathbf{x}))=\alpha_{j}+\langle \mathbf{\beta},\mathbf{x}\rangle\quad(j\in\{1,2,\ldots,J-1\}),\]

where \(\text{logit}(z)=\log\frac{z}{1-z}\) is a logit function, and \(\alpha_{1},\alpha_{2},\ldots,\alpha_{J-1}\in\mathbb{R},\mathbf{\beta}\in\mathbb{R }^{d}\) are parameters to be estimated. This model satisfies the parallelism assumption (McCullagh, 1980), which states that the regression coefficients are equal across all thresholds. However, the parallelism assumption is often considered to be violated (see, e.g., Long and Freese (2006)). For example, in restaurant ratings, basic factors such as good hygiene may be considered important in lower scores, while factors like ingredient origin and wine selection may be considered more significant in higher scores. See Williams (2016) for a discussion on the violation of the parallelism assumption in a real-world situation.