# Supplemental material

##  I  Formulation

Here we provide the full GR formalism for computing the QNMs. We define the
static spherically symmetric background metric as

$\displaystyle ds^{2}=-e^{\nu}dt^{2}+e^{\lambda}dr^{2}+r^{2}d\Omega^{2},$ (1)

where $\nu$ and $\lambda$ are functions of $r$ while $d\Omega$ is the unit
2-sphere line element. The Einstein field equations of the anisotropic fluid
background give (e.g., [1])

$\displaystyle\nu^{\prime}=$ $\displaystyle 2\frac{m+4\pi
r^{3}p_{r}}{r^{2}}e^{\lambda},$ (2) $\displaystyle p_{r}^{\prime}=$
$\displaystyle-(\rho+p_{r})\frac{\nu^{\prime}}{2}-\frac{2\sigma}{r},$ (3)
$\displaystyle\lambda^{\prime}=$ $\displaystyle 2\frac{-m+4\pi
r^{3}\rho}{r^{2}}e^{\lambda},$ (4) $\displaystyle e^{\lambda}=$
$\displaystyle\left(1-\frac{2m}{r}\right)^{-1},$ (5)

where the superscript prime denotes derivative with respect to $r$.

In the pulsating star, the metric and the matter dynamics are governed by the
perturbed Einstein field equations and the stress-energy conservation:

$\displaystyle\delta G_{\alpha\beta}=8\pi\delta T_{\alpha\beta},$ (6)
$\displaystyle\delta(\nabla_{\alpha}T^{\alpha\beta})=0,$ (7)

where $\nabla_{\alpha}$ denotes the covariant derivative, $G_{\alpha\beta}$ is
the Einstein tensor and $T_{\alpha\beta}$ is the stress-energy tensor. The
prefix $\delta$ represents Eulerian perturbations and is related to the
Lagrangian perturbations $\Delta$ through the Lie derivative [2],
$\Delta=\delta+\mathcal{L}_{\xi}$, given the Lagrangian displacement vector
$\xi^{\alpha}$ of the fluid.

We employ the same convention of the perturbed metric as in [3, 4], defined in
the Regge-Wheeler gauge [5]:

$\displaystyle\delta g_{\alpha\beta}dx^{\alpha}dx^{\beta}=$
$\displaystyle-\sum_{\ell,m}\big{[}e^{\nu}r^{\ell}H_{0}dt^{2}+2i\omega
r^{\ell+1}H_{1}dtdr$
$\displaystyle+e^{\lambda}r^{\ell}H_{2}dr^{2}+r^{\ell+2}Kd\Omega^{2}\big{]}e^{i\omega
t}Y_{\ell m}.$ (8)

Here $\left(H_{0},H_{1},H_{2},K\right)$ are functions of $r$, and $Y_{\ell m}$
are the spherical harmonics. The radial parts of the expansion depend on the
index $\ell$, which is suppressed for conciseness in the following
expressions. This applies to all the radial parts of the expansions in
spherical harmonics.

The stress-energy tensor takes the same form as [6]. One major assumption
about the perturbed density and radial pressure is the adiabatic relation

$\displaystyle\Delta p_{r}=\frac{\gamma p_{r}}{\rho+p_{r}}\Delta\rho,$ (9)

where $\gamma$ is the adiabatic index, and is assumed to be

$\displaystyle\gamma=\frac{\rho+p_{r}}{p_{r}}\left(\frac{\partial
p_{r}}{\partial\rho}\right)_{s}=\frac{\rho+p_{r}}{p_{r}}\left(\frac{\partial
p_{r}}{\partial\rho}\right)_{\text{eq}}.$ (10)

The subscript ‘$s$’ represents an adiabatic derivative while the subscript
‘eq’ denotes the derivative of the background $p_{r}$ against $\rho$. This
essentially means the EOS remains a single variable function in the dynamical
timescale. The Lagrangian displacement vector is defined as

$\displaystyle\xi^{\alpha}\partial_{\alpha}=$
$\displaystyle\sum_{\ell,m}r^{\ell-1}e^{i\omega
t}\Big{[}e^{-\lambda/2}WY_{\ell m}\partial_{r}$
$\displaystyle-\frac{V}{r}\left(\frac{\partial Y_{\ell
m}}{\partial\theta}\partial_{\theta}+\csc^{2}\theta\frac{\partial Y_{\ell
m}}{\partial\phi}\partial_{\phi}\right)\Big{]},$ (11)

where $V$ and $W$ are functions of $r$.

Using Eqs. (6)–(11), we arrive at the set of equations governing the stellar
pulsations:

$\displaystyle H_{1}^{\prime}=$ $\displaystyle\left[4\pi(\rho-
p_{r})e^{\lambda}r-\frac{2me^{\lambda}}{r^{2}}-\frac{\ell+1}{r}\right]H_{1}+\frac{e^{\lambda}}{r}K+\frac{e^{\lambda}}{r}H_{0}-\frac{16\pi(\rho+p_{r})}{r}e^{\lambda}\left(1-\bar{\sigma}\right)V,$
(12) $\displaystyle K^{\prime}=$
$\displaystyle\frac{\ell(\ell+1)}{2r}H_{1}+\left(\frac{\nu^{\prime}}{2}-\frac{\ell+1}{r}\right)K-8\pi(\rho+p_{r})\frac{e^{\lambda/2}}{r}W+\frac{1}{r}H_{0},$
(13) $\displaystyle W^{\prime}=$ $\displaystyle
re^{\lambda/2}(1-\bar{\sigma})K+\left(-\frac{\ell+1}{r}+\frac{2\bar{\sigma}}{r}\right)W+\frac{re^{(\lambda-\nu)/2}}{\gamma
p_{r}}X+\frac{re^{\lambda/2}}{2}H_{0}-\frac{\ell(\ell+1)}{r}e^{\lambda/2}(1-\bar{\sigma})V,$
(14) $\displaystyle X^{\prime}=$
$\displaystyle\frac{\rho+p_{r}}{2}e^{\nu/2}\left[r\omega^{2}e^{-\nu}+\frac{\ell(\ell+1)}{2r}(1-2\bar{\sigma})\right]H_{1}+\frac{\rho+p_{r}}{2}e^{\nu/2}\left[\left(\frac{3}{2}-2\bar{\sigma}\right)\nu^{\prime}-\left(1-6\bar{\sigma}\right)\frac{1}{r}-\frac{4\bar{\sigma}^{2}}{r}\right]K$
$\displaystyle-\frac{\rho+p_{r}}{r}e^{(\lambda+\nu)/2}\left[4\pi(\rho+p_{r})+\omega^{2}e^{-\nu}-F\right]W-\frac{1}{r}\left(\ell-2\frac{\rho+p_{r}}{\gamma
p_{r}}\bar{\sigma}\right)X$
$\displaystyle+\frac{\rho+p_{r}}{2}e^{\nu/2}\left(\frac{1}{r}-\frac{\nu^{\prime}}{2}\right)H_{0}+\frac{\ell(\ell+1)e^{\nu/2}p_{r}^{\prime}}{r^{2}}(1-\bar{\sigma})V+\frac{2e^{\nu/2}}{r}S.$
(15)

Here, the overbarred anisotropy parameter is defined by

$\bar{\sigma}=\frac{\sigma}{\rho+p_{r}},$ (16)

while $X$ is the radial part of the expansion

$\sum_{\ell,m}r^{\ell}XY_{\ell m}e^{i\omega t}=-e^{\nu/2}\Delta p_{r},$ (17)

and $F$ in Eq. (15) is given by

$\displaystyle
F=e^{-\lambda/2}\left\\{\frac{r^{2}}{2}\left(\frac{e^{-\lambda/2}\nu^{\prime}}{r^{2}}\right)^{\prime}-e^{-\lambda/2}\left[\left(\frac{6}{r^{2}}-\frac{2\nu^{\prime}}{r}\right)\bar{\sigma}-\frac{2}{r^{2}}\left(\frac{r\sigma^{\prime}}{\rho+p_{r}}\right)-\frac{4}{r^{2}}\bar{\sigma}^{2}\right]\right\\}.$
(18)

The Eulerian perturbation of the anisotropy parameter, $S$, is defined by

$\sum_{\ell,m}r^{\ell}SY_{\ell m}e^{i\omega t}=\delta\sigma,$ (19)

and is related to other perturbation variables by

$\displaystyle S=$ $\displaystyle-\left[\left(\frac{\partial\sigma}{\partial
p_{r}}\right)_{\text{eq}}+(\rho+p_{r})\left(\frac{A}{p_{r}^{\prime}}+\frac{1}{\gamma
p_{r}}\right)\left(\frac{\partial\sigma}{\partial\rho}\right)_{\text{eq}}\right]\frac{e^{-\lambda/2}p_{r}^{\prime}}{r}W$
$\displaystyle-\left[\left(\frac{\partial\sigma}{\partial
p_{r}}\right)_{\text{eq}}+\frac{\rho+p_{r}}{\gamma
p_{r}}\left(\frac{\partial\sigma}{\partial\rho}\right)_{\text{eq}}\right]e^{-\nu/2}X-\left(\frac{\partial\sigma}{\partial\mu}\right)_{\text{eq}}e^{-\lambda}H_{0}.$
(20)

Here, $A$ is the relativistic Schwarzschild discriminant, which we take to be
zero due to our assumption on $\gamma$ (i.e., assuming the EOS is a single
variable function of $p_{r}$), namely

$\displaystyle
A\equiv\frac{\rho^{\prime}}{\rho+p_{r}}-\frac{p_{r}^{\prime}}{\gamma
p_{r}}=0.$ (21)

We further assume the partial derivatives on $\sigma$ follow the equilibrium
background [6], similar to the case of $\gamma$. In general, these derivatives
should depend on the thermodynamic relations in the perturbation timescale
[7]. The Einstein field equations also give $H_{2}=H_{0}$ as in the isotropic
case. The variables ($H_{0}$,$V$) are related to the dependent variables
through the algebraic relations

$\displaystyle\left[3m+\frac{(\ell-1)(\ell+2)}{2}r+4\pi
r^{3}p_{r}\right]H_{0}=$ $\displaystyle 8\pi
r^{3}e^{-\nu/2}X-\left[\frac{\ell(\ell+1)}{2}(m+4\pi
r^{3}p_{r})-\omega^{2}r^{3}e^{-\lambda-\nu}\right]H_{1}$
$\displaystyle+\left[\frac{(\ell-1)(\ell+2)}{2}r-\omega^{2}r^{3}e^{-\nu}-\frac{e^{\lambda}}{r}(m+4\pi
r^{3}p_{r})(3m-r+4\pi r^{3}p_{r})\right]K$ $\displaystyle-16\pi
re^{-\lambda/2}(\rho+p_{r})\bar{\sigma}W,$ (22) $\displaystyle\quad\quad X=$
$\displaystyle\omega^{2}(\rho+p_{r})e^{-\nu/2}(1-\bar{\sigma})V+\frac{\rho+p_{r}}{2}e^{\nu/2}H_{0}-\frac{p_{r}^{\prime}}{r}e^{(\nu-\lambda)/2}W-e^{\nu/2}S.$
(23)

Let us summarize how the above equations are derived. Equations (12) and (13)
are the $(t,\theta)$-component and the $(t,r)$-component of Eq. (6). Equation
(14) can be obtained by combining the adiabatic relation Eq. (9) with the
$t$-component of Eq. (7). It can also be derived with an alternative approach
starting from the continuity equation as described in [8]. Equation (15) is
obtained by eliminating $H_{0}^{\prime}$, $K^{\prime}$ from the $r$-component
of Eq. (7) using the $(t,r)$ and $(\theta,r)$-components of Eq. (6). The
algebraic relation Eq. (22) is derived by eliminating the $H_{0}^{\prime}$,
$K^{\prime}$ terms with $(t,r)$, $(r,r)$, and $(\theta,r)$-components of Eq.
(6), and Eq. (23) is found from the $\theta$-component of Eq. (7). Equation
(20) comes from the relation

$\displaystyle\delta\sigma=\frac{\partial\sigma}{\partial p_{r}}\delta
p_{r}+\frac{\partial\sigma}{\partial\rho}\delta\rho+\frac{\partial\sigma}{\partial\mu}\delta\mu.$
(24)

To solve for the QNMs, we integrate Eqs. (12)–(15) numerically to obtain the
perturbed solution of the NS interior. We begin the integration near the
stellar center. Assuming the solutions are regular yields the following
constraints:

$\displaystyle H_{1}^{(0)}=$
$\displaystyle\frac{16\pi}{\ell(\ell+1)}(\rho^{(0)}+p_{r}^{(0)})W^{(0)}+\frac{2}{\ell+1}K^{(0)},$
(25) $\displaystyle X^{(0)}=$
$\displaystyle(\rho^{(0)}+p_{r}^{(0)})e^{\frac{\nu^{(0)}}{2}}\Bigg{\\{}\bigg{[}\frac{4\pi}{3}(\rho^{(0)}+3p_{r}^{(0)})+\frac{2\sigma^{(2)}}{\rho^{(0)}+p_{r}^{(0)}}-\frac{\omega^{2}e^{-\nu^{(0)}}}{\ell}\bigg{]}W^{(0)}+\frac{K^{(0)}}{2}\Bigg{\\}},$
(26) $\displaystyle S^{(0)}=$ $\displaystyle 0,$ (27)

where the superscript $(n)$ denotes the coefficient of the $n$th order
expansion of the quantity in series of $r$ about the stellar center. At the
surface, the vanishing Lagrangian pressure perturbation gives $X(R)=0$. The
interior solution is then matched with the metric perturbation in vacuum at
the stellar surface. We follow the Zerilli’s method described in [3] (also
check [9] for amendments of several typos) to solve for the complex frequency
corresponding to purely outgoing solutions at far-field region.

Note that Eq. (27) combined with Eq. (20) effectively places an extra
constraint on $\sigma$ near the center:

$\displaystyle\left(\frac{\partial\sigma}{\partial
x}\right)_{\text{eq}}=\mathcal{O}(r^{2}),$ (28)

where $x$ represents ($p_{r}$, $\rho$, $\mu$). This is similar to how the
background structure equation (Eq. (3)) restricts $\sigma$ to go as $r^{2}$ at
the center (at least $r$ to avoid irregularity). The H anisotropy model in Eq.
(LABEL:eq:H-model) is modified from the original form in [10] to satisfy the
above condition.

##  II An alternative derivation with the continuity equation

In this section, we provide an alternative way to derive the pulsation
equations, Eq. (14) in specific. Instead of using the $t$-component of Eq.
(7), we utilize the continuity equation of particle number density and the
relation of thermodynamics. At the end, we explain how this method allows
incorporating a more general form of the EOS in the perturbed configuration.

We begin by reviewing the derivation of the pulsation equations for
_isotropic_ stars. It is common to introduce the particle number density, $n$,
which is governed by the continuity equation (see, e.g., [2]):

$\displaystyle\nabla_{\alpha}(nu^{\alpha})=0\implies\frac{\Delta
n}{n}=-\frac{1}{2}h^{\alpha\beta}\Delta g_{\alpha\beta}.$ (29)

The number density is an independent variable in the EOS, i.e., $p=p(n)$,
$\rho=\rho(n)$, but does not enter the equation of motion directly once we
have $\rho$ as a function of $p$. The pressure $p$ here is the isotropic
pressure. The Lagrangian perturbation of $n$ is related to that of $\rho$
through the thermodynamic relation

$\displaystyle\Delta\rho=(\rho+p)\frac{\Delta n}{n}.$ (30)

Combining Eqs. (29), (30) and the adiabatic relation
$\gamma\Delta\rho/(\rho+p)=\Delta p/p$ gives the $t$-component of the stress
energy conservation (Eq. (7)) of an isotropic fluid.

While Eq. (29) still holds in the anisotropic case, Eq. (30) needs to be
modified, since the work done on the fluid does not depend only on the volume
change, but is directional dependent111Mondal and Bagchi [11] incorrectly
assumed that Eq. (30) is valid also for anisotropic stars.. This requires us
to explicitly write down the change in energy due to work done in each
direction (see, e.g., [12, 13], for the generalized thermodynamic relations in
an anisotropic medium):

$\displaystyle\Delta\rho=(\rho+p_{r})\frac{\Delta n}{n}+\sigma(\Delta
U^{\theta}{}_{\theta}+\Delta U^{\phi}{}_{\phi}),$ (31)

where $\Delta U_{\alpha\beta}$ is the perturbative Lagrangian strain tensor
[14] 222In fluid dynamics, it is more common to use the strain rate tensor to
describe the deformation of fluid elements during the flow. Since each fluid
element in the pulsating star oscillates about its equilibrium position, the
strain rate just differs from the strain by a factor of $i\omega e^{-\nu/2}$.,
given by

$\displaystyle\Delta U_{\alpha\beta}=\Delta
h_{\alpha\beta}=\frac{1}{2}h_{\alpha}{}^{\mu}h_{\beta}{}^{\nu}\Delta
g_{\mu\nu},$ (32)

and $\Delta U^{\alpha}{}_{\beta}=g^{\alpha\mu}\Delta U_{\mu\beta}$. Equation
(31), substituted with the adiabatic relation, Eq. (9), reads

$\displaystyle\frac{\Delta p_{r}}{p_{r}}=\gamma\left[\frac{\Delta
n}{n}+\bar{\sigma}(\Delta U^{\theta}{}_{\theta}+\Delta
U^{\phi}{}_{\phi})\right].$ (33)

One can show that this is equivalent to Eq. (14) using Eqs. (29) and (32),
which are explicitly written as

$\displaystyle\frac{\Delta n}{n}=$
$\displaystyle\sum_{\ell,m}\Bigg{[}\frac{1}{2}H_{0}+K-\frac{e^{-\lambda/2}}{r}W^{\prime}-\frac{\ell+1}{r^{2}}e^{-\lambda/2}W-\frac{\ell(\ell+1)}{r^{2}}V\Bigg{]}r^{\ell}Y_{\ell
m}e^{i\omega t},$ (34) $\displaystyle\Delta U^{\theta}{}_{\theta}+\Delta
U^{\phi}{}_{\phi}=$
$\displaystyle\sum_{\ell,m}\left[-K+2e^{-\lambda/2}\frac{W}{r^{2}}+\frac{\ell(\ell+1)}{r^{2}}V\right]r^{\ell}Y_{\ell
m}e^{i\omega t}.$ (35)

Notice that, in general, the anisotropic fluid can have different adiabatic
constants in each direction depending on the anisotropy EOS. That is,

$\displaystyle\frac{\Delta p_{r}}{p_{r}}=-\sum_{i}\gamma_{i}\Delta
U^{i}{}_{i},$ (36)

where $\gamma_{i}$ denotes the adiabatic index in the $i$-th spatial
direction. This suggests a further generalization of the pulsation equations
described in the formulation section. We leave such studies for future work.

##  III Integral relation of the eigenvalue and eigenfunctions

The eigenfrequencies and the eigenfunctions of the pulsation equations are
related through the integral equations

$\displaystyle\omega^{2}\int_{0}^{\infty}(I_{1}+J_{1})dr=$
$\displaystyle\int_{0}^{\infty}(I_{2}+J_{2})dr+I_{3}(R)+J_{3}(R)$
$\displaystyle+I_{4}(\infty),$ (37)

where $J_{1}$ to $J_{3}$ vanish in the isotropic limit and are non-zero only
inside the star. In this section, we provide the explicit forms of the terms
$I_{1}$ to $I_{4}$ and $J_{1}$ to $J_{3}$.

The terms $I_{1}$ to $I_{4}$ are obtained from Eq. (16) of [15], and are
explicitly written as:

$\displaystyle I_{1}=$ $\displaystyle
e^{\frac{\lambda-\nu}{2}}r^{2}\left\\{(\rho+p_{r})\left[\frac{1}{r^{4}}\left|\tilde{W}\right|^{2}+\frac{\ell(\ell+1)}{r^{2}}\left|\tilde{V}\right|^{2}\right]-\frac{\ell(\ell+1)}{16\pi
e^{\lambda}r^{2}}\left|\tilde{H}_{1}\right|^{2}-\frac{1}{16\pi}\left[\left|\tilde{K}\right|^{2}+2\operatorname{Re}\left(\tilde{K}\tilde{H}_{0}^{*}\right)\right]\right\\},$
(38) $\displaystyle I_{2}=$ $\displaystyle
e^{\frac{\lambda+\nu}{2}}r^{2}\Bigg{\\{}-\frac{(\rho+p_{r})\nu^{\prime}A}{2e^{\lambda}r^{4}}\left|W\right|^{2}+\frac{1}{\gamma
p_{r}}\left|\tilde{P}\right|^{2}+\left[\frac{\ell(\ell+1)}{16\pi
r^{2}}-\frac{3(\rho+p_{r})}{4}\right]\left|\tilde{H}_{0}\right|^{2}-\frac{1}{16\pi
e^{\lambda}}\left|\tilde{K}^{\prime}\right|^{2}$
$\displaystyle+\frac{\rho+p_{r}}{\gamma
p_{r}}\operatorname{Re}\left(\tilde{P}\tilde{H}_{0}^{*}\right)-\frac{(\rho+p_{r})A}{e^{\lambda/2}r^{2}}\operatorname{Re}(\tilde{W}\tilde{H}_{0}^{*})+\frac{1}{8\pi
e^{\lambda}}\operatorname{Re}\left[\left(\nu^{\prime}\tilde{H}_{0}+\tilde{H}_{0}^{\prime}\right)\tilde{K}^{\prime\,*}\right]\Bigg{\\}},$
(39) $\displaystyle I_{3}=$
$\displaystyle\frac{e^{\nu}\rho}{2}\left[2e^{-\nu/2}\operatorname{Re}\left(\tilde{W}\tilde{H}_{0}^{*}\right)+\frac{\nu^{\prime}}{r^{2}}\left|\tilde{W}\right|^{2}\right],$
(40) $\displaystyle I_{4}=$
$\displaystyle\frac{e^{\nu}r^{2}}{16\pi}\left[\tilde{K}^{*}\left(\tilde{H}_{0}^{\prime}+\nu^{\prime}\tilde{H}_{0}-\tilde{K}^{\prime}\right)-\tilde{K}^{\prime}\tilde{H}_{0}^{*}\right].$
(41)

Here, $\tilde{P}$ is defined through $\delta
p_{r}=\sum_{\ell,m}\tilde{P}Y_{\ell m}e^{i\omega t}$, and the symbol
$\operatorname{Re}$ denotes taking the real part. The perturbation variables
are redefined in the following ways to simplify the expressions:

$\displaystyle r^{\ell}K=$ $\displaystyle\tilde{K},\quad
r^{\ell}H_{0}=\tilde{H}_{0},\quad r^{\ell}V=-\tilde{V},$ $\displaystyle
r^{\ell}S=$ $\displaystyle\tilde{S},r^{\ell+1}H_{1}=\tilde{H}_{1},\quad
r^{\ell+1}W=\tilde{W}.$ (42)

We see that $I_{1}$ to $I_{3}$ are real while $I_{4}$ is in general complex if
we impose the purely outgoing boundary condition at infinity [16, 15].

The terms $J_{1}$ to $J_{3}$ are derived using the procedures described in
Appendix A of [15], with the Einstein equations and stress-energy conservation
for the anisotropic case:

$\displaystyle J_{1}=$
$\displaystyle-\ell(\ell+1)e^{\frac{\lambda-\nu}{2}}(\rho+p_{r})\bar{\sigma}\left|\tilde{V}\right|^{2},$
(43) $\displaystyle J_{2}=$ $\displaystyle
e^{\frac{\nu}{2}}\Bigg{[}(\rho+p_{r})\bar{\sigma}\left(-\tilde{K}^{\prime}+\frac{2e^{-\frac{\lambda}{2}}A}{r^{3}}\tilde{W}\right)-\frac{2\bar{\sigma}}{r}\left(1+\frac{\rho+p_{r}}{\gamma
p_{r}}\right)\tilde{P}_{r}+\frac{2}{r}\tilde{S}\Bigg{]}\tilde{W}^{*}$
$\displaystyle+e^{\frac{\nu+\lambda}{2}}\left(\bar{\sigma}\tilde{P}_{r}+\frac{\rho+p_{r}}{2}\bar{\sigma}\tilde{H}_{0}-\tilde{S}\right)\left[r^{2}\tilde{K}^{*}+\ell(\ell+1)\tilde{V}^{*}\right],$
(44) $\displaystyle J_{3}=$
$\displaystyle\frac{2e^{\nu}\rho}{r^{3}}\bar{\sigma}\left|\tilde{W}\right|^{2}.$
(45)

The perturbation variables of $J_{1}$ and $J_{3}$ are paired in the same way
as $I_{1}$, while $J_{2}$ is in general complex as the complex perturbation
variables cannot be paired in the same ways as $I_{1}$ to $I_{3}$. Although it
is not obvious that the complex pulsation variables within $J_{2}$ cannot be
paired up to form real terms, we verified this by numerically computing the
integral using the solutions of the QNMs.

Another perspective is to focus on $\tilde{S}$, which depends on the
anisotropy ansatz itself. The choice of the functional form for $\sigma$
affects the perturbation variable $\tilde{S}$, which in general causes $J_{2}$
to be complex. In [17], it is shown that the non-radial modes are stable as
long as the anisotropy is finite in the Newtonian incompressible star with
$\tilde{S}=\beta\tilde{P}$. We point out here that it holds specifically for
this anisotropy ansatz and therefore does not contradict our findings in this
paper. In addition, the instability comes from the imaginary part of the
eigenfrequency, which is caused by the GR effect.

##  IV Additional numerical data

Figure 1: The real (top panel) and imaginary (bottom panel) frequencies of the
first six $\ell=2$ QNMs of the Poly-H model with central density 7.455$\times
10^{14}~{}\text{g cm}^{-3}$. The left panel has $\beta=0$ and the right panel
has $\beta=10$. We also present the normal mode frequencies computed using the
relativistic Cowling approximation (Cowling) described in [6].

In this section, we provide additional numerical data of the QNMs computed for
the EOSs and anisotropy models shown in the main text. We demonstrate that the
anisotropy has a small effect on the real part of the frequency, even though
it changes the stability of the mode. We also quantify the validity of the
Cowling approximation for computing the oscillation modes of anisotropic stars
for the first time.

In Fig. 1, we show the real and imaginary parts of the QNM frequencies for the
first six $\ell=2$ modes of the Poly-H model. The real part of the frequencies
of the isotropic and the anisotropic case is of similar values for all the
modes. For the isotropic case, the imaginary part of the modes is all above
zero and decreases with the mode order. Meanwhile, that of the anisotropic
case does not show a monotonic behavior and can become negative.

Table 1:  Numerical data of the first three $\ell=2$ QNM frequencies of the anisotropic NS models. The central densities of the Poly models and the SLy4 models are set at $7.455\times 10^{14}\text{g cm}^{-3}$ and $9.88\times 10^{14}\text{g cm}^{-3}$ respectively. Model | $\beta$ |  $M$ ($M_{\odot}$) | Re($\omega$) (kHz) | Im($\omega$) (Hz)
---|---|---|---|---
|  |  | $f$ | $p_{1}$ | $p_{2}$ | $f$ | $p_{1}$ | $p_{2}$
Poly | 0 | 1.40 | 1.574 | 4.344 | 6.784 | 0.550 | $5.93\times 10^{-2}$ | $1.95\times 10^{-4}$
Poly-H | $-10.0$ | 2.18 | 1.945 | 4.364 | 6.104 | 2.262 | $-1.573$ | 1.212
Poly-H | 10.0 | 1.10 | 1.233 | 4.215 | 6.948 | 0.156 | $-0.259$ | $-0.265$
Poly-BL | $-0.5$ | 1.74 | 1.791 | 4.347 | 6.561 | 1.231 | 0.351 | $7.80\times 10^{-3}$
Poly-BL | 0.5 | 1.16 | 1.325 | 4.324 | 6.931 | 0.221 | $-2.01\times 10^{-2}$ | $-3.59\times 10^{-3}$
SLy4 | 0 | 1.40 | 1.934 | 6.304 | 9.348 | 0.830 | $3.77\times 10^{-2}$ | $2.25\times 10^{-5}$
SLy4-H | $-10.0$ | 2.52 | 2.527 | 5.715 | 7.672 | 5.561 | 13.24 | $-11.78$
SLy4-H | 10.0 | 1.06 | 1.322 | 6.147 | 8.877 | 0.129 | $-1.082$ | $-0.361$
SLy4-BL | $-0.5$ | 1.75 | 2.226 | 6.070 | 9.029 | 2.046 | 0.186 | 0.121
SLy4-BL | 0.5 | 1.16 | 1.556 | 6.437 | 9.109 | 0.274 | $-3.90\times 10^{-2}$ | $-1.24\times 10^{-2}$

The full GR formalism can be reduced to the relativistic Cowling limit given
in [6]. Notice that the approximation procedure is not by simply taking all
metric perturbations to zero. The steps involved are the same as the isotropic
case described in, e.g., [18, 19], where we set $H_{0}$, $K$, $H_{1}$ to zero
in Eqs. (14), (15), and (23), and further drop the term
$-4\pi(\rho+p_{r})^{2}e^{(\lambda+\nu)/2}W/r$ in Eq. (15) 333The reason for
removing this term comes from Eq. (13) in the Cowling limit, assuming $H_{1}$
does not reduce to zero in the same way as $H_{0}$ and $K$. An argument about
$H_{1}$ being larger than the other metric perturbations in the Newtonian
limit is given in [20]. The resulting formulas are equivalent to Eqs. (27),
(29) and (30) of [6] with a barotropic EOS, and can further be reduced to the
isotropic limit [21]. In Fig. 1, we also show the frequencies obtained by the
Cowling approximation. The $f$-mode frequencies show 20-30% deviations from
the full GR real frequencies. The percentage deviation decreases if we go to
higher radial order $p$-modes. In the case of the $p_{5}$-modes, the deviation
reduces to a $1\%$ level.

More numerical data are provided in Table. 1, including the real and imaginary
frequencies of the first three $\ell=2$ QNMs, as a reference for readers
interested in a numerical comparison. The $\beta=0$ case of the SLy4 EOS has
been checked against numerical data available (e.g., [22]), and attains good
agreement. We see that with the same $\rho$–$p_{r}$ relation, the real part of
the frequencies is not very sensitive to the change in $\beta$, while the
imaginary part can change by orders of magnitude and even change the sign.

## References

  * Bowers and Liang [1974] R. L. Bowers and E. P. T. Liang, Anisotropic spheres in general relativity, Astrophys. J. 188, 657 (1974).
  * Friedman and Schutz [1975] J. L. Friedman and B. F. Schutz, On the stability of relativistic systems, Astrophys. J. 200, 204 (1975).
  * Lindblom and Detweiler [1983] L. Lindblom and S. L. Detweiler, The quadrupole oscillations of neutron stars, Astrophys. J., Suppl. Ser. 53, 73 (1983).
  * Detweiler and Lindblom [1985] S. Detweiler and L. Lindblom, On the nonradial pulsations of general relativistic stellar models, Astrophys. J. 292, 12 (1985).
  * Regge and Wheeler [1957] T. Regge and J. A. Wheeler, Stability of a schwarzschild singularity, Phys. Rev. 108, 1063 (1957).
  * Doneva and Yazadjiev [2012] D. D. Doneva and S. S. Yazadjiev, Nonradial oscillations of anisotropic neutron stars in the cowling approximation, Phys. Rev. D 85, 124023 (2012).
  * Thorne [1967] K. S. Thorne, The General Relativistic Theory of Stellar Structure and Dynamics, in  _High Energy Astrophysics, Volume 3_, Vol. 3, edited by C. Dewitt, E. Schatzman, and P. Véron (1967) p. V.
  * [8] See Supplemental Material at URL-will-be-inserted-by-publisher for the detailed formalism of the pulsation problem, the explicit form of the integral formula relating the eigenfrequencies and eigenfunctions of the pulsation problem, and additional numerical data of the quasinormal modes. (see also references [2, 3, 5, 7, 21, 20, 19, 12, 14, 16, 22] therein).
  * Lü and Suen [2011] J.-L. Lü and W.-M. Suen, Determining the long living quasi-normal modes of relativistic stars, Chin. Phys. B 20, 040401 (2011).
  * Horvat  _et al._ [2010] D. Horvat, S. Ilijić, and A. Marunović, Radial pulsations and stability of anisotropic stars with a quasi-local equation of state, Class. Quantum Gravity 28, 025009 (2010).
  * Mondal and Bagchi [2023] S. Mondal and M. Bagchi, Non-radial oscillation of anisotropic neutron stars in full general relativity (2023).
  * Landau and Lifshits [1959] L. Landau and E. Lifshits,  _Theory of Elasticity_ (Pergamon, 1959).
  * Karlovini  _et al._ [2004] M. Karlovini, L. Samuelsson, and M. Zarroug, Elastic stars in general relativity: Ii. radial perturbations, Class. Quantum Gravity 21, 1559–1581 (2004).
  * Carter and Quintana [1972] B. Carter and H. Quintana, Proc. R. Soc. A 331, 57–83 (1972).
  * Detweiler and Ipser [1973] S. L. Detweiler and J. R. Ipser, A variational principle and a stability criterion for the non-radial modes of pulsation of stellar models in general relativity, Astrophys. J. 185, 685 (1973).
  * Price and Thorne [1969] R. Price and K. S. Thorne, Non-radial pulsation of general-relativistic stellar models. ii. properties of the gravitational waves, Astrophys. J. 155, 163 (1969).
  * Hillebrandt and Steinmetz [1976] W. Hillebrandt and K. O. Steinmetz, Anisotropic neutron star models: stability against radial and nonradial pulsations., Astron. Astrophys. 53, 283 (1976).
  * Lindblom and Splinter [1990] L. Lindblom and R. J. Splinter, The Accuracy of the Relativistic Cowling Approximation, Astrophys. J. 348, 198 (1990).
  * Zhao  _et al._ [2022] T. Zhao, C. Constantinou, P. Jaikumar, and M. Prakash, Quasinormal $g$ modes of neutron stars with quarks, Phys. Rev. D 105, 103025 (2022).
  * Finn [1988] L. S. Finn, Relativistic stellar pulsations in the Cowling approximation, Mon. Not. R. Astron. Soc. 232, 259 (1988), https://academic.oup.com/mnras/article-pdf/232/2/259/3316314/mnras232-0259.pdf .
  * McDermott  _et al._ [1983] P. N. McDermott, H. M. van Horn, and J. F. Scholl, Nonradial g-mode oscillations of warm neutron stars, Astrophys. J. 268, 837 (1983).
  * Kunjipurayil  _et al._ [2022] A. Kunjipurayil, T. Zhao, B. Kumar, B. K. Agrawal, and M. Prakash, Impact of the equation of state on $f$\- and $p$\- mode oscillations of neutron stars, Phys. Rev. D 106, 063005 (2022).


# Leveraging Open-Source Large Language Models for encoding Social
Determinants of Health using an Intelligent Router

Akul Goel1 Surya Narayanan Hari1 Belinda Waltman2 Matt Thomson1
1 California Institute of Technology
2 Los Angeles County Department of Health Services
<EMAIL_ADDRESS>
###### Abstract

Social Determinants of Health (SDOH) play a significant role in patient health
outcomes. The Center of Disease Control (CDC) introduced a subset of ICD-10
codes called Z-codes in an attempt to officially recognize and measure SDOH in
the health care system. However, these codes are rarely annotated in a
patient’s Electronic Health Record (EHR), and instead, in many cases, need to
be inferred from clinical notes. Previous research has shown that large
language models (LLMs) show promise on extracting unstructured data from EHRs.
However, with thousands of models to choose from with unique architectures and
training sets, it’s difficult to choose one model that performs the best on
coding tasks. Further, clinical notes contain trusted health information
making the use of closed-source language models from commercial vendors
difficult, so the identification of open source LLMs that can be run within
health organizations and exhibits high performance on SDOH tasks is an urgent
problem. Here, we introduce an intelligent routing system for SDOH coding that
uses a language model router to direct medical record data to open source LLMs
that demonstrate optimal performance on specific SDOH codes. The intelligent
routing system exhibits state of the art performance of 97.4% accuracy
averaged across 5 codes, including homelessness and food insecurity, on par
with closed models such as GPT-4o. In order to train the routing system and
validate models, we also introduce a synthetic data generation and validation
paradigm to increase the scale of training data without needing privacy
protected medical records. Together, we demonstrate an architecture for
intelligent routing of inputs to task-optimal language models to achieve high
performance across a set of medical coding sub-tasks.

##  1 Introduction

Social determinants of health (SDOH) are defined as non-medical factors that
strongly influence health outcomes. SDOH are increasingly regarded as
significant risk factors across a broad range of health outcomes and generate
measures of well-being [14]. SDOH generally include a broad range of factors
like economic stability and access to resources such as food, housing,
education, health care. Ideally a holistic, whole-person approach to physical
health should include a focus on these social determinants of health because
it is not possible to treat acute or chronic conditions in a vacuum. For
example, it may be challenging for an unhoused patient to afford, purchase,
and administer medications for chronic conditions when their primary, daily
survival focus is around finding a safe place to sleep at night. Given the
central importance of SDOH, there is an increased focus on screening for SDOH
and applying interventions in the form of social services to impact SDOH. An
important limitation for SDOH scoring is related to data - historically, many
of the notes related to SDOH domains were free-texted into the Social History
section of the EMR, so it may be difficult to extract for both reporting and
clinical intervention purposes.

Large Language Models (LLMs) show promise for extracting data from
unstructured medical notes. They have already been used on a variety of
clinical tasks such as predicting readmission rates from Electronic Health
Records (EHRs) and answering clinical questions [2][12][11]. There has even
been a publication to date using LLMs to identify generally ’adverse’ SDOH
codes[4]. However, achieving high accuracy results generally use closed-model
architectures like GPT4 or Claude, or require significant fine-tuning
[4][10][12]. Closed-model architectures - LLMs whose training data, weights,
and prompting strategy are proprietary, and are hosted on industrial cloud
computing infrastructure, require the transmission of health data to be taken
advantage of. As an alternative to closed source models, open source language
models have proliferated in recent years with models like LLAMA and Mistral
achieving performance on par with closed source models [13, 7]. However, there
are currently over 100,000 open source language models on the Hugging Face
platform, and it is difficult to discover open source LLMs that might be
optimal for a given task [5, 6]. Further, fine-tuning open-source models
requires valuable clinical data, where the lack thereof or quality-issues of
this data can cause model performance to suffer [3].

Here we introduce an intelligent routing architecture that leverages multiple
non-fine-tuned open-source LLMs to achieve state of the art accuracy. Our
router not only navigates the problem of open-source models being subject to a
variety of training architectures and possessing mixed quality data
[15][1][8], but in fact takes advantage of this by choosing LLMs where
training data may have lent itself better to higher accuracy on certain SDOH
codes. We also introduce a new synthetic data generation and validation
paradigm to address the issue of lack of high-quality clinical data that is
often hidden behind privacy protected medical records. Our router system
trained on this synthetic data, without fine-tuning or using closed-source
models, is able to achieve 97.4% accuracy, on par with much bigger, top-of-
the-line models like GPT-4o.

##  2 Results

### Identification of open source models for specific SDOH codes

Language models are trained on distinct data domains and show differential
performance on tasks including the identification of specific SDOH codes [5,
6]. SDOH codes span a variety of underlying topics from housing to family
relationships to incarceration status. Hence, coding is well suited to an
ensemble approach where expert models are used to analyze clinical notes for
specific SDOH factors. Therefore, we developed an intelligent routing system
for SDOH coding that consists of a router model and an ensemble of down-stream
open source models that show expert performance on coding for specific SDOH
factors.

To identify optimal open source models, we generated a data set that contains
both 500 medical notes from the MIMIC-III dataset [9], as well as synthetic
examples generated using an LLM. We, first, analyzed the performance of a set
of open source language models on each of seven codes (Figure 1), and, then,
trained a router to route incoming coding tasks to the optimal down-stream
model (Figure 8).

Figure 1 shows the performance of ten open-source large language models (LLMs)
across seven social determinants of health code (SDOH) on a dataset of 50
notes from the MIMIC-III database [9]. From this graph, we were able to
identify at least one model that achieved >80% accuracy on each code,
indicating overall that open-source LLMs have the ability to extract SDOH
information from unstructured medical charts. However, we observed variable
performance on these notes, where a different model performs the best on a
given code. We also see that some codes, such as unemployment, demonstrate
high accuracy across models, while others, like relative needing care, possess
more variable and lower overall accuracy scores.

Figure 1: LLMs exhibit differential performance on SDOH coding task. Plot
shows performance in accuracy of ten different open-source LLMs on seven SDOH
codes, where accuracy is defined as (True Positives + True Negatives) / (Total
Number of Notes). True positive is defined as correctly identifying a note as
needing a specific SDOH code, and true negative is defined as correctly
identifying a note as not needing a specific SDOH code. Models are indicated
by legend (in distinct colors). Different models emerge as the best model
depending on the specific SDOH task, with at least one model achieving 80%
accuracy on each code. Analysis was performed on 50 medical notes from the
MIMIC-III dataset [9].

Only a few sentences from the medical notes (<1% of the sentences) contained
evidence of specific SDOH codes, so we generated and validated synthetic data.
Furthermore, after manually labeling 500 medical notes from the MIMIC-III
dataset[9], we distilled our validation set to test only 5 SDOH codes
(homelessness, food insecurity, imprisonment or other incarceration, marital
estrangement, and relative needing care) for a proof of concept. Training a
router to pick a best model for a given SDOH factor from the medical note, we
observed the performance shown in Figure 3. On a 1000 sentence validation set,
containing sentences from 500 labeled medical notes found in the MIMIC-III
dataset [9] and synthetic data, our router picked a model that achieved >90%
accuracy and F1 score for every single code. The highest performing code was
homelessness, where the router picked model identified this code with 99.0%
accuracy (0.984 F1 score). The lowest performing code was imprisonment or
other incarceration, at 94.7% accuracy (0.918 F1 score). Comparing the
performance of the router picked model to GPT-4o, our chosen model beats that
of GPT for 4 of the 5 codes, only exhibiting marginally worse performance for
the relative needing care SDOH factor (97.5% vs 97.8% accuracy for our router
picked model vs GPT-4o).

SDOH Code | Model Chosen
---|---
Homelessness | NousResearch/Nous-Hermes-2-Yi-34B
Food Insecurity | Zero-one-ai/Yi-34B-Chat
Imprisonment or Other Incarceration | NousResearch/Nous-Hermes-2-Yi-34B
Marital Estrangement | NousResearch/Nous-Hermes-2-Yi-34B
Relative Needing Care | Meta-llama/Llama-2-13b-chat-hf
Figure 2: Router chooses different model based on SDOH code. Table showing
model chosen by router for each of 5 SDOH codes. Their performances on the
test set are represented in the ‘router model’ bar graph in Fig. 3 Figure 3:
Router chosen model exhibits high accuracy across codes, with comparable
performance to GPT-4o. a) Performance in accuracy of router chosen model
(blue) and GPT-4o (gold) against 5 SDOH codes on the dataset described in Fig.
6. Accuracy is defined as (True Positive + True Negative) / (Total Number of
Sentences) where true positives are sentences correctly identified as having
evidence of a specific SDOH code, and true negatives are defined as sentences
correctly identified as having no evidence of a specific SDOH code. Router
exhibits comparable accuracy to GPT-4o, slightly beating it in 4 of the 5
codes. b) Performance in F1 score of router chosen model (blue) and GPT-4o
(gold) against 5 SDOH codes on the dataset described in Fig. 6. The F1 score
is defined as (2*Precision*Recall) / (Recall + Precision), where precision is
calculated by (True Positive)/ (True Positive + False Positive), and recall is
calculated as (True Positive)/ (True Positive + False Negative). True positive
is defined as correctly identifying a sentence as needing a specific SDOH
code, and true negative is defined as correctly identifying a sentence as not
needing a specific SDOH code. False positive is defined as incorrectly
identifying a sentence as needing a specific SDOH code, and false negative is
defined as incorrectly identifying a sentence as not needing a specific SDOH
code. Router chosen model exhibits F1 scores to GPT-4o, slightly beating it in
4 of the 5 codes. Note that the label “imprisonment” was shortened from
“imprisonment or other incarceration” on both graphs for visualization
purposes.

In Figure 2, we see that the router selected a different best open-source
performing model based on the SDOH code to achieve the performance seen in
Figure 3. The most common model picked was NousResearch/Nous-Hermes-2-Yi-34B,
chosen for 3 of the 5 SDOH codes. The LLMs Zero-one-ai/Yi-34B-Chat and Meta-
llama/Llama-2-13b-chat-hf were chosen for one code each. All of these models
were evaluated without fine-tuning, are completely open-source, and can be
found on HuggingFace.

##  3 Methods

Our data came from patient electronic health records (EHRs) stored in the
MIMIC-III dataset [9]. We extracted medical charts with a ‘Social History’
section, on the assumption that these notes would be more likely to contain
evidence of an SDOH code. The factors we tested included homelessness, food
insecurity, imprisonment or other incarceration, low income, and marital
estrangement, relatives needing care, and unemployment. A medical coder
annotated 500 medical notes for by parsing through the notes, and indicating
information verbatim from the medical notes that corresponded to a specific
SDOH code. Noticing that these codes were assigned based on a phrase from the
note, we determined that a sentence had enough context for an LLM to
appropriately identify the codes bearing evidence in the medical records. We
prompted a number of open source models with the Together API using the prompt
template in Figure 4.

(a)

(b)

Figure 4: Models were prompted with a sentence from the medical note to
classify. a) The above figure shows the prompt template used to prompt all
models on all SDOH codes. The specific SDOH we were aiming to classify was
input into the SDOH keyword or phrase (green) field, and a sentence from the
medical note the model was to classify was placed in the SENTENCE (blue)
field. b) This figure shows a specific example of a prompt for SDOH code
’imprisonment or other incarceration’ (green), with a sentence (blue) from the
medical note corresponding to it.

Even after annotating 500 medical notes, sentences with evidence of specific
SDOH codes were sparse (<1% of the total number of sentences). In order to
address this, we introduced a novel synthetic data generation and validation
paradigm, outlined in Figure 5. Our synthetic data generation scheme involved
passing in random sentences corresponding to a specific SDOH code from the
medical note into Claude-3-opus as references. Using two distinct prompts, one
explicitly asking Claude to not use the SDOH keyword or phrase for complexity,
we generated a couple hundred synthetic sentences corresponding to SDOH codes.
We subsequently also had Claude verify the synthetic sentences it generated by
asking Claude to determine if the generated synthetic sentence actually had
evidence of a specific SDOH code, dropping the sentences that did not pass
this test. This scheme ultimately resulting in a validation set distribution
seen in Figure 6.

Figure 5: Claude-3-opus generated and verified synthetic data. Figure shows flow chart of synthetic data generation and verification by Claude-3-opus. Random sentences corresponding to a specific SDOH code were given as examples from labeled medical notes. Based on these examples, Claude was asked to generate synthetic sentences via two different prompts, one explicitly asking the model to not use the SDOH keyword for complexity. Each synthetic sentence was subsequently checked by Claude for evidence of the specific SDOH code and sentences that did not pass were discarded. Sentences that passed the verification were added to an SDOH’s synthetic data collection. Random sentences for examples were picked from 500 labeled medical notes from the MIMIC-III dataset [9].  SDOH Code | Gold Data | Synthetic Data | Negative Data
---|---|---|---
Homelessness | 28 | 318 | 704
Food Insecurity | 2 | 349 | 702
Imprisonment or Other Incarceration | 17 | 313 | 660
Marital Estrangement | 53 | 312 | 730
Relative Needing Care | 3 | 320 | 646
Figure 6: Test Set displayed approximately 33%/67% splits for each code. Table
shows data distribution across 5 SDOH codes, with various numbers of gold data
(sentences from medical note containing SDOH code), synthetic data (sentences
generated and validated by Claude in Fig. 5 containing SDOH code), and
negative data (random subset of sentences from medical note not containing
SDOH code). This validation set ultimately contained approximately 33%
positive label (defined as gold + synthetic data) and 67% negative label
(negative data) splits for each SDOH task, with around 1000 total sentences
for each code. Gold data and negative data were sentences pulled from 500
labeled medical notes from the MIMIC-III dataset [9].

The validation set consisted of approximately 33/67% splits of
positive/negative labels, with approximately 1000 sentences for each SDOH
code. Positive labels consisted of both data containing evidence of an SDOH
code from the medical note (gold data) and verified synthetic data (generated
and verified by scheme shown in Figure 5). Negative data consisted of
sentences from medical notes not containing evidence of an SDOH code. This was
the dataset used both to train the router, and was tested against to
ultimately determine the performance seen in Figure 3. Interestingly, we
observed that Claude had the ability to verify synthetic data it generated
itself, resulting in ‘dropped’ data numbers seen in Figure 7.

SDOH Code | Claude Dropped
---|---
Homelessness | 12
Food Insecurity | 51
Imprisonment or Other Incarceration | 7
Marital Estrangement | 8
Relative Needing Care | 0
Figure 7: Verification scheme resulted in Claude dropping synthetic data.
Table for 5 SDOH codes showing numbers of synthetic data originally dropped by
Claude in scheme shown in Fig. 5. Data distributions after “Claude Dropped” is
shown in Fig. 6 and was the dataset used to generate Figs. 3

We used the synthetic data generated by the paradigm introduced in Figure 5 in
order to train an “Oracle Router”. This intelligent router (whose schematic is
shown in Figure 8) takes in a SDOH keyword or phrase, and outputs the open-
source LLM that will perform the best on a given task. The router was trained
on both synthetic data (scheme shown in Figure 5) and labelled sentences from
the MIMIC-III dataset [9], and executes it’s function by choosing the model
that maximizes accuracy for a given SDOH coding task.

Figure 8: Router identifies best model for given SDOH code. Router model
architecture. Oracle router takes in keyword or phrase from given SDOH and
outputs best model for the task. Figure shows color coded arrow from SDOH code
to the model chosen by the router (ie, for food insecurity, Zero-one-
ai/Yi-34B-Chat is considered the best model).

The metrics we used were the following: We evaluated models both based on
accuracy and F1 score. In order to calculate these values, we define true
positive (TP), true negative (TN), false positive (FP), and false negative
(FN) as follows:

  * •

TP: Sentence or note correctly identified as containing evidence of SDOH code

  * •

TN: Sentence or note correctly identified as not containing evidence of SDOH
code

  * •

FP: Sentence or note incorrectly identified as containing evidence of SDOH
code

  * •

FN: Sentence or note incorrectly identified as not containing evidence of SDOH
code

Thus, the formulas for accuracy and F1 score are as follows:

Accuracy
$\displaystyle=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}=\frac{\text{TP}+\text{TN}}{\text{Total}}$
(1) Precision $\displaystyle=\frac{\text{TP}}{\text{TP}+\text{FP}}$ (2) Recall
$\displaystyle=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (3) F1
$\displaystyle=\frac{2\times\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}$
(4)

##  4 Conclusion

We demonstrate the the high performance of our intelligent router, which
utilizes the power of multiple open-source accuracy to perform slightly better
than state-of-the-art models sucuh as GPT-4o. We also introduced a novel
synthetic data generation and validation scheme. The high accuracy we achieved
was without any prompt-tuning or fine-tuning that could potentially boost
performance even more. We also plan to expand this system to account for more
SDOH codes. However, in this paper, we demonstrated an the power of an
architecture utilizing an intelligent routing of inputs to optimal language
models for identifying SDOH codes in unstructured medical notes.

## References

  * Bommasani et al. [2021] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang.  On the Opportunities and Risks of Foundation Models, 2021.  URL https://arxiv.org/abs/2108.07258.  Version Number: 3.
  * Clusmann et al. [2023] Jan Clusmann, Fiona R. Kolbinger, Hannah Sophie Muti, Zunamys I. Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia Löffler, Sophie-Caroline Schwarzkopf, Michaela Unger, Gregory P. Veldhuizen, Sophia J. Wagner, and Jakob Nikolas Kather.  The future landscape of large language models in medicine.  _Commun Med_ , 3(1):1–8, October 2023.  ISSN 2730-664X.  doi: 10.1038/s43856-023-00370-1.  URL https://www.nature.com/articles/s43856-023-00370-1.  Publisher: Nature Publishing Group.
  * Cook et al. [2021] Lily A Cook, Jonathan Sachs, and Nicole G Weiskopf.  The quality of social determinants data in the electronic health record: a systematic review.  _J Am Med Inform Assoc_ , 29(1):187–196, October 2021.  ISSN 1067-5027.  doi: 10.1093/jamia/ocab199.  URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8714289/.
  * Guevara et al. [2024] Marco Guevara, Shan Chen, Spencer Thomas, Tafadzwa L. Chaunzwa, Idalid Franco, Benjamin H. Kann, Shalini Moningi, Jack M. Qian, Madeleine Goldstein, Susan Harper, Hugo J. W. L. Aerts, Paul J. Catalano, Guergana K. Savova, Raymond H. Mak, and Danielle S. Bitterman.  Large language models to identify social determinants of health in electronic health records.  _npj Digit. Med._ , 7(1):1–14, January 2024.  ISSN 2398-6352.  doi: 10.1038/s41746-023-00970-0.  URL https://www.nature.com/articles/s41746-023-00970-0.  Publisher: Nature Publishing Group.
  * Hari and Thomson [2023a] Surya Narayanan Hari and Matt Thomson.  Herd: Using multiple, smaller LLMs to match the performances of proprietary, large LLMs via an intelligent composer, 2023a.  URL https://arxiv.org/abs/2310.19902.  Version Number: 1.
  * Hari and Thomson [2023b] Surya Narayanan Hari and Matt Thomson.  Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models, 2023b.  URL https://arxiv.org/abs/2308.11601.  Version Number: 2.
  * Jiang et al. [2023] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.  Mistral 7B, 2023.  URL https://arxiv.org/abs/2310.06825.  Version Number: 1.
  * Jin et al. [2022] Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng.  Dataless Knowledge Fusion by Merging Weights of Language Models, 2022.  URL https://arxiv.org/abs/2212.09849.  Version Number: 5.
  * Johnson et al. [2015] Alistair Johnson, Tom Pollard, and Roger Mark.  MIMIC-III Clinical Database, 2015.  URL https://physionet.org/content/mimiciii/1.4/.
  * Lybarger et al. [2021] Kevin Lybarger, Mari Ostendorf, and Meliha Yetisgen.  Annotating Social Determinants of Health Using Active Learning, and Characterizing Determinants Using Neural Event Extraction.  _J Biomed Inform_ , 113:103631, January 2021.  ISSN 1532-0464.  doi: 10.1016/j.jbi.2020.103631.  URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7856628/.
  * Mehandru et al. [2024] Nikita Mehandru, Brenda Y. Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J. Butte, and Ahmed Alaa.  Evaluating large language models as agents in the clinic.  _npj Digit. Med._ , 7(1):1–3, April 2024.  ISSN 2398-6352.  doi: 10.1038/s41746-024-01083-y.  URL https://www.nature.com/articles/s41746-024-01083-y.  Publisher: Nature Publishing Group.
  * Omiye et al. [2024] Jesutofunmi A. Omiye, Haiwen Gui, Shawheen J. Rezaei, James Zou, and Roxana Daneshjou.  Large Language Models in Medicine: The Potentials and Pitfalls : A Narrative Review.  _Ann Intern Med_ , 177(2):210–220, February 2024.  ISSN 1539-3704.  doi: 10.7326/M23-2772.
  * Touvron et al. [2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.  LLaMA: Open and Efficient Foundation Language Models, 2023.  URL https://arxiv.org/abs/2302.13971.  Version Number: 1.
  * Waltman [2024] Belinda Waltman.  The Margins Matter.  _JAMA_ , 331(1):23, January 2024.  ISSN 0098-7484.  doi: 10.1001/jama.2023.25361.  URL https://jamanetwork.com/journals/jama/fullarticle/2812973.
  * Wang et al. [2024] Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, and Yang Liu.  OpenChat: Advancing Open-source Language Models with Mixed-Quality Data, March 2024.  URL http://arxiv.org/abs/2309.11235.  arXiv:2309.11235 [cs].


* [33] Matthias Röger and Yoshihiro Tonegawa, _Convergence of phase-field approximations to the Gibbs-Thomson law_ , Calculus of Variations and Partial Differential Equations 32 (2008), no. 1, 111–136. MR 2377408
  * [34] Jacob Rubinstein, Peter Sternberg, and Joseph B. Keller, _Fast reaction, slow diffusion, and curve shortening_ , SIAM Journal on Applied Mathematics 49 (1989), no. 1, 116–133. MR 978829
  * [35] Christian Scharrer, _Some geometric inequalities for varifolds on Riemannian manifolds based on monotonicity identities_ , Ann. Global Anal. Geom. 61 (2022), no. 4, 691–719. MR 4423121
  * [36] Leon Simon, _Lectures on geometric measure theory_ , Proceedings of the Centre for Mathematical Analysis, Australian National University, vol. 3, Australian National University, Centre for Mathematical Analysis, Canberra, 1983\. MR 756417
  * [37] Peter Sternberg and Kevin Zumbrun, _Connectivity of phase boundaries in strictly convex domains_ , Archive for Rational Mechanics and Analysis 141 (1998), no. 4, 375–400. MR 1620498
  * [38] Yoshihiro Tonegawa, _Phase field model with a variable chemical potential_ , Proceedings of the Royal Society of Edinburgh. Section A. Mathematics 132 (2002), no. 4, 993–1019. MR 1926927
  * [39] by same author, _Domain dependent monotonicity formula for a singular perturbation problem_ , Indiana University Mathematics Journal 52 (2003), no. 1, 69–83. MR 1970021
  * [40] by same author, _A diffused interface whose chemical potential lies in a Sobolev space_ , Annali della Scuola Normale Superiore di Pisa. Classe di Scienze. Serie V 4 (2005), no. 3, 487–510. MR 2185866
  * [41] by same author, _On stable critical points for a singular perturbation problem_ , Communications in Analysis and Geometry 13 (2005), no. 2, 439–459. MR 2154826
  * [42] Yoshihiro Tonegawa and Neshan Wickramasekera, _Stable phase interfaces in the van der Waals–Cahn–Hilliard theory_ , Journal für die Reine und Angewandte Mathematik. [Crelle’s Journal] 668 (2012), 191–210. MR 2948876
  * [43] Changwei Xiong, _Comparison of Steklov eigenvalues on a domain and Laplacian eigenvalues on its boundary in Riemannian manifolds_ , J. Funct. Anal. 275 (2018), no. 12, 3245–3258. MR 3864501
  * [44] Jinju Xu, _Gradient estimates for semi-linear elliptic equations with prescribed contact angle problem_ , Journal of Mathematical Analysis and Applications 455 (2017), no. 1, 361–369. MR 3665104
  * [45] Jinju Xu and Dekai Zhang, _Gradient estimates of mean curvature equation with Neumann boundary condition in domains of Riemannian manifold_ , International Journal of Mathematics 28 (2017), no. 8, 1750065, 13. MR 3681126