#  Estimating mean profiles and fluxes in high-speed turbulent boundary layers
using inner/outer-layer transformations

Asif Manzoor Hasan111Ph.D. Candidate, Process and Energy Department Delft
University of Technology, Leeghwaterstraat 39, 2628 CB, Delft, The Netherlands
Johan Larsson222Professor, Department of Mechanical Engineering. Associate
Fellow AIAA. University of Maryland, College Park, MD 20742, USA  Sergio
Pirozzoli333Professor, Dipartimento di Ingegneria Meccanica e Aerospaziale
Sapienza Università di Roma, Via Eudossiana 18, 00184 Roma, Italy  Rene
Pecnik444Professor, Process and Energy Department Delft University of
Technology, Leeghwaterstraat 39, 2628 CB, Delft, The Netherlands

## Nomenclature

$\tau_{w}$ = | wall shear stress |  $Re_{\tau}$ = |  $\rho_{w}u_{\tau}\delta/\mu_{w}$, friction Reynolds number
---|---|---|---
$\rho$ = | density |  $Re_{\tau}^{*}$ = |  $\bar{\rho}u_{\tau}^{*}\delta/\bar{\mu}$, semi-local friction Reynolds number
$\mu$ = | dynamic viscosity |  $M_{\tau}$ = |  $u_{\tau}/\sqrt{\gamma RT_{w}}$, friction Mach number
$k$ = | thermal conductivity |  $M_{\infty}$ = |  $u_{\infty}/\sqrt{\gamma RT_{\infty}}$, free-stream Mach number
$c_{p}$ = | specific heat capacity at constant pressure |  $r$ = | recovery factor
$\gamma$ = | specific heat capacity ratio |  $c_{f}$ = |  $2\tau_{w}/(\rho_{\infty}u_{\infty}^{2})$, skin-friction coefficient
$T$ = | temperature |  $c_{h}$ = |  $q_{w}/(c_{p}\rho_{\infty}u_{\infty}(T_{w}-T_{r}))$, heat-transfer coefficient
$R$ = | specific gas constant |  $q_{w}$ = | wall heat flux
$u_{\tau}$ = |  $\sqrt{\tau_{w}/\rho_{w}}$, friction velocity |  $y$ = | wall-normal coordinate
$u_{\tau}^{*}$ = |  $\sqrt{\tau_{w}/\bar{\rho}}$, semi-local friction velocity |  $y^{*}$ = |  $y/\delta_{v}^{*}$, semi-local wall-normal coordinate
$\delta_{v}$ = |  $\mu_{w}/(\rho_{w}u_{\tau})$, viscous length scale |  $Pr$ = |  $\mu c_{p}/k$, Prandtl number
$\delta_{v}^{*}$ = |  $\bar{\mu}/(\bar{\rho}u_{\tau}^{*})$, semi-local viscous length scale |  $\kappa$ = | von Kármán constant
$\delta$ = | (or $\delta_{99}$), boundary layer thickness |  $\Pi$ = | Coles’ wake parameter
$\theta$ = | momentum thickness |  $C$ = | log-law intercept
$\delta^{*}$ = | displacement thickness |  $\mu_{t}$ = | eddy viscosity
$U$ = | transformed (incompressible) velocity |  |
$u$ = | untransformed velocity |  |
$Re_{\theta}$ = |  $\rho_{\infty}u_{\infty}\theta/\mu_{\infty}$, momentum thickness Reynolds number |  |
$Re_{\delta^{*}}$ = |  $\rho_{\infty}u_{\infty}\delta^{*}/\mu_{\infty}$, displacement thickness Reynolds number |  |
$Re_{\delta_{2}}$ = | $\rho_{\infty}u_{\infty}\theta/\mu_{w}$ |  |
Subscripts |  |
$w$ = | wall |  $\infty$ = | free-stream
$e$ = | boundary layer edge ($y=\delta$) |  $r$ = | recovery
Superscripts |  |
$+$ = | wall-scaled |  $\overline{(\cdot)}$ = | Reynolds averaging

##  1 Introduction

Accurately predicting drag and heat transfer for compressible high-speed flows
is of utmost importance for a range of engineering applications. This requires
the precise knowledge of the entire velocity and temperature profiles. A
common approach is to use compressible velocity scaling laws (transformation),
that inverse transform the velocity profile of an incompressible flow,
together with a temperature-velocity relation. Current methods [1, 2]
typically assume a single velocity scaling law, neglecting the different
scaling characteristics of the inner and outer layers. In this Note, we use
distinct velocity transformations for these two regions. In the inner layer,
we utilize a recently proposed scaling law that appropriately incorporates
variable property and intrinsic compressibility effects [3], while the outer
layer profile is inverse-transformed with the well-known Van Driest
transformation [4]. The result is an analytical expression for the mean shear
valid in the entire boundary layer, which combined with the temperature-
velocity relationship in Zhang et al. [5], provides predictions of mean
velocity and temperature profiles at unprecedented accuracy. Using these
profiles, drag and heat transfer is evaluated with an accuracy of +/-4% and
+/-8%, respectively, for a wide range of compressible turbulent boundary
layers up to Mach numbers of 14.

##  2 Proposed method

An incompressible velocity profile is composed of two parts: (1) the law of
the wall in the inner layer, and (2) the velocity defect law in the outer
layer. We can model the law of the wall either by composite velocity profiles
[6, 7, 8], or by integrating the mean momentum equation using a suitable eddy
viscosity model [9, 10]. Here, we follow the latter approach and utilize the
Johnson-King [10] eddy viscosity model. Likewise, there are several
formulations available to represent the defect law [11, 12, 13], of which we
use Coles’ law of the wake [11].

Once the reference incompressible velocity profile is obtained, we inverse
transform it using our recently proposed velocity transformation [3] for the
inner layer, and the Van Driest (VD) transformation [4] for the outer layer.
They are combined as follows:

$d\bar{u}^{+}=f_{3}^{-1}f_{2}^{-1}f_{1}^{-1}d\bar{U}^{+}_{inner}+f_{1}^{-1}d\bar{U}^{+}_{wake},$
(1)

where the factors $f_{1}$, $f_{2}$, and $f_{3}$ constitute the transformation
kernel proposed in Hasan et al. [3] that accounts for both variable property
and intrinsic compressibility effects, given as

$\frac{d\bar{U}^{+}}{{d\bar{u}^{+}}}=\underbrace{\left({\frac{1+\kappa
y^{*}{D}(M_{\tau})}{1+\kappa{y^{*}}{D(0)}}}\right)}_{f_{3}}\underbrace{{\left({1-\frac{y}{\delta_{v}^{*}}\frac{d\delta_{v}^{*}}{dy}}\right)}}_{f_{2}}\underbrace{\sqrt{\frac{\bar{\rho}}{\rho_{w}}}}_{f_{1}}\,,$
(2)

where

$D(M_{\tau})=\left[1-\mathrm{exp}\left({\frac{-y^{*}}{A^{+}+f(M_{\tau})}}\right)\right]^{2}.$
(3)

The value of $A^{+}$ differs based on the choice of the von Kármán constant
$\kappa$, such that the log-law intercept is reproduced for that $\kappa$ [8].
With $\kappa=0.41$, the value of $A^{+}=17$ gives a log-law intercept of 5.2
[14], whereas, with $\kappa=0.384$, $A^{+}=15.22$ gives a log-law intercept of
4.17. The additive term $f(M_{\tau})$ accounts for intrinsic compressibility
effects. Hasan et al. [3] proposed $f(M_{\tau})=19.3M_{\tau}$, that is
independent of the chosen value of $\kappa$. In Eq. (1),
$d\bar{U}^{+}_{inner}$ is modeled using the Johnson-King eddy viscosity model
as ${dy^{*}}/[{1+\kappa y^{*}D(0)}]$, which after integration, recovers the
incompressible law of the wall, and $d\bar{U}^{+}_{wake}=\Pi/\kappa\sin(\pi
y/\delta)\,\pi\,d(y/\delta)$ is the derivative of the Coles’ wake function
[7].

Inserting the expressions for $d\bar{U}^{+}_{inner}$, $d\bar{U}^{+}_{wake}$ in
Eq. (1), using $dy^{*}/dy=f_{2}/\delta_{v}^{*}$,
$u_{\tau}^{*}=u_{\tau}f_{1}^{-1}$, and upon rearrangement, we get the
dimensional form of the mean velocity gradient as

$\frac{d\bar{u}}{dy}=\frac{u_{\tau}^{*}}{\delta_{v}^{*}}\frac{1}{1+\kappa
y^{*}D(M_{\tau})}+\frac{u_{\tau}^{*}}{\delta}\,\frac{\Pi}{\kappa}\,\pi\,\sin\left(\pi\frac{y}{\delta}\right).$
(4)

Eq. (4) provides several useful insights. Analogous to an incompressible flow,
the mean velocity in a compressible flow is controlled by two distinct length
scales, $\delta_{v}^{*}$ and $\delta$, characteristic of the inner and outer
layers, respectively. The two layers are connected by a common velocity scale
$u_{\tau}^{*}$ (the semi-local friction velocity), leading to a logarithmic
law in the overlap region between them. Moreover, in the overlap layer, the
denominator of the first term on the right-hand side reduces to $\kappa y$,
consistent with Townsend’s attached-eddy hypothesis. The second term on the
right-hand-side is the wake term accounting for mean density variations, where
Coles’ wake parameter $\Pi$ depends on the Reynolds number, as discussed in
the subsection below.

###  2.1 Characterizing low-Reynolds-number effects on the wake parameter

Figure 1: The wake parameter $\Pi$ computed using the DNS data and plotted as
a function of (a) $Re_{\delta_{2}}$ (b) $Re_{\theta}$ (c) $Re_{\delta^{*}}$
and (d) $Re^{*}_{\tau_{\infty}}$ for 19 incompressible [15, 16, 17, 18] and 26
compressible [19, 20, 21, 22, A. Ceci, private communication] turbulent
boundary layers.

For incompressible boundary layers, Coles’ wake parameter is known to strongly
depend on $Re_{\theta}$ at low Reynolds numbers [23, 13, 24]. For compressible
boundary layers, the ambiguity of the optimal Reynolds number definition poses
a challenge to characterize the wake parameter. Fernholz and Finley [25],
mainly using experimental data at that time, observed that the momentum-
thickness Reynolds number with viscosity at the wall ($Re_{\delta_{2}}$) is
the suitable definition to scale $\Pi$. However, intuitively, $\Pi$ should
scale with Reynolds number based on the free-stream properties [26, 24]. Given
the recent availability of Direct Numerical Simulation (DNS) data at moderate
Reynolds numbers for both compressible and incompressible flows, we revisit
the question of which Reynolds number best describes the wake parameter.

First, we evaluate $\Pi$ for several incompressible and compressible DNS cases
from the literature and then report it as a function of different definitions
of the Reynolds number, searching for the definition yielding the least spread
of the data points. For incompressible flows, the wake strength can be
determined as
${\Pi}=0.5{\kappa}\left(\bar{U}^{+}(y=\delta)-{1}/{\kappa}\ln(\delta^{+})-C\right)$,
where $C$ is the log-law intercept for the chosen $\kappa$. For compressible
flows, the wake strength is based on the VD transformed velocity [25, 26] as
${\Pi}=0.5\kappa\left(\bar{U}_{vd}^{+}(y=\delta)-(\bar{U}_{vd}^{+})^{log}(y=\delta)\right)$,
where $\bar{U}_{vd}^{+}$ is obtained from the DNS data. The reference log law
$(\bar{U}_{vd}^{+})^{log}$, unlike for incompressible flows, cannot be
computed as ${1}/{\kappa}\ln(y^{+})+C_{vd}$, because $C_{vd}$ is found to be
non-universal for diabatic compressible boundary layers [27, 28]. Hence,
$(\bar{U}_{vd}^{+})^{log}$ can be obtained either by fitting a logarithmic
curve to $\bar{U}_{vd}^{+}$ [25], or by inverse transforming the
incompressible law of the wall. Here, we follow the latter approach by using
the compressibility transformation of Hasan et al. [3].

The value of the von Kármán constant $\kappa$ plays a crucial role in
estimating $\Pi$. Spalart [29] noted that a strong consensus on $\kappa$ is
needed to accurately estimate $\Pi$. However, such a consensus is yet missing
[30]. Recently, Nagib and Chauhan [8] showed that $\kappa=0.384$ is a suitable
choice for incompressible boundary layers, verified to be true also for
channels [31] and pipes [32]. However, due to historical reasons and wide
acceptance of $\kappa=0.41$, we will proceed with this value. The same
procedure can straightforwardly be repeated with a different value of
$\kappa$.

Figure 1 shows the wake parameter for twenty-six compressible and nineteen
incompressible boundary layer flows, as a function of $Re_{\delta_{2}}$,
$Re_{\theta}$, $Re_{\delta^{*}}$ and $Re_{\tau_{\infty}}^{*}$. The spread in
the data points is found to be quite large for all the definitions, as $\Pi$
is the difference of two relatively large quantities, namely
$\bar{U}_{vd}^{+}$ and $(\bar{U}_{vd}^{+})^{log}$ at the boundary layer edge,
as outlined above. Note that even incompressible boundary layers are not
devoid of this scatter [29, 13]. Figure 1(a) shows the presence of two
distinct branches, hence $Re_{\delta_{2}}$ does not seem to be suitable to
characterize $\Pi$, unlike reported in previous literature [25, 1]. Among the
four definitions of Reynolds number, $Re_{\theta}$ seems to show the least
spread. Figure 1(b) also reports several functional forms of
$\Pi=f(Re_{\theta})$. Use of the modified Kármán-Schoenherr friction formula
[33] for indirect evaluation of $\Pi$, does not show saturation at high
Reynolds numbers as observed in Coles [23] for incompressible flows. The
Cebeci-Smith relation [24] underpredicts $\Pi$, but reproduces saturation at
high Reynolds numbers. We thus propose a relation similar to that proposed by
[24], with modified constants to achieve a better fit with data from recent
incompressible DNS [17, 18]. The relation is

${\Pi}=0.69\,\left[1-\exp(-0.243\sqrt{z}-{0.15}\,z)\right],\quad\mathrm{where}\quad
z=Re_{\theta}/425-1.$ (5)

Inset in Fig. 1(b) compares the skin-friction curve computed using Eq. (5)
with the modified Kármán-Schoenherr skin-friction formula [33]. The distance
between the two curves is large at low Reynolds numbers, but less so at higher
Reynolds numbers. As expected, the incompressible DNS data of Jiménez et al.
[17], Sillero et al. [18], follow the friction curve computed using Eq. (5).

###  2.2 Implementation of the proposed method

For convenience, Eq. (4) can also be expressed in terms of the dimensional
variables $\tau_{w}$, $\bar{\mu}$ and $\bar{\rho}$ as,

$\frac{d\bar{u}}{dy}=\frac{\tau_{w}}{\bar{\mu}+\underbrace{\sqrt{\tau_{w}\bar{\rho}}\kappa
yD(M_{\tau})}_{\mu_{t}}}+\frac{\sqrt{\tau_{w}/\bar{\rho}}}{\delta}\,\frac{\Pi}{\kappa}\,\pi\,\sin\left(\pi\frac{y}{\delta}\right),$
(6)

where $\mu_{t}$ is the Johnson-King eddy viscosity model corrected for
intrinsic compressibility effects, derived from Hasan et al. [3]
transformation. It can be readily used in turbulence modeling, for instance,
as a wall-model in Large Eddy Simulations. Note that different eddy viscosity
models can be used in Eq. (6), for example, Prandtl’s mixing length model (see
Appendix A). Eq. (6) covers the entire boundary layer, and it can be
integrated in conjunction with a suitable temperature model such as the one
proposed by Zhang et al. [5], which is given as

$\frac{\bar{T}}{T_{w}}=1+\frac{T_{r}-T_{w}}{T_{w}}\left[(1-s\,{Pr})\left(\frac{\bar{u}}{u_{\infty}}\right)^{2}+s\,{Pr}\left(\frac{\bar{u}}{u_{\infty}}\right)\right]+\frac{T_{\infty}-T_{r}}{T_{w}}\left(\frac{\bar{u}}{u_{\infty}}\right)^{2},$
(7)

where $s\,{Pr}=0.8$, $T_{r}/T_{\infty}=1+0.5r(\gamma-1)M_{\infty}^{2}$, and
$r=Pr^{1/3}$. Moreover, a suitable viscosity law (e.g., power or Sutherland’s
law), and the ideal gas equation of state $\bar{\rho}/\rho_{w}=T_{w}/\bar{T}$
have to be used to compute mean viscosity and density profiles, respectively.
The inputs that need to be provided are the Reynolds number ($Re_{\theta}$),
free-stream Mach number ($M_{\infty}$), wall cooling/heating parameter
($T_{w}/T_{r}$) and (optionally) the dimensional wall or free-stream
temperature for Sutherland’s law. It is important to note that Eq. (7), and
all solver inputs are based on the quantities in the free-stream, and not at
the boundary layer edge. For more insights on the solver, please refer to the
source code available on GitHub [34].

##  3 Results

Figure 2 shows the predicted velocity and temperature profiles for a selection
of high Mach number cases. As can be seen, the DNS and the predicted profiles
are in good agreement, thus corroborating our methodology. The insets in
Figure 2 show the error in the predicted skin-friction and heat-transfer
coefficients for thirty compressible cases from the literature. For most
cases, the friction coefficient ($c_{f}$) is predicted with $+/-4\%$ accuracy,
with a maximum error of -5.3%. The prediction of the heat-transfer coefficient
($c_{h}$) shows a slightly larger error compared to $c_{f}$, potentially due
to additional inaccuracies arising from the temperature-velocity relation. In
most cases, $c_{h}$ is predicted with $+/-8\%$ accuracy, with a maximum error
of 10.3%.

The proposed method is modular in that it can also be applied using other
inner-layer transformations [35, 36, 28] with minor modifications as discussed
in Appendix B. This is shown in Figure 3, which compares the proposed approach
with another modular approach of Kumar and Larsson [2], both with different
inner layer transformations. Additionally, the figure includes results
obtained with the method of Huang et al. [1] using the VD transformation, and
the widely recognized Van Driest II skin-friction formula [37]. Figure 3 also
shows the root-mean-square error, determined as
$\mathrm{RMS}=\sqrt{{1}/{N}\sum\varepsilon_{c_{f}}^{2}},$ where $N$ is the
total number of DNS cases considered. The Van Driest II formula and the method
of Huang et al. have similar RMS error of about 6% 555Huang et al.’s method
with the more accurate temperature velocity relation in Zhang et al. [5] leads
to an RMS error of 12%., which is not surprising as both of them are built on
Van Driest’s mixing-length arguments. The errors are selectively positive for
majority of the cases, and it increases with higher Mach number and stronger
wall cooling. The source of this error mainly resides in the inaccuracy of the
VD velocity transformation in the near-wall region for diabatic flows. To
eliminate this shortcoming, Kumar and Larsson [2] developed a modular
methodology, which is quite accurate when the transformation of Volpiani et
al. [36] is used, but it is less accurate if other velocity transformations
are implemented. This inaccuracy is because the outer layer velocity profile
is also inverse-transformed according to the inner-layer transformation. In
the current approach, the velocity profile is instead inverse-transformed
using two distinct transformations, which take into account the different
scaling properties of the inner and outer layers, thus reducing the RMS error
with respect to Kumar and Larsson’s modular method for all the transformations
tested herein. The error using the proposed approach with the TL
transformation is preferentially positive for all the cases. This is due to
the log-law shift observed in the TL scaling, which is effectively removed in
the Hasan et al. transformation, thereby yielding an RMS error of 2.66%, which
is the lowest among all approaches.

Figure 2: Predicted velocity (top) and temperature (bottom) profiles (dashed
lines) compared to DNS (black solid lines) for the cases with the highest
reported Mach numbers in the respective publications: (left to right)
$M_{\infty}=13.64$, $T_{w}/T_{r}=0.18$ [19]; $M_{\infty}=4$, $T_{w}/T_{r}=1$
[20]; $M_{\infty}=7.87$, $T_{w}/T_{r}=0.48$ (A. Ceci, private communication);
$M_{\infty}=5.84$, $T_{w}/T_{r}=0.25$ [22]; $M_{\infty}=5.86$,
$T_{w}/T_{r}=0.76$ [21]. (Insets): Percent error in skin-friction (top) and
heat-transfer (bottom) prediction for 30 compressible turbulent boundary
layers from the literature [20, 19, 22, 21, 38, A. Ceci, private
communication]. The error is computed as
$\varepsilon_{c_{f}}=(c_{f}-c_{f}^{DNS})/c_{f}^{DNS}\,\times 100$ and likewise
for $\varepsilon_{c_{h}}$. Symbols are as in Figure 1.  Figure 3: Error in
skin-friction prediction using the proposed approach compared to different
state-of-the-art approaches. The letters on the X-axis denote the velocity
transformation used for that approach. HLPP, GFM, VIPL, TL, and VD stand for
the transformations proposed in Hasan et al. [3], Griffin et al. [35],
Volpiani et al. [36], Trettel and Larsson [28], and Van Driest [4],
respectively. The numbers are RMS values computed as outlined in the text.
Symbols are as in Figure 1. The shaded region shows an error bar of +/-5%.
Note that inputs for all the methods are based on properties in the free-
stream instead of at the edge of the boundary layer.

##  4 Conclusions

We have derived an expression for the mean velocity gradient in high-speed
boundary layers [Eq. (6)] that combines the inner-layer transformation
recently proposed by Hasan et al. [3] and the Van Driest [4] outer-layer
transformation, thus covering the entire boundary layer. The Coles’ wake
parameter in this expression is determined using an adjusted Cebeci and Smith
relation [Eq. (5)] with the definition of $Re_{\theta}$ as the most suitable
parameter to characterize low-Reynolds-number effects on $\Pi$. This method
allows remarkably accurate predictions of the mean velocity and temperature
profiles, leading to estimation of the friction and heat-transfer coefficients
which are within $+/-4\%$ and $+/-8\%$ error bounds with respect to DNS data,
respectively.

The skin-friction results are compared with that from other state-of-the-art
approaches considered in literature. Limited accuracy of the VD II and Huang
et al.’s methods is attributable to inaccuracy of the underlying VD
transformation in the near-wall region of diabatic boundary layers, whereas
inaccuracy in Kumar and Larsson’s approach is attributable to the
unsuitability of the inner layer velocity transformations in the outer layer.
By combining different scaling laws in the inner layer with the Van Driest
transformation in the outer layer, our method demonstrates improved results
for all the inner-layer transformations herein tested, with the lowest RMS
error of 2.66% achieved with Hasan et al.’s transformation.

The methodology developed in this note promises straightforward application to
other classes of wall-bounded flows like channels and pipes, upon change of
the temperature-velocity relation [e.g. 39], and using different values of the
wake parameter $\Pi$ [8]. Also, the method is modular in the sense that it can
be used with other temperature models and equations of state.

## Acknowledgements

We thank Dr. P. Costa for the insightful discussions. This work was supported
by the European Research Council grant no. ERC-2019-CoG-864660, Critical; and
the Air Force Office of Scientific Research under grants FA9550-19-1-0210 and
FA9550-19-1-7029.

## Appendix A: Mean shear using Prandtl’s mixing length model

The choice of the eddy viscosity model affects the first term on the right
hand side of Eq. (6). By analogy, the mean shear equation using Prandtl’s
mixing length model is thus as follows,

$\frac{d\bar{u}}{dy}=\frac{2\,\tau_{w}}{\bar{\mu}+\sqrt{\bar{\mu}^{2}+[2\sqrt{\tau_{w}\bar{\rho}}\kappa
yD(M_{\tau})]^{2}}}+\frac{\sqrt{\tau_{w}/\bar{\rho}}}{\delta}\,\frac{\Pi}{\kappa}\,\pi\,\sin\left(\pi\frac{y}{\delta}\right),$
(8)

where $D(M_{\tau})$ is the damping function corrected for intrinsic
compressibility effects as

$D(M_{\tau})=1-\mathrm{exp}\left({\frac{-y^{*}}{A^{+}+39M_{\tau}}}\right),$
(9)

with $A^{+}=25.53$ (or $26$) for $\kappa=0.41$, and where the additive term
$39M_{\tau}$ is obtained following similar steps as for the Johnson-King model
(see Ref. [3]).

## Appendix B: Implementation of the method using velocity transformations in
Ref. [35, 36]

In the logarithmic region and beyond, the first term on the right-hand side of
Eq. (6) reduces to $\sqrt{\tau_{w}/\bar{\rho}}/(\kappa y)$, which is the same
as Van Driest’s original arguments [4]. It is crucial to satisfy this
condition, otherwise the logarithmic profile extending to the outer layer
would not obey Van Driest’s scaling. The transformations of Griffin et al.
[35], Volpiani et al. [36] fail to satisfy this property. To address this
issue, we enforce Van Driest’s scaling in the outer layer by modifying Eq. (1)
as follows

$\begin{split}y^{+}_{T}\leq
50:\quad&d\bar{u}^{+}=\mathcal{T}^{-1}d\bar{U}^{+}_{inner}+f_{1}^{-1}d\bar{U}^{+}_{wake},\\\
y^{+}_{T}>50:\quad&d\bar{u}^{+}=f_{1}^{-1}d\bar{U}^{+}_{inner}+f_{1}^{-1}d\bar{U}^{+}_{wake},\end{split}$
(10)

where $\mathcal{T}$ denotes the inner-layer transformation kernel and
$y^{+}_{T}$ is the transformed coordinate. The value of $50$ is taken
arbitrarily as a start of the logarithmic region.

## References

  * Huang et al. [1993] Huang, P., Bradshaw, P., and Coakley, T., “Skin friction and velocity profile family for compressible turbulentboundary layers,” _AIAA journal_ , Vol. 31, No. 9, 1993, pp. 1600–1604.
  * Kumar and Larsson [2022] Kumar, V., and Larsson, J., “Modular Method for Estimation of Velocity and Temperature Profiles in High-Speed Boundary Layers,” _AIAA Journal_ , Vol. 60, No. 9, 2022, pp. 5165–5172.
  * Hasan et al. [2023] Hasan, A. M., Larsson, J., Pirozzoli, S., and Pecnik, R., “Incorporating intrinsic compressibility effects in velocity transformations for wall-bounded turbulent flows,” , 2023.
  * Van Driest [1951] Van Driest, E. R., “Turbulent boundary layer in compressible fluids,” _Journal of the Aeronautical Sciences_ , Vol. 18, No. 3, 1951, pp. 145–160.
  * Zhang et al. [2014] Zhang, Y.-S., Bi, W.-T., Hussain, F., and She, Z.-S., “A generalized Reynolds analogy for compressible wall-bounded turbulent flows,” _Journal of Fluid Mechanics_ , Vol. 739, 2014, pp. 392–420.
  * Musker [1979] Musker, A., “Explicit expression for the smooth wall velocity distribution in a turbulent boundary layer,” _AIAA Journal_ , Vol. 17, No. 6, 1979, pp. 655–657.
  * Chauhan et al. [2007] Chauhan, K., Nagib, H., and Monkewitz, P., “On the composite logarithmic profile in zero pressure gradient turbulent boundary layers,” _45th AIAA Aerospace Sciences Meeting and Exhibit_ , 2007, p. 532.
  * Nagib and Chauhan [2008] Nagib, H. M., and Chauhan, K. A., “Variations of von Kármán coefficient in canonical flows,” _Physics of fluids_ , Vol. 20, No. 10, 2008, p. 101518.
  * Van Driest [1956a] Van Driest, E. R., “On turbulent flow near a wall,” _Journal of the aeronautical sciences_ , Vol. 23, No. 11, 1956a, pp. 1007–1011.
  * Johnson and King [1985] Johnson, D. A., and King, L., “A mathematically simple turbulence closure model for attached and separated turbulent boundary layers,” _AIAA journal_ , Vol. 23, No. 11, 1985, pp. 1684–1692.
  * Coles [1956] Coles, D., “The law of the wake in the turbulent boundary layer,” _Journal of Fluid Mechanics_ , Vol. 1, No. 2, 1956, pp. 191–226.
  * Zagarola and Smits [1998] Zagarola, M. V., and Smits, A. J., “A new mean velocity scaling for turbulent boundary layers,” _Proceedings of FEDSM_ , Vol. 98, 1998, pp. 21–25.
  * Fernholz and Finleyt [1996] Fernholz, H., and Finleyt, P., “The incompressible zero-pressure-gradient turbulent boundary layer: an assessment of the data,” _Progress in Aerospace Sciences_ , Vol. 32, No. 4, 1996, pp. 245–311.
  * Iyer and Malik [2019] Iyer, P. S., and Malik, M. R., “Analysis of the equilibrium wall model for high-speed turbulent flows,” _Physical Review Fluids_ , Vol. 4, No. 7, 2019, p. 074604.
  * Schlatter et al. [2009] Schlatter, P., Örlü, R., Li, Q., Brethouwer, G., Fransson, J. H., Johansson, A. V., Alfredsson, P. H., and Henningson, D. S., “Turbulent boundary layers up to Re $\theta$= 2500 studied through simulation and experiment,” _Physics of fluids_ , Vol. 21, No. 5, 2009, p. 051702.
  * Schlatter and Örlü [2010] Schlatter, P., and Örlü, R., “Assessment of direct numerical simulation data of turbulent boundary layers,” _Journal of Fluid Mechanics_ , Vol. 659, 2010, pp. 116–126.
  * Jiménez et al. [2010] Jiménez, J., Hoyas, S., Simens, M. P., and Mizuno, Y., “Turbulent boundary layers and channels at moderate Reynolds numbers,” _Journal of Fluid Mechanics_ , Vol. 657, 2010, pp. 335–360.
  * Sillero et al. [2013] Sillero, J. A., Jiménez, J., and Moser, R. D., “One-point statistics for turbulent wall-bounded flows at Reynolds numbers up to $\delta^{+}=$ 2000,” _Physics of Fluids_ , Vol. 25, No. 10, 2013, p. 105102\.
  * Zhang et al. [2018] Zhang, C., Duan, L., and Choudhari, M. M., “Direct numerical simulation database for supersonic and hypersonic turbulent boundary layers,” _AIAA Journal_ , Vol. 56, No. 11, 2018, pp. 4297–4311.
  * Bernardini and Pirozzoli [2011] Bernardini, M., and Pirozzoli, S., “Wall pressure fluctuations beneath supersonic turbulent boundary layers,” _Physics of Fluids_ , Vol. 23, No. 8, 2011, p. 085102.
  * Cogo et al. [2022] Cogo, M., Salvadore, F., Picano, F., and Bernardini, M., “Direct numerical simulation of supersonic and hypersonic turbulent boundary layers at moderate-high Reynolds numbers and isothermal wall condition,” _Journal of Fluid Mechanics_ , Vol. 945, 2022, p. A30.
  * Ceci et al. [2022] Ceci, A., Palumbo, A., Larsson, J., and Pirozzoli, S., “Numerical tripping of high-speed turbulent boundary layers,” _Theoretical and Computational Fluid Dynamics_ , Vol. 36, No. 6, 2022, pp. 865–886.
  * Coles [1962] Coles, D., “The turbulent boundary layer in a compressible fluid. RAND Corp., Rep,” Tech. rep., R-403-PR, 1962.
  * Cebeci and Smith [1974] Cebeci, T., and Smith, A. M. O., _Analysis of turbulent boundary layers_ , Elsevier, 1974.
  * Fernholz and Finley [1980] Fernholz, H.-H., and Finley, P., “A critical commentary on mean flow data for two-dimensional compressible turbulent boundary layers,” Tech. rep., AGARD-AG-253, 1980.
  * Smits and Dussauge [2006] Smits, A. J., and Dussauge, J.-P., _Turbulent shear layers in supersonic flow_ , Springer Science & Business Media, 2006.
  * Bradshaw [1977] Bradshaw, P., “Compressible turbulent shear layers,” _Annual Review of Fluid Mechanics_ , Vol. 9, No. 1, 1977, pp. 33–52.
  * Trettel and Larsson [2016] Trettel, A., and Larsson, J., “Mean velocity scaling for compressible wall turbulence with heat transfer,” _Physics of Fluids_ , Vol. 28, No. 2, 2016, p. 026102.
  * Spalart [1988] Spalart, P. R., “Direct simulation of a turbulent boundary layer up to R$\theta$= 1410,” _Journal of fluid mechanics_ , Vol. 187, 1988, pp. 61–98.
  * Monkewitz and Nagib [2023] Monkewitz, P. A., and Nagib, H. M., “The hunt for the Kármán "constant” revisited,” , 2023.
  * Lee and Moser [2015] Lee, M., and Moser, R. D., “Direct numerical simulation of turbulent channel flow up to,” _Journal of fluid mechanics_ , Vol. 774, 2015, pp. 395–415.
  * Pirozzoli et al. [2021] Pirozzoli, S., Romero, J., Fatica, M., Verzicco, R., and Orlandi, P., “One-point statistics for turbulent pipe flow up to,” _Journal of fluid mechanics_ , Vol. 926, 2021.
  * Nagib et al. [2007] Nagib, H. M., Chauhan, K. A., and Monkewitz, P. A., “Approach to an asymptotic state for zero pressure gradient turbulent boundary layers,” _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ , Vol. 365, No. 1852, 2007, pp. 755–770.
  * Pecnik and Hasan [2023] Pecnik, R., and Hasan, A. M., “A jupyter notebook to estimate mean profiles and fluxes for high-speed boundary layers,” , 2023.  URL https://github.com/Fluid-Dynamics-Of-Energy-Systems-Team/DragandHeatTransferEstimation.git.
  * Griffin et al. [2021] Griffin, K. P., Fu, L., and Moin, P., “Velocity transformation for compressible wall-bounded turbulent flows with and without heat transfer,” _Proceedings of the National Academy of Sciences_ , Vol. 118, No. 34, 2021, p. e2111144118.
  * Volpiani et al. [2020] Volpiani, P. S., Iyer, P. S., Pirozzoli, S., and Larsson, J., “Data-driven compressibility transformation for turbulent wall layers,” _Physical Review Fluids_ , Vol. 5, No. 5, 2020, p. 052602.
  * Van Driest [1956b] Van Driest, E. R., _The problem of aerodynamic heating_ , Institute of the Aeronautical Sciences, 1956b.
  * Huang et al. [2020] Huang, J., Nicholson, G. L., Duan, L., Choudhari, M. M., and Bowersox, R. D., “Simulation and modeling of cold-wall hypersonic turbulent boundary layers on flat plate,” _AIAA Scitech 2020 Forum_ , 2020, p. 0571.
  * Song et al. [2022] Song, Y., Zhang, P., Liu, Y., and Xia, Z., “Central mean temperature scaling in compressible turbulent channel flows with symmetric isothermal boundaries,” _Physical Review Fluids_ , Vol. 7, No. 4, 2022, p. 044606.


#  Is It Really Long Context if All You Need Is Retrieval?
Towards Genuinely Difficult Long Context NLP

Omer Goldman, Alon Jacovi11footnotemark: 1, Aviv Slobodkin11footnotemark: 1,
Aviya Maimon11footnotemark: 1, Ido Dagan, Reut Tsarfaty
Bar-Ilan University
<EMAIL_ADDRESS>Equal contribution

###### Abstract

Improvements in language models’ capabilities have pushed their applications
towards longer contexts, making long-context evaluation and development an
active research area. However, many disparate use-cases are grouped together
under the umbrella term of “long-context”, defined simply by the total length
of the model’s input, including – for example – Needle-in-a-Haystack tasks,
book summarization, and information aggregation. Given their varied
difficulty, in this position paper we argue that conflating different tasks by
their context length is unproductive. As a community, we require a more
precise vocabulary to understand what makes long-context tasks similar or
different. We propose to unpack the taxonomy of long-context based on the
properties that make them more difficult with longer contexts. We propose two
orthogonal axes of difficulty: (I) Diffusion: How hard is it to find the
necessary information in the context? (II) Scope: How much necessary
information is there to find? We survey the literature on long-context,
provide justification for this taxonomy as an informative descriptor, and
situate the literature with respect to it. We conclude that the most difficult
and interesting settings, whose necessary information is very long and highly
diffused within the input, is severely under-explored. By using a descriptive
vocabulary and discussing the relevant properties of difficulty in long-
context, we can implement more informed research in this area. We call for a
careful design of tasks and benchmarks with distinctly long context, taking
into account the characteristics that make it qualitatively different from
shorter context.

Is It Really Long Context if All You Need Is Retrieval?
Towards Genuinely Difficult Long Context NLP

Omer Goldman††thanks: Equal contribution, Alon Jacovi11footnotemark: 1, Aviv
Slobodkin11footnotemark: 1, Aviya Maimon11footnotemark: 1, Ido Dagan, Reut
Tsarfaty Bar-Ilan University<EMAIL_ADDRESS>

##  1 Introduction

Figure 1: A taxonomy of long context tasks based on the distribution of the
needed information in the text. Tasks with larger scope and higher diffusion
are more difficult and more indicative of LLMs long context capabilities.

The ability to deal with ever-longer contexts has been one of the most notable
trends among the emerging capabilities of large language models (LLMs).
Starting with a few hundred tokens as the maximal input length of the first
attention-based LLMs Devlin et al. (2019); Raffel et al. (2020), contemporary
models are – technically – able to process up to 128k and even 1M tokens
(Gemini Team Google, 2024; OpenAI, 2024).

The demand to evaluate LLMs in this setting has led to a line of research on
designing long-context tasks and benchmarks, in order to systematically
understand models’ capabilities and drive their development. However, the
field has generally a sole recurring descriptor to define such measurements
by—simply, the length of the context. For example, long-context benchmarks
group tasks mostly by length in words (e.g., Shaham et al., 2022; Bai et al.,
2023; Zhang et al., 2024). This leads to qualitatively different measurements
being conflated together, with conclusions about long-context capabilities
being extended from one class of tasks to others. The community is, of course,
aware that, for example, tasks which require a small part of the input are
different from tasks that require a large part of it. But we ask the more
general question: What are the properties that differentiate tasks when
conditioned on their context length? What can we accomplish with such a
distinction?

In this position paper, we claim that the current landscape of works on long-
context evaluation will greatly benefit from a more fine-grained
characterization of long-context task design. We argue that judging LLMs by
their ability to process long sequences, while disregarding the task they
process them for, overlooks the characteristics that make longer inputs more
difficult, and interesting to research, to begin with (§2).

For example, Needle in a Haystack tasks (NIAH; Ivgi et al., 2023; Mohtashami
and Jaggi, 2023) involve queries whose main challenge is finding the relevant
information in a long context, without requiring much further processing.
Synthetic NIAH datasets are, of course, easier than their natural equivalents
Ivgi et al. (2023), but the “natural vs. artificial” classification is not
informative in our setting, since it applies equally for tasks regardless of
context-length. What, then, is an informative property? What makes long-
context tasks different from each other? For example, multiple-needle variants
of NIAH Hsieh et al. (2024), or those that position the “needles” closer or
farther apart Levy et al. (2024). Evidently, “the number of tokens in the
input” is not a sufficient descriptor.

To resolve this roadblock, we present a taxonomy of long-context tasks for the
different factors that make them harder when controlling for context length
(§3). This taxonomy is derived by surveying the long-context literature and
surfacing the most salient points of distinction between various tasks. We
focus on (I) how difficult it is to find and extract the required information
from the input (its diffusion in the input), and (II) the absolute quantity of
required information to solve the task (its scope). See Figure 1 for a
summary.

To understand this categorization and its utility, we review the literature on
long-context evaluation and position the works with respect to those factors.
We find that the most challenging setting, in which a large quantity of
required information is present in diffused manner that is difficult to
extract, is significantly under-explored (§4).

Finally, acknowledging the inherent and legitimate reasons behind the focus on
context length as the sole descriptor of difficulty, we discuss possible paths
forward for designing more reliable measurements of long-context capabilities
when utilizing a more nuanced vocabulary (§5).

##  2 Task Design in Long Context

Evaluating the performance of NLP models over very long contexts is a fast-
changing area of research Bishop et al. (2024); Wu et al. (2024). Measurements
are regularly updated to account for new capabilities which improve with
extrapolation architectures Vaswani et al. (2017); Su et al. (2024) and
training data He et al. (2023); Chen et al. (2023). Evaluators were tasked
with designing measurements of long-context capabilities cheaply, efficiently,
and quickly, while matching realistic use-cases as much as possible. The most
common way of differentiating long-context tasks, besides the context’s
length, is whether they are naturally-constructed or synthetically-constructed
Tay et al. (2020); Bai et al. (2023); Hsieh et al. (2024).

### Natural construction.

A simple yet effective way of “moving the goalpost” for context length is by
modeling long-context tasks based on short-context tasks. This was done, for
example, with QA (Kočiský et al., 2018, cf. Dunn et al., 2017), summarization
(Huang et al., 2021, cf. Narayan et al., 2018), and NLI (Koreeda and Manning,
2021, cf. Williams et al., 2018). Specialized domains like legal Bruno and
Roth (2022); Nguyen et al. (2024) and literature (Wang et al., 2022;
Kryscinski et al., 2022) often involve longer texts, turning typically short-
context tasks such as QA and NLI into long-context scenarios. Another more
native methodology is to create new tasks which inherently require a long
context, such as multi-document summarization Fabbri et al. (2019); Angelidis
et al. (2021), survey generation Gao et al. (2024), and structured data
aggregation Caciularu et al. (2024). Both methodologies share the constraint
that, due to their natural construction (i.e., using natural text), once
created, they are difficult to modify for longer contexts as models’ long-
context capabilities improve.

### Synthetic construction.

A more flexible approach, sacrificing natural construction for length control,
is to use distractors to synthetically increase the context length. This
method allows for cheap and efficient (in terms of task construction cost)
evaluation of models’ full context length capabilities, with difficulty
adjusted by controlling the distractors. Tasks like Needle-in-a-Haystack
(NIAH; Ivgi et al., 2023; Kamradt, 2023) and PassKey retrieval (Mohtashami and
Jaggi, 2023) were created to evaluate a model’s ability to pinpoint specific
information amid lengthy distractors. Flexible and effective against existing
models, they became standard benchmarks for evaluating new long-context models
(GLM Team, 2024; Jiang et al., 2024). Followup studies have complicated these
tasks by increasing the number of critical details to locate (Arora et al.,
2023; Liu et al., 2024a) and changing their position within the input Liu et
al. (2024b); Levy et al. (2024).

### Limitations of the status quo.

NIAH-like tasks aim to assess information retrieval capabilities, yet many
“naturally constructed” QA and reading-comprehension tasks with trivial
questions about a long context accomplish the same goal. At the same time,
“multiple needles” NIAH can increase difficulty not by increasing the quantity
of needles or length of input, but by adding distractors between needles Levy
et al. (2024). What can systematically explain the different variables at
play, in order to inform better task design in the future?

Clearly, there are underlying qualitative differences that discriminate
between these various tasks besides their natural and synthetic constructions,
and besides their actual context length. Therefore, we require a more
informative vocabulary to discuss the goals of each task design, what it
accomplishes, and what it does not, in terms of measuring long-context
capabilities.

##  3 What Makes Long Context More than Retrieval?

We require a taxonomy to capture task difficulty variations beyond mere
“number of tokens”. We focus on the information that is canonically required
to solve the task as the conditioning variable. Our classification can be
summarized via the following two questions, when asked about a given task:

(I) How difficult is it to find and extract the required information?

(II) How much information is needed to be found?

For instance, consider the task of summarizing a book, in comparison to a NIAH
task of identifying a numerical detail in a long financial report (e.g., “how
much did the company earn in 2015?”). Although both tasks involve long texts,
information requirement and accessibility vary significantly. The NIAH task
focuses on localized, identifiable information, while summarization requires
extracting key details dispersed throughout the text, tangled together with
irrelevant content. Therefore, we can say that the book summarization task is
more difficult on both axes (I) and (II).

Below we give more formal descriptions of the two axes characterized by the
questions above.

### (I) Diffusion.

Although the question above intuitively defines “difficulty of information
finding”, we offer a more concrete description. Between two similar tasks, we
consider the information harder to find in one task compared to another if:
(1) It is more obscured (e.g., linguistically, semantically, contextually,
etc); (2) it is more sparse, such that it is interspersed with non-required
information; (3) its indicators are less redundant, such that there are less
places where the same information is available.

### (II) Scope.

The property of scope is simpler, and refers to the minimal quantity of
information needed to solve the task. Importantly, we are not concerned with
precise metric for “quantity of information” at this stage – it can refer to
quantity of tokens, sentences, relations, cells in a table, etc. Any metric
that reliably captures difficulty for an established solver is sufficient for
our purposes.

### Illustrative example.

To illustrate, consider the Wikipedia entry for New York City and a simple
question: “What is the estimated population of the city?” Since the answer
needs a small snippet of information, we say that the task has small scope.
And since it is easily-accessible, we say that it has low diffusion. Consider,
instead, the question “how many syllables are in this document?” – since this
question requires the entire document to answer, we say that it has large
scope, but if we consider counting syllables as straightforward, then we say
its diffusion is still low. Finally, with the question “Was the city’s mayor
elected before or after the city was affected by Hurricane Sandy?” – since it
requires snippets from two different areas of the text, we can say that when
compared to the question about the city’s population, the diffusion is higher,
but not as high as for the question “What makes the city a prominent place on
the world stage?” which poses a challenge on both axes.

##  4 Challenging Long Context Is Under-Explored

Revisiting the works surveyed in §2, they clearly differ with respect to both
scope and diffusion.

### With respect to diffusion.

The information needed for tasks ranges from easily accessible to highly
diffused and difficult to detect. On low diffusion we have NIAH Kamradt
(2023); Mohtashami and Jaggi (2023) and a myriad of factual single-hop QA
datasets (Tseng et al., 2016; Kočiský et al., 2017; Kwiatkowski et al., 2019;
Dasigi et al., 2021, inter alia) in which the answer is relatively accessible.
Adding more snippets of information separated by distractors, either in the
form of several needles Arora et al. (2023); Hsieh et al. (2024) or of hops in
a multi-hop question Trivedi et al. (2022); Zhao et al. (2022), complicates
the information detection due to the need to find at least two snippets Levy
et al. (2024), thereby increasing diffusion. Diffusion can also be increased
by making the detection of the information less straightforward (e.g., Pang et
al., 2022) or requiring aggregation Shaham et al. (2023). Lastly,
summarization tasks are of a very high diffusion Huang et al. (2021); Wang et
al. (2022), as they require the non-trivial detection of salient facts that
are interwoven with the irrelevant text.

### With respect to scope.

Tasks overwhelmingly target relatively small scope. In addition to the
aforementioned NIAH tasks and their variants, many QA datasets apply as well
(Li et al., 2023; Zhao et al., 2023; Reddy et al., 2024, inter alia). A
somewhat higher scope is achieved by datasets for query-based summarization
Zhong et al. (2021); Wang et al. (2022), and QA datasets with more obfuscated
answers that require reading the text surrounding the answer for its
verification An et al. (2023); He et al. (2023). Although much higher on the
scope ladder, book summarization is still limited in its scope as datasets
include texts that are only of up to 20$k$ tokens Huang et al. (2021); Chen et
al. (2022); Shaham et al. (2023). Currently, tasks with the highest scope,
requiring information across the entire input length, are artificial and of
low diffusion, like common words extraction Hsieh et al. (2024).

### Conclusion.

We conclude that (1) the majority of tasks designed to challenge LLMs in a
long-context setting target either scope or diffusion, such that (2) tasks
that push current models’ capabilities on both axes are under-represented in
the current landscape.

##  5 Discussion: Towards Genuinely Difficult Long-Context Task Design

### Challenges.

Designing meaningful long-context tasks amidst rapid model progress is
profoundly challenging, making the deficiency in tasks representing difficulty
on both the diffusion and scope axes unsurprising. One source of this
challenge is the lack of diverse, coherent long texts, as models’ context
windows can now be comparable to the length of the New
Testament111www.readinglength.com/book/isbn-0190909005 and the
Odyssey.222www.readinglength.com/book/isbn-0140268863 The methodologies
discussed in §2 for creating long context tasks – lengthening short context
tasks and synthetically creating length-adjustable tasks – are preferred for
their straightforward definition and the incremental adjustments they require
for existing data. They rely on the common understanding of machine
comprehension as formulated with short context in mind Dunietz et al. (2020),
and therefore they are intuitive and easy to comprehend for NLP researchers
without domain expertise (e.g., in law or biomedical fields that have long
contexts).

### Future work.

The goals laid forward in this work are clear: For more durable and robust
measurement of long-context capabilities, we must design tasks that explicitly
target both the diffusion and scope capabilities of models. How can this be
achieved? As mentioned, one possible avenue is to rely more on tasks that
require domain expertise, such as legal documents Bruno and Roth (2022),
financial reports Reddy et al. (2024), biomedical publications Stylianou et
al. (2021), and so on. In specialized domains, it is common that diffusion
will be naturally higher Zhao et al. (2022). Tasks that involve implicit
aggregations over structured data, such as table manipulation Caciularu et al.
(2024), are possible avenues for increasing both scope and diffusion
synthetically by leveraging the data structure. In this work, we argue that an
explicit vocabulary for such properties of difficulty is what can enable more
informed techniques to design difficult tasks in the future.

##  6 Conclusions

We present a taxonomy of factors that make long-context tasks more challenging
compared to short ones. This is in contrast with the existing literature that
refers only to the length of the input as the hallmark of long context, and as
a result ends up conflating tasks of different character when assessing the
ability of models to understand longer text. We reviewed works on evaluation
in a long-context setting and found that the most challenging setting, in
which the information needed is of large scope and is highly diffused within
the input, is under-explored. Finally, we suggested some leads for future work
to tackle this imbalance towards a more informative long context evaluation.

##  7 Limitations

### Formality.

In the context of this work, we have deliberately adhered to a taxonomy based
on an intuitive description, in the interest of utility to a wide diversity of
research and flexibility for future work. Difficulty in searching for and
extracting information, and quantity of information, are both vague terms that
can only be grounded in the context of a specific family of tasks and use-
cases. We intend for this work to serve as a call to action and a tool for a
shared vocabulary in the interest of more informed long-context task design in
the future, rather than to anchor the taxonomy to a specific and fragile point
in time.

### Retrieval is still interesting.

Although we argue that small scope and low diffusion tasks are the least
indicative of the model’s ability to long-context capabilities, tasks that are
well-served by implicit retrieval or by traditional retrieval-based pipelines
are certainly relevant and useful in a variety of common use-cases Stylianou
et al. (2021); Bruno and Roth (2022); Gao et al. (2023).

### Other uses for a long-context window.

This paper deals only with long inputs that serve as inputs to a task. The
long context of course can have other purposes as well, like containing many
in-context examples Bertsch et al. (2024) or containing other modalities and
structures Jiang et al. (2023).

## References

  * An et al. (2023) Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, and Xipeng Qiu. 2023.  L-eval: Instituting standardized evaluation for long context language models.  _Preprint_ , arXiv:2307.11088.
  * Angelidis et al. (2021) Stefanos Angelidis, Reinald Kim Amplayo, Yoshihiko Suhara, Xiaolan Wang, and Mirella Lapata. 2021.  Extractive opinion summarization in quantized transformer spaces.  _Transactions of the Association for Computational Linguistics_ , 9:277–293.
  * Arora et al. (2023) Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, and Christopher Ré. 2023.  Zoology: Measuring and improving recall in efficient language models.  _arXiv preprint arXiv:2312.04927_.
  * Bai et al. (2023) Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2023.  Longbench: A bilingual, multitask benchmark for long context understanding.  _Preprint_ , arXiv:2308.14508.
  * Bertsch et al. (2024) Amanda Bertsch, Maor Ivgi, Uri Alon, Jonathan Berant, Matthew R. Gormley, and Graham Neubig. 2024.  In-context learning with long-context models: An in-depth exploration.  _Preprint_ , arXiv:2405.00200.
  * Bishop et al. (2024) Jennifer A Bishop, Qianqian Xie, and Sophia Ananiadou. 2024.  Longdocfactscore: Evaluating the factuality of long document abstractive summarisation.  _Preprint_ , arXiv:2309.12455.
  * Bruno and Roth (2022) William Bruno and Dan Roth. 2022.  Lawngnli: A long-premise benchmark for in-domain generalization from short to long contexts and for implication-based retrieval.  _Preprint_ , arXiv:2212.03222.
  * Caciularu et al. (2024) Avi Caciularu, Alon Jacovi, Eyal Ben-David, Sasha Goldshtein, Tal Schuster, Jonathan Herzig, Gal Elidan, and Amir Globerson. 2024.  Tact: Advancing complex aggregative reasoning with information extraction tools.  _Preprint_ , arXiv:2406.03618.
  * Chen et al. (2022) Mingda Chen, Zewei Chu, Sam Wiseman, and Kevin Gimpel. 2022.  SummScreen: A dataset for abstractive screenplay summarization.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8602–8615, Dublin, Ireland. Association for Computational Linguistics.
  * Chen et al. (2023) Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023.  Longlora: Efficient fine-tuning of long-context large language models.  _ArXiv_ , abs/2309.12307.
  * Dasigi et al. (2021) Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021.  A dataset of information-seeking questions and answers anchored in research papers.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4599–4610, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dunietz et al. (2020) Jesse Dunietz, Greg Burnham, Akash Bharadwaj, Owen Rambow, Jennifer Chu-Carroll, and Dave Ferrucci. 2020.  To test machine comprehension, start by defining comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7839–7859, Online. Association for Computational Linguistics.
  * Dunn et al. (2017) Matthew Dunn, Levent Sagun, Mike Higgins, V Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017.  Searchqa: A new q&a dataset augmented with context from a search engine.  _arXiv preprint arXiv:1704.05179_.
  * Fabbri et al. (2019) Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. 2019.  Multi-news: a large-scale multi-document summarization dataset and abstractive hierarchical model.  _Preprint_ , arXiv:1906.01749.
  * Gao et al. (2024) Fan Gao, Hang Jiang, Rui Yang, Qingcheng Zeng, Jinghui Lu, Moritz Blum, Dairui Liu, Tianwei She, Yuang Jiang, and Irene Li. 2024.  Large language models on wikipedia-style survey generation: an evaluation in nlp concepts.  _Preprint_ , arXiv:2308.10410.
  * Gao et al. (2023) Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023.  Rarr: Researching and revising what language models say, using language models.  _Preprint_ , arXiv:2210.08726.
  * Gemini Team Google (2024) Gemini Team Google. 2024.  Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.  _Preprint_ , arXiv:2403.05530.
  * GLM Team (2024) GLM Team. 2024.  GLM-4-9b-chat technical report.
  * He et al. (2023) Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, Yibo Liu, Yuxin Liang, Hao Wang, Qianguo Sun, Songxin Zhang, Zejian Xie, and Jiaxing Zhang. 2023.  Never lost in the middle: Improving large language models via attention strengthening question answering.  _Preprint_ , arXiv:2311.09198.
  * Hsieh et al. (2024) Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024.  Ruler: What’s the real context size of your long-context language models? _Preprint_ , arXiv:2404.06654.
  * Huang et al. (2021) Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021.  Efficient attentions for long document summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1419–1436, Online. Association for Computational Linguistics.
  * Ivgi et al. (2023) Maor Ivgi, Uri Shaham, and Jonathan Berant. 2023.  Efficient long-text understanding with short-text models.  _Transactions of the Association for Computational Linguistics_ , 11:284–299.
  * Jiang et al. (2024) Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024.  Mixtral of experts.  _Preprint_ , arXiv:2401.04088.
  * Jiang et al. (2023) Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Xin Zhao, and Ji-Rong Wen. 2023.  StructGPT: A general framework for large language model to reason over structured data.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 9237–9251, Singapore. Association for Computational Linguistics.
  * Kamradt (2023) Gregory Kamradt. 2023.  Needle in a haystack - pressure testing LLMs.  GitHub.
  * Koreeda and Manning (2021) Yuta Koreeda and Christopher Manning. 2021.  ContractNLI: A dataset for document-level natural language inference for contracts.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 1907–1919, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Kočiský et al. (2017) Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2017.  The narrativeqa reading comprehension challenge.  _Preprint_ , arXiv:1712.07040.
  * Kočiský et al. (2018) Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018.  The NarrativeQA Reading Comprehension Challenge.  _Transactions of the Association for Computational Linguistics_ , 6:317–328.
  * Kryscinski et al. (2022) Wojciech Kryscinski, Nazneen Rajani, Divyansh Agarwal, Caiming Xiong, and Dragomir Radev. 2022.  BOOKSUM: A collection of datasets for long-form narrative summarization.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 6536–6558, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019.  Natural questions: A benchmark for question answering research.  _Transactions of the Association for Computational Linguistics_ , 7:452–466.
  * Levy et al. (2024) Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024.  Same task, more tokens: the impact of input length on the reasoning performance of large language models.  _Preprint_ , arXiv:2402.14848.
  * Li et al. (2023) Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023.  Loogle: Can long-context language models understand long contexts? _Preprint_ , arXiv:2311.04939.
  * Liu et al. (2024a) Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. 2024a.  World model on million-length video and language with ringattention.  _arXiv preprint arXiv:2402.08268_.
  * Liu et al. (2024b) Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024b.  Lost in the middle: How language models use long contexts.  _Transactions of the Association for Computational Linguistics_ , 12:157–173.
  * Mohtashami and Jaggi (2023) Amirkeivan Mohtashami and Martin Jaggi. 2023.  Landmark attention: Random-access infinite context length for transformers.  In _Workshop on Efficient Systems for Foundation Models@ ICML2023_.
  * Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
  * Nguyen et al. (2024) Chau Nguyen, Phuong Nguyen, Thanh Tran, Dat Nguyen, An Trieu, Tin Pham, Anh Dang, and Le-Minh Nguyen. 2024.  Captain at coliee 2023: Efficient methods for legal information retrieval and entailment tasks.  _Preprint_ , arXiv:2401.03551.
  * OpenAI (2024) OpenAI. 2024.  GPT-4 technical report.  _Preprint_ , arXiv:2303.08774.
  * Pang et al. (2022) Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, and Samuel Bowman. 2022.  QuALITY: Question answering with long input texts, yes! In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5336–5358, Seattle, United States. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of machine learning research_ , 21(140):1–67.
  * Reddy et al. (2024) Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krumdick, Charles Lovering, and Chris Tanner. 2024.  Docfinqa: A long-context financial reasoning dataset.  _Preprint_ , arXiv:2401.06915.
  * Shaham et al. (2023) Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, and Omer Levy. 2023.  ZeroSCROLLS: A zero-shot benchmark for long text understanding.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 7977–7989, Singapore. Association for Computational Linguistics.
  * Shaham et al. (2022) Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, and Omer Levy. 2022.  SCROLLS: Standardized CompaRison over long language sequences.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 12007–12021, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Stylianou et al. (2021) Nikolaos Stylianou, Panagiotis Kosmoliaptsis, and Ioannis Vlahavas. 2021.  Improved biomedical entity recognition via longer context modeling.  In _Artificial Intelligence Applications and Innovations: 17th IFIP WG 12.5 International Conference, AIAI 2021, Hersonissos, Crete, Greece, June 25–27, 2021, Proceedings 17_ , pages 45–56. Springer.
  * Su et al. (2024) Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024.  Roformer: Enhanced transformer with rotary position embedding.  _Neurocomputing_ , 568:127063.
  * Tay et al. (2020) Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. 2020.  Long range arena: A benchmark for efficient transformers.  _Preprint_ , arXiv:2011.04006.
  * Trivedi et al. (2022) Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022.  Musique: Multihop questions via single-hop question composition.  _Preprint_ , arXiv:2108.00573.
  * Tseng et al. (2016) Bo-Hsiang Tseng, Sheng-Syun Shen, Hung-Yi Lee, and Lin-Shan Lee. 2016.  Towards machine comprehension of spoken content: Initial toefl listening comprehension test by machine.
  * Vaswani et al. (2017) Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Neural Information Processing Systems_.
  * Wang et al. (2022) Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R. Bowman. 2022.  SQuALITY: Building a long-document summarization dataset the hard way.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 1139–1156, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Wu et al. (2024) Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, and Estevam Hruschka. 2024.  Less is more for long document summary evaluation by llms.  _Preprint_ , arXiv:2309.07382.
  * Zhang et al. (2024) Xinrong Zhang, Yingfa Chen, Shengding Hu, Zihang Xu, Junhao Chen, Moo Khai Hao, Xu Han, Zhen Leng Thai, Shuo Wang, Zhiyuan Liu, and Maosong Sun. 2024.  $\infty$bench: Extending long context evaluation beyond 100k tokens.  _Preprint_ , arXiv:2402.13718.
  * Zhao et al. (2022) Yilun Zhao, Yunxiang Li, Chenying Li, and Rui Zhang. 2022.  MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6588–6600, Dublin, Ireland. Association for Computational Linguistics.
  * Zhao et al. (2023) Yilun Zhao, Yitao Long, Hongjun Liu, Linyong Nan, Lyuhao Chen, Ryo Kamoi, Yixin Liu, Xiangru Tang, Rui Zhang, and Arman Cohan. 2023.  Docmath-eval: Evaluating numerical reasoning capabilities of llms in understanding long documents with tabular data.  _ArXiv_ , abs/2311.09805.
  * Zhong et al. (2021) Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. 2021.  QMSum: A new benchmark for query-based multi-domain meeting summarization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5905–5921, Online. Association for Computational Linguistics.


# Ease and Equity of Point of Interest Accessibility via Public Transit in the
U.S.

Alexander Li  Georgia Institute of Technology ,  Mengyang Liu  Georgia
Institute of Technology ,  Aurimas Racas  Georgia Institute of Technology ,
Tejas Santanam  Georgia Institute of Technology ,  Junaid Syed  Georgia
Institute of Technology and  Przemek Zientala  Georgia Institute of Technology

(2019)

††journalyear: 2019††copyright: rightsretained††conference: CSE6242; Data and
Visual Analytics††doi: ;

##  1\. Introduction and Problem

The tool developed as a result of this paper analyzes the ease and equity of
access to major POI categories (e.g. vaccination centers, grocery stores,
hospitals) using public transit in major US cities. We built an interactive
website that enables easy exploration of current access equity and allows
performing scenario analysis by introducing/removing POIs. Accessibility
indices were calculated using a 2SFCA (2-step floating catchment area)
approach, and ML methods were utilized for exploratory statistical analysis of
the results.

Many populations rely on public transit for access to work, food, family, and
more. By giving city planners a tool to see the gaps in public transit access,
we hope to improve consideration of transportation accessibility in future
infrastructure projects. Currently, decision-making around site placement in
urban environments often fails to consider accessibility and equity as major
criteria. An even larger issue lies with the fact that even transit planning,
at least in Atlanta, is largely accessibility-blind and is focused more on
ridership counts at different points in the city. However, what that approach
fails to consider is places that have lower ridership due to accessibility and
connectivity issues with POIs that people need to access. This can lead to a
cycle where infrastructure development and site placement is not focused on
the opportunity areas where ridership can develop if access to resources like
work, food, vaccination, and more is prioritized.

Our work aims to provide a holistic overview of public transit system
effectiveness and equity in major US cities, which may reveal previously
unknown gaps. We will provide interactive interfaces that will allow decision-
makers and the general public to explore and understand what drives public
transit network effectiveness and how difficult would it be to make
improvements to accessibility and equity. We base our work on data sources
that are also available outside the U.S. whenever possible, and thus our work
may serve as a framework for future work that extends to non-U.S. geographies.

##  2\. Survey

Access to facilities by public transit is key to reducing the vulnerability of
population segments that do not have access to a private car, with researchers
calling for more attention to public transit in accessibility modelling since
late 2000’s (2008).

COVID-19 generated a flurry of research showing the importance of public
transit. Multiple studies looked at vaccination rates, showing that they are
lower in rural areas (Murthy et al., 2021), minority communities (Reitsma et
al., 2021), and persist despite efforts to address them (KFF, 2022). Public
transit was shown to be an important factor. Research on informal population
in Rome found access to public transit especially important to elderly people
(Bentivegna et al., 2022); a review of US state-level COVID-19 vaccination
strategies found that public transit was a critical component in reducing
inequities, with strategies used including ride subsidies, financial support
for the public transit authorities, and deployment of ride-sharing services
(Thoumi et al., 2021).

Importance of public transit access is not limited to vaccination centers.
Spatial access is critical to ensuring quality health services (Wan et al.,
2012), especially in socially vulnerable regions (Vo et al., 2020). Planners
and policy makers are interested in identifying areas with low grocery store
coverage (Jiao and Azimian, 2021), understanding how public transit impacts
equity of job location access (Horner and Mefford, 2005) and measuring access
to educational facilities (Yenisetty and Bahadure, 2020), among others.

With public transit providers suffering from lower ridership and service cuts
due to COVID-19 (Chen et al., 2020), an updated and comprehensive research
into public transit accessibility and its equity is critical, with such
information useful to policy makers, city planning officials and for everyday
citizens looking to be better informed on the social problems in their
neighborhoods.

Most recent research focuses on vaccination center (VC) ease of access and
typically relies on catchment area approaches. In the US, ”vaccine deserts”
were identified using enhanced two-step catchment area approaches (E2SFCA),
and concluded that VCs were not effectively targeting the most vulnerable
groups (Rader et al., 2021). A study in Florida utilized two step floating
catchment (2SFCA) and E2SFCA methods to show hospital access differences
across the state (Ghorbanzadeh et al., 2021). In England, research has
concluded that taking travel time into account reveals significant VC coverage
disparities not visible in the official statistics (Duffy et al., 2022). In
Germany, it was found that the VCs are distributed equitably in urban/rural
areas, and the key determinant to access is the transport mode used instead
(Neumeier, 2022).

There is existing research focused on other types of POIs, too. In Brazil,
research looked at healthcare system capacity under COVID-19 incorporating
data about income and racial inequities, they utilized the B2SFCA method and
found that intensive care unit equipment availability was substantially lower
among Black and poor communities (Pereira et al., 2021). A study in Tamil Nadu
utilized the 3SFCA method to calculate the accessibility index of healthcare
facilities and find optimal locations to setup new ones (Rekha et al., 2017).
Research in Iran has shown that spatial analysis techniques (E2SFCA) can be
used to select public hospitals and public healthcare centers for optimal
coverage of the population (Mohammadi et al., 2021). Research in Hong Kong
utilized gravity models to measure the spatial accessibility of urban parks
and found that public transport significantly increased the spatial inequality
of urban park accessibility between public and private housing residents
(Chang et al., 2019). A study in US proposed the KD2SFCA method by
incorporating the kernel density function into the 2SFCA method to highlight
the geographic disparities in accessibility to food stores (Dai and Wang,
2011).

##  3\. Methodology

###  3.1. Data

####  3.1.1. Data sources

Our data consists of three primary categories. The first one is the US Census
(American Community Survey - 2020) which contains information about
demographics and census block group polygons. This data is collected by the US
government on a census block group level and obtained via SafeGraph. We
extracted population amounts, race breakdowns, no-car percentages, income
levels by bracket, age breakdown, and sex breakdown for each census block
group. The next category is Point of Interest (POI) data from OpenStreetMap
(OSM) (collected using Overpass API). We extracted the latitude/longitude and
names of schools, theaters, restaurants, hospitals, and grocery stores. This
data is joined with vaccination site data obtained from the Center for Disease
Control (CDC). Finally, the last category of data consists of public transit
scheduling and stop information obtained via the General Transit Feed
Specification (GTFS) data feeds that are provided by public transit operators
(we used transitfeeds.com website to identify where the datafeeds are
available).

The aforementioned datasets were collected for five of the largest metro areas
in the US (New York City, Los Angeles, Chicago, Dallas, and Atlanta). The
total data size in our database is 11 gigabytes. Our dataset consisted of
60,000 vaccination centers, 14,000 restaurants, 4,500 grocery
stores/supermarkets, 4,200 (pre-)schools, 700 clinics/hospitals, and 500
cinemas/theatres.

###  3.2. Data preparation

####  3.2.1. Catchment area estimation

For each POI, we estimated their coverage areas (referred to as catchment
areas thereon) by generating isochrones - areas delineating a boundary
reachable within a given travel time. We generated 30-minute isochrones based
on morning, afternoon and evening public transit schedules. We used a
Dockerized OpenTripPlanner server for this purpose; it relies on
aforementioned GTFS transit feeds and OSM street-network information to
generate accurate isochrones.

####  3.2.2. Hierarchical spatial indexing system

To ensure easier and consistent calculations and more readable visualizations,
we mapped all our data points (demographics data, POI coordinates and
catchment areas) to the H3 hierarchical spatial indexing system (developed by
Uber). We used H3 hexagons with an area size of $\approx 0.1km^{2}$,
corresponding to H3 resolution level 9. This resulted in M:N relationships
between census polygons and H3 hexagons.

Figure 1. Overlap types between H3 hexagons and census polygons. Note that,
with decreasing hexagon area, the relationship of polygon to hexagon IDs
becomes closer to pure 1:N

Using smaller hexagons resulted in smaller margins of error in our analysis.
In the even that information (e.g. demographic data of a census block group)
spanned multiple H3 hexagons, the allocation was made in proportion to the
percentage area covered in each hexagon. In total, we calculated information
for 137,000 unique H3 hexagons across the United States.

###  3.3. Accessibility index calculation

The accessibility index was calculated using a 2SFCA (2-step floating
catchment area) approach (Ghorbanzadeh et al., 2021). Out of all the possible
methods, this method was chosen as it is the simplest method for policymakers
to understand. Since we wanted our tool to have implementable value, we felt
that the 2SFCA approach would not confuse policymakers while still conveying
significant insights into accessibility. While the other methodologies may be
more complex, the accessibility in understanding for a non-technical person
seemed to be a larger benefit. The calculation for each census block (group)
was done as follows:

1\. For each POI ID, get all catchment areas. For each catchment area,
calculate POI-to-population ratio as follows:

$R_{j}=\frac{|supplyunits|}{\sum_{j}catchmentpop_{j}}$

2\. For each H3 hexagon, sum $R_{j}$ of all catchment areas which overlap with
the hexagon. Weigh that sum by proportion of population for a given group as a
fraction of entire population in that hexagon. The sum is the final
accessibility score:

$A_{h}=\sum_{j\in j\cap h}R_{j}$

###  3.4. User interface

We built an interactive website where users can explore the accessibility
statistics and engage in scenario-like analysis. The main technologies used
include Vue.js and Deck.gl Javascript libraries, supported with a FastAPI
Python backend. A screenshot of the user interface is shown in the figure
below.

Figure 2. User Interface

Upon entering the site, users are shown the accessibility of vaccination sites
in Atlanta. In the control panel dropdown on the left, the users can adjust
different factors and select different cities, POIs (Cinemas and Theaters,
Schools and Pre-schools, Grocery Stores, Vaccination Centers, Hospitals and
Clinics, Restaurants), times of day (morning, afternoon, evening), and
demographic factors (Race, Age & Sex, Income, Vehicle Availability). The users
can then go through different demographic options to see how transit
accessibility to the chosen POI type differs for different demographic groups.
Visually, the H3 hexagons on the map will display in different colors to
indicate accessibility scores to that POI type, and upon hovering, a tooltip
will show for each hexagon containing the selected demographic information.
The tool will also allow users to place new potential POIs and remove existing
POIs and see the recalculated impact on the demographic accessibility metrics
displayed on the charts on the right hand side of the page. This kind of
situational analysis is a unique feature of our tool. New POIs will be added
in a different color from existing POIs so that the user does not conflate the
two types. Users can also click on each POI to see the 30-minute isochrone on
the map. This isochrone can also be generated for new hypothetical POIs added
by the user. Additionally, the analysis displayed can be changed based on time
of day, POI type, and desired demographic analysis. For example, the placement
of some POIs might use age as a motivating factor (hospitals might want to
locate closer to older age groups). Or perhaps, the placement process might
require time of day analysis (theaters are mostly attended in the evening).
Thus, the tool allows decision makers to view and get a deeper understanding
on the factors that matter to them. The tool also contains analysis for
Atlanta, New York City, Los Angeles, Chicago, and Dallas. The user interface
is simple to use and allows decision makers to clearly see where the gaps in
access are, as well as how significant those gaps are, for both transit system
planning and POI location planning purposes.

###  3.5. Innovations

The primary innovation is the ability to quantify and compare accessibility
metrics across cities, POI categories, and populations segments beyond solely
data reporting. Additionally, the focus on creating an interactive visual tool
using H3 polygons for population discretization for decision makers is
something that hasn’t been done before to our knowledge. Since census blocks
are of differing sizes and can have changing boundaries every census year, H3
hexagons allow for a standardized way to compare different parts of the city
in equally sized geographies. Furthermore, the scenario analysis that
recalculates accessibility statistics based on adding and removing POIs is a
unique feature. Lastly, the granularity down to the census block group level
in analysis allows for accuracy in equity analysis that commonly has only been
done to the metropolitan statistical area (MSA) level.

##  4\. Experiments and Evaluation

###  4.1. Comparisons and Atlanta Takeaways

In Figure 3 below, we can see the comparison of weighted accessibility across
cities broken down by POI type.

Figure 3. Weighted Accessibility by POI Type and City

Weighted accessibility is the accessibility for each H3 hexagon normalized by
population. Atlanta actually has the highest accessibility for theaters,
hospitals, and schools out of the five cities analyzed. In addition, Atlanta
does not have the lowest accessibility for any of the POI types. New York City
has the highest accessibility for grocery stores, restaurants, and vaccination
centers. Los Angeles has the lowest accessibility for all POI types, a
testament to the dysfunction of its transit system. However, even though a
city has a higher accessibility score, that does not mean it works for
everyone. For example, even though Atlanta has the highest accessibility for
clinics and hospitals, the accessibility of those hospitals is twice as high
for White populations in Atlanta as compared to Black populations, as seen in
the figure below. The accessibility scores can be validated by, for example,
the fact that New York City has the highest restaurant accessibility. That
intuitively makes sense due to the population and restaurant density New York
is known for, thus serving as a sanity check for our numbers.

Figure 4. Hospital Accessibility Index by Race in Atlanta

Similarly, access to grocery stores in Atlanta is 70% higher for White
populations than Black populations. For example, in Chicago, the gap in access
for grocery stores is much smaller between White and Black populations, while
for hospitals in Chicago, Black populations have a higher accessibility than
White populations. Even Los Angeles, which has the worst overall accessibility
across POIs, has similar accessibility for White and Black populations to all
POIs. This highlights a suboptimal transit system in Los Angeles, but at least
that system serves people equitably. In Atlanta, the major issue is the lack
of transit infrastructure and reachability in the South and Southwest portions
of the city. MARTA often lacks reach in those neighborhoods that are home to
majority Black populations. This creates a transit divide where entire groups
of people are left disconnected from the city. The fact that the ML models run
were able to predict majority race easily solely on accessibiltiy score for a
hexagon underscores this fact. Just like racial divides, there are also
divides in income in Atlanta. For grocery stores and hospitals, those who make
over $200,000 a year have the highest accessibility via public transit.
However, in Los Angeles, those who make under $10,000 a year have the highest
accessibility to grocery stores and hospitals, which are often the people who
use public transit the most.

###  4.2. Statistical analysis

Based on the accessibility metrics, we wanted to see if we could predict the
majority race based on accessibility index for each H3 hexagon in Atlanta. As
an example, we present results for vaccination centers, with time of day being
the afternoon. We developed a Random Forest model for this classification
task. We found that the model was able to predict majority race in case of
Black and While populations with high certainty (F1 scores 0.85) indicating
that there is an association between race and accessibility and thus pointing
to the inequity of accessibility between them. The model did not perform well
in case of minority races (e.g. Asian majority was predicted with a F1 score
of 0.11, other races had an F1 score of 0) which is not unexpected given that
there are very few locations where these races are in the majority. A
confusion matrix for test set can be seen below in Figure 5 illustrating this
finding.

Figure 5. Test set confusion matrix. Note the high predictive power for Black
and White populations.

We also analyzed the inequity dynamics by using demographic characteristics of
H3 hexagons to predict accessibility scores to answer the opposite question.
We used an Extra Tree Regressor model for the prediction. The most important
demographic characteristics for accessibility prediction, in order, were race,
age/sex, vehicle availability, and income. There were also explorations made
using Linear Regression models, but the Extra Tree Regressor performed the
best (RMSE of 0.2).

We plan to release the application for use, and success will be measured by
the interest of policy makers, city planners, and corporations in the tool, as
well as the value they find in the tool’s analysis.

##  5\. Conclusion and Discussion

Throughout this project, we have been able to create a tool that empowers city
and transit planners to make better decisions with regards to accessibility
and equity. By providing granular analysis and an easy-to-understand visual
tool, we were able to impress officials in the Atlanta government. There is
also an ability to help companies in Atlanta better access their labor
markets, as outlined by leadership at ATLDOT. Depending on POI types, the
analysis may literally lead to saving lives (2022).

In future city and transit planning operations by the city, extra care needs
to be taken to ensure that these currently underserved populations are
receiving the vital transportation connections needed for their livelihood.
Despite relatively higher overall accessibility scores seen in the previous
section, Atlanta is actually one of the worst in this regard as compared to
other cities. We hope these eye-opening stats will drive change in the way
Atlanta develops.

The main limitation of our tool relates to insufficient data, especially as it
pertains to the completeness of POI locations. Another limitation of our work
is that only consider public transit and not other modes of transport, which
may prevent our work from being complete for certain types of decision-making
situations. However, at a minimum, everyone has the ability to use public
transit, so it provides a good check for whether or not it is even possible
for someone to be served by a POI.

Overall, Atlanta’s transit system and city planning vision has a lot of
potential to grow in the coming years. We can hope that the city can seize the
opportunity to better serve racial minority and lower-income populations and
increase their access to vital POIs.

## References

  * (1)
  * who (2022) 2022\.  Vaccine equity.  https://www.who.int/campaigns/vaccine-equity
  * Bentivegna et al. (2022) Enrico Bentivegna, Silvia Di Meo, Anita Carriero, Nadia Capriotti, Alberto Barbieri, and Paolo Martelletti. 2022.  Access to COVID-19 Vaccination during the Pandemic in the Informal Settlements of Rome.  _International Journal of Environmental Research and Public Health_ 19, 2 (2022), 719.
  * Chang et al. (2019) Zheng Chang, Jiayu Chen, Weifeng Li, and Xin Li. 2019\.  Public transportation and the spatial inequality of urban park accessibility: New evidence from Hong Kong.  _Transportation Research Part D: Transport and Environment_ 76 (2019), 111–122.  https://doi.org/10.1016/j.trd.2019.09.012
  * Chen et al. (2020) Katherine L Chen, Madeline Brozen, Jeffrey E Rollman, Tayler Ward, Keith Norris, Kimberly D Gregory, and Frederick J Zimmerman. 2020.  Transportation Access to Health Care During the COVID-19 Pandemic: Trends and Implications for Significant Patient Populations and Health Care Needs.  (2020).
  * Dai and Wang (2011) Dajun Dai and Fahui Wang. 2011.  Geographic Disparities in Accessibility to Food Stores in Southwest Mississippi.  _Environment and Planning B: Planning and Design_ 38, 4 (2011), 659–677.  https://doi.org/10.1068/b36149 arXiv:https://doi.org/10.1068/b36149
  * Duffy et al. (2022) Catherine Duffy, Andy Newing, and Joanna Górska. 2022\.  Evaluating the Geographical Accessibility and Equity of COVID-19 Vaccination Sites in England.  _Vaccines_ 10, 1 (2022), 50.
  * Ghorbanzadeh et al. (2021) Mahyar Ghorbanzadeh, Kyusik Kim, Eren Erman Ozguven, and Mark W Horner. 2021.  Spatial accessibility assessment of COVID-19 patients to healthcare facilities: A case study of Florida.  _Travel Behaviour and Society_ 24 (2021), 95–101.
  * Horner and Mefford (2005) Mark W Horner and Jessica N Mefford. 2005.  Examining the spatial and social variation in employment accessibility: A case study of bus transit in Austin, Texas.  In _Access to destinations_. Emerald Group Publishing Limited.
  * Jiao and Azimian (2021) Junfeng Jiao and Amin Azimian. 2021.  Measuring accessibility to grocery stores using radiation model and survival analysis.  _Journal of Transport Geography_ 94 (2021), 103107.
  * KFF (2022) KFF. 2022.  Latest data on covid-19 vaccinations by race/ethnicity.  https://www.kff.org/coronavirus-covid-19/issue-brief/latest-data-on-covid-19-vaccinations-by-race-ethnicity/
  * Martin et al. (2008) David Martin, Hannah Jordan, and Paul Roderick. 2008\.  Taking the bus: incorporating public transport timetable data into health care accessibility modelling.  _Environment and Planning A_ 40, 10 (2008), 2510–2525.
  * Mohammadi et al. (2021) Alireza Mohammadi, Abolfazl Mollalo, Robert Bergquist, and Behzad Kiani. 2021.  Measuring COVID-19 vaccination coverage: an enhanced age-adjusted two-step floating catchment area model.  _Infectious diseases of poverty_ 10, 1 (2021), 1–13.
  * Murthy et al. (2021) Bhavini Patel Murthy, Natalie Sterrett, Daniel Weller, Elizabeth Zell, Laura Reynolds, Robin L Toblin, Neil Murthy, Jennifer Kriss, Charles Rose, Betsy Cadwell, et al. 2021\.  Disparities in COVID-19 vaccination coverage between urban and rural counties—United States, December 14, 2020–April 10, 2021.  _Morbidity and Mortality Weekly Report_ 70, 20 (2021), 759\.
  * Neumeier (2022) Stefan Neumeier. 2022\.  Accessibility of COVID-19 Vaccination Centers in Germany via Different Means of Transport.  _KN-Journal of Cartography and Geographic Information_ (2022), 1–18.
  * Pereira et al. (2021) Rafael HM Pereira, Carlos Kauê Vieira Braga, Luciana Mendes Servo, Bernardo Serra, Pedro Amaral, Nelson Gouveia, and Antonio Paez. 2021\.  Geographic access to COVID-19 healthcare in Brazil using a balanced float catchment area approach.  _Social Science & Medicine_ 273 (2021), 113773.
  * Rader et al. (2021) Benjamin Rader, Christina M Astley, Kara Sewalk, Paul L Delamater, Kathryn Cordiano, Laura Wronski, Jessica Malaty Rivera, Kai Hallberg, Megan F Pera, Jonathan Cantor, et al. 2021\.  Spatial Accessibility Modeling of Vaccine Deserts as Barriers to Controlling SARS-CoV-2.  _medRxiv_ (2021).
  * Reitsma et al. (2021) Marissa B Reitsma, Jeremy D Goldhaber-Fiebert, and Joshua A Salomon. 2021.  Quantifying and benchmarking disparities in COVID-19 vaccination rates by race and ethnicity.  _JAMA network open_ 4, 10 (2021), e2130343–e2130343.
  * Rekha et al. (2017) R Shanmathi Rekha, Shayesta Wajid, Nisha Radhakrishnan, and Samson Mathew. 2017.  Accessibility analysis of health care facility using geospatial techniques.  _Transportation Research Procedia_ 27 (2017), 1163–1170.
  * Thoumi et al. (2021) A Thoumi, H Tewarson, and K Johnson. 2021.  Prioritizing Equity for COVID-19 Vaccinations: Promising Practices from States to Reduce Racial and Ethnic Disparities.  _Washington, DC: Duke-Margolis Center for Health Policy and National Governors Association Center for Best Practices_ (2021).
  * Vo et al. (2020) Au Vo, Rahul Bhaskar, Tai-Yin Chi, and Guillaume Faddoul. 2020\.  Identifying socially vulnerable regions with persistent low accessibility to emergency care through a spatial decision framework.  _Journal of Decision Systems_ 29, 4 (2020), 201–222.
  * Wan et al. (2012) Neng Wan, Bin Zou, and Troy Sternberg. 2012.  A three-step floating catchment area method for analyzing spatial access to health services.  _International Journal of Geographical Information Science_ 26, 6 (2012), 1073–1089.
  * Yenisetty and Bahadure (2020) Pavan Teja Yenisetty and Pankaj Bahadure. 2020.  Spatial accessibility measures to educational facilities from public transit: a case of Indian cities.  _Smart and Sustainable Built Environment_ (2020).