Network-Controlled Repeater - An Introduction

Fco. Italo G. Carvalho, Raul Victor de O. Paiva, Tarcisio F. Maciel, Senior Member, IEEE,
Victor F. Monteiro, Member, IEEE, Fco. Rafael M. Lima, Senior Member, IEEE, Darlan C. Moreira,
Diego A. Sousa, Behrooz Makki, Senior Member, IEEE, Magnus Åström and Lei Bao

In 5G wireless cellular networks, millimeter wave spectrum opens room for several potential improvements in throughput, reliability, latency, among other aspects.
However, it also brings challenges, such as a higher influence of blockage which may significantly limit the coverage.
In this context, NCR are network nodes with low complexity that represent a technique to overcome coverage problems.
In this paper, we introduce the NCR concept and study its performance gains and deployment options.
Particularly, presenting the main specifications of NCR as agreed in 3GPP Rel-18, we analyze different NCR deployments in an urban scenario and compare its performance with alternative deployments.
As demonstrated, with a proper network planning  and beamforming design, NCR is an attractive solution to cover blind spots the BS may have.

Network-controlled repeater, 3GPP, network densification, 5G, beamforming, millimeter wave communications

§ INTRODUCTION

5G and beyond aim to provide everyone everywhere with high QoS.
To cope with such requirements, different techniques are used, among which network densification, beamforming and mmWave communications.
Particularly, it is expected that, in near future, to assist the gNB, a wireless network will be densified with different types of beamforming-capable nodes such as IAB nodes, NCR (sometimes called NetCRs), etc.

During the 3GPP Rel- 16-18, IAB has been specified as the main 5G relaying technique, which is based on decode-and-forwarding <cit.>.
In a few words, the goal of IAB is to provide flexible wireless backhaul using 3GPP NR technology, providing not only backhaul but also the existing cellular services in the same node.
Although the coverage of IAB is typically larger than the coverage of, e.g., Rel-17 and Rel-18 repeaters, it is a relatively complex and expensive node compared to other relaying nodes.
Thus, depending on the deployment, there may be need for simpler and more affordable nodes with low complexity for, e.g., coverage hole removal.
Indeed, Rel-17 repeater <cit.> is one option, which simply amplifies-and-forwards every signal it receives.

Historically, repeaters are considered as a source of interference (since they forward everything including noise).
So, it is beneficial to limit their interference via, e.g., proper beamforming.
Additionally, as explained in the sequel, enabling the gNB to control the repeaters makes it possible to deploy them in mmWave spectrum where beamforming is a necessity, improve their data transmission and/or energy efficiency.
These have been the main motivations for 3GPP to define and specify the requirements/capabilities for NCR in Rel-18, as repeaters with beamforming capabilities which are under network control.

In this paper, we introduce the NCR, as defined by 3GPP Rel-18, and study their potential gains/deployment options.
Highlighting the main Rel-18 NCR-specific agreements on different protocol layers, we analyze different NCR deployments in an urban scenario.
Furthermore, we compare the system-level performance of NCR-assisted networks with those in alternative network deployments.
As we show, NCR are attractive nodes to assist existing networks, especially for covering blind spots that are naturally inherent in present mmWave networks.
Additionally, network planning and beamforming are shown to have large impact on the performance of NCR-assisted networks.

It is worth noting that the most relevant works to our paper are <cit.>, in which the concept of a heterogeneous smart electromagnetic environment with different types of network nodes is studied <cit.>, network planning is optimized for the cases with NCR and RIS <cit.> and conceptual comparisons between NCR and RIS are presented <cit.>.

§ 3GPP NCR

In general, NCR are repeaters with beamforming capability that can receive and process SCI from the network.
NCR are of interest in both outdoor and indoor networks as well as for outdoor-to-indoor or indoor-to-outdoor communications.
Different from IAB which is based on decode-and-forward relaying, NCR are beamforming-capable amplify-and-forward relay nodes and, thereby, amplify the noise/interferences.
The SCI allows the NCR to perform their amplify-and-forward operation efficiently.
In other words, NCR can be considered as network-controlled “beam benders” relative to the gNB.
That is, although deployed at a distance from their controlling gNB, they are logically part of it for all management purposes, i.e., they are deployed under the control of the operator.

Particularly, 3GPP has specified NCR based on the following assumptions <cit.>:

* NCR are in-band RF repeaters used for extension of network coverage on FR 1[FR1 is defined as bands in the sub-6 GHz spectrum (although 7125 MHz is the maximum) while FR2 defines bands in the mmWave range.] and FR 2 bands based on the NCR model given in <cit.>;

* NCR are deployed in single hop stationary deployments;

* NCR are transparent to the UE, in the sense that the UE is not made aware of the NCR by the network, following its normal behavior as in the cases without NCR;

* NCR can maintain the gNB-repeater link and the repeater-UE link simultaneously.

gNB to UE connection via an NCR.

Figure <ref> demonstrates the schematic of an NCR as defined in <cit.>.
The NCR consists of two functions, namely, MT and Fwd.
The NCR-MT is the entity responsible for communicating with its controlling gNB to exchange information, e.g., SCI.
The link between the gNB and the NCR-MT is referred to as C-link which is based on NR Uu interface.
The NCR-Fwd is the entity that performs amplify-and-forward in the UL/DL signals between the gNB and the UE.
The links between the gNB and NCR-Fwd and between the NCR-Fwd and UE are referred to as backhaul and access links, respectively.
The behavior of the NCR-Fwd is solely controlled based on the SCI received by the NCR-MT from the controlling gNB.
The 3GPP Rel-18 NCR work-item has mainly focused on specifying the signaling of the SCI for controlling the NCR-Fwd behavior related to beamforming, UL-DL TDD operation and ON-OFF configuration.

As one of its key properties, an NCR can be turned ON/OFF to save energy/manage interference.
Here, by default an NCR is OFF, in which case the NCR-Fwd is not expected to forward signals.
An NCR goes into ON state only in the time resources with an access link beam indication.
Both slot- and symbol-level granularities are feasible for the time-domain resource indication and determination of the access link beam.

The NCR receives a signal from the gNB in DL (or, from the UE in UL).
Within the NCR, the signal is amplified and possibly also filtered before it is transmitted to the UE in DL (or, to the gNB in UL).
Beamforming is used for both reception and transmission, resulting in improved SINR.
Which beamforming to apply and when to apply it is provided to the NCR over the SCI.
Consequently, the NCR is not required to have any signal and channel awareness for signals forwarded by the NCR-Fwd.
The forwarding operation is performed entirely in the analog domain, implying a latency in the order of tens of nanoseconds, and it is fully manageable (if at all needed) with existing timing alignment procedures.
To receive SCI, the NCR-MT uses the normal Uu interface and must therefore support a subset of UE capabilities.

Like fixed IAB-nodes, NCR are stationary nodes deployed according to the network planning done by the operator.
The planning can be expected to result in a stable control/backhaul link between the controlling gNB and NCR.
Thus, the required features/capabilities of NCR-MT are like those in IAB-MT (see <cit.> and <cit.> for detailed specifications for IAB).

As baseline, the same large-scale properties of the channel are expected to be experienced by the C-link and backhaul link, at least when the NCR-MT and NCR-Fwd operate in the same carrier as in Rel-18.
The DL communication of C-link and backhaul link can be simultaneously performed or TDMed.
The UL communication of C-link and backhaul link, on the other hand, can be performed at least TDMed.
The simultaneous UL transmission of the C-link and backhaul link is subject to NCR capability.

Fixed and adaptive beams can be considered at NCR for C-link and backhaul-link, since support for a fixed beam is part of the basic NCR features, and, as a capability, the NCR-MT may perform adaptive beamforming on the C-link/backhaul link.
On the one hand, in simultaneous operation, NCR-Fwd follows NCR-MT's beam indication (TCI or SRI).
On the other hand, in non-simultaneous operation, NCR-Fwd follows dedicated backhaul beam indication or pre-defined rules.
Conditioned on capability, the backhaul link may additionally be configured with separate TCI states.

The beam control of the NCR in the access link should be carefully addressed to minimize the impact on cell-common and UE-specific signals/channels which are forwarded towards the UE.
According to the specification, the forward resource can be configured periodically, semi-persistent, or dynamically.
Periodic and semi-persistent beam configuration can be used, e.g., for repeating cell-specific signals, with rarely/never changed configuration and are configured per periodicity, where each periodicity includes a set of time offsets and durations.
Additionally, dynamic access link beam configuration may be used for forwarding dynamically scheduled UE-specific channels, which are indicated as pairs of beam and time resource indices, where the full set of time resources is preconfigured by RRC.

The DL/UL transmission direction of the NCR-Fwd function can be principally derived from the NCR-MT configurations.
Finally, NCR do not support dynamic TDD since that would require signal awareness and a significantly more complicated NCR design.

Different ranges of NCR' links have been defined.
At the UE-side of NCR, wide-, medium- and local-area range have been defined.
At the gNB-side, on the other hand, wide- and local-area range have been defined.
These types of links’ properties can be independently declared.
The main differences between wide- and local-area links are the level of network planning and transmit power.
Wide-area links benefit from high transmit power/amplification gain and are well-planned where, for instance, the gNB-side of the NCR is located on the rooftop or similar, to achieve LOS over a long distance to a controlling gNB.
For the local-area type, on the other hand, the node may be mounted at low heights, e.g., lamppost, etc., and may have a power in the order of a typical UE.

In the initial discussions to define the Rel-18 NCR study-item, there have been discussions whether RIS should also be included into the NCR discussions.
However, it was decided not to include and leave them for possible future releases.
There are similarities and differences between NCR and RIS.
In simple words, RIS are NCR with negative amplification gain, and both emit the incoming signals without decoding.
Compared to NCR, RIS are expected to be simpler nodes with less focused beamforming capability/accuracy and without active amplification (or small amplification in the cases with active RIS).
As opposed to NCR, RIS do not amplify the signal but also not the noise/interference.
Additionally, RIS have only one reflection matrix, while NCR can do separate beamforming at the transmitter and receiver sides.
This should be an advantage for the NCR when controlling the interference in, e.g., multi-user scenarios.

§ PERFORMANCE EVALUATION OF NCR-ASSISTED NETWORKS

To study the performance of NCR, we consider both UL and DL of a TDD 5G NR network, where NCR are used to extend coverage and overcome possible coverage holes due to blockage.
We assume the same type of data traffic in both link directions.

Consider a network composed of $N$ NCR and $B$ gNB serving $U$ UE, either directly or through one of the NCR, over $K$ RB, which correspond to the minimum allocable time-frequency system resources.
At each allocation period, namely TTI, each gNB assigns RB to its served UE orthogonally, either directly or via an NCR.
Thus, the links of the same gNB do not interfere with each other but may interfere with links from other gNB reusing the same RB.
It is assumed that NCR do not perform signal processing, i.e., do not filter out incoming signals that are transmitted to UE who are not served by them.
In other words, NCR amplify and forward both desired and interfering signals.
The gNB and NCR are equipped with antenna arrays while UE have a single antenna.

Considering DL communication, the signal directly received by a UE from its serving gNB or indirectly via the gNB-controlled NCR might be formed by the superposition of several signal components, i.e.,

* signal of interest coming from the serving gNB, in case of direct communication, and/or an amplified and forwarded signal of interest coming from the NCR serving the UE;

* amplified AWGN of the receiver of the serving NCR, which is forwarded to the UE in case of indirect communication;

* AWGN of the UE receiver;

* interfering signal components coming directly from neighbor gNB;

* interfering signals amplified and forwarded by neighbor NCR; and

* amplified AWGN components from neighbor NCR receivers, which are forwarded towards the UE.

These components are subject to effects impaired by wireless channels, e.g., path loss, shadowing, fast fading, etc.
They are also subjected to the gains of transmit/receive filters adopted by the gNB and NCR.
A similar set of signals can be considered in UL communication.

In the considered scenario, the UE association to either gNB or NCR is performed as follows.
Beam sweepings between gNB and NCR and between NCR and UE are periodically performed with different periodicities, selected considering the profile of each type of link <cit.>.
Then, the best beam pairs for each link (backhaul and/or access) are selected and used until the next beam sweeping
In this context, gNB and NCR are static nodes with LOS established during the network planning phase.
Then, the gNB-NCR link usually has a long coherence time and, therefore, requires less beam update (or even never requires).
Oppositely, the beamforming at the access link may require frequent updates since the channel varies faster due to the UE' mobility <cit.>.
The UE follows its typical behavior to measure the RSRP of the received reference signals and reports the measurements to its serving gNB.
Based on the received RSRP measurement reports, the gNB evaluates whether to directly transmit to the UE or to serve the UE through its NCR.

A key objective of NCR is to extend/improve the coverage of 5G networks which tends to be less consistent at mmWave bands.
Herein, we aim to contribute to the research on NCR by assessing the gains of introducing different numbers of
NCR into networks covered originally only by conventional gNB.

Scenarios of interest.

For this purpose, we analyze the system performance of different setups.
Two base scenarios, as shown in NCR_LOS_FIG:scenario, are considered.
In Scenario A, we consider two macro-gNB at the central block and two NCR.
Scenario A is split into two cases: one case in which the shown gNB and NCR are present and another case where only the gNB are used, which we refer to as macro-only scenario.
Similarly, Scenario B contains four macro-gNB at the central block, and four NCR at the central blocks of each side road.
Scenario B is also split into a case with only gNB (macro-only scenario) and another case with gNB and NCR.
The system-level performance of these four cases is evaluated through simulations and characterized in terms of SINR,
MCS usage and throughput.

A simplified version of the Madrid grid <cit.> is considered.
The scenarios in NCR_LOS_FIG:scenario have nine 120m $\times$ 120m blocks.
They are surrounded by 3m wide sidewalks and separated from each other by 14m wide streets.
The UE are uniformly positioned on the sidewalks.
When one of them reaches a block corner, it has 60% of chance of continuing straight ahead and 20% of chance to turn left or right.
These probabilities are proportionally re-scaled whenever some of the directions are not available at a block corner.

The channel model of <cit.>, which is compliant with <cit.>, is used to model the links between all entities.
It is consistent in space, time, and frequency dimensions and considers small-scale fading, log-normal shadowing and
distance-dependent path loss propagation components.
Links from the gNB to the NCR and to the UE follow the 3GPP UMa profile <cit.>.
Links from the NCR to the UE follow the 3GPP UMi <cit.>.
From the gNB to the NCR, we consider LOS links, while those from the gNB and the NCR to the UE can be LOS or NLOS, depending on the UE' positions.
The NCR have a fixed amplification gain of 90dB subject to a maximum output power constraint.
Other system and node parameters are summarized in NCR_LOS_TAB:Simul_Param.

Simulation parameters.

Parameter                              3cValue

Layout                                          3lSimplified Madrid grid <cit.>

Carrier frequency                               3l28

System bandwidth                                3l50

Subcarrier spacing                              3l60

Number of subcarriers per RB              3l$12$

Number of RB                             3l$66$

Slot duration                                   3l0.25

OFDM symbols per slot                     3l$14$

Channel generation procedure                    3lCf. <cit.>

Path loss                                       3lCf. <cit.>

Fast fading                                     3lCf. <cit.> and

AWGN power per subcarrier                  3l-174dBm

Noise figure                                    3l9

Mobility model                                  3lPedestrian <cit.>

Number of UE                              3l72

CBR traffic packet size                   3l$3072$ bits

CBR traffic packet inter-arrival time     3l4slots

Nodes                        gNB                   NCR                   UE

Height                                          25                     10                     1.5

Transmit power                                  35m                 33m                 24m

Fixed DL/UL amplification gain                                  -                 90                 -

Antenna tilt                                    $12^{\circ}$                        $12^{\circ}$                        $0^{\circ}$

Antenna array                                   URA $8\times 8$                     URA $8\times 8$                     Single antenna

Antenna element pattern                         3GPP 3D <cit.>     3GPP 3D <cit.>     Omni

Max. antenna element gain                       8i                  8i                  0i

Speed                                           0km/h                        0km/h                        3km/h

SINR and throughput values for Scenarios A & B at the 10$^{\text{th}}$, 50$^{\text{th}}$ and 90$^{\text{th}}$ percentiles.

5c|Scenario A
5cScenario B

3c|SINR (dB)
2c|Throughput (MBits/s)
3c|SINR (dB)
2cThroughput (MBits/s)

2c|Direct links
1c|Forwarded links
2c|Direct links
1c|Forwarded links

w/o NCRs
1c|w/ NCRs
w/o NCRs
w/ NCRs
w/o NCRs
1c|w/ NCRs
w/o NCRs
w/ NCRs

2c|Direct links
1c|Forwarded links

2c|Direct links
1c|Forwarded links

w/o NCRs
1c|w/ NCRs
w/o NCRs
w/ NCRs
w/o NCRs
1c|w/ NCRs
w/o NCRs
w/ NCRs

Figure <ref> presents the CDF of the SINR of the direct and forwarded links for both Scenarios A and B.
Solid curves refer to direct links between gNB and UE, while dashed curves refer to links forwarded by NCR.
The blue and green curves are obtained for Scenario A without and with NCR, respectively, while the yellow and red ones relate to Scenario B analogously.

CDF of SINR for the direct and forwarded link for Scenarios A and B on Downlink and Uplink, respectively.

Considering the DL, at the 10$^{\text{th}}$ percentile of SINR CDF shown in NCR_LOS_FIG:Simulation-results-sinr-cdf-all and summarized in NCR_LOS_TAB:Results-Percentiles, Scenario A achieved SINR of 2.97dB for the direct links in the macro-only case.
In Scenario A with NCR, at the 10$^\text{th}$ percentile, we observe an SINR of 8.93dB for the direct links and 25.05dB for the forwarded links, respectively.
Therefore, compared to the cases with macro-only, the worst connections are improved by around 6dB when connected to the gNB and by 22dB when served through the NCR.
This is intuitive because the introduction of the NCR reduces the effective distance between UE and their serving nodes.
Regarding the 50$^{\text{th}}$ percentile of the SINR in the same scenarios, the value of 18.13dB for the macro-only case improves to 25.72dB for the direct links and to 34.94dB for the NCR-forwarded links, achieving gains of $\approx$ 7.60dB and 16.81dB, respectively.
With respect to the 90$^{\text{th}}$ percentile of the SINR in the same scenarios, the value of 46.64dB for the macro-only case improves to 47.74dB for the direct links and to 47.85dB for the NCR-forwarded links, i.e., we have gains of $\approx$ 1dB and 1.20dB, respectively.

As for the UL, Scenario A at the 10$^{\text{th}}$ percentile of SINR CDF, also shown in NCR_LOS_FIG:Simulation-results-sinr-cdf-all and summarized in NCR_LOS_TAB:Results-Percentiles, had an SINR of -1.72dB for the direct links considering the deployment composed only of gNB, while having 1.51dB and 19.50dB for the direct and forwarded links when using NCR.
Therefore, by using NCR, the worst connections are improved by around 3.23dB when connected to the gNB and by 21.22dB when served through the NCR.
For the 50$^{\text{th}}$ percentile of the SINR, the value of 9.65dB for the macro-only case improves to 13.32dB for the direct links and 26.70dB for the forwarded links, achieving gains of $\approx$ 3.70dB and 17.05dB, respectively.
Furthermore, for the 90$^{\text{th}}$ percentile of the SINR, the value of 35.66dB for the macro-only case improves to 36.04dB for the direct links and 39.52dB for the forwarded links, i.e., we have gains of $\approx$ 0.38dB and 3.86dB, respectively.

For Scenario B DL, we have similar trends in terms of SINR at the 10$^{\text{th}}$ percentile with a value of 6.92dB for the direct links in the macro-only deployment, while for the case with NCR we observe 6.05dB for direct links and 22.95dB for the forwarded links.
For the 50$^{\text{th}}$ percentile of the SINR, the value of 18.79dB for the macro-only case improves to 22.05dB for the direct links and 36.81dB for the forwarded links, achieving gains of $\approx$ 3.26dB and 18.02dB, respectively.
In Scenario B, it is perceived a more expressive relative growth at the 90$^{\text{th}}$ percentile of the SINR values between the cases without and with NCR.
We observe 35.50dB for direct links in the macro-only case and 43.76dB and 48.70dB for the direct and forwarded links DL, respectively, in the case with NCR.
The same trend is observed for the cases with UL communication.

While the absolute values of SINR in Scenario B are generally lower than those seen in Scenario A, the relative improvements are higher.
This is because Scenario B has four macros which introduce more interference in the system and lead to lower overall SINR values.
In this more challenging interference scenario, the relative gains that can be brought by NCR are naturally larger.
Notice that for the forwarded links, which remain relatively separated from each other in both Scenarios A and B, the NCR' performance remains similar to each other.
In this way, the presence of NCR improves the QoS experienced by the weak UE significantly, in harmony with the main objectives considered by 3GPP Rel-18.
However, to limit the interference, one needs to determine the proper number of NCR and deploy them with proper planning.
Moreover, due to the lower transmit power of the UE, compared to gNB, the UL direction is the one that benefits the most from the deployment of NCR.

It is interesting to note that we have observed the same qualitative conclusions when considering different MCS (although not demonstrated in the figures).
For the highest MCS, i.e., MCS 15, we have observed a 40% usage for Scenario A without NCR and 70% for the Scenario A with NCR while for Scenario B, we have observed a 30% usage with macro-only and 50% with NCR.
In absolute terms, Scenario A has in general larger usage of the highest MCS than Scenario B, while the relative
increase (fourfold) for the highest MCS usage in Scenario B is larger.

Figure <ref> shows the CDF of the throughput values for Scenarios A and B.
Solid and dashed lines refer to DL and UL, respectively.
The blue and green curves refer to Scenario A, without and with NCR, respectively, while, analogously, the yellow and red curves refer to Scenario B.

CDF of throughput.

As demonstrated in NCR_LOS_FIG:throughput-cdf and NCR_LOS_TAB:Results-Percentiles,
for Scenario A in UL, the 10$^\text{th}$ percentile throughput is 0.10MBits/s and 1.81MBits/s without and with NCR, respectively. The 50$^\text{th}$ percentile throughput is 2.21MBits/s and 3.01MBits/s without and with NCR, respectively, while the 90$^\text{th}$ percentile throughput is 3.15MBits/s and 3.78MBits/s, respectively.
That means an eighteen-fold improvement for the worst links in UL when NCR are used in Scenario A.
That is, the UE with poor direct link to the gNB (for instance, cell-edge UE) benefit most from the presence of NCR, although even the cell-center UE still benefit slightly from the presence of the NCR.

For Scenario B in UL, the 10$^\text{th}$ percentile throughput is 1.87MBits/s and 2.67MBits/s without
and with NCR, respectively. The 50$^\text{th}$ percentile throughput is 3.31MBits/s and 3.69MBits/s without
and with NCR, respectively, while the 90$^\text{th}$ percentile throughput is 4.68MBits/s and 5.08MBits/s, respectively.
The different values of throughput gain between Scenarios A and B are explained by the higher interference level in the latter which strongly impacts the worst links.
On the other hand, since in Scenario B without and with NCR we have larger number of nodes reusing the spectrum, the overall throughput obtained in Scenario B in both DL and UL is larger in absolute values, compared to those obtained in Scenario A (see NCR_LOS_FIG:throughput-cdf).
Note that very high but different SINR values may lead to the same throughput, since this latter is ultimately limited by the capacity of the highest MCS.

§ ON THE CHALLENGES OF NCR-ASSISTED NETWORKS

Although the system-level evaluations show great potential for coverage improvement via the implementation of NCR, there are still challenges to be considered before the NCR can be implemented on a large-scale.
Some of them are listed in the following:

* Lack of testbed evaluations: To the best of our knowledge, there is yet no practical implementation of NCR.
Once NCR are manufactured, testbed evaluations and field measurements can provide better understanding of the potentials and challenges of NCR.

* (Self-)Interference management: Rel-17 repeaters support up to 90dB amplification gains and NCR are expected to support at least the same number.
In such cases, self-interference as well as interference to/from the other nodes can be an issue which requires accurate interference management mechanisms.
Here, network planning and beamforming are expected to reduce the effect of (self-)interference remarkably.
However, in scenarios where the spatial deployment is not able to filter (self-)interference links, (self-)interference effects need to be considered.

* Cost-efficiency tradeoff: Although NCR can improve the network coverage, its performance, in terms of both cost and efficiency, needs to be further evaluated compared to its alternative competitors such as IAB and RIS.
Particularly, while IAB is a more complex node than NCR, it covers a larger area.
Thus, compared to NCR, a fewer number of IAB nodes may be required to cover an area, and the total cost of ownership, on the network level, for these technologies need to be carefully investigated.
On the other hand, NCR is expected to outperform the RIS in terms of performance, with limited or no cost increment (see <cit.> for a conceptual comparison between the RIS and NCR).

§ CONCLUSIONS AND FUTURE PERSPECTIVES

In this paper, we introduced the NCR concept, the main standardization effort on it, as well as its potential and challenges in different deployments.
As shown, employing NCR improves the performance of both direct and forwarded links considering different performance metrics.
Particularly, cell-edge UE and UL communications are the ones with the highest benefit from the presence of NCR in mmWave networks.
Furthermore, the results show the importance of network planning where the relative performance gains achievable using NCR depend on the number and positions of these nodes selected for deployment.
Depending on the number of nodes, resource reuse and interference levels, the introduction of NCR may result in UL throughput gain of up to ten times for the cell-edge UE.

§ ACKNOWLEDGMENTS

This work was supported by Ericsson Research, Sweden, and Ericsson Innovation Center, Brazil, under UFC.51 Technical Cooperation Contract Ericsson/UFC. The work of Victor F. Monteiro was supported by CNPq under Grant 308267/2022-2. The work of Tarcisio F. Maciel was supported by CNPq under Grant 312471/2021-1. The work of Francisco R. M. Lima was supported by FUNCAP (edital BPI) under Grant BP4-0172-00245.01.00/20.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

3rd Generation Partnership Project (3GPP)

titleNR; Integrated access and backhaul radio transmission and Reception

journaltitleIEEE Open J. Commu. Soc.
titlePaving the Way Towards Mobile IAB: Problems, Solutions and Challenges

journaltitleIEEE Open J. Commu. Soc.
titleOn Integrated Access and Backhaul Networks: Current Status and Potentials

3rd Generation Partnership Project (3GPP)

titleNR; NR Repeater Radio Transmission and Reception

journaltitleIEEE Trans. Antennas Propag.
titleTowards a Heterogeneous Smart Electromagnetic Environment for Millimeter-Wave Communications: An Industrial Viewpoint

Turin, Italy

booktitleProc. of the Internat. Symp. on Modeling and Opt. in Mobile, Ad Hoc, and Wireless Netw. (WiOPT)
titleTowards Reliable mmWave 6G RAN: Reconfigurable Surfaces, Smart Repeaters, or Both?

titleA Comparison between Network-Controlled Repeaters and Reconfigurable Intelligent Surfaces

3rd Generation Partnership Project (3GPP)

noteTSG RAN meeting no. 97-e
titleNew WID on NR network-controlled repeaters

3rd Generation Partnership Project (3GPP)

titleStudy on NR network-controlled repeaters

3rd Generation Partnership Project (3GPP)

title5G;NR;User Equipment (UE) radio access capabilities

3rd Generation Partnership Project (3GPP)

titleNR; Radio resource control (RRC) protocol specification

titleSimulation guidelines

journaltitleIEEE Access
titleA Stochastic Channel Model With Dual Mobility for 5G Massive Networks

3rd Generation Partnership Project (3GPP)

titleStudy on Channel Model for Frequencies from 0.5 to 100 GHz

3rd Generation Partnership Project (3GPP)

titleStudy on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR

§ BIOGRAPHIES

* Fco. Italo G. Carvalho<EMAIL_ADDRESS>is an undergraduate student at the Federal University of Ceará (UFC), Fortaleza, Brazil and is with the Wireless Telecommunications Research Group (GTEL), UFC.

* Raul Victor de O. Paiva<EMAIL_ADDRESS>is a PhD student at UFC and is with GTEL.

* Tarcisio F. Maciel<EMAIL_ADDRESS>is a Professor at UFC and is with the GTEL. He is interested on multiuser/multiantenna communications.

* Victor F. Monteiro<EMAIL_ADDRESS>is a Professor at UFC and is with the GTEL. He is focused on network architecture and protocols.

* Fco. Rafael M. Lima<EMAIL_ADDRESS>is a Professor at UFC and is with the GTEL. He  has focused on radio resource management.

* Darlan C. Moreira<EMAIL_ADDRESS>is a senior researcher at GTEL focusing on simulation programming.

* Diego A. Sousa<EMAIL_ADDRESS>is a Professor at the Federal Institute of Education, Science, and Technology of Ceará (IFCE), Paracuru, Brazil and is with the GTEL. He has focused on radio resource management.

* Behrooz Makki<EMAIL_ADDRESS>is a Senior Researcher in Ericsson, focusing on different aspects of network densification.

* Magnus Åstr$\ddot{\text{o}}$m<EMAIL_ADDRESS>is a Research Leader in Ericsson, developing different aspects of 5G and beyond.

* Lei Bao<EMAIL_ADDRESS>is a Senior Researcher in Ericsson, focusing on wireless backhaul.


# EJRA_Critique

Oliver Linton, Raghavendra Rau, Patrick Baert, Peter Bossaerts,
Jon Crowcroft, G.R. Evans, Paul Ewart, Nick Gay,

Paul Kattuman, Stefan Scholtes, Hamid Sabourian, and Richard J. Smith
Universities of Cambridge and Oxford

(May 2024)

# Is the EJRA proportionate and therefore justified? A critical review of the
EJRA policy at Cambridge

Oliver Linton, Raghavendra Rau, Patrick Baert, Peter Bossaerts,
Jon Crowcroft, G.R. Evans, Paul Ewart, Nick Gay,

Paul Kattuman, Stefan Scholtes, Hamid Sabourian, and Richard J. Smith
Universities of Cambridge and Oxford

(May 2024)

This paper critically evaluates the HESA (Higher Education Statistics Agency)
Data Report for the Employer Justified Retirement Age (EJRA) Review Group at
the University of Cambridge ([1]), identifying significant methodological
flaws and misinterpretations. Our analysis reveals issues such as application
of data filters only to the Cambridge sample, inconsistent variable treatment,
and erroneous statistical conclusions. The Report suggests EJRA increased job
creation rates at Cambridge, but we show Cambridge consistently had lower job
creation rates for Established Academic Careers compared to other Russell
Group universities, both before and after EJRA implementation in 2011. This
suggests that EJRA is not a significant factor driving job creation rates.
Since other universities without an EJRA exhibit higher job creation rates,
this suggests job creation can be sustained without such a policy. We conclude
that the EJRA did not achieve its intended goal of increasing opportunities
for young academics and may have exacerbated existing disparities compared to
other leading universities. We recommend EJRA be abolished at Cambridge since
it does not meet its justified aims and could be viewed as unlawful age
discrimination.

_Keywords_ : Discrimination, Job Creation; Retirement; Treatment Effect,
Universities

_JEL Classification_ : C12, C21, C23, J21, J26, J63, J83.

##  1 Introduction

The University of Cambridge in the United Kingdom employs an Employer
Justified Retirement Age (EJRA) policy, mandating retirement for all academic
and academic-related staff at the end of the academic year in which they turn
67. Following the abolition of the Default Retirement Age (DRA) by the
Coalition government in 2011 following on from the Equality Act (2010),
employers were permitted to impose mandatory retirement only on the condition
that it could be objectively justified as a proportional means of achieving a
legitimate social policy aim. Implemented in 2012, after consultation with its
staff, the Cambridge EJRA has four stated goals:

  * •

To ensure inter-generational fairness and career progression;

  * •

To enable effective succession-planning;

  * •

To promote innovation in research and knowledge creation; and

  * •

To preserve academic autonomy and freedom.

While the twenty two other Russell Group universities in the U.K. did not
choose to introduce an EJRA in 2011, Cambridge and Oxford universities did so,
setting them apart from the other Russell Group universities (and indeed other
employers).

The effectiveness of the EJRA in achieving its aims is predicated upon its
effect upon the rate at which vacancies arise from retirement and other
causes. Estimates of the effect of the EJRA on the vacancy creation rate (VCR)
may be made using models employing simplifying assumptions. A simple model,
presented by Paul Ewart during debates in Congregation at Oxford111See
https://tinyurl.com/yc68vksc, page 443., assumes a fixed number of posts
subject to the EJRA and a uniform age distribution i.e. equal numbers of staff
in each year group by age. (The Data Report confirms that this reflects the
age distribution also at Cambridge.)

Then, initially assuming that vacancies arise only by retirements at the EJRA
when a cohort reaches the retirement age and are filled by younger academics
at the average age of appointment, the process can be viewed as a queue where
the numbers joining equal the numbers leaving by retirement. If the EJRA was
abolished, then the average age at which people retired would increase, at
most, by about 3 years, according to survey data. In the steady state, the
length of the queue would increase, but the number in each age group would
decrease correspondingly to maintain the same number of staff. Hence the
number of vacancies each year would decrease by the same amount.

The effect of an EJRA then is to increase the Vacancy Creation rate (VCR)
relative to the situation with no mandatory retirement. It is on this basis
that the University seeks to justify its EJRA policy as a means to achieving
its stated aims. However, since the policy discriminates on the basis of a
protected characteristic, age, it is unlawful unless it can be objectively
justified as a proportionate means of achieving those aims. Using this simple
“queuing” model, it is possible to estimate the effect of the EJRA on the VCR
as a measure of its proportionality.

This article critically evaluates the Data Report, identifying significant
methodological flaws and misinterpretations. The Data Report suggests that
EJRA at the University of Cambridge increased job creation rates. Our analysis
reveals issues such as inappropriate data filters, inconsistent variable
treatment, and erroneous statistical conclusions. We show that Cambridge
consistently had lower job (vacancy) creation rates for Established Academic
Careers (EAC) compared to other Russell Group universities, both before and
after the EJRA implementation, implying that EJRA is not a significant factor
driving job creation rates. Since other Russell Group universities, which do
not have an EJRA, exhibit higher job creation rates, this suggests that job
creation can be sustained or even thrive without such a policy.

##  2 Qualitative assessment of the EJRA

Using the data quoted in the Data Report, the average age at appointment of 40
and retirement at the EJRA at 67 gives an average career length of 27 years.
Assuming, on the basis of survey and other data, that on average people will
work for a further 3 years, the average career length will increase by 3/27
i.e. 1/9 or about 11% and the VCR will decrease by a corresponding proportion.
Therefore, for a fixed number of posts, the EJRA increases the VCR by about
11%. However, as confirmed by the Data Report, at least 50% of vacancies arise
for reasons other than retirement, so the effect of the EJRA on the VCR is
reduced to 5.5%. In addition, about 50% of those reaching the EJRA will retire
voluntarily and so the relative effect of the EJRA on the VCR is reduced still
further to about 2.75%.

The Ewart Employment Tribunal judged that an increase in vacancy creation rate
of between 2 – 4% was “trivial” and not a proportionate means of achieving the
aims.222See https://tinyurl.com/2ywaft57 This finding was upheld by the
Employment Appeal Tribunal333See https://tinyurl.com/3eea3bkm and was further
confirmed in a recent ET cases (2023) involving four claimants at Oxford
University.444See https://tinyurl.com/447ecv5a, https://tinyurl.com/5ahknzkk,
https://tinyurl.com/5efvjakv, and https://tinyurl.com/3zxt2kd7 It was
acknowledged by Oxford University’s statistical analysis in these recent ET
tribunals that the modelling method presented by Ewart “gave similar results
to the methodology based on Little’s Law of queuing, once the correct
assumptions were used.”555Witness statement of Malgorzata Turner in Employment
Tribunals referred to in footnote 4 above.

It is worth noting that the recommendation of the Review Report that the EJRA
should be changed to set at 69 instead of 67 will result in the policy being
even less proportionate. This is because the average extension of 3 years will
be to a career of 29 years, rather than 27 years, a percentage reduction of
the VCR due to retirements alone from 11% (3/27) to 10% (3/29) and an overall
reduction in the VCR of 2.5%. However, if people continue to retire at the
current average age, then the extension will be only 1 year past 69, giving a
career extension of 1/29 or 3% and an overall reduction of the VCR of 0.75%
when vacancies for other reasons and voluntary retirements at the EJRA are
taken into account.

The “steady state” conditions of this simple model do not take account of the
transient effect induced by a sudden abolition of the EJRA or a change in the
age from say 67 to 69. In the latter case, and using the same assumption of a
uniform age distribution, there would be no vacancies for two years after
which the situation would evolve into the steady state over time. This effect
is treated in a dynamic modelling approach using Little’s Law that would also
have to take account of the vacancies created by reasons other than retirement
and the average duration of extensions of employment post-EJRA.

It is clear, however, given the similarity in the situations at Oxford and
Cambridge; same age at appointment of 40 and 50% of vacancies arising from
reasons other than retirement etc., one expects that the increase the vacancy
creation rate by the EJRA at Cambridge will also be in the range 2 – 4%. Any
significant deviation from this estimate would suggest that some filtering of
the statistics data has been made to change the assumptions built into the
model.

We next turn to the specific goals of the EJRA and examine whether the EJRA is
a proportionate means of achieving those goals. Recall that the EJRA sets four
goals:

Promoting Innovation and Knowledge Creation

The EJRA is predicated on an antiquated university model with a fixed number
of professorships, necessitating one professor’s departure for another’s
ascension. This premise aligns with the “lump of labor fallacy” – the
erroneous belief that there is a fixed amount of work to be distributed,
leading to the misconception that increased productivity causes unemployment.
However, this notion is fallacious ([2]). Arguing against productivity
enhancements due to potential job losses is irrational. The creation of new
professorships is contingent on funding availability and academic interest,
not vacancy generation through retirements. Hiring new faculty members can
potentially stimulate novel research avenues and ways of thinking, fostering
innovation and knowledge creation, rather than merely replacing outgoing
scholars. In addition, the senior outgoing scholars are likely able to
generate significant amounts of funds, all of which are lost to the
university.

Enabling Effective Succession Planning It is acknowledged that at least 50% of
vacancies arise before the mandatory retirement age and these are managed
successfully, even if they do not occur at predictable times. It is therefore
difficult to imagine that vacancies arising from voluntary retirement could
not be equally well managed. Indeed, it is likely that academics wishing to
retire would be willing and capable of giving adequate prior notice of their
intention to retire and so mitigate any potential issues related to succession
planning.

The EJRA hinders the University’s ability to attract senior external
candidates, as the prospect of mandatory retirement within a few years can
deter potential applicants. Moreover, the policy impedes academic productivity
well before retirement, as academics may be unable to supervise doctoral
students or apply for grants at least five years prior to the retirement age
due to the impending termination of their employment. Consequently, productive
senior academics often seek employment elsewhere well before reaching the
EJRA, resulting in a detrimental brain drain of highly sought-after and
accomplished scholars from the University. This has the impact of reducing
predictability since senior academics seeking positions elsewhere will be
unlikely to notify the university ahead of time that they are actively looking
elsewhere.

Ensuring Intergenerational Fairness and Career Progression

It is improbable that junior academics at Cambridge directly benefit from the
forced retirement of their senior colleagues. Senior positions are advertised
globally, and the potential for internal promotions is often outweighed by
external applicants. Thus, even if the model underpinning the Review Report is
sound (which it clearly isn’t), the intergenerational fairness argument would
not apply within the University context but rather on a global scale, where
Cambridge’s retirees ostensibly create opportunities for younger academics
worldwide. Needless to say, this would only be a drop in the ocean and by no
means a proportionate measure. Curiously, this rationale does not extend to
administrative positions, which are exempt from the EJRA. Additionally, the
policy disproportionately disadvantages women, whose career trajectories are
often prolonged due to family obligations, resulting in a shorter timeframe
before reaching the retirement age after achieving career milestones. In
figure 1, the university argues that the EJRA policy promoted in the HESA
review increases the lectureship opportunities available to post-doctoral
colleagues ([5]).

Figure 1: University website for postdoc students at Cambridge: Retrieved 20
May 2024

Since the Data Report suggests increasing the retirement age by two years to
69, it is difficult to understand how lectureship opportunities will be
increased. One possibility is that the University plans on increasing the
number of senior positions such as professorships. Unfortunately, a simple
graph of the number of senior professorships at Cambridge, depicted in figure
2 shows a very sharp decline in these positions after the EJRA period in
Cambridge.

Figure 2: Number of professorships created at Cambridge (15 year trailing
average)

Preserving Academic Autonomy and Freedom Finally, the EJRA also undermines
academic autonomy and freedom. To secure an extension beyond the retirement
age, academics must demonstrate productivity, which may incentivise risk-
averse behavior and incremental research rather than pursuing potentially
groundbreaking but uncertain avenues. This dynamic has the potential of
stifling innovation and knowledge creation, which should be a core objective
of a leading research university.

In summary, the arguments highlight the potential adverse impacts of the EJRA
on innovation, succession planning, intergenerational fairness, and academic
freedom, contending that the policy may be counterproductive to the
University’s stated aims and could hinder its pursuit of excellence in
research and education.

##  3 Quantitative assessment of the Data Report

The Data Report sought to establish that EJRA has a beneficial effect on
vacancy creation based on historical data and statistical techniques. In
general terms and when properly applied, the method of analysis (difference-
in-differences) used is capable of measuring the impact of a policy by
comparing changes, from before to after the policy change, between a group
that experienced the policy and a group that did not. In the analysis of
whether the removal of EJRA at Cambridge would affect job creation, one group
was the RRG (Remaining Russell Group) where EJRA was removed, and the other
group comprised solely of Cambridge where an EJRA was introduced so that
mandatory retirement continued, and the difference-in-differences calculation
was applied to vacancy creation rates in the RRG and in Cambridge, before and
after EJRA removal in the RRG. This section shows that, contrary to sound
practice, Cambridge’s data was selectively filtered while other universities’
data were not treated the same way. The analysis as applied did not adequately
control for pre-existing trends, or for differences between universities. In
fact, Cambridge consistently had lower vacancy creation rates compared to
other universities, both before and after EJRA removal in the RRG. It is shown
below that the conclusion that the removal of the EJRA policy would
significantly impact the vacancy creation rate in Cambridge simply does not
follow from the analysis.

The second part of the Data Report uses a system dynamics model to simulate
forward what would be the likely consequences of the abolition of the EJRA at
Cambridge in terms of vacancies. In the second part of this section, we point
out significant limitations in that analysis as well.

###  3.1 Effect of EJRA on Job Creation Rate

The authors of the Data Report obtained data from HESA on the number of
employees in different categories for each year from 2007-2008 academic year
to 2021-2022 academic year for each of the 24 Russell Group universities
including Cambridge and Oxford. Oxford is excluded from the analysis with a
comparison made only between Cambridge and RRG. The authors describe on page 7
how they applied various “filters” to the HESA data using Cambridge HR data to
construct the group that is eligible for EJRA, which was not identified from
the HESA data alone. They did not do and could not do such a data editing for
the RRG universities and so it is not clear that a fair comparison was made.
To characterize this choice, if an RRG university wanted to introduce an EJRA,
would they also apply it only to a subset of the individuals recorded in the
HESA dataset?

The Data Report examines two classes of staff at Cambridge: Established
Academic staff (EAC) and Established Academic-related staff (EAR). The EAC
group includes academic staff whose primary function is teaching and/or
research. Specifically, the Data Report defines EAC as employees with:

  * •

Academic employment function coded as 1 (Teaching Only), 3 (Teaching and
Research), or 9 (Neither, e.g. Vice-Chancellor)

  * •

Salary point 38 and above

  * •

Full-time or full-time term-time only employment mode

  * •

Open-ended/permanent terms of employment

The EAR group includes non-academic professional staff supporting teaching and
research. The Data Report defines EAR as employees with:

  * •

Academic employment function coded as 4 (Not an academic contract) or X (for
data before 2011/12)

  * •

Salary levels point 33 and above

  * •

Wholly or partly financed by the higher education provider

  * •

Open-ended/permanent terms of employment

In effect, the EAC group represents the core group of established academic
teaching and research staff, while the EAR group covers the non-academic
professional support staff, both at relatively senior levels within the
university. The purpose of the Data Report is to show that the university
created a significantly higher number of jobs in the Cambridge EAC group than
would have arisen if Cambridge did not have an EJRA by comparing with other
universities who do not operate an EJRA.

The main variable of interest is defined as

$r_{it}^{jc}=\frac{N_{it}^{jc}}{N_{it}},$ (1)

where $N_{it}$ is the total employees in a given group (for example,
university $i)$ at year (actually academic year) $t,$ while $N_{it}^{jc}$ is
the total number of new appointees at university $i$ in year $t$ (presumably
new appointees in the specific group). This is the measure of job creation
that is identified as the outcome of the EJRA policy.

This variable is shown in Figure A1 for both the EAC and EAR groups (see
figure 3 reproduced below).

Figure 3: Figure A1 reproduced from the Data Report ([1])

Figure 3 further breaks this down to EAC STEM and EAC non STEM groups. These
figures show the time series of the 22 RRG universities and Cambridge over the
period 2007-2008 to 2021-2022 (a period of 15 years) along with the average
across RRG and a vertical line indicating 2012, the date from which the
abolition of retirement age at RRG is binding. Regardless of the figure
examined, the key feature is that Cambridge has had a lower job creation rate
in comparison with every single other RRG university. This is acknowledged on
page 8 of the Data Report, “UoC has a lower job creation rate for EAC compared
to most other HEPs (Higher Education Providers). “Figure A1 shows the average
job creation rate is 3% for UoC between AY 2007/08 and 2021/22, while the
average for RRG is 5%.” In one sense, the reader can stop here since the raw
data clearly indicates a result completely divergent from its core message
that the EJRA has created significantly more jobs at Cambridge. However, there
are potential flaws in this interpretation that suggest we look further.

Specifically, in the Data Report they argue that these numbers are the average
over all fifteen years and it is plausible that Cambridge has improved its
performance relatively since 2011, post the introduction of EJRA. A simple way
of capturing this pattern would be to compare the average (over time) of
Cambridge performance after 2011 with the average of the RRG universities.
Unfortunately, looking at Figure 3, it is clear that Cambridge would lose that
contest too: In every single year, Cambridge created fewer EAC jobs than the
average of the RRG universities and in every year, it has been in the bottom
5.

Yet another possibility is that something else is to blame - perhaps for
example, there were other factors, presumably macroeconomic, that affected job
creation over this period. In that case, what we need to do is to compare the
average performance of Cambridge post 2011 with the average performance of
Cambridge pre-2011 and do the same for the RRG universities. This is the heart
of why their argument appears to work since one can see that just prior to
2012 the RRG universities created a lot of new positions and their performance
subsequent to 2012 was relatively worse. Actually, the key feature of the
pre-2012 data was the big dip following the 2008 Global Financial Crisis (GFC)
and the strong rebound, especially for RRG universities, just running up to
2012.

The main approach they use is a so-called natural experiment, which is
intended to capture how clinical trials are carried out. The treatment or
policy variable is the introduction of the EJRA by Cambridge in 2011, and the
treatment group is Cambridge with pre-treatment period 2007/2008-2011/2012 and
post treatment period 2012/2013 (it takes a year for the abolition of
retirement to take effect) to 2021/2022. The control group is the RRG
universities. It is important to note that this design addresses the issue
about abolition of EJRA as it compares universities without EJRA with
Cambridge that does. It does not provide any evidence about the effect of
raising the age of EJRA to 69 from 67. As outlined in the introduction above,
increasing the age of the EJRA results in its being even less proportionate as
a means of achieving the aims.

The main issues for application of this method here are:

  1. 1.

Treatment is not randomly assigned as in a clinical trial. Cambridge, the
authors themselves argue (and the authors of the Oxford EJRA argued before
them), is different and those differences lead them to choose the EJRA policy
in the first place. Fundamentally, one cannot separate out whether the
difference between Cambridge and the RRG universities is due to the EJRA or
due to the fundamental difference already present.

  2. 2.

The treatment is really 2012 itself, not the introduction of the EJRA (or
rather the division is into the pre- and post-2012 periods). The EJRA was not
the only thing that changed after 2012 compared to before 2012 and that could
influence vacancy rates.

  3. 3.

The sample sizes are too small. There is only one treated patient, Cambridge,
and only 22 untreated patients, the RRG. Imagine testing a vaccine on one
patient and applying a placebo to twenty two. Pfizer’s Covid-19 vaccine
clinical trial had over 46,000 participants. The statistics here all rely on
large sample approximations, and here the smallest group average is made from
5 time series observations.

  4. 4.

The unit of measurement is at the university level. In a clinical trial, one
is looking at individuals and it is reasonable to assume that although they
are different in fundamental ways, their health outcomes are essentially
independent across individuals and only depends on the treatment. It is hard
to believe that is the case for university’s economic decisions, which is what
is needed here. For example, all the universities are admitting undergraduates
through UCAS and they are all trying to recruit from the same pool of
academics.

In conclusion, the methods adopted are not appropriate for the data they have.
The results they obtained are not credible. The fundamental fact is that
Cambridge creates fewer new academic jobs in comparison with RRG whether or
not they both had a retirement age. This is clear from Figure A1.

Figure A30 from the Data Report (reproduced in Figure 4) gives their estimates
of the treatment effect separately for each year along with confidence bands.
This is defined as the difference in job creation between Cambridge and RRG
relative to the same difference for 2012 (the zero value for 2012 is
suppressed from the figure). According to this graph, the introduction of EJRA
had a positive relative effect even five years before it was introduced! In
the language of natural experiments, this evidences that the pre treatment
parallel trend assumption is violated, which they acknowledge, and which leads
them to implement a further couple of inappropriate methods for dealing with
this issue. In the Appendix we give a more formal treatment of the statistical
issues. Another interesting aspect of this figure is that there is a declining
trend in job creation rates in the post-Brexit period (from 2017 onwards)
interrupted by a spike upwards during the Covid year of 2020. Even taking
their methods at face value, by 2021, the effect has totally disappeared, so
whatever was driving the results has vanished. Another interpretation of this
data is just that, as in any time series, there are periods where the series
is above trend and periods where it is below trend and these cycles are
determined by a complex set of reasons that have not been investigated in this
report.

Figure 4: Figure A30 reproduced from HESA Data Report ([1])

###  3.2 Future Effects of the EJRA Policy

The Data Report forecasts the future effects of a policy change based on [3]
that builds on Little’s law of queuing ([4]) to study the effect of retirement
policy at MIT. This is a more detailed model than that used by Ewart and
outlined above and essentially says that in a stationary system, the long run
average value of a stock variable (total number of employees) is equal to the
product of the long term average net arrival rate to the system and the
average residence time in the system. Abolishing the EJRA, for example, would
lengthen the steady state time in the system for each employee and would
therefore imply, if the total number of employees is held fixed, that the
number of vacancies and new hires would fall. The system would take time to
adjust to the new long run equilibrium, the so-called steady state. To build
this model, one has to estimate the exit rates due to lateral moves, death,
and retirement in the absence of a mandate, which is done from the average
values observed in RRG over the 15 years of HESA data.

One implication of this model would be that the statistical model in (2) is
misspecified, since it implies that there is a constant effect on vacancies
from the EJRA whereas the queuing model says that there is a dynamic response
that takes time to take full effect. Secondly, the queuing model is treated as
exact, whereas the key parameters are uncertain and estimated from data. In
that respect, the estimates reported in the Data Report are subject to wide
variability as is any social science phenomenon you are trying to forecast 40
years hence.

The model is also flawed in treating everyone in the queue as the same,
failing to account for the differing contributions and income levels generated
by faculty at different career stages. Senior faculty who stay on the payroll
contribute significantly to the Research Excellence Framework (REF) returns,
which in turn bring in pro-rata funding from central sources. Senior faculty
tend to attract more PhD students due to their track records and projects.
Their research funding attracts overheads, which are typically calculated as a
percentage, resulting in more funding for more senior faculty. The Higher
Education Quality Enhancement (HEQE) Quality-Related (QR) funding is also
based on the actual cost of the faculty member, not a generic FTE. The model
errs by taking a “unit person/Full-Time Equivalent” approach. The correct
approach would consider an FTE at the typical income level that a faculty
member at that career stage generates.

The results are reported in Table 1 of the Data Report. In the first ten
years, there are on average 27.7 fewer new appointments. In the decade from
2053-2062 there are 13.3 fewer vacancies; which is apparently the chosen
steady state regime. To put this in perspective, consider Table 3.1, which
shows the total number of established academic staff at Cambridge over the
period 2013-2023. The fourth column shows that in 2023 a total of 33 members
of academic staff left office at the end of their EJRA year, ostensibly due to
the EJRA itself, rather than to their own personal preferences. However, from
2022 to 2023, the total number of established staff grew from 1642 to 1669, a
net addition of 27. In fact, since 2013, the number of established staff has
grown by approximately 1% per year. If we extrapolate that trend in
establishment size to 2053/2062, the loss in new vacancies would be more than
matched by the the number of new positions every year. One final point to bear
in mind when looking at the long term steady state, the State pension age is
set to increase to 69 in 2046-2048 due to the increase in longevity.

##  4 Conclusion

This paper critically evaluates the findings of the Data Report for the EJRA
Review Group, identifying crucial methodological flaws and misinterpretations.
The Data Report suggests that the Employer Justified Retirement Age (EJRA) at
the University of Cambridge increased job creation rates and contributed
positively to the academic workforce. Our analysis reveals several key issues,
including inappropriate filters applied to the data, inconsistent treatment of
variables, and erroneous conclusions drawn from statistical models.

The Data Report applies more stringent filters to Cambridge data compared to
other Russell Group universities (RRG). Certain categories of staff are
included in the Cambridge analysis but excluded from the RRG analysis, leading
to inappropriate comparisons. Relevant data on job creation and retirements
are excluded without clear justification, potentially deflating or inflating
the observed job creation rates. Furthermore, the job creation rate is defined
differently for Cambridge and the RRG, resulting in incommensurate
comparisons.

The Data Report also contains inconsistencies in how variables are treated and
measured. The time periods analysed for Cambridge were not directly comparable
to those for the RRG, with trends and changes post-2011 not uniformly analysed
across all institutions. Key variables such as retirement age and job creation
rates are defined and calculated differently, failing to account for
structural differences between institutions. Additionally, the Data Report
does not uniformly adjust for macroeconomic factors and other external
influences, leading to an inaccurate representation of EJRA’s effect.

Importantly, the Difference-in-Differences model used in the Data Report does
not adequately control for pre-existing trends and differences between
Cambridge and the RRG, resulting in unjustifiable findings on EJRA’s impact on
job creation. The Data Report’s claim that EJRA led to a 1.6 percentage point
increase in job creation does not hold under proper controls and comparisons.
The dynamic model used is inconsistent with the static regression approach,
assuming a constant effect over time and ignoring potential variable
interactions. The Data Report’s results lack robustness checks, confidence
intervals, and measures of uncertainty, with statistical significance
disappearing when alternative methods (such as the wild bootstrap) were
applied, indicating unreliable conclusions.

Our analysis shows that Cambridge University consistently had lower job
creation rates for Established Academic Careers (EAC) compared to the RRG
average, both before and after EJRA implementation in 2011. Cambridge’s
average job creation rate was approximately 3% per year, while the RRG average
was about 5%. There was no relative improvement in job creation rates at
Cambridge post-EJRA implementation. Every year since 2011, Cambridge has
created fewer EAC jobs than the RRG average, contradicting the Data Report’s
claim of a positive impact.

Overall, we can conclude that the EJRA policy did not achieve its intended
goal of increasing job creation for young academics and may have exacerbated
existing disparities in job creation rates compared to other leading
universities. Cambridge consistently had lower job creation rates for
Established Academic Careers (EAC) compared to other Russell Group
universities, both before and after the EJRA implementation, implying that
EJRA is not a significant factor driving job creation rates. Other Russell
Group universities, which do not have an EJRA, exhibit higher job creation
rates, suggesting that job creation can be sustained or even thrive without
such a policy.

We are grateful to Simon Baron-Cohen for helpful comments.

Dedication. We would like to dedicate this paper to the memory of Ross
Anderson who devoted much of his tireless energy to combatting the spurious
arguments put forth in favour of the EJRA at Cambridge.

##  5 Appendix

The main model that is used for establishing a significant effect is presented
in Section 3.1 of the Data Report, the first displayed equation. We can
rewrite this equation in more standard notation as

$r_{it}^{jc}=\alpha_{i}+\gamma_{t}+\delta D_{it}+\varepsilon_{it},$ (2)

The variable $D_{it}$ is the treatment dummy, equal to one for Cambridge
post-2012 and equal to zero for Cambridge pre-2012, and equal to zero for all
the RRG universities from 2007-2008 to 2021-2022. This approach is called a
reversal design and it is used since the retirement policy at Cambridge did
not change, while it did at the RRG (it was abolished).

The parameter of interest is $\delta$, which captures the effect of EJRA on
job creation (which is implicitly constant across time). The statistical null
hypothesis is that $\delta=0$, i.e., that the EJRA had no effect on job
creation versus the general alternative. Table A.4 column 1 reports the
results of this regression separately for EAC and EAR groups.

The authors compute the so-called difference in differences

$\widehat{\delta}=\left(\overline{Cam}_{post}-\overline{Cam}_{pre}\right)-\left(\overline{\overline{RRG}}_{post}-\overline{\overline{RRG}}_{pre}\right),$
(3)

where the overbar indicates averages over time and the double bar indicates
averages over universities in RRG. It is important to note that there are only
five observations in $\overline{Cam}_{pre}$ and ten observations in
$\overline{Cam}_{post}$. In Figure A30, they report

$\widehat{\delta}_{t}=Cam_{t}-{Cam}_{2012}-(\overline{RRG}_{t}-\overline{RRG}_{2012}).$
(4)

In this case there is only one observation in $Cam_{t}$. By comparing with
2012, they carefully use the point of maximum difference to pivot around. A
more standard approach would be to compare with the pre-policy average
$\overline{Cam}_{pre}-\overline{\overline{RRG}}_{pre}$.

The authors’ main estimates in column 1 of Table A.4. indicate that $\delta$
is positive and statistically significant under their calculated standard
errors. However, when they apply the wild bootstrap standard errors in Section
3.4, they report that the statistical significance vanishes. The authors
acknowledge that the usual parallel trends assumption is violated in this
data, as is clear from Figure A1 and A30. This would imply the estimates in
the first column of Table A.4 are suspect. To address this issue, they do the
following described on page 9. They fit the regression model

$r_{it}^{jc}=a_{i}+b_{i}t+e_{it},$ (5)

for each university using the data before 2012 (5 observations) to obtain
$\widehat{a}_{i},\widehat{b}_{i}$ and then calculate the residual
$\widehat{r}_{it}^{jc}=r_{it}^{jc}-\widehat{a}_{i}-\widehat{b}_{i}t$ for all
$i,t.$ Then, they take $\widehat{r}_{it}^{jc}$ as the dependent variable in
(2).

This is absurd. To understand why, imagine constructing a linear trend from
five observations on any noisy time series and then extrapolating a further
ten observations based on that trend. This is extremely bad practice from a
time series point of view, and any public body such as the Bank of England
could not argue policy on the basis of such evidence. In Figure A3, one can
see two RRG universities have rather extreme boosts in job creation followed
by a sharp reversal in one case and slower reversal in the other case, all
before 2012. So whatever led them to increase job creation and decrease job
creation could be repeated at any time after 2012. Logically, if you
extrapolated the trend backwards, it appears that for some universities in
2000, the job creation would be negative. Hence the foundation of this
pretrend assumption/method is very weak. Mechanically, however, the result of
this process is to drag down the RRG values as those fake pretrends did not
continue and this gives a boost to the estimated treatment effect. The
treatment effect here is to be interpreted relative to a pre-established
trend, so the figure of 4.408 in Table A.4 column 2 is not directly comparable
with the result from column 1.

The second method they adopt to address the pre 2012 “trend” is based on the
regression

$r_{it}^{jc}=\alpha+\beta s_{it}+u_{it},$ (6)

where $s_{it}$ is the log of FTE student numbers at university $i$ in year
$t$. In this case, the $\beta$ is assumed constant over universities and again
estimated using the pre-2011 data. They then calculate
$\widetilde{r}_{it}^{jc}=r_{it}^{jc}-\widehat{\beta}s_{it}$ for all $i,t$ and
then apply the DID to this residual.

In this specification, it is odd to use the log of student numbers to predict
the ratio of new hires. If we assume that the staff-student ratio is targeted
to be constant we would have (where $N$ is the number faculty and $S$ is
student numbers):

$0=d\left(\frac{N}{S}\right)=\frac{dN}{S}-\frac{N}{S}\frac{dS}{S}=\frac{N}{S}\left(\frac{dN}{N}-\frac{dS}{S}\right),$
(7)

so that $dN/N$ should be equal to $d\log S$, not $\beta\log S.$ Empirically,
$s=\log S$ is more related to $\log N$ than $d\log N$. It is also surprising
that in this case, they do not allow $\beta$ to vary with university, perhaps
because the result would have been different had they done that. The estimates
from this equation are not credible either. Interestingly, in column 3 of
Table A.4 they show that for this method the EJRA for EAR staff produces a
significant increase in vacancies.

The inference methods are “robust standard errors” in the main part and a wild
bootstrap method in Section 3.4. The standard errors in Table A.4 effectively
assume that $\varepsilon_{it}$ is independent across $i$ and $t,$ that is,
after accounting for the “main effects” $\alpha_{i}$ (university specific
effect) and $\gamma_{t}$ (time specific effect), the remaining movements are
uncorrelated. This is clearly a bad assumption in this case as the
universities in the RRG are not separate individuals whose health outcomes do
not affect others’ outcomes but directly competing institutions who are
subject to the same rules and procedures (apart from EJRA). The pre-trend
estimation is not taken into account of in the standard errors. Instead, the
authors just supply the residuals to STATA and get the usual standard errors
that would arise if there were no pre-estimation. Clearly, estimating two
parameters from five observations is pushing the boundaries of credibility.
Credible standard errors would be much larger than those presented. Indeed, as
acknowledged in Section 3.4 (robustness checks), when the wild bootstrap is
used (this takes account of heteroscedasticity but not cross university and
cross time correlations), the statistical significance disappears. Likewise,
when a synthetic control method is used, that is, finding a linear combination
of RRG that best matches Cambridge in the pre treatment period with respect to
vacancy rate, they report that the statistical significance disappears.

## References

  * [1] University Cambridge  “HESA Data Report for EJRA Review Group” Confidential Report, 2024
  * [2] Paul Krugman  “Trump, Immigration and the Lump of Labor Fallacy” The New York Times, 2024  URL: ttps://www.nytimes.com/2024/02/06/opinion/trump-immigration-lump-of-labor.html
  * [3] R. Larson and M. Diaz  “Nonfixed Retirement Age for University Professors: Modeling Its Effects on New Faculty Hires”  In _Service Science_ 4, 2012, pp. 69–78
  * [4] John D.. Little  “A Proof for the Queuing Formula: L= $\lambda$,W”  In _Operations Research_ 9.3, 1961, pp. 383–387  URL: http://www.jstor.org/stable/167570
  * [5] University of Cambridge  “Postdoc Academy” Accessed: 2024-05-20, https://www.postdocacademy.cam.ac.uk/, 2024


# Analytic Optimization-Based Microbubble Tracking in
Ultrasound Super-Resolution Microscopy

###### Abstract

Ultrasound localization microscopy (ULM) refers to a promising medical imaging
modality that systematically leverages the advantages of contrast-enhanced
ultrasound (CEUS) to surpass the diffraction barrier and delineate the
microvascular map. Localization and tracking of microbubbles (MBs), two
significant steps of ULM, facilitate generating the vascular map and the
velocity distribution, respectively. Herein, we propose a novel MB tracking
technique considering temporal pairing as a bubble-set registration problem.
Iterative registration is performed between the bubble sets in two consecutive
time instants by analytically optimizing a cost function that takes position
and point-spread function (PSF) similarities as well as physically plausible
levels of bubbles’ movement into account. Furthermore, we infer MBs’ parity in
a fuzzy manner instead of binary. The proposed technique performs well in
validation experiments with two synthetic and two in vivo datasets provided by
the Ultrasound Localisation and TRacking Algorithms for Super Resolution
(ULTRA-SR) Challenge.

Index Terms—  Ultrasound localization microscopy, Contrast-enhanced
ultrasound, Microbubble tracking, Bubble-set registration, Analytic
optimization.

(a) CEUS image with bubble centers (b) Bubble density (c) Speed (d) Color bar,
density (e) Color bar, speed ($ms^{-1}$)

Fig. 1: Results for the low-frequency synthetic dataset. (a) shows a CEUS
image with the localized bubble centers, whereas (b) and (c) depict the
localization density and speed maps, respectively.

##  1 Introduction

Inspired by its optical analog, ultrasound localization microscopy (ULM) [1]
has evolved to break conventional ultrasound’s diffraction barrier and provide
a super-resolution (SR) vascular map. ULM is a minimally-invasive imaging
technique that processes contrast-enhanced ultrasound (CEUS) image sequences
to reveal the microvascular network of interest. SR ultrasound research has
seen rapid advancement during the last decade. Thus, ULM has established
itself as a promising modality to assist the assessment of vascular health [2,
3] and the diagnoses of fatal diseases [4].

The localization and tracking of gas-filled microbubbles (MBs) circulating
through the bloodstream are two vital steps of ULM. Detecting the MBs and
localizing their centers facilitates generating the localization density map
representing the vascular network under investigation. Among many MB
localization techniques available in the literature, the thresholding and
amplitude-weighted coordinate averaging-based approach [1] is a mainstream
one. Along this line, a pipeline technique involving rough MB center
localization based on the correlation map between the CEUS image and the
point-spread function (PSF) followed by a center of mass calculation step has
been proposed [5]. Sparsity-based deconvolution was proposed in [6] to improve
the isolation of overlapping bubbles. In [7], the probe channels have been
randomly subsampled to reduce ULM’s computational load and hardware
complexity, minimally affecting the MB localization accuracy. Deep neural
network (DNN)-based techniques [8] have also been proposed for improved MB
tracking.

Tracking the localized MBs is required to produce the velocity map. MB
tracking is predominantly accomplished by block-matching [1] or bipartite
pairing-based [9] techniques. A Kalman filter-based robust microbubble
tracking technique has been developed in [5]. Another robust technique
employing an echo-Lagrangian particle filter has been proposed in [10]. Hansen
et al. [11] have devised a cross-correlation-based motion compensation
technique for improved MB tracking. A Euclidian distance-based nearest
neighbor algorithm is exploited in [12], whereas a forward-backward strategy
is adopted in [13] for tracking the MBs in ULM.

For localizing the MBs, we use a standard correlation- and amplitude-weighted
center of mass-based technique. For MB tracking, we have developed an analytic
optimization-based technique inspired by its success in several research
fields [14, 15, 16, 17, 18]. We look at the temporal MB tracking problem from
the standpoint of bubble-set registration and make bubble-pairing decisions
probabilistically. The reference and target bubble sets are registered through
iterative optimization of a cost function penalizing position dissimilarity,
PSF disparity, and large MB movement. The proposed ULM technique is validated
against low- and high-frequency synthetic datasets, in vivo rat brain, and
human lymph node datasets.

(a) CEUS image with bubble centers (b) Bubble density (c) Speed (d) Color bar,
density (e) Color bar, speed ($ms^{-1}$)

Fig. 2: Results for the high-frequency synthetic dataset. (a) presents a CEUS
image with the localized bubble centers, whereas (b) and (c) show the
localization density and speed maps, respectively.

(a) CEUS image with bubble centers (b) Bubble density (c) Speed (d) Color bar,
density (e) Color bar, speed ($ms^{-1}$)

Fig. 3: Results for the in vivo rat brain dataset. (a) shows a CEUS image with
the localized MB centers, whereas (b) and (c) depict the bubble density and
speed maps, respectively.

##  2 Methods

In this section, we first describe our MB localization and the bubble density
map rendering technique. Then we outline the proposed MB tracking algorithm
and its mathematical details.

###  2.1 MB Localization and Bubble Density Map

A suitable bubble signal was extracted from one of the CEUS images and
considered the PSF. The local peaks of the correlations between the PSF and
the CEUS image patches determined the bubble centers’ integer locations. The
subpixel localization precision was achieved by amplitude-weighted averaging
of the grid coordinates around the integer center location. Finally, a
Gaussian around each localized MB center was considered, and all such
Gaussians were added to produce the SR bubble density map.

(a) CEUS image with bubble centers (b) Bubble density (c) Speed ($ms^{-1}$)

Fig. 4: Results for the lymph node dataset. (a) manifests a CEUS image with
the localized MB centers, whereas (b) and (c) present the MB density and speed
maps, respectively. (d) and (e) show the density and speed color bars,
respectively.

###  2.2 MB Tracking and Speed Map

Let $r_{i}$, $i\in\\{1,2,3,...,m\\}$ and $t_{j}$, $j\in\\{1,2,3,...,n\\}$
denote the reference and target bubble sets, respectively. $x_{i}$ and $y_{j}$
refer to the reference and target MB center locations, respectively. We aim to
determine the correspondence between the reference and target MBs by obtaining
the transformation function $f$ that registers the two bubble sets. To that
end, we alternatingly iterate between optimizing a cost function $C$ to obtain
$f$ and updating the bubble-pairing probability $p_{i,j}$. As given in Eq. 1,
$C$ penalizes position and PSF disparities, and forces the estimated bubble
movements to be small.

$\displaystyle C=\alpha\sum\limits_{j=1}^{n}\sum\limits_{i=1}^{m}p_{i,j}\lVert
x_{i}-f(y_{j})\rVert^{2}+$ (1)
$\displaystyle\beta\sum\limits_{j=1}^{n}\sum\limits_{i=1}^{m}p_{i,j}\lVert
PSF_{x_{i}}-PSF_{f(y_{j})}\rVert^{2}+\gamma\sum\lVert f\rVert^{2}$

where $\alpha$, $\beta$, and $\gamma$ control the relative importance of the
constraints. $C$ is analytically minimized using the Gauss-Newton optimization
technique. In each iteration, $p_{i,j}$ is updated by combining the location
and PSF similarity-based matching probabilities:

$\displaystyle p_{i,j}=p_{i,j,loc}\times p_{i,j,psf}$ (2)

where $p_{i,j,loc}$ and $p_{i,j,psf}$ are defined as follows.

$\displaystyle p_{i,j,loc}={1\over w_{1}}e^{-{[d(x_{i},f(y_{j}))]^{2}\over
2w_{1}}}$ (3)

$\displaystyle p_{i,j,psf}={1\over w_{2}}e^{-{\sum[\textrm{$PSF$
disparity}]^{2}\over 2w_{2}}}$ (4)

Here, $w_{1}$ and $w_{2}$ are tunable weights, whereas $d(\cdot)$ stands for
the Euclidian distance. Row and columns of the $m\times n$-sized probability
matrix are iteratively normalized so that the entries of each row and column
sum up to one. A reference bubble is finally paired to one of the target
bubbles based on the highest matching probability.

The velocity of each MB is estimated by frame-to-frame tracking. The
velocities of all MBs localized within a circle centered at a particular grid
point are gathered to form the final velocity map. A principal component
analysis (PCA) is performed to reject the velocities more than three standard
deviations away from the two orthogonal axes. After this outlier rejection
step, a distance-based Gaussian-weighted averaging is performed on the
remaining velocities to find the final velocity at the grid location. This
pipeline provides the velocity at each grid point, and the amplitudes of the
$2D$ velocities render the speed map.

##  3 Results and Discussion

We have validated the proposed ULM technique against low- and high-frequency
synthetic as well as in vivo rat brain and human lymph node datasets provided
by the ULTRA-SR Challenge. The low- and high-frequency datasets were simulated
with a GE M5Sc-D phased array and a Verasonics L11-4v linear array transducer
setting the center frequencies to 2.84 MHz and 7.24 MHz, respectively. The
brain and lymph node datasets, respectively, were acquired at 15.63 MHz and
5.6 MHz transmit frequencies. The number of CEUS images used for generating
the SR images corresponding to the low- and high-frequency synthetic and in
vivo rat and human datasets were 500, 500, 8000, and 1382, respectively.

###  3.1 Synthetic Datasets

Figs. 1 and 2 show representative CEUS images with localized bubble centers,
SR localization density maps, and speed images for the low- and high-frequency
synthetic datasets, respectively. The localized MB centers and the bubble
density maps validate the merit of the localization technique employed in this
work. The speed maps clearly distinguish between the high- and low-speed
simulated vessels. In addition, the speed distributions correspond well with
the CEUS videos’ visual assessment. The solid and dotted vessels on the
localization density images appear as low- and high-speed vessels,
respectively, on the speed maps, which further demonstrate the proposed
tracking technique’s potential.

###  3.2 In Vivo Rat Brain Dataset

The in vivo rat brain results have been shown in Fig. 3. It is evident from
the localized centers that almost all visible MBs are detected. The
localization density map visualizes the microvasculature and the relatively
large vessels. Note that the vascular network delineated by our technique is
consistent with the previously published rat brain vasculature [19]. The speed
map yields good contrast between the high- and low-speed vascular regions. In
addition, the speed distribution agrees well with the visual judgment of the
MBs’ speed in different spatial regions of the CEUS sequences.

###  3.3 In Vivo Human Lymph Node Dataset

Fig. 4 presents a CEUS image with MB centers and the SR density and speed maps
for the human lymph node dataset. Given the difficulty of this particular
dataset, the localization technique exhibits good performance, which is
evident in the localization density map. The speed map distinguishes the low-
speed vessels from the high-speed ones and shows good correspondence with our
visual perception of the comparative speed distribution in the CEUS videos.

##  4 Conclusion

A ULM pipeline is proposed where MB is localized using a cross-correlation-
based technique. We have devised a novel MB tracking algorithm that registers
the reference bubble set to the target one by iteratively optimizing a cost
function. Instead of binary, MB pairing is modeled as a fuzzy system where
bubbles are temporally matched through probabilistic inferences. The proposed
technique performs well on synthetic and in vivo datasets.

## ACKNOWLEDGMENT

This work is partly supported by the Natural Sciences and Engineering Research
Council of Canada (NSERC). Md Ashikuzzaman holds the B2X Doctoral Research
Fellowship provided by the Fonds de Recherche du Québec - Nature et
Technologies (FRQNT). We thank the organizers of the ULTRA-SR Challenge for
providing the synthetic and in vivo datasets.

## References

  * [1] Christensen-Jeffries _et al._ , “In vivo acoustic super-resolution and super-resolved velocity mapping using microbubbles,” _IEEE transactions on medical imaging_ , vol. 34, no. 2, pp. 433–440, 2014.
  * [2] C. Demené _et al._ , “Transcranial ultrafast ultrasound localization microscopy of brain vasculature in patients,” _Nature biomedical engineering_ , vol. 5, no. 3, pp. 219–228, 2021.
  * [3] O. Demeulenaere _et al._ , “Coronary flow assessment using 3-dimensional ultrafast ultrasound localization microscopy,” _JACC: Cardiovascular Imaging_ , 2022.
  * [4] Y. Hao _et al._ , “Non-rigid motion correction for ultrasound localization microscopy of the liver in vivo,” in _2019 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2019, pp. 2263–2266.
  * [5] S. Tang _et al._ , “Kalman filter-based microbubble tracking for robust super-resolution ultrasound microvessel imaging,” _IEEE Trans. UFFC_ , vol. 67, no. 9, pp. 1738–1751, 2020.
  * [6] J. Yan _et al._ , “Super-resolution ultrasound through sparsity-based deconvolution and multi-feature tracking,” _IEEE Transactions on Medical Imaging_ , 2022.
  * [7] E. Hardy _et al._ , “Sparse channel sampling for ultrasound localization microscopy (sparse-ulm),” _Physics in Medicine & Biology_, vol. 66, no. 9, p. 095008, 2021.
  * [8] K. G. Brown _et al._ , “Faster super-resolution ultrasound imaging with a deep learning model for tissue decluttering and contrast agent localization,” _Biomedical physics & engineering express_, vol. 7, no. 6, p. 065035, 2021.
  * [9] P. Song _et al._ , “Improved super-resolution ultrasound microvessel imaging with spatiotemporal nonlocal means filtering and bipartite graph-based microbubble tracking,” _IEEE Trans. UFFC_ , vol. 65, no. 2, pp. 149–167, 2017.
  * [10] M. Jeronimo, M. Najjari, and D. Rival, “Echo-lagrangian particle tracking: an ultrasound-based method for extracting path-dependent flow quantities,” _Measurement Science and Technology_ , vol. 31, no. 5, p. 054008, 2020.
  * [11] K. B. Hansen _et al._ , “Robust microbubble tracking for super resolution imaging in ultrasound,” in _IEEE IUS_. IEEE, 2016, pp. 1–4.
  * [12] K. Johnson, I. Oezdemir, and K. Hoyt, “Three-dimensional evaluation of microvascular networks using contrast-enhanced ultrasound and microbubble tracking,” in _IEEE IUS_. IEEE, 2020, pp. 1–3.
  * [13] I. Taghavi _et al._ , “Microbubble tracking with a forward-backward strategy,” in _SPIE Medical Imaging_ , vol. 12038. SPIE, 2022, pp. 66–71.
  * [14] M. Ashikuzzaman, C. J. Gauthier, and H. Rivaz, “Global ultrasound elastography in spatial and temporal domains,” _IEEE Trans. Ultrasonics, Ferroelectrics, and Frequency Control_ , vol. 66, no. 5, pp. 876–887, 2019.
  * [15] M. Ashikuzzaman _et al._ , “Combining first- and second-order continuity constraints in ultrasound elastography,” _IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control_ , vol. 68, no. 7, pp. 2407–2418, 2021.
  * [16] M. Ashikuzzaman and H. Rivaz, “Second-order ultrasound elastography with l1-norm spatial regularization,” _IEEE Trans. Ultrasonics, Ferroelectrics, and Frequency Control_ , vol. 69, no. 3, pp. 1008–1019, 2022.
  * [17] M. Ashikuzzaman, T. J. Hall, and H. Rivaz, “Incorporating gradient similarity for robust time delay estimation in ultrasound elastography,” _IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control_ , vol. 69, no. 5, pp. 1738–1750, 2022.
  * [18] H. Chui and A. Rangarajan, “A new point matching algorithm for non-rigid registration,” _Computer Vision Im. Und._ , vol. 89, no. 2-3, pp. 114–141, 2003.
  * [19] V. Hingot _et al._ , “Microvascular flow dictates the compromise between spatial resolution and acquisition time in ultrasound localization microscopy,” _Scientific reports_ , vol. 9, no. 1, pp. 1–10, 2019.