[115]. In (C64), the $s_{\textrm{c}}^{2}$ should be modified (compared with
Eq. (1)) as,

$s_{\textrm{c}}^{2}\approx\widehat{P}_{\textrm{c}}\left(1+\frac{1}{3}\frac{1+3\widehat{P}_{\textrm{c}}^{2}+4\widehat{P}_{\textrm{c}}}{1-3\widehat{P}_{\textrm{c}}^{2}}\right)\left(1+\ell_{1}\widehat{P}_{\textrm{c}}+\ell_{2}\widehat{P}_{\textrm{c}}^{2}\right),$
(C65)

where $\ell_{1}$ and $\ell_{2}$ are two coefficients to be determined. When
writing out (C65), the applicable region of the perturbative expansion should
be set as
$|\ell_{2}\widehat{P}_{\textrm{c}}^{2}|\lesssim|\ell_{1}\widehat{P}_{\textrm{c}}|$
and $|\ell_{1}\widehat{P}_{\textrm{c}}|\lesssim 1$, or equivalently
$\widehat{P}_{\textrm{c}}\lesssim\min\\{|\ell_{1}\ell_{2}^{-1}|,|\ell_{1}^{-1}|\\}$.
The reduced radius $\widehat{R}$ is estimated from
$\widehat{P}_{\textrm{c}}+b_{2}\widehat{R}^{2}+b_{4}\widehat{R}^{4}=0$, see
treatments given in the Appendix of Ref. [115].

Similarly [115], the NS mass is given as
$M_{\textrm{NS}}\sim\widehat{R}^{3}/\sqrt{\varepsilon_{\textrm{c}}}$. Taking
the derivative of $M_{\textrm{NS}}$ with respect to $\varepsilon_{\textrm{c}}$
and making it be zero gives the $s_{\textrm{c}}^{2}$, i.e.,

$\displaystyle s_{\textrm{c}}^{2}\approx$
$\displaystyle\frac{4}{3}\widehat{P}_{\textrm{c}}+\frac{4\ell_{1}+923}{708}\widehat{P}_{\textrm{c}}^{2}+\frac{37\ell_{1}-4\ell_{1}^{2}+4\ell_{2}+583}{354}\widehat{P}_{\textrm{c}}^{3}.$
(C66)

By comparing it with the corresponding perturbative expansion of (C65) to the
same order, namely,

$s_{\textrm{c}}^{2}\approx\frac{4}{3}\widehat{P}_{\textrm{c}}+\left(\frac{4}{3}+\frac{4}{3}\ell_{1}\right)\widehat{P}_{\textrm{c}}^{2}+\left(2+\frac{4}{3}\ell_{1}+\frac{4}{3}\ell_{2}\right)\widehat{P}_{\textrm{c}}^{3},$
(C67)

one obtains $\ell_{1}=-21/940\approx-0.02$ and $\ell_{2}\approx-0.25$. The
effective applicable region is estimated as
$\widehat{P}_{\textrm{c}}\lesssim\widehat{P}_{\textrm{c}}^{\textrm{eff}}\approx\min\\{|\ell_{1}\ell_{2}^{-1}|,|\ell_{1}^{-1}|\\}=|\ell_{1}\ell_{2}^{-1}|\approx
0.09$.

Although $\widehat{P}_{\textrm{c}}\lesssim 0.09$ is required, one can find
that for $\widehat{P}_{\textrm{c}}\approx 0.24$ the central SSS
$s_{\textrm{c}}^{2}\approx 0.438$ which is very close to about 0.446 predicted
by Eq. (1). Furthermore, setting $s_{\textrm{c}}^{2}\leq 1$ from the causality
condition now leads to $\widehat{P}_{\textrm{c}}\lesssim 0.381$ (it should be
noticed, however, this upper bound far exceeds the effective region of
$\widehat{P}_{\textrm{c}}\lesssim\widehat{P}_{\textrm{c}}^{\textrm{eff}}\approx
0.09$).

##  Appendix D Order of Magnitude Estimate for $a_{4}$

In this appendix, we give an order of magnitude estimate for the coefficient
$a_{4}$ appearing in the expansion of $\widehat{\varepsilon}\approx
1+a_{2}\widehat{r}^{2}+a_{4}\widehat{r}^{4}$. The starting point is that both
$\widehat{\varepsilon}$ and $\widehat{\rho}=\rho/\rho_{\textrm{c}}$ are
decreasing functions of radial distance from the center. We may use the
following elementary result: If $y(x)=1+Ax+Bx^{2}$ ($A<0$) is a decreasing
function defined for positive $x$, then the first-order derivative
$y^{\prime}(x)$ should be negative, and consequently,

$B\leq-{A}/{2x_{\max}},$ (D68)

with $x_{\max}$ the maximum value of $x$ the $y$ could take.

Applying the criterion (D68) to $\widehat{\varepsilon}\approx
1+a_{2}\widehat{r}^{2}+a_{4}\widehat{r}^{4}$ and expression (53) in the main
text and treating them as functions of $\widehat{r}^{2}$ gives,

$\displaystyle a_{4}\leq$
$\displaystyle-\frac{b_{2}}{2s_{\textrm{c}}^{2}}\frac{1}{\widehat{R}^{2}},~{}~{}\textrm{and,}~{}~{}a_{4}\leq\frac{b_{2}}{2s_{\textrm{c}}^{2}}\left(\frac{b_{2}}{1+\widehat{P}_{\textrm{c}}}-\frac{1}{\widehat{R}^{2}}\right),$
(D69)

respectively, where $\widehat{R}=R/Q$ is the reduced NS radius and
$a_{2}=b_{2}/s_{\textrm{c}}^{2}$ is used here. In order to estimate the order
of magnitude of $a_{4}$ conservatively, we take both $\widehat{R}$ and
$\widehat{P}_{\textrm{c}}$ as small as possible (determined by the structure
of (D69)). For instance, by taking $R\approx 10\,\textrm{km}$ and
$\varepsilon_{\textrm{c}}\approx 0.5\,\textrm{GeV}/\textrm{fm}^{3}$, we obtain
$\widehat{R}\approx 0.91$ as $Q=(4\pi G\varepsilon_{\textrm{c}})^{-1/2}\approx
11\,\textrm{km}$. Similarly, we adopt the reduced central pressure as
$\widehat{P}_{\textrm{c}}\approx 0.16$ (for massive NSs), so the first
inequality of (D69) leads to $a_{4}\lesssim 0.67$ while the second gives
$a_{4}\lesssim 0.80$. Therefore, $a_{4}\sim\mathcal{O}(1)$. On the other hand,
both $\widehat{\varepsilon}$ and $\widehat{\rho}$ are obviously decreasing
functions of $\widehat{r}$ if $a_{4}<0$.

## References

  * [1] L. Landau and E. Lifshitz, Fluid Mechanics, Pergamon Press, 1987, Section 64.
  * [2] J. Walecka, Ann. Phys. 83, 491 (1974).
  * [3] S. Chin, Ann. Phys. 108, 301 (1976).
  * [4] G. Baym and S. Chin, Phys. Lett. B62, 241 (1976).
  * [5] B. Freedman and L. McLerran, Phys. Rev. D 16, 1130 (1977); 1147 (1977); 1169 (1977).
  * [6] V. Baluni, Phys. Rev. D 17, 2092 (1978).
  * [7] R. Wiringa, V. Fiks, and A. Fabrocini, Phys. Rev. C 38, 1010 (1988).
  * [8] A. Akmal, V. Pandharipande, and D. Ravenhall, Phys. Rev. C 58, 1804 (1998).
  * [9] A. Migdal, Rev. Mod. Phys. 50, 107 (1978).
  * [10] P. Morley and M. Kislinger, Phys. Rep. 51, 63 (1979).
  * [11] E. Shuryak, Phys. Rep. 61, 71 (1980).
  * [12] D. Bailin and A. Love, Phys. Rep. 107, 325 (1984).
  * [13] J. Lattimer and M. Prakash, Astrophys. J. 550, 426 (2001).
  * [14] J. Lattimer and M. Prakash, Science 304, 536 (2004).
  * [15] J. Lattimer and M. Prakash, Phys. Rep. 442, 109 (2007).
  * [16] A. Steiner et al., Phys. Rep. 411, 325 (2007).
  * [17] M. Alford et al., Rev. Mod. Phys. 80, 1455 (2008).
  * [18] B.A. Li, L.W. Chen, and C.M. Ko, Phys. Rep. 464, 113 (2008).
  * [19] A. Watts et al., Rev. Mod. Phys. 88, 021001 (2016).
  * [20] M. Oertel et al., Rev. Mod. Phys. 89, 015007 (2017).
  * [21] I. Vida$\widetilde{\textrm{n}}$a, Proc. Roy. Soc. Lond. A 474, 0145 (2018).
  * [22] C. Drischler, J. Holt, and C. Wellenhofer, Annu. Rev. Nucl. Part. Sci. 71, 403 (2021).
  * [23] G. Burgio et al. , Prog. Nucl. Part. Phys. 120, 103879 (2021).
  * [24] A. Lovato et al., arXiv:2211.02224 (2022).
  * [25] A. Sorensen et al., arXiv:2301.13253 (2023).
  * [26] V. Pandharipande, D. Pines, and R. Smith, Astrophys. J. 208, 550 (1976).
  * [27] N. Glendenning, Astrophys. J. 293, 470 (1985).
  * [28] N. Glendenning, Phys. Rev. D 46, 1274 (1992).
  * [29] H. Heiselberg, C. Pethick, and E. Staubo, Phys. Rev. Lett. 70, 1355 (1993).
  * [30] N. Glendenning and J. Bielich, Phys. Rev. Lett. 81, 4564 (1998).
  * [31] M. Alford, K. Rajagopal, and F. Wilczek, Phys. Lett. B422, 247 (1998).
  * [32] T. Sch$\ddot{\textrm{a}}$fer and F. Wiczlek, Phys. Rev. D 60, 114033 (1998).
  * [33] J. Bielich et al., Phys. Rev. Lett. 89, 171101 (2002).
  * [34] A. Kurkela et al., Astrophys. J. 789, 127 (2014).
  * [35] E. Fraga, A. Kurkela, and A. Vuorinen, Astrophys. J. Lett. 781, L25 (2014).
  * [36] E. Annala et al., Nat. Phys. 16, 907 (2020).
  * [37] Ya. Zeldovich, J. Exptl. Theoret. Phys. (U.S.S.R.), 41, 1609 (1961); and Ya. Zeldovich and I. Novikov, Stars and Relativity, Dover Press, 1996, Chap. 6.
  * [38] G. Baym et al., Rep. Prog. Phys. 81, 056902 (2018).
  * [39] L. Baiotti, Prog. Part. Nucl. Phys. 109, 103714 (2019).
  * [40] M. Orsaria et al., J. Phys. G 46, 073002 (2019).
  * [41] B.A. Li et al., Eur. Phys. J. A 55, 117 (2019).
  * [42] V. Dexheimer et al., J. Phys. G 48, 073001 (2021).
  * [43] J. Lattimer, Ann. Rev. Nucl. Part. Phys. 71, 433 (2021).
  * [44] T. Kojo, AAPPS Bulletin, 31, 3428 (2021).
  * [45] A. Sedrakian, Particles 6, 713-730 (2023)
  * [46] B. Abbott et al., Phys. Rev. Lett. 119, 161101 (2017).
  * [47] B. Abbott et al., Phys. Rev. Lett. 121, 161101 (2018).
  * [48] A. Bauswein et al., Astrophys. J. Lett. 850, L34 (2017).
  * [49] D. Radice et al., Astrophys. J. Lett. 852, L29 (2018).
  * [50] E. Most et al., Phys. Rev. Lett. 120, 261103 (2018).
  * [51] F. Fattoyev, J. Piekarewicz, and C. Horowitz, Phys. Rev. Lett. 120, 172702 (2018).
  * [52] S. Bose et al., Phys. Rev. Lett. 120, 031102 (2018).
  * [53] E. Annala et al., Phys. Rev. Lett. 120, 172703 (2018).
  * [54] Y. Zhou, L.W. Chen, and Z. Zhang, Phys. Rev. D 99, 121301(R) (2019).
  * [55] A. Bauswein et al., Phys. Rev. Lett. 125, 141103 (2020).
  * [56] T. Dietrich et al., Science 370, 1450 (2020).
  * [57] N.B. Zhang and B.A. Li, Eur. Phys. J. A 55, 39 (2019); Astrophys. J. 879, 99 (2019); Astrophys. J. 921, 111 (2021).
  * [58] H. Tan, J. Noronha-Hostler, and N. Yunes, Phys. Rev. Lett. 125, 261104 (2020).
  * [59] G. Raaijmakers et al., Astrophys. J. Lett. 893, L21 (2020).
  * [60] G. Raaijmakers et al., Astrophys. J. Lett. 918, L29 (2021).
  * [61] P. Landry, R. Essick and K. Chatziioannou, Phys Rev. D 101, 123007 (2020).
  * [62] R. Essick et al., Phys. Rev. C 102, 055803 (2020).
  * [63] P. Pang et al., Astrophys. J. 922, 14 (2021).
  * [64] I. Bombaci et al., Phys. Rev. Lett. 126, 162702 (2021).
  * [65] S. Huth et al., Nature 606, 276 (2022).
  * [66] E. Annala et al., Phys. Rev. X 12, 011058 (2022).
  * [67] Z. Cao et al., Phys. Rev. D 106, 083007 (2022).
  * [68] M. Breschi et al., Phys. Rev. Lett. 128, 161102 (2022).
  * [69] E. Fonseca et al., Astrophys. J. Lett. 915, L12 (2021).
  * [70] T. Riley et al., Astrophys. J. Lett. 918, L27 (2021).
  * [71] M. Miller et al., Astrophys. J. Lett. 918, L28 (2021).
  * [72] T. Salmi et al., Astrophys. J. 941, 150 (2022).
  * [73] T. Riley et al., Astrophys. J. Lett. 887, L21 (2019).
  * [74] M. Miller et al., Astrophys. J. Lett. 887, L24 (2019).
  * [75] C. Raithel, F. $\ddot{\textrm{O}}$zel, and D. Psaltis, Astrophys. J. 831, 44 (2016).
  * [76] C. Raithel, F. $\ddot{\textrm{O}}$zel, and D. Psaltis, Astrophys. J. 844, 156 (2017).
  * [77] C. Raithel, F. $\ddot{\textrm{O}}$zel, and D. Psaltis, Astrophys. J. Lett. 857, L23 (2018).
  * [78] C. Raithel1, F. $\ddot{\textrm{O}}$zel, and D. Psaltis, Astrophys. J. 908, 103 (2021).
  * [79] S. De et al., Phys. Rev. Lett. 121, 091102 (2018).
  * [80] M. Ruiz, S. Shapiro, and A. Tsokaros, Phys. Rev. D 97, 021501(R) (2018).
  * [81] Y. Lim and J. Holt, Phys. Rev. Lett. 121, 062701 (2018).
  * [82] Y. Lim and J. Holt, Eur. Phys. J. A 55, 209 (2019).
  * [83] C. Drischler et al., Phys. Rev. Lett. 125, 202702 (2020).
  * [84] A. Nathanail, E. Most, and L. Rezzolla, Astrophys. J. Lett. 908, L28 (2021).
  * [85] M.Z. Han et al., Astrophys. J. 919, 11 (2021).
  * [86] Y.J. Huang et al., Phys. Rev. Lett. 129, 181101 (2022).
  * [87] B. Biswas, Astrophys. J. 926, 75 (2022).
  * [88] E. Most et al., Phys. Rev. Lett. 122, 061101 (2019).
  * [89] A. Bauswein et al., Phys. Rev. Lett. 122, 061102 (2019).
  * [90] G. Monta$\widetilde{\textrm{n}}$a et al., Phys. Rev. D 99, 103009 (2019).
  * [91] G. Malfatti et al., Phys. Rev. D 100, 015803 (2019).
  * [92] S. Greif et al., Mon. Not. Roy. Astron. Soc. 485, 5363 (2019).
  * [93] G. Baym et al., Astrophys. J. 885, 42 (2019).
  * [94] W.J. Xie and B.A. Li, Astrophys. J. 899, 4 (2020); Phys. Rev. C 103, 035802 (2021).
  * [95] L. Weih, M. Hanauske, and L. Rezzolla, Phys. Rev. Lett. 124, 171103 (2020).
  * [96] T.Q. Zhao and J. Lattimer, Phys. Rev. D 102, 023021 (2020).
  * [97] K. Fukushima, T. Kojo, and W. Weise, Phys. Rev. D 102, 096017 (2020).
  * [98] J. Kapusta and T. Welle, Phys. Rev. C 104, L014801 (2021).
  * [99] C. Drischler et al., Phys. Rev. C 103, 045808 (2021).
  * [100] C. Drischler, S. Han, and S. Reddy, Phys. Rev. C 105, 035808 (2022).
  * [101] T. Kojo, G. Baym, and T. Hatsuda, Astrophys. J. 934, 46 (2022).
  * [102] Y. Fujimoto and K. Fukushima, Phys. Rev. D 105, 014025 (2022).
  * [103] I. Legred et al., Phys. Rev. D 104, 063003 (2021).
  * [104] M. Al-Mamun et al., Phys. Rev. Lett. 126, 061101 (2021).
  * [105] A. Perego et al., Phys. Rev. Lett. 129, 032701 (2022).
  * [106] M.Z. Han et al., Science Bulletin 68, 913 (2023).
  * [107] R. Somasundaram, I. Tews, and J. Margueron, Phys. Rev. C 107, 025801 (2023).
  * [108] L. Brandes, W. Weise, and N. Kaiser, Phys. Rev. D 107, 014011 (2023).
  * [109] K. Fukushima, T. Kojo, and W. Weise, Phys. Rev. D 102, 096017 (2010).
  * [110] Y. Fujimoto et al., Phys. Rev. Lett. 130, 091404 (2023).
  * [111] N.B. Zhang and B.A. Li, Eur. Phys. J. A 59, 86 (2023).
  * [112] N.B. Zhang and B.A. Li, Phys. Rev. C 108, 025803 (2023).
  * [113] C. Raithel and E. Most, Phys. Rev. Lett. 130, 201403 (2023).
  * [114] R. Chiba and T. Kojo, arXiv:2304.13920 (2023).
  * [115] B.J. Cai, B.A. Li, and Z. Zhang, Astrophys. J. 952, 147 (2023).
  * [116] P. Hohler and M. Stephanov, Phys. Rev. D 80, 066002 (2009).
  * [117] A. Cherman, T. Cohen, and A. Nellore, Phys. Rev. D 80, 066003 (2009).
  * [118] R. Romani et al., Astrophys. J. Lett. 934, L18 (2022).
  * [119] Y. Fujimoto et al., Phys. Rev. Lett. 129, 252702 (2022).
  * [120] R. Tolman, Phys. Rev. 55, 364 (1939).
  * [121] J. Oppenheimer and G. Volkoff, Phys. Rev. 55, 374 (1939).
  * [122] C. Misner, K. Thorne, and J. Wheeler, Gravitation, Princeton University Press, 2017, Section 23.5.
  * [123] L. Landau, Phys. Z. Sowjetunion, 1, 285 (1932).
  * [124] C. Rhoades and R. Ruffini, Phys. Rev. Lett. 32, 324 (1974).
  * [125] J. Hartle and A. Sabbadini, Astrophys. J. 213, 831 (1977).
  * [126] J. Hartle, Phys. Rep. 46, 201 (1978).
  * [127] L. Lindblom, Astrophys. J. 278, 364 (1984).
  * [128] L. Lindblom, Astrophys. J. 398, 569 (1992).
  * [129] V. Kalogera and G. Baym, Astrophys. J. 470, L61 (1996).
  * [130] J. Lattimer and M. Prakash, Phys. Rev. Lett. 94, 111101 (2005).
  * [131] J. Lattimer and M. Prakash, in From Nuclei to Stars, edited by S. Lee, WorldScientific Press, 2011, pp. 275–304.
  * [132] S. Shapiro and S. Teukolsky, Black Holes, White Dwarfs and Neutron Stars, Wiley-VCH, 1983, Chap. 2.
  * [133] S. Chandrasekhar, An Introduction to the Study of Stellar Structure, Dover Press, 2010, Chap. 3.
  * [134] B. Serot and J. Walecka, Adv. Nucl. Phys. 16, 1 (1986).
  * [135] B.J. Cai and B.A. Li, arXiv:2210.10924 (2022).
  * [136] D. Vautherin and D. Brink, Phys. Rev. 5, 626 (1972).
  * [137] Z. Zhang and L.W. Chen, Phys. Rev. C 94, 06426 (2016).
  * [138] J. Stone and P. Reinhard, Prog. Nucl. Part. Phys. 58, 587 (2007).
  * [139] G. Baym, C. Pethick, and P. Sutherland, Astrophys. J. 170, 299 (1971).
  * [140] K. Iida and K. Sato, Astrophys. J. 477, 294 (1997).
  * [141] J. Xu et al., Astrophys. J. 697, 1549 (2009).
  * [142] F. Coester et al., Phys. Rev. C 1, 769 (1970).
  * [143] J.L. Liang, C. Ecker, and L. Rezzolla, Astrophys. J. 949, 11 (2023).
  * [144] S.P. Tang et al., arXiv:2309.15441 (2023).
  * [145] F. $\ddot{\textrm{O}}$zel and P. Freire, Ann. Rev. Astron. Astrophys. 54, 401 (2016).
  * [146] C. Capano et al., Nat. Astron. 4, 625 (2020).
  * [147] P. Bedaque and A. Steiner, Phys. Rev. Lett. 114, 031103 (2015).
  * [148] Z.Q. Miao et al., Astrophys. J. Lett. 917, L22 (2021).
  * [149] H. Tan et al., Phys. Rev. Lett. 128, 161101 (2022).
  * [150] H. Tan et al., Phys. Rev. D 105, 023018 (2022).
  * [151] C. Ecker and L. Rezzolla, Astrophys. J. Lett. 939, L35 (2022).
  * [152] C. Ecker and L. Rezzolla, Mon. Not. Roy. Astron. Soc. 519, 2615 (2023).
  * [153] L. McLerran and S. Reddy, Phys. Rev. Lett. 122, 122701 (2019).
  * [154] S. Altiparmak, C. Ecker, and L. Rezzolla, Astrophys. J. Lett. 939, L34 (2022).
  * [155] J. Alsing, H. Silva, and E. Berti, Mon. Not. Roy. Astron. Soc. 478, 1377 (2018).
  * [156] I. Tews et al., Astrophys. J. 860, 149 (2018).
  * [157] M. Leonhardt et al., Phys. Rev. Lett. 125,142502 (2020).
  * [158] J. Stone et al., Mon. Not. Roy. Astron. Soc. 502, 3476 (2021).
  * [159] K. Otto, M. Oertel, and B. Schaefer, Phys. Rev. D 101, 103021 (2020).
  * [160] T. Motta, P. Guichon, and A. Thomas, Nucl. Phys. A1009, 122157 (2021).
  * [161] We thank the referee of this manuscript for suggesting us to emphasize these points here.
  * [162] F. $\ddot{\textrm{O}}$zel et al., Astrohphys. J. 820, 28 (2016).
  * [163] R. Abbott et al., Astrophys. J. L 896, L44 (2020).
  * [164] H. M$\ddot{\textrm{u}}$ther, M. Prakash, and T. Ainsworth, Phys. Lett. B199, 469 (1987).
  * [165] M. Prakrash, J. Cooke, and J. Lattimer, Phys. Rev. D 52, 661 (1995).
  * [166] H. M$\ddot{\textrm{u}}$ller and B. Serot, Nucl. Phys. A606, 508 (1996).
  * [167] L. Engvik et al., Astrophys. J. 469, 794 (1996).
  * [168] F. Douchin and P. Haensel, Astron. Astrophys. 380, 151 (2001).
  * [169] B. Todd-Rutel and J. Piekarewicz, Phys. Rev. Lett. 95, 122501 (2005).
  * [170] M. Alford et al., Astrophys. J. 629, 969 (2005).
  * [171] B. Lackey, M. Nayyar, and B. Owen, Phys. Rev. D 73, 024021 (2006).
  * [172] M. Hempel and J. Bielich, Nucl. Phys. A837, 210 (2010).
  * [173] P.C. Chu and L.W. Chen, Astrophys. J. 780, 135 (2014).
  * [174] S. Banik, M. Hempel, and D. Bandyopadhyay, Astrophys. J. Supp. Ser. 214, 22 (2014).
  * [175] F. Li et al., Astrophys. J. 929, 183 (2022).
  * [176] B. Abbott et al., Astrophys. J. Lett. 892, L3 (2020).
  * [177] J. Lattimer et al., Astrophys. J. 355, 241 (1990).
  * [178] H. Buchdahl, Phys. Rev. 116, 1027 (1959).
  * [179] A. Lightman et al., Problem Book in Relativity and Gravitation, Princeton University Press, 1975, Problem 16.13.
  * [180] X. Zhou et al., Nat. Phys. 1477, 69 (2023).
  * [181] M. Bejger and P. Haensel, Astron. Astrophys. 405, 747 (2003).
  * [182] J. Lattimer and B. Schutz, Astrophy. J. 629, 979 (2005).
  * [183] A. Steiner, J. Lattimer, and E. Brown, Eur. Phys. J. A 52, 18 (2016).
  * [184] C. Breu and L. Rezzolla, Mon. Not.Roy. Astron. Soc. 459, 646 (2016).
  * [185] P. Demorest et al., Nature 467, 1081 (2010).
  * [186] J. Antoniadis et al., Science 340, 1233232 (2013).
  * [187] T. Gorda, O. Komoltsev, and A. Kurkela, Astrophys. J. 950, 107 (2023).
  * [188] M. Marczenko et al., Phys. Rev. C 107, 025802 (2023).
  * [189] L. Rezzolla, E. Most, and L. Weih, Astrophys. J. Lett. 852, L25 (2018).
  * [190] B. Margalit and B. Metzger, Astrophys. J. Lett. 850, L19 (2017).
  * [191] M. Ruiz, S. Shapiro, and A. Tsokaros, Phys. Rev. D 97, 021501(R) (2018).
  * [192] J. Takatsy et al., arXiv:2303.00013 (2023).
  * [193] C. Musolino, C. Ecker, and L. Rezzolla, arXiv:2307.03225 (2023).
  * [194] C. Provid$\hat{\textrm{e}}$ncia et al., arXiv:2307.05086 (2023).
  * [195] E. Annala et al., arXiv:2303.11356 (2023).
  * [196] L. Brandes, W. Weise, and N. Kaiser, arXiv:2306.06218 (2023).
  * [197] M. Shibata, Numerical Relativity, World Scientific Press, 2015, Chap. 8 and Chap. 9.
  * [198] J. Bjorken, Phys. Rev. D 27, 140 (1983).
  * [199] A. Kurkela, P. Romatschke, and A. Vuorinen, Phys. Rev. D 81, 105021 (2010).
  * [200] The National Academies Press, New Worlds, New Horizons in Astronomy and Astrophysics, 2011,
https://www.nap.edu/catalog/12951/new-worlds-new-horizons-in-astronomy-and-
astrophysics


0.2pt 55

††footnotetext: Copyright © 2020 for this paper by its authors. Use permitted
under Creative Commons License Attribution 4.0 International (CC BY 4.0). CLEF
2020, 22-25 September 2020, Thessaloniki, Greece.11institutetext:  Queen Mary
University of London, United Kingdom 22institutetext: University of Warwick,
United Kingdom 33institutetext: Alan Turing Institute, United Kingdom
44institutetext: University of Oxford, United Kingdom 55institutetext: Imam
Abdulrahman bin Faisal University, Saudi Arabia

# QMUL-SDS at CheckThat! 2020:
Determining COVID-19 Tweet Check-Worthiness Using an Enhanced CT-BERT with
Numeric Expressions

Rabab Alkhalifa 1155 Theodore Yoong 44 Elena Kochkina 2233 Arkaitz Zubiaga 11
Maria Liakata 112233

###### Abstract

This paper describes the participation of the QMUL-SDS team for Task 1 of the
CLEF 2020 CheckThat! shared task. The purpose of this task is to determine the
check-worthiness of tweets about COVID-19 to identify and prioritise tweets
that need fact-checking. The overarching aim is to further support ongoing
efforts to protect the public from fake news and help people find reliable
information. We describe and analyse the results of our submissions. We show
that a CNN using COVID-Twitter-BERT (CT-BERT) enhanced with numeric
expressions can effectively boost performance from baseline results. We also
show results of training data augmentation with rumours on other topics. Our
best system ranked fourth in the task with encouraging outcomes showing
potential for improved results in the future.

##  1 Introduction

The vast majority of people seek information online and consider it a
touchstone of guidance and authority [1]. In particular, social media has
become the key resource to go to for following updates during times of crisis
[2]. Any registered user can share posts on social media without content
verification, potentially exposing thousands or millions of other users to
harmful misinformation. To prevent the undesired consequences of
misinformation spread, there is a need to develop tools to assess the validity
of social media posts. This problem has been particularly accentuated in light
of the COVID-19 pandemic, accompanied by the rising spread of unverified
claims and conspiracy theories about the virus and untested dangerous
treatments. Compounded with the devastating effects from the virus alone, the
social harms from misinformation spread can be particularly injurious [3]. The
CheckThat! shared task provided a benchmark evaluation lab to develop systems
for check-worthiness detection, with the aim of prioritising claims to be
provided to fact-checkers.

In this paper, we present our approaches in tackling the check-worthiness
detection task as outlined in Task 1 of the CLEF-2020 CheckThat! Lab. We
evaluated several variants of our Convolutional Neural Network (CNN) model
with different pre-processing approaches and several BERT embeddings. We also
tested the benefits of including the use of external data to augment the
training data provided. We submitted three models that have shown the best
performance on the development set. Our best performing model utilised a
COVID-Twitter-BERT (CT-BERT) enhanced with numeric expressions, which was
ranked fourth in the task.

##  2 Related Work

We organise the related work into two subsections relevant to our proposed
methods and the systems we submitted to the evaluation lab: claim check-
worthiness and rumour detection.

###  2.1 Determination of Claim Check-worthiness

While there is no general streamlined approach to fact-checking, the fact-
checking pipeline can be divided into different sub-tasks based on a number of
contexts [4]. The first of the tasks and the one concerning our work consists
in producing a list of claims ranked by importance (check-worthiness), in an
effort to prioritise claims to be fact-checked. Systems for claim detection
(as a classification task) and ranking by check-worthiness include (i)
ClaimBuster [5], which combines numerous features such as TF-IDF, POS tags and
NER on a Support Vector Machine to produce importance scores for each claim,
and (ii) ClaimRank [6], which uses a large set of features both from
individuals sentences and from surrounding context. More recent methods, such
as [7], have made use of embedding-based methods such as InferSent for
detecting claims by leveraging contextual features within sentences. In
previous editions of CheckThat! [8], the shared task did not involve the
detection of claims, as claims were already given as input.

Previous work on claim detection by Konstantinovskiy et al. [7] suggested the
use of numeric expressions as a strong baseline for detection of claims.
Indeed they showed that the use of numeric expressions as a feature leads to
high precision, despite achieving lower recall and overall F1 score than other
methods. This is due to the prevailing presence of numeric expressions in
check-worthy claims, as opposed to non-check-worthy claims and non-claims.
Given the emphasis of the CheckThat! shared task on precision-based evaluation
(using mean average precision as a metric), we opted for incorporating numeric
expressions in our model.

###  2.2 Rumour Detection

A rumour is generally defined as an unverified piece of information that
circulates. In the same way that a check-worthiness detection looks at claims
to be verified, e.g. in the context of a TV debate, rumour detection consists
in detecting pieces of information that are in circulation while they still
lack verification, generally in the context of breaking news, making it a
time-sensitive task [9]. Rumours differ from check-worthy claims in their
nature as well as relevance to the fact-checkers, as not all rumours are
necessarily of interest to fact-checkers. Still, both tasks have significant
commonalities.

In our approaches to check-worthiness determination, we try to leverage
existing data for rumour detection, consisting of rumours and non-rumours,
with the aim of providing additional knowledge that would enrich the task (see
§3.1).

Rumour detection, as the task of detecting unverified pieces of information,
has been studied before, for instance through the RumourEval shared tasks held
at SemEval 2019 [10]. Prior to that, Zubiaga et al. [11] introduced a
sequential rumour detection model that leveraged Conditional Random Fields
(CRF) for leveraging event context, as well as Zhao et al. [12] that looked at
evidence from others responding to tweets with comments of the form of “is
this really true?”, which would be indicative of a tweet containing rumourous
content.

##  3 Task Description

The task we explore was introduced by Barrón-Cedeño et al. [13] and is
formulated as follows:

Given a topic and a stream of potentially-related tweets, rank the tweets
according to their check-worthiness for the topic, where a check-worthy tweet
is a tweet that includes a claim that is of interest to a large audience
(especially journalists) and may have a harmful effect.

For example, consider the target topic–tweet pair:

Target topic: COVID-19 Tweet: Doctors in #Italy warn Europe to “get ready” for
#coronavirus, saying 10% of #COVID19 patients need ICU care, and hospitals are
overwhelmed. Label: Check-worthy

Although Task 1 is available in both English and Arabic, we focussed solely on
the English task [14]. Tweets in this dataset for this task exclusively
covered the pandemic caused by the Coronavirus Disease 2019 (COVID-19). The
ultimate objective of ranking the tweets identified as check-worthy claims is
to enable prioritisation of claims to fact-checkers.

The task can be formally described as the following binary classification
problem. We define the training set consisting of $n$ labelled tweets as
$\mathcal{D}=\\{({\bf x}_{i},y_{i}),1\leq i\leq
n\\}\in\left(\mathcal{X}\times\\{0,1\\}\right)^{n}$. Here, ${\bf x}_{i}$ is
the $i^{\text{th}}$ feature vector in feature space $\mathcal{X}$ which
contains the tweet features such as the $i^{\text{th}}$ tweet itself ${\bf
t}_{i}$, the topic, and whether it is a claim or not; and $y_{i}\in\\{0,1\\}$
is the label indicating check-worthiness of ${\bf t}_{i}$. The objective is to
obtain a map $h:\mathcal{X}\mapsto\\{0,1\\}$, based on the class probability
measure $\mathbb{P}(y|{\bf t})$, which is subsequently used to rank the tweets
in the test set.

###  3.1 Datasets

Dataset |  |
---
No. of check-
worthy tweets
(rumours)
|  No. of non-check-
---
worthy tweets
(non-rumours)
Total
CLEF Train | 231 | 441 | 672
CLEF Development | 59 | 91 | 150
CLEF Test | 80 | 60 | 140
PHEME | 2402 | 4023 | 6425
Twitter 15 | 1012 | 362 | 1374
Twitter 16 | 536 | 199 | 735
Table 1: Number of posts and class distribution in the datasets used

In our experiments, we made use of three datasets of Twitter posts in English,
which include the dataset provided by the organisers (CLEF) and two external
publicly available datasets (PHEME, Twitter 15 and Twitter 16) to augment the
training set. The PHEME, Twitter 15 and Twitter 16 datasets were chosen for
augmentation as these are relatively large datasets annotated for rumour
detection task, which is very similar to claim check-worthiness as described
in section 2. Table 1 shows the number of tweets used in each of the datasets
and the class distribution.

The _CLEF_ dataset contains tweets related to the topic of COVID-19. They were
annotated by the task organisers as either check-worthy or not check-worthy
thus defining a binary classification task.111Annotation rules can be found
here: https://github.com/sshaar/clef2020-factchecking-task1#data-annotation-
process This dataset is rather small and is limited to individual tweets
concerned with a single topic. The dataset is imbalanced with the majority of
tweets being not check-worthy.

The _PHEME_ dataset [9, 15] contains Twitter conversations discussing rumours
(defined as unverified check-worthy claims spreading widely on social media)
and non-
rumours.222https://figshare.com/articles/PHEME_dataset_for_Rumour_Detection_and_Veracity_Classification/6392078
This dataset contains conversations related to 9 major newsworthy events, such
as shooting in Charlie Hebdo, shooting in Ottowa, crash of Germanwings plane.
In this work, we use only the source tweets of the conversations in the PHEME
dataset (they are conveying the essence of a rumour, rather than the following
discussion) in order to have the same input structure as the CLEF dataset. We
performed experiments augmenting the training set with both rumours and non-
rumours from the PHEME dataset. We found that adding rumours only is more
beneficial than adding the full PHEME dataset.

The _Twitter 15 and Twitter 16_ datasets [16] contain Twitter conversations,
discussing True, False and Unverified rumours as well as non-rumours on
various topics. Here, we do not use all 4-class labels, but instead convert
True, False and Unverified classes into single check-worthy class. We also use
only source tweets to augment the CLEF training set.

###  3.2 Evaluation

The CLEF dataset is split into training, development and testing sets. The
check-worthiness task is evaluated as a ranking task, i.e. the participant
systems should produce a list of tweets with the estimated score for check-
worthiness. The official evaluation metric is Mean Average Precision (MAP),
but the precisions at rank $k$ ($P@5,P@10,P@30$) are also reported. Baseline
results provided by the organisers are Random Classification (MAP = 0.35) and
SVM with $N$-gram Prediction (MAP = 0.69).

##  4 Our Approach

In our approach, we fed a pre-trained word-level vector representation into a
CNN model. Using vector representations of words as inputs offers high
flexibility, allowing swaps between different pre-trained word vectors during
model initialisation to be maintained without additional overhead. For our
work, we tested multiple feature representations in order to assess which one
would be best for our problem, these include combinations of frequency-based,
vector-based representations and authors’ profiles. Since most evaluation
tweets were not supplied with author’s profiles, and frequency-based models
may not be of great generalisability for unseen data, we reduced our
exploration to two dynamic word embeddings, ELMo [17] and BERT [18]. Since the
former did not provide any increase in performance, we chose the latter to be
further explored with different pre-processing techniques.

Figure 1: General Model Architecture.

With these settings, we evaluated our model using two variations of the BERT
pre-trained architecture, uncased BERT (uncased-BERT) and COVID-Twitter-BERT
(CT-BERT) [19] (see Figure 1). While both are transformer-based models, CT-
BERT is pre-trained on COVID-19 Twitter data using whole word-masked modelling
and next sentence prediction.

In the following sections we describe the main characteristics of our designed
system including pre-processing, feature representation, model architecture
and hyperparameters used.

###  4.1 Pre-processing

We performed standard Twitter data pre-processing in order to improve our
system performance. For each model, we implemented different pre-processing
variants depending on the vector representation leading to best performance.
The pre-processing steps can be summarised as follows.

  1. (i)

Segment2Token: We split each sentence into tokens considering the type of
every segment and breaking it into individual tokens. Digits, URLs, accounts
and hashtags were replaced by $\langle number\rangle$ , $\langle url\rangle$,
$\langle account\rangle$, $\langle hashtag\rangle$. respectively. This was
implemented using a simple split function with different expression finding
methods. By analysing generated segments, we settled on different treatment
for special tokens in every tweet. For example, hyperlinks were either
completely removed from the dataset or replaced by special tokens.
Furthermore, digits and all other numerical expressions that contained ‘%’ or
‘$’ were either removed or tokenised. Tokenising numerical expressions allows
the model to generalise better (see §5). For example:

Tweet: $[NEWS]$ Naver #BAEKHYUN EXO Baekhyun donates 50 million won to prevent
the spread of Corona 19 @weareoneEXO #EXO Segment2Token: $[NEWS]$ Naver
$\langle hashtag\rangle$ EXO Baekhyun donates $\langle number\rangle$ won to
prevent the spread of Corona 19 $\langle account\rangle$ $\langle
hashtag\rangle$.

  2. (ii)

Segment2Root: In NLP, the $\tilde{\chi}^{2}$-statistical measure tests term-
dependency of the tweet being about one of the classes as in [20]. We used it
to analyse the segments of the tweets. In these settings, for few account
handles and hashtags with high $\tilde{\chi}^{2}$-score, we manually combined
them depending on their semantic meaning. For instance:

Hashtags: #coronavirus, #COVID19’, #COVID-19, #COVID19, #Coronavirus, #Corona-
virus Hashtag2Root: coronavirus

In this case, different hashtags about COVID-19 were all consolidated under
the ‘coronavirus’ umbrella term.

  3. (iii)

Word2id: As with all BERT models, we included classification embedding tokens
for every tweet: [CLS] at the beginning and a separator token [SEP] at the
end. We then decomposed ${\bf t}_{i}$ into a sequence of numerical tokens
using BERT tokenisation methods. This was done by mapping each token to a
unique integer in the corpus’ vocabulary.

  4. (iv)

Padding: We ensured that the input sequences in every batch was the same
length. This was achieved through increasing the length of some of the
sequences by adding more tokens. We tried to reduce the padding by allowing
our model to decide the padding length based on a given batch size (set to 10)
and the longest sequence within the given batch. For example, if the longest
sequence length for a given batch is 20, then all other shorter sentences will
be padded to match its length.

Finally, a look-up table was used for each token from the generated
representation, ready to be fed into the model.

###  4.2 Model Hyperparameters

The CNN architecture requires tuning various hyperparameters. These include
input representations, number of layers and filters, pooling, and activation
functions. We utilised a BERT language model in accompaniment to the the
general CNN architecture. Within the model design, we propose variations of a
three-layer CNN with 32 filters with different window sizes: 2, 4 and 7. The
multiple filters act as feature extractors.

Additionally, we used an Adam optimiser with learning rate fixed at
$2\text{e}^{-5}$ and number of training epochs set to 8. Our $N$-gram kernels
encompass a Rectified Linear Unit (ReLU) activation function, given by
$\max(0,x)$. All pooling layers use a max-pooling operation. For the binary
classification, we utilise a sigmoid activation function $\sigma(x)$ for the
output layer, defined as

$\sigma(x)=\frac{1}{1+\text{e}^{-x}}$

To determine the final output labels, we classify check-worthiness based on
the indicator variable

$h(x)=\begin{cases}1\quad\text{if
}\sigma(x)\geq\displaystyle\frac{1}{2}\text{, indicating a check-worthy
tweet,}\\\ \\\ 0\quad\text{if }\sigma(x)<\displaystyle\frac{1}{2}\text{,
suppressing the tweet as non-check-worthy.}\end{cases}$

##  5 Results and Discussion

In the following section, we discuss the selection of the models we tried and
ultimately submitted to the shared task. We also evaluate and compare their
performance.

###  5.1 Model Selection on the CLEF Development Set

We performed our model selection using the development set. Details of the
pre-processing steps and embeddings applied to each of the eight models we
tested are given in Table 2. The performance of the models according to the
various precision metrics are shown in Table 3.

####  5.1.1 CLEF Benchmark Data Experiments:

Text distortion has been used by [21], where their methods were more
successful than using the full text in the classification process. Taking
inspiration from their work, we used tokenisation for different segments of
the tweet where account handles, hashtags, URLs and digits were assigned
special tokens and added to the model vocabulary. The goal of this step was to
avoid over-fitting the training data.

In Model 7, we experimented with ELMo embeddings, which gave the best
performance in terms of MAP, in our tests without additional pre-processing
and outperforms random baseline (MAP = 0.35). However, Model 7 did not
outperform the $N$-gram baseline (MAP = 0.69), and thus we did not choose it
for the test set submission.

In Model 4, we trained word embeddings on the training set along with the
model and combined them with TF-IDF representations. This led to improvements
over Models 5-7 and over the $N$-gram baseline.

For Model 1, we used intensive pre-processing in tandem with a CNN model with
three filters of sizes 2, 4 and 7. On the other hand, in Model 2, only numeric
expressions were tokenised and the CNN model only had two filters of sizes 2
and 4. Models 1 and 2 displayed the best performance on the development set,
and were hence chosen for submission on the testing set.

Model No. | Pre-processing | Embeddings
---|---|---
1 |  Frequently mentioned entities replaced with their account name. Hashtags with repeated topics combined using the $\tilde{\chi}^{2}$-score, other URLs and hashtags tokenised with special tokens. | CT-BERT
2 |  Special tokens for digits. Account handles, URLs and hashtags removed. | CT-BERT
3 |  Training set merged with rumours from PHEME. Special tokens for digits. |  BERT-EN (Uncased)
4 |  Digits, account handles, URLs and hashtags removed. |  CLEF Train Embeddings \+ TF-IDF
5 |  Training set merged with Twitter 15 and Twitter 16 datasets. Special tokens for digits. |  BERT-EN (Uncased)
6 |  Training set merged with PHEME, Twitter 15 and Twitter 16 datasets. Special tokens for digits. |  BERT-EN (Uncased)
7 |  No pre-processing was applied. | ELMo
8 |  Trained using PHEME, Twitter 15 and Twitter 16 datasets only, without CLEF training data. |  BERT-EN (Uncased)
Table 2: Description of the models tested.

####  5.1.2 Data Augmentation Experiments:

While the task definition states the presence of the target topic when
identifying tweet check-worthiness, the dataset provided only covers a single
topic, COVID-19. We experimented with training data augmentation using check-
worthy tweets from external datasets (PHEME and Twitter15, Twitter 16 as
described in section 3.1) (see models 3, 5, 6 in Table 2). These datasets
cover different topics, accounts and vocabulary, so incorporating them could
contribute to future generalisability of the model. We also performed
experiments using only existing external datasets of rumours and non-rumours
and omitting the CLEF training data to test the generalisability of the
currently available datasets and models to the emergence of new rumour topics
(see model 8 in Table 2). The results are presented in Table 3.

Model No. |  Average precision |  _R_ -precision ($R=59$) | Precision@_k_
---|---|---|---
|  @1 |  @3 |  @5 |  @10 |  @20 |  @50
1 |  0.81 | 0.71 |  1.00 |  1.00 |  1.00 |  1.00 |  0.95 |  0.74
2 |  0.80 | 0.71 |  1.00 |  1.00 |  1.00 |  1.00 |  0.95 |  0.76
3 |  0.75 | 0.69 |  1.00 |  0.67 |  0.60 |  0.80 |  0.85 |  0.74
4 |  0.74 | 0.63 |  1.00 |  1.00 |  0.80 |  0.90 |  0.85 |  0.68
5 |  0.65 | 0.59 |  1.00 |  1.00 |  0.80 |  0.80 |  0.75 |  0.62
6 |  0.56 | 0.57 |  0.00 |  0.33 |  0.20 |  0.50 |  0.70 |  0.58
7 |  0.53 | 0.56 |  0.00 |  0.33 |  0.40 |  0.50 |  0.55 |  0.58
8 |  0.45 | 0.44 |  1.00 |  0.33 |  0.40 |  0.40 |  0.50 |  0.44

Table 3: Model performance on the development set

As expected, Model 8, which did not use the CLEF training data, performed
worse compared to the models that did make use of the training data provided.
However, it outperformed the random baseline (MAP = 0.35) by 10%, showing that
there is enough overlap in the task definitions and inherent nature of
rumours/check-worthy claims to provide meaningful signal for model training.
Models 3, 5 and 6, which augmented the training data, did not perform as well
as the models using only the training data. Models 5 and 6 did not outperform
the $N$-gram baseline (MAP = 0.69) provided by the
organisers333https://github.com/sshaar/clef2020-factchecking-task1#baseline.
Model 3 only adds rumours to the training data, thus shifting the class
balance in the dataset, and performed better than adding both rumours and non-
rumours from PHEME to the training set.

These results show the importance of model training or fine-tuning on the
evaluation domain. The lack of performance improvement could be also due to
the differences in the definitions of the tasks and rumours/check-worthy
claims by each of the datasets. Moreover, different fact-checking
organisations would naturally make different choices when analysing the same
data. In [7], they found that educational background can lead to bias in
annotation efforts for fact-checking. The subjectiveness that underlies check-
worthiness thereby adds further complications to the task of ranking by
importance. These results also highlight the especially challenging aspects of
the need for generalising to new unseen topics, as well as leveraging data
from a related task such as that of rumour detection, in detecting tweet
check-worthiness.

###  5.2 Results on the CLEF Test

We selected the best three models based on MAP (see Table 3). Table 4 shows
the official results obtained by our systems on the testing set.

Model No. |  Average precision |  $R$-precision ($R=59$) | Precision@_k_
---|---|---|---
|  @1 |  @3 |  @5 |  @10 |  @20 |  @50
1 |  0.71 | 0.63 |  1.00 |  1.00 |  1.00 |  0.90 |  0.80 |  0.64
2 |  0.78 | 0.70 |  1.00 |  1.00 |  1.00 |  1.00 |  0.85 |  0.70
3 |  0.73 | 0.63 |  1.00 |  1.00 |  1.00 |  0.90 |  0.85 |  0.68

Table 4: Performance of the submitted models on the testing set

####  5.2.1 Models 1 and 2:

We found that avoiding extensive tokenisation (Model 1) while merely
tokenising numeric expressions yields better results in the test set and
allows the model to learn more general patterns from the training set (Model
2). For example:

Tweet: France, Spain and Germany are about 9 to 10 days behind Italy in
$\\#COVID19$ progression; the UK and the US follow at 13 to 16 days.
Digit2Token: France, Spain and Germany are about $\langle number\rangle$ to
$\langle number\rangle$ days behind Italy in corona virus progression; the UK
and the US follow at $\langle number\rangle$ to $\langle number\rangle$ days.

Moreover, our approach uses a less complex model which keeps the weights small
and results in better overall performance. Therefore, in order for our model
to generalise to unseen tweets in the test set, numeric expressions should be
unified and model complexity needs to be maintained.

####  5.2.2 Model 3:

In this submission, we augmented the training data with rumours from the PHEME
dataset. While the results of this submission on the development set were the
lowest out of the selected three, on the testing set it outperforms Model 1.
This shows that external data from a related task adds meaningful signal for
model training and contributes to system generalisability.

##  6 Conclusion

This paper describes our efforts as participants of the Task 1 of the
CheckThat! 2020 evaluation lab, in which we ranked fourth, which was held in
conjunction with the CLEF conference. We describe our proposed model that
leverages the COVID-Twitter-19 BERT (CT-BERT) word embeddings and performs a
special treatment for rare tokens with a CNN relying on the tweet alone.

The experimental results show that the performance of our model increases
significantly by tokenising numerical expressions. The present work is
restricted in choosing the best feature representation. In the future, this
work can be enhanced in different possible directions. For example,
incorporating pragmatic information related to author’s profile information.
Thus, simulate actual users’ behaviour in verifying claims in social media.

Given the small size of the training data provided by the organisers, we also
performed additional experiments leveraging external datasets with the aim of
augmenting the training data. External data we incorporated had the challenge
of being datasets pertaining to rumour detection and on different topics,
hence with slight differences with the task and domain at hand. Our
experiments with data augmentation did not lead to improved performance,
highlighting that inclusion of external data of a different nature (i.e. in
terms of task and domain) is particularly challenging and, if they can provide
an improvement to the check-worthiness detection task, more careful
integration and adaptation will be necessary.

##  7 Acknowledgments

This research utilised Queen Mary’s Apocrita HPC facility, supported by QMUL
Research-IT. http://doi.org/10.5281/zenodo.438045. This work was also
partially supported by The Alan Turing Institute under the EPSRC grant
EP/N510-129/1.

## References

  * [1] L. M. S. Miller and R. A. Bell, “Online health information seeking: the influence of age, information trustworthiness, and search challenges,” _Journal of aging and health_ , vol. 24, no. 3, pp. 525–541, 2012.
  * [2] L. Palen, “Online social media in crisis events,” _Educause quarterly_ , vol. 31, no. 3, pp. 76–78, 2008.
  * [3] J. Y. Cuan-Baltazar, M. J. Muñoz-Perez, C. Robledo-Vega, M. F. Pérez-Zepeda, and E. Soto-Vega, “Misinformation of covid-19 on the internet: infodemiology study,” _JMIR public health and surveillance_ , vol. 6, no. 2, p. e18444, 2020.
  * [4] M. Babakar and W. Moy, “The State of Automated Factchecking,” Full Fact, London, UK, Tech. Rep., 2016.
  * [5] N. Hassan, G. Zhang, F. Arslan, J. Caraballo, D. Jimenez, S. Gawsane, S. Hasan, M. Joseph, A. Kulkarni, A. K. Nayak _et al._ , “ClaimBuster: the first-ever end-to-end fact-checking system,” _Proceedings of the VLDB Endowment_ , vol. 10, no. 12, pp. 1945–1948, 2017.
  * [6] P. Gencheva, P. Nakov, L. Màrquez, A. Barrón-Cedeño, and I. Koychev, “A context-aware approach for detecting worth-checking claims in political debates,” in _Proceedings of the International Conference Recent Advances in Natural Language Processing_. INCOMA Ltd., 2017, pp. 267–276.
  * [7] L. Konstantinovskiy, O. Price, M. Babakar, and A. Zubiaga, “Towards automated factchecking: Developing an annotation schema and benchmark for consistent automated claim detection,” _ACM Digital Threats: Research and Practice_ , 2020.
  * [8] M. Hasanain, R. Suwaileh, T. Elsayed, A. Barrón-Cedeno, and P. Nakov, “Overview of the clef-2019 checkthat! lab: Automatic identification and verification of claims. task 2: Evidence and factuality.” in _CLEF (Working Notes)_ , 2019.
  * [9] A. Zubiaga, A. Aker, K. Bontcheva, M. Liakata, and R. Procter, “Detection and resolution of rumours in social media: A survey,” _ACM Computing Surveys (CSUR)_ , vol. 51, no. 2, pp. 1–36, 2018.
  * [10] G. Gorrell, E. Kochkina, M. Liakata, A. Aker, A. Zubiaga, K. Bontcheva, and L. Derczynski, “Semeval-2019 task 7: Rumoureval, determining rumour veracity and support for rumours,” in _Proceedings of the 13th International Workshop on Semantic Evaluation_ , 2019, pp. 845–854.
  * [11] A. Zubiaga, M. Liakata, and R. Procter, “Exploiting context for rumour detection in social media,” in _International Conference on Social Informatics_. Springer, 2017, pp. 109–123.
  * [12] Z. Zhao, P. Resnick, and Q. Mei, “Enquiring minds: Early detection of rumors in social media from enquiry posts,” in _Proceedings of the 24th international conference on world wide web_ , 2015, pp. 1395–1405.
  * [13] A. Barrón-Cedeño, T. Elsayed, P. Nakov, G. Da San Martino, M. Hasanain, R. Suwaileh, and F. Haouari, “Checkthat! at clef 2020: Enabling the automatic identification and verification of claims in social media,” in _European Conference on Information Retrieval_. Springer, 2020, pp. 499–507.
  * [14] A. Barrón-Cedeño, T. Elsayed, P. Nakov, G. Da San Martino, M. Hasanain, R. Suwaileh, F. Haouari, N. Babulkov, B. Hamdan, A. Nikolov, S. Shaar, and Z. Sheikh Ali, “Overview of CheckThat! 2020: Automatic identification and verification of claims in social media,” ser. LNCS (12260), A. Arampatzis, E. Kanoulas, T. Tsikrika, S. Vrochidis, H. Joho, C. Lioma, C. Eickhoff, A. Névéol, L. Cappellato, and N. Ferro, Eds. Springer, 2020.
  * [15] E. Kochkina, M. Liakata, and A. Zubiaga, “All-in-one: Multi-task learning for rumour verification,” in _Proceedings of the 27th International Conference on Computational Linguistics_ , 2018, pp. 3402–3413.
  * [16] J. Ma, W. Gao, and K.-F. Wong, “Detect rumors in microblog posts using propagation structure via kernel learning,” in _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , vol. 1, 2017, pp. 708–717.
  * [17] M. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , 2018, pp. 2227–2237.
  * [18] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 2019, pp. 4171–4186.
  * [19] M. Müller, M. Salathé, and P. Kummervold, “Covid-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter,” _arXiv preprint arXiv:2005.07503_ , 2020.
  * [20] A. Olteanu, C. Castillo, F. Diaz, and S. Vieweg, “Crisislex: A lexicon for collecting and filtering microblogged communications in crises,” in _Eighth international AAAI conference on weblogs and social media_ , 2014\.
  * [21] B. Ghanem, M. Montes-y Gómez, F. Rangel, and P. Rosso, “Upv-inaoe-autoritas-check that: Preliminary approach for checking worthiness of claims,” in _CLEF_ , 2018.


# Vanquishing the computational cost of passive gamma emission tomography
simulations: a physics-aware reduced order modeling approach

Nicola Cavallini  European Commission, Joint Research Centre, Via Enrico
Fermi, Ispra, I -211027 (Va), Italy  Riccardo Ferretti  European Commission,
Joint Research Centre, Via Enrico Fermi, Ispra, I -211027 (Va), Italy  Gunnar
Bostrom  European Commission, Joint Research Centre, Via Enrico Fermi, Ispra,
I -211027 (Va), Italy  Stephen Croft  Lancaster University, Bailrigg,
Lancaster, UK  Aurora Fassi  European Commission, Joint Research Centre, Via
Enrico Fermi, Ispra, I -211027 (Va), Italy  Giovanni Mercurio  European
Commission, Joint Research Centre, Via Enrico Fermi, Ispra, I -211027 (Va),
Italy  Stefan Nonneman  European Commission, Joint Research Centre, Via Enrico
Fermi, Ispra, I -211027 (Va), Italy  Andrea Favalli  European Commission,
Joint Research Centre, Via Enrico Fermi, Ispra, I -211027 (Va), Italy  Los
Alamos National Laboratory, P.O. Box 1663, Los Alamos, NM 87545, USA

###### Abstract

Passive Gamma Emission Tomography (PGET) has been developed by the
International Atomic Energy Agency as a way to directly image the spatial
distribution of individual fuel pins in a spent nuclear fuel assembly and so
determine potential diversion.

Constructing the analysis and interpretation of PGET measurements rely on the
availability of comprehensive datasets. Experimental data are expensive,
limited, and so are augmented by Monte Carlo simulations. The main issue
concerning Monte Carlo simulations is the high computational cost to simulate
the 360 angular views of the tomography. Similar challenges pervade numerical
science.

To address this challenge, we have developed a physics-aware reduced order
modeling approach. It provides a framework to combine a small subset of the
360 angular views with a computationally inexpensive proxy solution, that
brings the essence of the physics, to obtain a real-time high-fidelity
solution at all angular views, but at a fraction of the computational cost.

## Introduction

Passive Gamma Emission Tomography (PGET) is a measurement technique developed
for spent nuclear fuel verification to help meet nuclear non-proliferation and
safeguards requirements. It is part of the armory of technical measures put in
place by the International Atomic Agency (IAEA), in the framework of the
Treaty on the Non-Proliferation of Nuclear Weapons (NPT), to ensure that each
Member State complies with the requirements of spent nuclear fuel stewardship
and accountancy. The PGET system is designed to perform partial defect and
bias defect verification of a spent nuclear fuel assembly while it is shielded
underwater prior to being moved to long term storage or into a geological
repository. The IAEA approved PGET in 2017 for inspections [17, 16, 26, 18, 5,
9].

The PGET detects the passive gamma-ray emissions from the assembly arising
from the build-up of fission products generated during the use in the reactor.
The most important gamma-ray signatures come from the fission products ^137Cs,
^134Cs, ^154Eu, ^106Ru, and ^144Ce [10]. They are important because they
confer information about the initial enrichment, burnup and cooling time of
the assembly while being practical to measure. Only a few fission products are
produced in sufficient quantity, and have intense yet long-lived and
penetrating gamma-ray emissions to be of general interest for nondestructive
assay applications.

The PGET design consists of two highly collimated linear arrays comprising up
to 91 cadmium-zinc-telluride (CdZnTe) gamma detectors. The detector pitch is 4
mm. Each detector is behind a tungsten collimator 15 mm wide, 100 mm thick,
and 5 mm high in front of the detector, increasing to 70 mm at the exit. The
two detector heads are on a rotating disk on opposite sides with a 2 mm
offset. This 2 mm stagger is devoted to obtaining 182 data collections,
referred to as counts, per each view. The technical details of the PGET
detector system are reported in [3, 2, 26].

The two detectors arrays rotate continuously around the spent fuel assembly.
The number of events are integrated both over energy and rotation windows.
Typical values for the energy windows are: <400 keV, 400-600 keV, 600-700 keV,
and 700-1200 keV, while rotations are integrated over each of the 360 degrees.
The end product of a PGET acquisition sequence is a sinogram for each energy
window, a matrix characterized by a number of rows equal to the number of
detectors and a number of columns equal to the number of angular views, 360 in
a standard acquisition. An accurate sinogram is needed as the input to
tomographic reconstruction algorithms aimed at reconstructing spent fuel axial
cross-sections in order to detect cases of anomalies or diversion, even to the
single rod level (so-called bias defect).

It is worth mentioning that compared with medical and industrial tomography
systems, PGET involves peculiar challenges due to the wide range of gamma
activities that characterize the spent nuclear fuel and the high gamma
attenuation of the nuclear materials. These parameters are outside the control
of the experimenter and are challenging in the case of detecting anomalies in
the center of the assemblies [25].

Development and evaluation studies of algorithms for the analysis and
interpretation of PGET measurements, for example, based on artificial
intelligence [9, 15], rely on the availability of extensive and comprehensive
datasets of experimental and/or simulated data. Experimental data are
extremely limited in availability and breadth of potential diversion
scenarios, and are also expensive to generate. Realistic simulations of spent
nuclear fuel measurements are therefore essential in evaluating the
performance of the algorithms behind the interpretation of PGET measurements,
especially in robustly identifying potential diversion situations.

Monte Carlo simulations, for example using the Monte Carlo N-Particle (MCNP)
code[14], allow the construction of a realistic and extended dataset of
simulated cases and related sinograms; however, the computational cost is
high. High-fidelity simulations, in fact, require the treatment of the full
physics radiation transport in a detailed geometry of the PGET system. Miller
et al. and Wittman et al. pointed out that a high-fidelity MCNP simulation of
the PGET measurement of a fuel assembly, the basic 360 angular views of a
sinogram, take about 6-7 days in a cluster composed of 128 nodes and 8192
cores [18, 30, 21].

To address this challenge, we have developed a physics-aware reduced order
modeling approach. In this approach we combine (1) a limited number of the 360
angular views (limited views tomography) and (2) a real time analytically-
built approximated sinogram (based on the Lambert-Beer law) at all the 360
views, that brings the essence of the physics. The computational cost is cut
by only simulating a sparse subset of angular views. The computational benefit
is proportional to the number of skipped views (e.g., simulating 60 views
instead of 360 reduces the cost to about one-sixth). The Physics-Aware Reduced
Order Model approach then enables the reconstruction of all the angular views.
This is achieved using Proper Orthogonal Decomposition (POD)[22, 19, 7, 11], a
subset of the techniques available in Reduced Order Modeling numerical
methods, which identifies the dominant patterns in the data by applying
Singular Value Decomposition (SVD) to the matrix of the limited views data. We
named our method Physics-Aware Proper Orthogonal Decomposition (PA-POD).

The method’s performance is tested and measured against the data released by
IAEA under the PGET Tomographic and Analysis Challenge (2019 [1, 29]. We used
the IAEA field dataset as surrogate of realistic simulations. The dataset is
used as the ground truth to which apply our PA-POD method. We sample limited
set of angular views, and we reconstruct the sinogram at each angular view via
PA-POD. The goodness of the method is obtained by comparison between ground
truth and the reconstruction. Among the collection of IAEA data, three cases
are of particular interest because they refer to field scenarios. They are
named competition three, four, and five, and related to a VVER (water-water
energetic reactor), a PWR (Pressurized Water Reactor), and a BWR (Boiling
Water Reactor) fuel assembly, respectively. In the Results section we report
the results of the PA-POD method applied to the IAEA PWR assembly case, as PWR
nuclear plants are the large majority of nuclear plants in the world. All the
results presented refer to the 600-700 keV window: it contains the ^137Cs
gamma emission line of 661.7 keV, a primary gamma emission of spent nuclear
fuel with a relatively long half-life of 30 years[10].

## Our Approach

We consider the sinogram matrix $\mathbf{S}$, our ground truth, as a discrete
representation of an unknown continuous solution $\mathbf{s}(y,\theta)$, where
$y$ represents the spatial coordinate of the detector and $\theta$ is the
angular coordinate. POD approximates the solution as a linear combination of
dominant patterns, or modes, ${\mathbf{u}_{i}(y)}$ estimated using only a
reduced set of angular views. The approximated solution is:

$\mathbf{\tilde{s}}(y,\theta)=\sum_{i=0}^{k-1}\mathbf{u}_{i}(y)\,\mathbf{c}_{i}(\theta),$
(1)

with $k$ the number of selected modes. All the possible linear combination of
the modes $\\{\mathbf{u}_{i}(y)\\}$, with $i=0,\ldots,k-1$, construct the so-
called POD solution space (or POD space), where $k$ is its dimesionality. In
the following steps, we fix the notation and summarize our PA-POD approach.

  1. 1.

Database Creation: Take the set of $N_{s}$ simulated/measured limited angular
views at the angular coordinates $\theta_{n}$ and construct the set of pairs
$\\{\mathbf{s}(y,\theta_{n}),\theta_{n}\\}$, with $n=0,\ldots,N_{s}-1$. They
are stored in a database matrix $\mathbf{\hat{S}}_{N\times N_{s}}$, where $N$
is the number of detectors.

  2. 2.

POD Solution Space Construction: Apply Singular Value Decomposition to the
database matrix: $\mathbf{\hat{S}}=\mathbf{U}\,\bm{\Sigma}\,\mathbf{V}^{*}$.
Meaning decompose the database matrix into two unitary matrices $\mathbf{U}$
and $\mathbf{V}$ and a diagonal matrix $\bm{\Sigma}$. Here ∗ denotes the
complex conjugate transpose. The diagonal elements $\delta_{j}$ of
$\bm{\Sigma}$, or singular values, are non-negative and ordered from the
largest to the smallest. They are usually normalized in the form
$\sigma_{j}=\frac{\delta_{j}}{\sum_{l=0}^{N-1}\delta_{l}}$. We define
information variance relative to the $j$-th singular value as the sum of all
the first $\sigma_{j}$. It is a measure of how much of the total data
variability is captured by the first $j$ modes [4, 11]. The $i$-th column of
$\mathbf{U}$ stores the mode $\mathbf{u}_{i}(y)$. To construct a suitable POD
solution space we balance between the number of modes and the information
variance. This reduces the dimensionality of the problem and avoids selecting
higher modes [4] which are generally associated with noise rather than
information.

  3. 3.

Coefficients Estimation: We use $\mathbf{U^{*}}_{k\times N}$, the conjugate
transpose of the first $k$ columns of the modes matrix $\mathbf{U}$, to
project into the POD space $\mathbf{R}_{N\times 360}$, a computationally
inexpensive approximation of the full sinogram $\mathbf{S}$. We refer to this
approximation as the Real-Time Approximate Forward Model, shortened as Real-
Time Model. In the PGET case it consists in a Lambert exponential attenuation
model applied to a voxelization of the fuel assembly geometry. It simplifies
the gamma photon transport and enables a real time approximation at any
possible PGET angular view. This model was initially proposed by [2], details
on our own implementation are described in section Methods. As a result we
obtain the estimation of the set of coefficients at the 360 views,
$\\{\mathbf{c}_{i}(\theta_{m})\\}$ in equation (1), with $i=0,\ldots,k-1$, and
$m=0,\ldots,359$. In matrix form: $\mathbf{\tilde{C}}_{k\times
360}=\mathbf{U^{*}}_{k,N}\,\mathbf{R}_{N\times 360}$. The coefficients
$\tilde{\mathbf{{C}}}$ are row wise scaled to match $\mathbf{\hat{S}}$ values.

  4. 4.

Solution Evaluation: The final approximated sinogram is represented by
equation(1), that can be written in matrix form:
$\mathbf{\tilde{S}}=\mathbf{U}\,\mathbf{\tilde{C}}$.

We define and report the error on the reconstructed image rather than on the
sinogram because in the PGET analysis the sinogram quality is judged by the
quality of its reconstruction. As a metric we choose to quantify the error
matrix $\mathbf{e}$ of the filtered back projection (FBP, mathematically
represented with the operator $\mathcal{F}(\cdot)$) applied to the ground
truth, to the Real-Time Approximated Forward Model, and to our PA-POD
estimation. Specifically, we used the ramp-filtered version of FBP [13, 23].
In explicit terms, we have the following pixel wise error map definitions:

$\mathbf{e}_{\mathrm{Real-Time\
Model}}=\left|\frac{\mathcal{F}(\mathbf{R})-\mathcal{F}(\mathbf{S})}{\mathcal{F}(\mathbf{S})}\right|;\quad\mathbf{e}_{\mathrm{PA-
POD}}=\left|\frac{\mathcal{F}(\mathbf{\tilde{S}})-\mathcal{F}(\mathbf{S})}{\mathcal{F}(\mathbf{S})}\right|.$

We apply a mask (matrix) to select the $\mathcal{F}(\mathbf{S})$ values that
are greater than 15% of the maximum. The mask covers the assembly cross
section and its premises, this area is addressed as $\Omega_{\mathrm{tot}}$.

Furthermore, we use the cumulative error distribution as an integral error
measure of maps $\mathbf{e}_{\mathrm{Real-Time\ Model}}$ and
$\mathbf{e}_{\mathrm{PA-POD}}$. Given an error threshold
$\epsilon_{\mathrm{th}}$, we count the number of pixels with a smaller or
equal error, we label it with $\Omega|_{\epsilon_{\mathrm{th}}}$. We define
the pixel fraction as the ratio between this area and $\Omega_{\mathrm{tot}}$:

$\mathrm{pixel\
fraction}=\frac{\Omega|_{\epsilon_{\mathrm{th}}}}{\Omega_{\mathrm{tot}}},$ (2)

the greater its value, the better the performance. The metric we use in this
work does not require any arbitrary filtering as the commonly used Structural
Similarity Index [28].

## PWR Case Results

Ground TruthIAEA competition, PWR case sinogram: $\mathbf{S}\ =\
\left[\begin{array}[]{ c }\\\ \\\ \\\ \\\ \\\ \\\ \\\
\end{array}\right.$$\left.\begin{array}[]{ c }\\\ \\\ \\\ \\\ \\\ \\\ \\\
\end{array}\right]_{N\times 360}$$\theta=7^{\circ}$ $\theta=15^{\circ}$
$\theta=22^{\circ}$ $\theta_{N_{s}-1}$ Database Matrixis constructed using a
set of limited views,example angles
$[7^{\circ},15^{\circ},22^{\circ},\theta_{N_{s}-1}]_{N_{s}}$:$\begin{aligned}
\mathbf{\hat{S}}={}&\left[\begin{array}[]{ c c c c c }|&|&|&&|\\\
\mathbf{s}(7^{\circ})&\mathbf{s}(15^{\circ})&\mathbf{s}(22^{\circ})&\cdots&\mathbf{s}(\theta_{N_{s}-1})\\\
|&|&|&&|\end{array}\right]_{N\times N_{s}}=\\\ \stackrel{{\scriptstyle
SVD}}{{=}}&\left[\begin{array}[]{ c c c c c }|&|&|&&|\\\
\mathbf{u}_{0}&\mathbf{u}_{1}&\mathbf{u}_{2}&\cdots&\mathbf{u}_{N-1}\\\
|&|&|&&|\end{array}\right]_{N\times
N_{s}}\bm{\Sigma}\mathbf{V}^{*}\end{aligned}$Singular Value Decomposition is
applied to $\mathbf{\hat{S}}$to identify dominant patterns, i.e. construct POD
solution space:POD Solution Space Construction,Dimensionality ReductionThe
approximated solution is combination of selected $k$ basis
functions$\tilde{\mathbf{s}}(y,\theta)=\sum_{i=0}^{k-1}\mathbf{u}_{i}(y)\
\mathbf{c}_{i}(\theta),$in matrix form:$\mathbf{\tilde{S}}=\mathbf{U}_{N\times
k}\ \mathbf{\tilde{C}}_{k\times 360}$Real-Time Forward Approximate
ModelComplete sinogram of the PWR casecalculated with the Real-Time Modelat
all the angular views. $\mathbf{R}=\left[\begin{array}[]{ c }\\\ \\\ \\\ \\\
\\\ \\\ \\\ \end{array}\right.$$\left.\begin{array}[]{ c }\\\ \\\ \\\ \\\ \\\
\\\ \\\ \end{array}\right]_{N\times 360}$Coefficients EstimationWe project the
real time solution into the POD space:$\tilde{\mathbf{C}}_{k\times
360}=\mathbf{U^{*}{}}_{k\times N}\mathbf{R}_{N\times 360}.$Solution
Evaluation$\tilde{\mathbf{S}}_{N\times 360}=\mathbf{U}_{N\times
k}\tilde{\mathbf{C}}_{k\times 360}$$\mathbf{\tilde{S}}=\left[\begin{array}[]{
c }\\\ \\\ \\\ \\\ \\\ \\\ \\\ \end{array}\right.$$\left.\begin{array}[]{ c
}\\\ \\\ \\\ \\\ \\\ \\\ \\\ \end{array}\right]_{N\times 360}$ Figure 1: The
figure reports the major steps in our PA-POD method. On top left we represent
the actual sinogram for the PWR assembly from the IAEA competition (our ground
truth), as high fidelity data, named $\mathbf{S}$ in our formulation. With the
orange color we highlight the subset of limited views that construct the
database matrix. We present the POD solution space and plot the first three
modes $\mathbf{u}_{0},\mathbf{u}_{1},\mathbf{u}_{2}$ for the case of
$N_{s}=k=60$ randomly chosen views. On bottom left, the sinogram $\mathbf{R}$,
on the same pin configuration, as obtained by the Real-Time Approximate
Forward Model. At the bottom right the PA-POD formulation and resulting
sinogram $\tilde{\mathbf{S}}$.

The following results report the application of our PA-POD approach, starting
from a sparse sampling of the 360 angular views of the PWR assembly sinogram
included in the IAEA competition dataset. Specifically, it consists of a
10$\times$10 pins assembly where a 3$\times$3 block of pins has been removed;
thus it is also a case of a diversion scenario. The Real-Time Model relies on
the investigation domain discretisation, see section Methods for the details,
the reported results are computed with the following mesh sizes: $\Delta z=10$
mm, $\Delta x=\Delta y=0.5$ mm, being $x,y,z$ the three coordinate axes of the
investigation domain. Both the ground truth sinogram and the Real-Time one are
normalized between zero and one.

The flowchart in figure 1 summarises our method and presents an overview of
the results of the PWR case. In particular, we report the ground truth
sinogram $\mathbf{S}$, the real-time approximated sinogram $\mathbf{R}$, the
first three modes $\mathbf{u}_{0},\mathbf{u}_{1},\mathbf{u}_{2}$ and the PA-
POD approximate sinogram $\tilde{\mathbf{S}}$. In the real-time approximated
forward model, we assume the intensity of pins of the PWR assembly is flat
where the pins are located, and zero in the 3 by 3 block in the lower left
center where the pins are removed. The PA-POD sinogram $\tilde{\mathbf{S}}$ is
constructed randomly sampling 60 views for the database matrix, and the first
60 modes are selected to construct the POD space. The orange views highlighted
in the top left corner have the objective of proving a visualisation of the
database matrix, the rotation angles are just examples.

Figure 2 represents the error and quantifies it. In picture (a) the ground
truth, while the error is spatially visualized in the maps (b) and (c). To
avoid any procedural biasing effect, we repeat the views sampling 100 times,
and at each pixel we plot the median value for the error. Figure (c) shows
that PA-POD can correct most of the high error pixels in between pins and
around the assembly. The PA-POD relative error map clearly indicates that the
approach reproduces the actual data within a relative error of the order of
10%.

Figure 2(d) and (e) complete the picture plotting the error cumulative
distribution, that we called pixel fraction to emphasize its geometrical
meaning, see equation 2. Figure 2(d) starts with a minimal number of pixels
having the same value as the ground truth due to the normalization of the two
sinograms. As soon as we consider larger errors, the area that PA-POD can
describe with a specific accuracy increases with a far steeper gradient than
the Real-Time Model. Figure 2 (b), shows the results with a finer-graded
detail. We focus on the 10% relative error and observe that PA-POD covers 58%
of the total area, while the Real- Time Model describes 29% with the same
accuracy. These results show that it is possible to approximately double the
accuracy of the real- time model using 16% of the total number of views.

Figure 2: Here we represent the ground truth (a) together with the error that
characterizes the real time model (b) and our PA-POD approximation (c). In the
PA-POD case we randomly sample sixty views of the spent fuel, we repeat the
sample one hundred times, and for each pixel, we collect the median of the
sampled data. (d) The improvement from the Real Time Model to PA-POD is
quantified by the area between the blue and orange lines. (d) More precisely,
considering a 10% error, PA-POD describes 57% of the total area, while the
Real Time Model describes 29% of the area.

The convergence of the PA-POD method is studied in further detail in figure
3(a). With $k=N_{s}$, we vary $N_{s}$ from 30 to 120, with a spacing of 10
units. For each value of $N_{s}$ we select 100 random uniform sets of views,
evaluate the pixel fraction at 10% and report the mean and the standard
deviation for each distribution. PA-POD expresses its best performance where
the sample is very sparse, in the range 30-60 samples. From 80 samples onwards
the convergence reaches a plateau, this behavior is due to the non-
interpolatory nature of the coefficients, see Discussion section for details.
Furthermore we notice that the standard deviation is smaller than $0.02$
expressed in pixel fraction at 10%, this implies that the performance is
consistent with respect the views choice. In figure 3(b) we present a closer
view of the pixel fraction distribution with respect the views choice. We
consider the pixel fraction at 10% error and we evaluate it for 1000 uniformly
random sets of sixty views, with $N_{s}=k=60$. The mean is 0.539 and the
standard deviation is 0.019 expressed in pixel fraction. This confirms the
method is robust with respect the views selection. In both figures 3(a) and
(b) we see that the mean values for the pixel fraction at the 10% error are
slightly smaller compared to the values in figure 2, 0.539, compared to 0.57.
Evaluating the error pixel by pixel, repeating and collecting the median, as
in figure 2, has a minimal regularization effect on the error metric, without
any consequence for our overall conclusions.

Figure 3: (a) With $k=N_{s}$, we vary $N_{s}$ from 30 to 120, with a spacing
of 10 units. For each value of $N_{s}$ we select 100 random uniform sets of
views, evaluate the pixel fraction at 10% and report the mean and the standard
deviation for each distribution. The method delivers its best performance
between 60 and 80 samples. In figure (b) a closer view of the pixel fraction
distribution with respect the sample choice. With $N_{s}=k=60$ we pick 1000
random uniform sets of views, evaluate the 10% pixel fraction for each set and
plot the distribution. We observe that the mean is 0.545 and the standard
deviation is 0.017, both the measures are expressed in terms of pixel
fraction.

Figure 4 plots the singular values for the PWR ground truth sinogram (a) and
the corresponding information variance (b). Figure 4(b) highlights that a
reduced set of modes captures the majority of the information variance. In
particular 8, 27, and 45 modes capture 80%, 90% and 95% of the total
information variance respectively. This result confirms the underlying
assumption of our work: a singoram, experimentally measured or numerically
computed, is described by a limited set of dominant patterns, that carefully
combined together can provide a complete solution with a sparse set of angular
views.

Figure 4:  (a) Logarithmic plot of the singular values spectrum for the IAEA
PWR fuel assembly sinogram, and (b) the related information variance. We
highlight that it takes respectively 8, 27, and 45 modes to capture 80%, 90%
and 95% of the total information variance.

## Discussion

Our approach was in part inspired by a particular flavor of Proper Orthogonal
Decomposition that uses interpolation to estimate the coefficients, namely POD
with Interpolation (PODI). In this domain of interest, the research effort is
devoted to finding the most accurate interpolator for each applicable case
[22, 19, 7, 11]. PODI is particularly appealing because it is agnostic of the
underlying physics of the phenomenon, it has no notion of the mathematical
method describing the prototype, in ROM literature these methods are called
non-intrusive [11]. In this section we will briefly summarize the steps that
define the method, we will show its limitations in the PGET case, and how we
addressed them while developing PA-POD.

In PODI the interpolatory coefficients are estimated by projecting the
database matrix onto the POD space:
$\mathbf{C}=\mathbf{U^{*}}\,\mathbf{\hat{S}}$. The columns of
$\mathbf{C}_{k\times N_{s}}$ construct a set of pairs
$\\{\mathbf{c}_{j},\theta_{j}\\}$, with $j=0,\ldots,N_{s}-1$. Consider
$\tilde{\theta}$ a rotation angle not included in our sample, we interpolate
the coefficients $c_{ij}$ to get $\\{\tilde{\mathbf{c}},\tilde{\theta}\\}$ and
reconstruct:

$\mathbf{\tilde{s}}(y,\tilde{\theta})=\sum_{i=0}^{k-1}\mathbf{u}_{i}(y)\cdot\tilde{\mathbf{c}}.$

$\mathbf{\tilde{C}}_{k\times 360}=\mathcal{I}(\mathbf{C}_{k\times N_{s}})$ is
the coefficients matrix for the full 360 degrees, it is the result of the
interpolation operator $\mathcal{I}$ applied to $\mathbf{\hat{S}}$ projected
into the POD space.

We compare the PODI and PA-POD approaches side by side in figure 5. In both
cases we look for the same solution structure:

$\mathbf{\tilde{S}}_{N\times 360}=\mathbf{U}_{N\times
k}\tilde{\mathbf{C}}_{k\times 360}$

In both cases, the coefficients are evaluated by projecting available data
into the POD space:

$\mathbf{C}_{k\times N_{s}}=\mathbf{U^{*}}_{k\times
N}\,\mathbf{\hat{S}}_{N\times
N_{s}}\quad\mathrm{vs.}\quad\tilde{\mathbf{C}}_{k\times
360}=\mathbf{U^{*}{}}_{k\times N}\mathbf{R}_{N\times 360},$

while PODI interpolates accurate but scarce data $\mathbf{\hat{S}}_{N\times
N_{s}}$, conversely, PA-POD relies on approximate but dense data provided by
the Real-Time Model $\mathbf{R}_{N\times 360}$.

Physics-Aware PODDefineReal-Time Approximated Forward Model,as the operator
$\mathcal{R}(\theta)$.Evaluate the real time modelfor all the
parameters:$\mathbf{R}_{N\times 360}=\mathcal{R}([0^{\circ},\dotsc,\
359^{\circ}])$POD with InterpolationProject the database matrixin the POD
space,$\mathbf{C}_{k\times N_{s}}=\mathbf{U^{*}}_{k\times
N}\,\mathbf{S}_{N\times N_{s}},$to construct $N_{s}$ pairs
$\\{\mathbf{c}_{i},\theta_{i}\\}_{N_{s}}$. Define an interpolator:
$\mathcal{I}$Evaluate, via the interpolator,the coefficients for allthe
parameters:$\tilde{\mathbf{C}}_{k\times
360}=\mathcal{I}\left(\mathbf{C}_{k\times N_{s}}\right)$Project the Real-Time
Modelinto the POD spaceto obtain the coefficients
matrix:$\tilde{\mathbf{C}}_{k\times 360}=\mathbf{U^{*}{}}_{k\times
N}\mathbf{R}_{N\times 360}$The approximation is a combinationof coefficients
and basis functions$\tilde{\mathbf{S}}_{N\times 360}=\mathbf{U}_{N\times
k}\tilde{\mathbf{C}}_{k\times 360}$ Figure 5: This picture shows a side by
side comparison between the PODI and PA-POD. In PODI we interpolate accurate
but scarce data, while with PA-POD we evaluate dense real time approximated
data.

In figure 6 we compare the FBP for several approximation techniques with the
ground truth, (a). The Real-Time Model (b) is an effective approximation in
describing the assembly geometry. Figures (c) and (d) are constructed with
$N_{s}=k=60$. PODI fails to capture the structure of the fuel assembly (c),
while PA-POD preserves its geometrical structure (d) and recovers part of the
secondary effects in the ground truth. Ring artifact is notable in the PA-POD
reconstruction, figure 6(d), this effect can be removed using the algorithm
provided in reference [27].

Figure 6: Filtered backprojection for the IAEA data (a), and the
approximations we are studying, $N_{s}=k=60$. Real-Time Model (b) and PA-POD
(d) preserve the geometrical structure of the spent fuel, while PODI cannot
provide a representative reconstruction (c).

We tested three different interpolators. The results are reported in figure 7,
the ground truth in figure (a), (b) linear interpolation on the coefficients
in the POD space, (c) linear interpolation on the data, (d) radial basis
functions interpolation on the coefficients, but none could preserve the
assembly geometry for $N_{s}=k=60$. In this phase of the exploration we relied
on the EZyRB package developed at Scuola Internazionale Superiore Studi
Avanzati [6].

Figure 7: This picture presents the filtered backprojection applied to three
different interpolations, with $N_{s}=60$ and $k=30$. (a) Ground truth; (b)
POD with linear interpolation of the coefficients; (c) Linear interpolation on
the data; (d) POD with radial basis functions interpolation. The geometrical
structure of the fuel assembly is severely compromised in all of three
approximations. None of them preserves the assembly pin structure.

The most important reason why interpolating methods show an inadequate
performance is the sampling scarcity. In particular, figures 8 and 9, show
that the interpolation points fall too far apart to be effectively
interpolated.

Figure 8 gives an overview of modes and coefficients, with $k=N_{s}=60$. It
shows the first four coefficients evaluated in three different ways. First, we
project the ground truth into the POD space:

$\mathbf{C}_{\mathrm{ground\ truth}}=\mathbf{U}^{*}\,\mathbf{S},$

this is the best possible approximation of the expected coefficients. Second,
we project the sample $\mathbf{\hat{S}}$ into the POD space, as prescribed by
the PODI approach. Third, we evaluate the coefficients with the PA-POD method,
projecting into the POD space an inexpensive approximation of the full
sinogram, namely the result of the Real-Time Model.

We notice that while the sample preserves an overall fitting with the ground
truth, important details are missed, causing the failure of the geometrical
reconstruction. On the other hand, PA-POD coefficients are not interpolatory
as in the PODI case but provide a better overall description of the assembly
geometry. We notice that the modes $\mathbf{U}(:,i)$ for the database matrix
$\mathbf{\hat{S}}$ are in good agreement with the ones obtained from the full
sinogram $\mathbf{S}$, figure 8.

Figure 8: Plots of the first four coefficients $\mathbf{C}(i,:)$ and modes
$\mathbf{U}(:,i)$ for the PWR sinogram, with $k=N_{s}=60$. A few discrepancies
are visible in the sampled coefficients, while the physics-aware ones have a
better definition especially in areas characterized by high oscillations. The
sampled modes, obtained applying SVD to $\mathbf{\hat{S}}$, are in good
agreement with ones obtained using the whole dataset $\mathbf{S}$.

Figure 9 shows several details of the coefficient matrices. The sampling is
not sufficiently detailed to capture the actual shape of the coefficient,
regardless of the accuracy of the interpolator, there is no way to recover
information that is not in the data unless physics awareness of the prototype
is restored, as we did with PA-POD.

It is evident, as depicted in figure 9, that PA-POD coefficients are not
interpolatory. As a consequence, the resulting sinogram is not exactly the
original one at the sampled locations.

Figure 9: In these plots a close up view of the ground truth coefficients, the
sampled ones, and the physics-aware ones are reported. Given $N_{s}=k=60$ with
randomly sampled views, we observed that the sampled coefficients do not
capture enough data to account for the local structure. The information
dropped during the sampling is in fact irrecoverable, unless we provide a
physics-aware way to reconstruct the coefficients. It is this notion of
“physics awareness” coefficients that inspired the main idea in this article.

### Future Directions

The ability of our PA-POD to create extended simulations at a low
computational cost enables the construction of a digital twin, where actual
measurements and simulated data are combined with a deep learning algorithm in
a continuous update as measurements and simulations are produced in real-time.
The benefit of doing so is to provide an integrated system, hardware, and
digital twin for real-time high-accuracy verification of irradiated nuclear
fuel assemblies using PGET. Our proposed approach can be readily adapted and
used for other applications in nuclear safeguards and beyond, such as the
creation of a dataset for the Tomographic Gamma Scanner technique [24], which
is a non-destructive assay technique, as well as in other
radiography/tomography approaches for nuclear material and global security
applications [12].

## Methods

### Reference data used in the paper (IAEA Competition)

Our method’s development and the associated verification have benefited from
high-fidelity data from real case scenarios. The dataset provided by the
International Atomic Energy Agency on the occasion of IAEA Tomographic and
Analysis Challenge fit the purpose. The data are publicly available [1, 29,
20]. Among several mockup data, few measurements are provided that were used
in the paper. In the paper, the original names of the IAEA data, “competition
3”, “competition 4”, and “competition 5” are renamed into VVER, PWR, BWR
respectively. In our presented results, we focused on the PWR case, 600-700
keV energy-deposition window, which it contains the full energy peak from the
^137Cs gamma emission at 661.7 keV, a major gamma emission of the spent
nuclear fuel [10].

### Real-Time Approximate Forward Model

Our Real-Time Model is built on the model of Backholm et al. [2]. A schematic
representation of the model set is depicted in figure 10. The investigation
domain $\Omega\subset{\mathbb{R}^{2}}$ is the axial cross-section of a nuclear
fuel assembly and we discretize it as a two-dimensional mesh. At each pixel
$p$ of the discretized domain, we define $\lambda\
\in\mathbb{R}^{N_{\mathrm{pix}}}$ the emission values and the attenuation
values $\mu\ \in\mathbb{R}^{N_{\mathrm{pix}}}$, $N_{\mathrm{pix}}$ is the
number of pixels. We consider the values of $\lambda$ and $\mu$ constant over
each pixel. At each detector $i$ we model the flux using a Lambert-Beer
(exponential) attenuation model [2]:

$F(\lambda,\mu)_{i}=H(\mu)_{i,p}\,\lambda_{p}=\left[r_{i,p}\,\mathrm{exp}\left(-c_{i,p}\,d^{T}_{i,p}\,\mu\right)\right]\,\lambda_{p}.$

The 360 views are obtained by rotating the pin positions, and so the assembly,
with respect to the detectors. The emission and attenuation maps are
consequently updated. The two-dimensional grid is further discretized,
dividing the half-space above and below each pixel, in voxels.

detector: $\displaystyle i$emission: $\displaystyle\lambda$pixel:
$\displaystyle p$detector: $\displaystyle i$attenuation
$\displaystyle\mu$collimatora) Emission Mapb) Attenuation Mapc) Solid Angled)
Elevation Angleelevation angle: $\displaystyle\alpha_{s}$detector:
$\displaystyle i$voxel: $\displaystyle s$2D griddetector: $\displaystyle
i$solid angle: $r_{s}^{3\mathrm{D}}$voxel: $\displaystyle s$2D grid Figure 10:
A schematic view of the major players in the real time model implementation.

Given this setting, the following quantities are defined.

  * •

$r_{i,p}$ is the response of the detector $i$ with respect to the pixel $p$.
Given a three-dimensional point, the barycenter of a voxel $s$, $r^{3D}_{s}$
is the solid angle between the voxel barycenter and the detector face, divided
by $4\pi$. Follows $r_{i,p}$ is the average of the responses for all the
voxels, insisting on the same pixel:
$r_{i,p}=\frac{1}{N_{p,\mathrm{vox}}}\sum_{s=1}^{N_{p,\mathrm{vox}}}r^{3D}_{s}$.

  * •

$d^{T}_{i,p}\,\mu$ is the integral of the discrete attenuation values on the
segment connecting the pixel $p$ and the detector $i$. In our implementation
we explicitly find the intersection between the connecting segment and the
grid, being the attenuation piecewise constant, a zero-order quadrature is
sufficient to exactly evaluate the integral.

  * •

$c_{i,p}$ is an attenuation correction factor, evaluated as:
$c_{i,p}=\frac{1}{r_{i,p}}\sum_{s=1}^{N_{p,\mathrm{vox}}}\frac{r^{3D}_{s}}{\mathrm{cos}(\alpha_{s})}$.
$\alpha_{s}$ is the angle between the voxel barycenter, the detector and the
two dimensional plane. It accounts for the response matrix $r_{i,p}$ and the
correction term $c_{i,p}$ are purely geometrical measures and are not affected
by the rotation of the fuel assembly. $d^{T}_{i,p}\,\mu$ instead depends on
the attenuation map, which depends on attenuation distribution, that changes
with the assembly rotation and has to be assembled for each rotation angle.

We tested three different mesh sizes, while $\Delta z=10\ \mathrm{mm}$ was the
same in all the three cases, the other dimensions are $\Delta x=\Delta y=(2.5\
\mathrm{mm},1.0\ \mathrm{mm},0.5\ \mathrm{mm})$. The mesh size does not affect
the accuracy of our results, the coarsest mesh size can be chosen to minimize
the computational cost.

Values for emission $\lambda$ attenuation $\mu$ are defined in [2]. Emission
is expressed in arbitrary units, and is 0 for water and 100 for the spent
fuel. Attenuation is $0.1356\ \mathrm{mm}^{-1}$ for the spent fuel, and
$0.0085\ \mathrm{mm}^{-1}$ for water. Virta et al. in [25] provide an
extensive exploration of such parameters in experimental settings. An
approximation of the Real-Time approximate forward model is that it does not
model the downscattering of the gamma rays.

The computational cost for a full sinogram according to this ray-tracing
scheme is a few minutes on a desktop machine, and the assembly of all the
matrices is a type of “embarrassingly parallel” task [8], it can be
parallelized both on a “per detector” basis or on a “per degree” basis.

## References

  * [1] IAEA tomographic challenge.  https://ideas.unite.un.org/iaea-tomography/Page/Home, 2019.  [Online; accessed 13-December-2022].
  * [2] R. Backholm, T. A. Bubba, S. Siltanen, C. Bélanger-Champagne, T. Helin, and P. Dendooven.  Simultaneous reconstruction of emission and attenuation in passive gamma emission tomography of spent nuclear fuel.  Inverse Problems and Imaging, 14(2):317–337, 2020.
  * [3] C. Belanger-Champagne, P. Peura, P. Eerola, T. Honkamaa, T. White, M. Mayorov, and P. Dendooven.  Effect of Gamma-Ray Energy on Image Quality in Passive Gamma Emission Tomography of Spent Nuclear Fuel.  IEEE Transactions on Nuclear Science, 66(1):487–496, 2019.
  * [4] S. L. Brunton and J. N. Kutz.  Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control.  Cambridge University Press, USA, 1st edition, 2019.
  * [5] D. Chernikova, K. Axell, S. Gavrilov, B. Cederwall, and A. Favalli.  A complementary method for traditional image reconstruction techniques with application to spent nuclear fuel assemblies.  In 59th Annual Meeting of the Institute of Nuclear Materials Management (INMM 2018). Institute of Nuclear Materials Management, 07 2018.
  * [6] N. Demo, M. Tezzele, and G. Rozza.  EZyRB: Easy Reduced Basis method.  The Journal of Open Source Software, 3(24):661, 2018.
  * [7] N. Demo, M. Tezzele, and G. Rozza.  A non-intrusive approach for the reconstruction of POD modal coefficients through active subspaces.  Comptes Rendus Mécanique, 347(11):873–881, Nov. 2019.
  * [8] Y. Deng.  Applied Parallel Computing.  World Scientific Publishing Co., Inc., USA, 2012.
  * [9] M. Fang, Y. Altmann, D. Della Latta, M. Salvatori, and A. Di Fulvio.  Quantitative imaging and automated fuel pin identification for passive gamma emission tomography.  Scientific Reports, 11(1):2442, Jan 2021.
  * [10] A. Favalli, D. Vo, B. Grogan, P. Jansson, H. Liljenfeldt, V. Mozin, P. Schwalbach, A. Sjöland, S. Tobin, H. Trellue, and S. Vaccaro.  Determining initial enrichment, burnup, and cooling time of pressurized-water-reactor spent fuel assemblies by analyzing passive gamma spectra measured at the clab interim-fuel storage facility in sweden.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 820:102–111, 2016.
  * [11] J. S. Hesthaven, G. Rozza, and B. Stamm.  Certified Reduced Basis Methods for Parametrized Partial Differential Equations.  Springer Briefs in Mathematics. Springer, Switzerland, 1 edition, 2015\.
  * [12] C.-K. Huang, D. P. Broughton, S. Palaniyappan, A. Junghans, M. Iliev, S. H. Batha, R. E. Reinovsky, and A. Favalli.  High-yield and high-angular-fluence neutron generation from deuterons accelerated by laser-driven collisionless shock.  Applied Physics Letters, 120(2):024102, 2022.
  * [13] A. C. Kak and M. Slaney.  Principles of computerized tomographic imaging.  Society for Industrial and Applied Mathematics, Philadelphia, 2001.
  * [14] J. A. Kulesza, T. R. Adams, J. C. Armstrong, S. R. Bolding, F. B. Brown, J. S. Bull, T. P. Burke, A. R. Clark, R. A. Forster, III, J. F. Giron, A. S. Grieve, C. J. Josey, R. L. Martz, G. W. McKinney, E. J. Pearson, M. E. Rising, C. J. Solomon, Jr., S. Swaminarayan, T. J. Trahan, S. C. Wilson, and A. J. Zukaitis.  MCNP® code version 6.3.0 theory & user manual.  Technical Report LA-UR-22-30006, Rev. 1, Los Alamos National Laboratory, Los Alamos, NM, USA, September 2022.
  * [15] A. Lahiri, M. L. Klasky, J. A. Fessler, and S. Ravishankar.  Sparse-view cone beam CT reconstruction using data-consistent supervised and adversarial learning from scarce training data, 2022.
  * [16] A. Lebrun, J. Beaumont, A. Berlizov, L. Bourva, Y. Dodane, D. Finker, M. Mayorov, and A. Sokolov.  The next generation of non-destructive assay tools for iaea safeguards verification.  In Proceeding of the Symposium on International Safeguards. International Atomic Energy Agency, 2022.
  * [17] M. Mayorov, T. White, A. Lebrun, J. Brutscher, J. Keubler, A. Birnbaum, V. Ivanov, T. Honkamaa, P. Peura, and J. Dahlberg.  Gamma emission tomography for the inspection of spent nuclear fuel.  In 2017 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pages 1–2, 2017.
  * [18] E. Miller, M. Vladimir, W. Timothy, C. Luke, D. Nikhil, W. Richard, and P. Pauli.  Assessing instrument performance for passive gamma emission tomography of spent fuel.  In INMM 59th Annual Meeting Paper Advanced Nondestructive Assay Techniques for Fuel Assemblies, 2018.
  * [19] G. Ortali, N. Demo, and G. Rozza.  Gaussian process approach within a data-driven POD framework for fluid dynamics engineering problems.  Mathematics in Engineering, 4(3):1–16, 2022.
  * [20] L. Presotto.  Tomographic reconstruction for the IAEA PGET detector: a competition entry.  In 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), pages 1–3, 2019.
  * [21] L. Smith, S. Jacobsson Svärd, V. Mozin, P. Jansson, E. Miller, T. White, N. Deshmukh, H. Trellue, R. Wittman, A. Davour, S. Grape, P. Andersson, S. Vaccaro, S. Holcombe, and T. Honkamaa.  A Viability Study of Gamma Emission Tomography for Spent Fuel Verification: JNT 1955 Phase I Technical Report.  Technical Report PNNL-25995, Pacific Northwest National Laboratory, USA, 10 2016.
  * [22] M. Tezzele, N. Demo, and G. Rozza.  Shape optimization through proper orthogonal decomposition with interpolation and dynamic mode decomposition enhanced by active subspaces.  In VIII International Conference on Computational Methods in Marine Engineering, 2019.
  * [23] S. Van der Walt, J. L. Schönberger, J. Nunez-Iglesias, F. Boulogne, J. D. Warner, N. Yager, E. Gouillart, and T. Yu.  scikit-image: image processing in python.  PeerJ, 2:e453, 2014.
  * [24] R. Venkataraman, M. Villani, S. Croft, P. McClay, R. McElroy, S. Kane, W. Mueller, and R. Estep.  An integrated tomographic gamma scanning system for non-destructive assay of radioactive waste.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 579(1):375–379, 2007.  Proceedings of the 11th Symposium on Radiation Measurements and Applications.
  * [25] R. Virta, R. Backholm, T. A. Bubba, T. Helin, M. Moring, S. Siltanen, P. Dendooven, and T. Honkamaa.  Fuel rod classification from passive gamma emission tomography (PGET) of spent nuclear fuel assemblies.  ESARDA Bulletin, 2020(61):10–21, 2020.
  * [26] R. Virta, T. A. Bubba, M. Moring, S. Siltanen, T. Honkamaa, and P. Dendooven.  Improved passive gamma emission tomography image quality in the central region of spent nuclear fuel.  Scientific Reports, 12(1):12473, Jul 2022.
  * [27] N. T. Vo, R. C. Atwood, and M. Drakopoulos.  Superior techniques for eliminating ring artifacts in x-ray micro-tomography.  Opt. Express, 26(22):28396–28412, Oct 2018.
  * [28] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  * [29] T. White, A. Lebrun, T. Honkamaa, M. Mayorov, P. Peura, J. Dahlberg, J. Keubler, V. Ivanov, and A. Turunen.  Application of passive gamma emission tomography (pget) for the verification of spent nuclear fuel.  In 59th Annual Meeting of the Institute of Nuclear Materials Management (INMM 2018). Institute of Nuclear Materials Management, 07 2018.
  * [30] R. Wittman, L. Campbell, N. Deshmukh, P. Jansson, E. Miller, and V. Mozin.  Benchmarking update to IAEA.  Technical Report PNNL-SA-13389, Pacific Northwest National Laboratory, 2018.