* Niedźwiecki et al. (2016) Niedźwiecki, A., Zdziarski, A. A., and Szanecki, M. (2016).  On the Lamppost Model of Accreting Black Holes.  _ApJ_ 821, L1.  10.3847/2041-8205/821/1/L1
  * Noda and Done (2018) Noda, H. and Done, C. (2018).  Explaining changing-look AGN with state transition triggered by rapid mass accretion rate drop.  _MNRAS_ 480, 3898–3906.  10.1093/mnras/sty2032
  * Oh et al. (2018) Oh, K., Koss, M., Markwardt, C. B., Schawinski, K., Baumgartner, W. H., Barthelmy, S. D., et al. (2018).  The 105-Month Swift-BAT All-sky Hard X-Ray Survey.  _ApJS_ 235, 4.  10.3847/1538-4365/aaa7fd
  * Panessa et al. (2019) Panessa, F., Baldi, R. D., Laor, A., Padovani, P., Behar, E., and McHardy, I. (2019).  The origin of radio emission from radio-quiet active galactic nuclei.  _Nature Astronomy_ 3, 387–396.  10.1038/s41550-019-0765-4
  * Parker et al. (2014) Parker, M. L., Wilkins, D. R., Fabian, A. C., Grupe, D., Dauser, T., Matt, G., et al. (2014).  The NuSTAR spectrum of Mrk 335: extreme relativistic effects within two gravitational radii of the event horizon?  _MNRAS_ 443, 1723–1732.  10.1093/mnras/stu1246
  * Pasham et al. (2019) Pasham, D. R., Remillard, R. A., Fragile, P. C., Franchini, A., Stone, N. C., Lodato, G., et al. (2019).  A loud quasi-periodic oscillation after a star is disrupted by a massive black hole.  _Science_ 363, 531–534.  10.1126/science.aar7480
  * Petrucci et al. (2020) Petrucci, P. O., Gronkiewicz, D., Rozanska, A., Belmont, R., Bianchi, S., Czerny, B., et al. (2020).  Radiation spectra of warm and optically thick coronae in AGNs.  _A &A_ 634, A85.  10.1051/0004-6361/201937011
  * Petrucci et al. (2001a) Petrucci, P. O., Haardt, F., Maraschi, L., Grandi, P., Malzac, J., Matt, G., et al. (2001a).  Testing Comptonization Models Using BeppoSAX Observations of Seyfert 1 Galaxies.  _ApJ_ 556, 716–726.  10.1086/321629
  * Petrucci et al. (2004) Petrucci, P. O., Maraschi, L., Haardt, F., and Nandra, K. (2004).  Physical interpretation of the NGC 7469 UV/X-ray variability.  _A &A_ 413, 477–487.  10.1051/0004-6361:20031499
  * Petrucci et al. (2001b) Petrucci, P. O., Merloni, A., Fabian, A., Haardt, F., and Gallo, E. (2001b).  The effects of a Comptonizing corona on the appearance of the reflection components in accreting black hole spectra.  _MNRAS_ 328, 501–510.  10.1046/j.1365-8711.2001.04897.x
  * Petrucci et al. (2013) Petrucci, P. O., Paltani, S., Malzac, J., Kaastra, J. S., Cappi, M., Ponti, G., et al. (2013).  Multiwavelength campaign on Mrk 509. XII. Broad band spectral analysis.  _A &A_ 549, A73.  10.1051/0004-6361/201219956
  * Petrucci et al. (2023) Petrucci, P. O., Piétu, V., Behar, E., Clavel, M., Bianchi, S., Henri, G., et al. (2023).  Simultaneous millimetric and X-ray intraday variability in the radio-quiet AGN MCG+08-11-11.  _A &A_ 678, L4.  10.1051/0004-6361/202347495
  * Petrucci et al. (2018) Petrucci, P. O., Ursini, F., De Rosa, A., Bianchi, S., Cappi, M., Matt, G., et al. (2018).  Testing warm Comptonization models for the origin of the soft X-ray excess in AGNs.  _A &A_ 611, A59.  10.1051/0004-6361/201731580
  * Porquet et al. (2019) Porquet, D., Done, C., Reeves, J. N., Grosso, N., Marinucci, A., Matt, G., et al. (2019).  A deep X-ray view of the bare AGN Ark 120. V. Spin determination from disc-Comptonisation efficiency method.  _A &A_ 623, A11.  10.1051/0004-6361/201834448
  * Poutanen and Svensson (1996) Poutanen, J. and Svensson, R. (1996).  The Two-Phase Pair Corona Model for Active Galactic Nuclei and X-Ray Binaries: How to Obtain Exact Solutions.  _ApJ_ 470, 249.  10.1086/177865
  * Predehl et al. (2021) Predehl, P., Andritschke, R., Arefiev, V., Babyshkin, V., Batanov, O., Becker, W., et al. (2021).  The eROSITA X-ray telescope on SRG.  _A &A_ 647, A1.  10.1051/0004-6361/202039313
  * Raginski and Laor (2016) Raginski, I. and Laor, A. (2016).  AGN coronal emission models - I. The predicted radio emission.  _MNRAS_ 459, 2082–2096.  10.1093/mnras/stw772
  * Raimundo and Fabian (2009) Raimundo, S. I. and Fabian, A. C. (2009).  Eddington ratio and accretion efficiency in active galactic nuclei evolution.  _MNRAS_ 396, 1217–1221.  10.1111/j.1365-2966.2009.14796.x
  * Raimundo et al. (2012) Raimundo, S. I., Fabian, A. C., Vasudevan, R. V., Gandhi, P., and Wu, J. (2012).  Can we measure the accretion efficiency of active galactic nuclei?  _MNRAS_ 419, 2529–2544.  10.1111/j.1365-2966.2011.19904.x
  * Ratheesh et al. (2021) Ratheesh, A., Matt, G., Tombesi, F., Soffitta, P., Pesce-Rollins, M., and Di Marco, A. (2021).  Exploring the accretion-ejection geometry of GRS 1915+105 in the obscured state with future X-ray spectro-polarimetry.  _A &A_ 655, A96.  10.1051/0004-6361/202140701
  * Reeves et al. (2021) Reeves, J. N., Braito, V., Porquet, D., Lobban, A. P., Matzeu, G. A., and Nardini, E. (2021).  The flaring X-ray corona in the quasar PDS 456.  _MNRAS_ 500, 1974–1991.  10.1093/mnras/staa3377
  * Reis and Miller (2013) Reis, R. C. and Miller, J. M. (2013).  On the Size and Location of the X-Ray Emitting Coronae around Black Holes.  _ApJ_ 769, L7.  10.1088/2041-8205/769/1/L7
  * Remillard and McClintock (2006) Remillard, R. A. and McClintock, J. E. (2006).  X-Ray Properties of Black-Hole Binaries.  _ARA &A_ 44, 49–92.  10.1146/annurev.astro.44.051905.092532
  * Reynolds (2021) Reynolds, C. S. (2021).  Observational Constraints on Black Hole Spin.  _ARA &A_ 59, 117–154.  10.1146/annurev-astro-112420-035022
  * Reynolds and Begelman (1997) Reynolds, C. S. and Begelman, M. C. (1997).  Iron Fluorescence from within the Innermost Stable Orbit of Black Hole Accretion Disks.  _ApJ_ 488, 109–118.  10.1086/304703
  * Ricci et al. (2023) Ricci, C., Chang, C.-S., Kawamuro, T., Privon, G., Mushotzky, R., Trakhtenbrot, B., et al. (2023).  A Tight Correlation Between Millimeter and X-ray Emission in Accreting Massive Black Holes from $<$100 Milliarcsecond-resolution ALMA Observations.  _arXiv e-prints_ , arXiv:2306.0467910.48550/arXiv.2306.04679
  * Ricci et al. (2018) Ricci, C., Ho, L. C., Fabian, A. C., Trakhtenbrot, B., Koss, M. J., Ueda, Y., et al. (2018).  BAT AGN Spectroscopic Survey - XII. The relation between coronal properties of active galactic nuclei and the Eddington ratio.  _MNRAS_ 480, 1819–1830.  10.1093/mnras/sty1879
  * Ricci et al. (2020) Ricci, C., Kara, E., Loewenstein, M., Trakhtenbrot, B., Arcavi, I., Remillard, R., et al. (2020).  The Destruction and Recreation of the X-Ray Corona in a Changing-look Active Galactic Nucleus.  _ApJ_ 898, L1.  10.3847/2041-8213/ab91a1
  * Ricci et al. (2021) Ricci, C., Loewenstein, M., Kara, E., Remillard, R., Trakhtenbrot, B., Arcavi, I., et al. (2021).  The 450 Day X-Ray Monitoring of the Changing-look AGN 1ES 1927+654.  _ApJS_ 255, 7.  10.3847/1538-4365/abe94b
  * Ricci and Trakhtenbrot (2022) Ricci, C. and Trakhtenbrot, B. (2022).  Changing-look Active Galactic Nuclei.  _arXiv e-prints_ , arXiv:2211.0513210.48550/arXiv.2211.05132
  * Ricci et al. (2017) Ricci, C., Trakhtenbrot, B., Koss, M. J., Ueda, Y., Del Vecchio, I., Treister, E., et al. (2017).  BAT AGN Spectroscopic Survey. V. X-Ray Properties of the Swift/BAT 70-month AGN Catalog.  _ApJS_ 233, 17.  10.3847/1538-4365/aa96ad
  * Risaliti et al. (2013) Risaliti, G., Harrison, F. A., Madsen, K. K., Walton, D. J., Boggs, S. E., Christensen, F. E., et al. (2013).  A rapidly spinning supermassive black hole at the centre of NGC 1365.  _Nature_ 494, 449–451.  10.1038/nature11938
  * Risaliti et al. (2011) Risaliti, G., Nardini, E., Salvati, M., Elvis, M., Fabbiano, G., Maiolino, R., et al. (2011).  X-ray absorption by broad-line region clouds in Mrk 766.  _MNRAS_ 410, 1027–1035.  10.1111/j.1365-2966.2010.17503.x
  * Rivers et al. (2013) Rivers, E., Markowitz, A., and Rothschild, R. (2013).  Full Spectral Survey of Active Galactic Nuclei in the Rossi X-ray Timing Explorer Archive.  _ApJ_ 772, 114.  10.1088/0004-637X/772/2/114
  * Ross et al. (2018) Ross, N. P., Ford, K. E. S., Graham, M., McKernan, B., Stern, D., Meisner, A. M., et al. (2018).  A new physical interpretation of optical and infrared variability in quasars.  _MNRAS_ 480, 4468–4479.  10.1093/mnras/sty2002
  * Ruan et al. (2019) Ruan, J. J., Anderson, S. F., Eracleous, M., Green, P. J., Haggard, D., MacLeod, C. L., et al. (2019).  The Analogous Structure of Accretion Flows in Supermassive and Stellar Mass Black Holes: New Insights from Faded Changing-look Quasars.  _ApJ_ 883, 76.  10.3847/1538-4357/ab3c1a
  * Rybicki and Lightman (1979) Rybicki, G. B. and Lightman, A. P. (1979).  _Radiative processes in astrophysics_ (John Wiley & Sons, Ltd)
  * Sagiv et al. (2014) Sagiv, I., Gal-Yam, A., Ofek, E. O., Waxman, E., Aharonson, O., Kulkarni, S. R., et al. (2014).  Science with a Wide-field UV Transient Explorer.  _AJ_ 147, 79.  10.1088/0004-6256/147/4/79
  * Sanfrutos et al. (2013) Sanfrutos, M., Miniutti, G., Agís-González, B., Fabian, A. C., Miller, J. M., Panessa, F., et al. (2013).  The size of the X-ray emitting region in SWIFT J2127.4+5654 via a broad line region cloud X-ray eclipse.  _MNRAS_ 436, 1588–1594.  10.1093/mnras/stt1675
  * Saxton et al. (2012) Saxton, R. D., Read, A. M., Esquej, P., Komossa, S., Dougherty, S., Rodriguez-Pascual, P., et al. (2012).  A tidal disruption-like X-ray flare from the quiescent galaxy SDSS J120136.02+300305.5.  _A &A_ 541, A106.  10.1051/0004-6361/201118367
  * Scepi et al. (2021) Scepi, N., Begelman, M. C., and Dexter, J. (2021).  Magnetic flux inversion in a peculiar changing look AGN.  _MNRAS_ 502, L50–L54.  10.1093/mnrasl/slab002
  * Schnittman and Krolik (2010) Schnittman, J. D. and Krolik, J. H. (2010).  X-ray Polarization from Accreting Black Holes: Coronal Emission.  _ApJ_ 712, 908–924.  10.1088/0004-637X/712/2/908
  * Shankar et al. (2004) Shankar, F., Salucci, P., Granato, G. L., De Zotti, G., and Danese, L. (2004).  Supermassive black hole demography: the match between the local and accreted mass functions.  _MNRAS_ 354, 1020–1030.  10.1111/j.1365-2966.2004.08261.x
  * Shankar et al. (2020) Shankar, F., Weinberg, D. H., Marsden, C., Grylls, P. J., Bernardi, M., Yang, G., et al. (2020).  Probing black hole accretion tracks, scaling relations, and radiative efficiencies from stacked X-ray active galactic nuclei.  _MNRAS_ 493, 1500–1511.  10.1093/mnras/stz3522
  * Shankar et al. (2009) Shankar, F., Weinberg, D. H., and Miralda-Escudé, J. (2009).  Self-Consistent Models of the AGN and Black Hole Populations: Duty Cycles, Accretion Rates, and the Mean Radiative Efficiency.  _ApJ_ 690, 20–41.  10.1088/0004-637X/690/1/20
  * Shappee et al. (2014) Shappee, B. J., Prieto, J. L., Grupe, D., Kochanek, C. S., Stanek, K. Z., De Rosa, G., et al. (2014).  The Man behind the Curtain: X-Rays Drive the UV through NIR Variability in the 2013 Active Galactic Nucleus Outburst in NGC 2617.  _ApJ_ 788, 48.  10.1088/0004-637X/788/1/48
  * Sironi and Beloborodov (2020) Sironi, L. and Beloborodov, A. M. (2020).  Kinetic Simulations of Radiative Magnetic Reconnection in the Coronae of Accreting Black Holes.  _ApJ_ 899, 52.  10.3847/1538-4357/aba622
  * Smith et al. (2016) Smith, K. L., Mushotzky, R. F., Vogel, S., Shimizu, T. T., and Miller, N. (2016).  Radio Properties of the BAT AGNs: the FIR-radio Relation, the Fundamental Plane, and the Main Sequence of Star Formation.  _ApJ_ 832, 163.  10.3847/0004-637X/832/2/163
  * Sniegowska et al. (2020) Sniegowska, M., Czerny, B., Bon, E., and Bon, N. (2020).  Possible mechanism for multiple changing-look phenomena in active galactic nuclei.  _A &A_ 641, A167.  10.1051/0004-6361/202038575
  * Soltan (1982) Soltan, A. (1982).  Masses of quasars.  _MNRAS_ 200, 115–122.  10.1093/mnras/200.1.115
  * Sridhar et al. (2021) Sridhar, N., Sironi, L., and Beloborodov, A. M. (2021).  Comptonization by reconnection plasmoids in black hole coronae I: Magnetically dominated pair plasma.  _MNRAS_ 507, 5625–5640.  10.1093/mnras/stab2534
  * Sridhar et al. (2023) Sridhar, N., Sironi, L., and Beloborodov, A. M. (2023).  Comptonization by reconnection plasmoids in black hole coronae II: Electron-ion plasma.  _MNRAS_ 518, 1301–1315.  10.1093/mnras/stac2730
  * Stern et al. (1995) Stern, B. E., Poutanen, J., Svensson, R., Sikora, M., and Begelman, M. C. (1995).  On the Geometry of the X-Ray–Emitting Region in Seyfert Galaxies.  _ApJ_ 449, L13.  10.1086/309617
  * Stern et al. (2018) Stern, D., McKernan, B., Graham, M. J., Ford, K. E. S., Ross, N. P., Meisner, A. M., et al. (2018).  A Mid-IR Selected Changing-look Quasar and Physical Scenarios for Abrupt AGN Fading.  _ApJ_ 864, 27.  10.3847/1538-4357/aac726
  * Svensson (1984) Svensson, R. (1984).  Steady mildly relativistic thermal plasmas - Processes and properties.  _MNRAS_ 209, 175–208.  10.1093/mnras/209.2.175
  * Tagliacozzo et al. (2023) Tagliacozzo, D., Marinucci, A., Ursini, F., Matt, G., Bianchi, S., Baldini, L., et al. (2023).  The geometry of the hot corona in MCG-05-23-16 constrained by X-ray polarimetry.  _arXiv e-prints_ , arXiv:2305.1021310.48550/arXiv.2305.10213
  * Tamborra et al. (2018) Tamborra, F., Matt, G., Bianchi, S., and Dovčiak, M. (2018).  MoCA: A Monte Carlo code for Comptonisation in Astrophysics. I. Description of the code and first results.  _A &A_ 619, A105.  10.1051/0004-6361/201732023
  * Tashiro et al. (2018) Tashiro, M., Maejima, H., Toda, K., Kelley, R., Reichenthal, L., Lobell, J., et al. (2018).  Concept of the X-ray Astronomy Recovery Mission.  In _Space Telescopes and Instrumentation 2018: Ultraviolet to Gamma Ray_ , eds. J.-W. A. den Herder, S. Nikzad, and K. Nakazawa. vol. 10699 of _Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series_ , 1069922.  10.1117/12.2309455
  * Tazaki et al. (2011) Tazaki, F., Ueda, Y., Terashima, Y., and Mushotzky, R. F. (2011).  Suzaku View of the Swift/BAT Active Galactic Nuclei. IV. Nature of Two Narrow-line Radio Galaxies (3C 403 and IC 5063).  _ApJ_ 738, 70.  10.1088/0004-637X/738/1/70
  * Tetarenko et al. (2016) Tetarenko, B. E., Sivakoff, G. R., Heinke, C. O., and Gladstone, J. C. (2016).  WATCHDOG: A Comprehensive All-sky Database of Galactic Black Hole X-ray Binaries.  _ApJS_ 222, 15.  10.3847/0067-0049/222/2/15
  * Titarchuk (1994) Titarchuk, L. (1994).  Generalized Comptonization Models and Application to the Recent High-Energy Observations.  _ApJ_ 434, 570.  10.1086/174760
  * Titarchuk and Lyubarskij (1995) Titarchuk, L. and Lyubarskij, Y. (1995).  Power-Law Spectra as a Result of Comptonization of the Soft Radiation in a Plasma Cloud.  _ApJ_ 450, 876.  10.1086/176191
  * Tomsick et al. (2019) Tomsick, J., Zoglauer, A., Sleator, C., Lazar, H., Beechert, J., Boggs, S., et al. (2019).  The Compton Spectrometer and Imager.  In _Bulletin of the American Astronomical Society_. vol. 51, 98.  10.48550/arXiv.1908.04334
  * Tonry et al. (2018) Tonry, J. L., Denneau, L., Heinze, A. N., Stalder, B., Smith, K. W., Smartt, S. J., et al. (2018).  ATLAS: A High-cadence All-sky Survey System.  _PASP_ 130, 064505.  10.1088/1538-3873/aabadf
  * Tortosa et al. (2018) Tortosa, A., Bianchi, S., Marinucci, A., Matt, G., and Petrucci, P. O. (2018).  A NuSTAR census of coronal parameters in Seyfert galaxies.  _A &A_ 614, A37.  10.1051/0004-6361/201732382
  * Tortosa et al. (2023) Tortosa, A., Ricci, C., Ho, L. C., Tombesi, F., Du, P., Inayoshi, K., et al. (2023).  Systematic broad-band X-ray study of super-Eddington accretion on to supermassive black holes - I. X-ray continuum.  _MNRAS_ 519, 6267–6283.  10.1093/mnras/stac3590
  * Trakhtenbrot et al. (2019) Trakhtenbrot, B., Arcavi, I., MacLeod, C. L., Ricci, C., Kara, E., Graham, M. L., et al. (2019).  1ES 1927+654: An AGN Caught Changing Look on a Timescale of Months.  _ApJ_ 883, 94.  10.3847/1538-4357/ab39e4
  * Urry and Padovani (1995) Urry, C. M. and Padovani, P. (1995).  Unified Schemes for Radio-Loud Active Galactic Nuclei.  _PASP_ 107, 803.  10.1086/133630
  * Ursini et al. (2019) Ursini, F., Bassani, L., Malizia, A., Bazzano, A., Bird, A. J., Stephen, J. B., et al. (2019).  The coronal temperature of NGC 4388 and NGC 2110 measured with INTEGRAL.  _A &A_ 629, A54.  10.1051/0004-6361/201936273
  * Ursini et al. (2023) Ursini, F., Marinucci, A., Matt, G., Bianchi, S., Marin, F., Marshall, H. L., et al. (2023).  Mapping the circumnuclear regions of the Circinus galaxy with the Imaging X-ray Polarimetry Explorer.  _MNRAS_ 519, 50–58.  10.1093/mnras/stac3189
  * Ursini et al. (2016) Ursini, F., Petrucci, P. O., Matt, G., Bianchi, S., Cappi, M., De Marco, B., et al. (2016).  High-energy monitoring of NGC 4593 with XMM-Newton and NuSTAR. X-ray spectral analysis.  _MNRAS_ 463, 382–392.  10.1093/mnras/stw2022
  * Uttley et al. (2014) Uttley, P., Cackett, E. M., Fabian, A. C., Kara, E., and Wilkins, D. R. (2014).  X-ray reverberation around accreting black holes.  _A &A Rev._ 22, 72.  10.1007/s00159-014-0072-0
  * Vaiana and Rosner (1978) Vaiana, G. S. and Rosner, R. (1978).  Recent advances in coronal physics.  _ARA &A_ 16, 393–428.  10.1146/annurev.aa.16.090178.002141
  * Vasudevan and Fabian (2007) Vasudevan, R. V. and Fabian, A. C. (2007).  Piecing together the X-ray background: bolometric corrections for active galactic nuclei.  _MNRAS_ 381, 1235–1251.  10.1111/j.1365-2966.2007.12328.x
  * Walton et al. (2021) Walton, D. J., Baloković, M., Fabian, A. C., Gallo, L. C., Koss, M., Nardini, E., et al. (2021).  Extreme relativistic reflection in the active galaxy ESO 033-G002.  _MNRAS_ 506, 1557–1572.  10.1093/mnras/stab1290
  * Walton et al. (2012) Walton, D. J., Reis, R. C., Cackett, E. M., Fabian, A. C., and Miller, J. M. (2012).  The similarity of broad iron lines in X-ray binaries and active galactic nuclei.  _MNRAS_ 422, 2510–2531.  10.1111/j.1365-2966.2012.20809.x
  * Walton et al. (2014) Walton, D. J., Risaliti, G., Harrison, F. A., Fabian, A. C., Miller, J. M., Arevalo, P., et al. (2014).  NuSTAR and XMM-NEWTON Observations of NGC 1365: Extreme Absorption Variability and a Constant Inner Accretion Disk.  _ApJ_ 788, 76.  10.1088/0004-637X/788/1/76
  * Wambsganss (2006) Wambsganss, J. (2006).  Part 4: Gravitational microlensing.  In _Saas-Fee Advanced Course 33: Gravitational Lensing: Strong, Weak and Micro_ , eds. G. Meylan, P. Jetzer, P. North, P. Schneider, C. S. Kochanek, and J. Wambsganss. 453–540
  * Weisskopf et al. (2022) Weisskopf, M. C., Soffitta, P., Baldini, L., Ramsey, B. D., O’Dell, S. L., Romani, R. W., et al. (2022).  The Imaging X-Ray Polarimetry Explorer (IXPE): Pre-Launch.  _Journal of Astronomical Telescopes, Instruments, and Systems_ 8, 026002.  10.1117/1.JATIS.8.2.026002
  * Wen et al. (2020) Wen, S., Jonker, P. G., Stone, N. C., Zabludoff, A. I., and Psaltis, D. (2020).  Continuum-fitting the X-Ray Spectra of Tidal Disruption Events.  _ApJ_ 897, 80.  10.3847/1538-4357/ab9817
  * Wevers (2020) Wevers, T. (2020).  Fainter harder brighter softer: a correlation between $\alpha$ox, X-ray spectral state, and Eddington ratio in tidal disruption events.  _MNRAS_ 497, L1–L6.  10.1093/mnrasl/slaa097
  * Wevers et al. (2021) Wevers, T., Pasham, D. R., van Velzen, S., Miller-Jones, J. C. A., Uttley, P., Gendreau, K. C., et al. (2021).  Rapid Accretion State Transitions following the Tidal Disruption Event AT2018fyk.  _ApJ_ 912, 151.  10.3847/1538-4357/abf5e2
  * Wilkins et al. (2016) Wilkins, D. R., Cackett, E. M., Fabian, A. C., and Reynolds, C. S. (2016).  Towards modelling X-ray reverberation in AGN: piecing together the extended corona.  _MNRAS_ 458, 200–225.  10.1093/mnras/stw276
  * Wilkins and Fabian (2011) Wilkins, D. R. and Fabian, A. C. (2011).  Determination of the X-ray reflection emissivity profile of 1H 0707-495.  _MNRAS_ 414, 1269–1277.  10.1111/j.1365-2966.2011.18458.x
  * Wilkins and Fabian (2012) Wilkins, D. R. and Fabian, A. C. (2012).  Understanding X-ray reflection emissivity profiles in AGN: locating the X-ray source.  _MNRAS_ 424, 1284–1296.  10.1111/j.1365-2966.2012.21308.x
  * Wilkins and Fabian (2013) Wilkins, D. R. and Fabian, A. C. (2013).  The origin of the lag spectra observed in AGN: Reverberation and the propagation of X-ray source fluctuations.  _MNRAS_ 430, 247–258.  10.1093/mnras/sts591
  * Wilkins and Gallo (2015) Wilkins, D. R. and Gallo, L. C. (2015).  Driving extreme variability: the evolving corona and evidence for jet launching in Markarian 335 .  _MNRAS_ 449, 129–146.  10.1093/mnras/stv162
  * Wilms et al. (2000) Wilms, J., Allen, A., and McCray, R. (2000).  On the Absorption of X-Rays in the Interstellar Medium.  _ApJ_ 542, 914–924.  10.1086/317016
  * XRISM Science Team (2022) XRISM Science Team (2022).  XRISM Quick Reference.  _arXiv e-prints_ , arXiv:2202.0539910.48550/arXiv.2202.05399
  * Yao et al. (2022) Yao, Y., Lu, W., Guolo, M., Pasham, D. R., Gezari, S., Gilfanov, M., et al. (2022).  The Tidal Disruption Event AT2021ehb: Evidence of Relativistic Disk Reflection, and Rapid Evolution of the Disk-Corona System.  _ApJ_ 937, 8.  10.3847/1538-4357/ac898a
  * Yu and Tremaine (2002) Yu, Q. and Tremaine, S. (2002).  Observational constraints on growth of massive black holes.  _MNRAS_ 335, 965–976.  10.1046/j.1365-8711.2002.05532.x
  * Zdziarski et al. (1996) Zdziarski, A. A., Johnson, W. N., and Magdziarz, P. (1996).  Broad-band $\gamma$-ray and X-ray spectra of NGC 4151 and their implications for physical processes and geometry.  _MNRAS_ 283, 193–206.  10.1093/mnras/283.1.193
  * Zdziarski et al. (2021) Zdziarski, A. A., Jourdain, E., Lubiński, P., Szanecki, M., Niedźwiecki, A., Veledina, A., et al. (2021).  Hybrid Comptonization and Electron-Positron Pair Production in the Black-hole X-Ray Binary MAXI J1820+070.  _ApJ_ 914, L5.  10.3847/2041-8213/ac0147
  * Zdziarski and Lightman (1985) Zdziarski, A. A. and Lightman, A. P. (1985).  Nonthermal electron-positron pair production and the ’universal’ X-ray spectrum of active galactic nuclei.  _ApJ_ 294, L79–L83.  10.1086/184513
  * Zdziarski et al. (1993) Zdziarski, A. A., Lightman, A. P., and Maciolek-Niedzwiecki, A. (1993).  Acceleration Efficiency in Nonthermal Sources and the Soft Gamma Rays from NGC 4151 Observed by OSSE and SIGMA.  _ApJ_ 414, L93.  10.1086/187004
  * Zhang et al. (2019a) Zhang, S., Santangelo, A., Feroci, M., Xu, Y., Lu, F., Chen, Y., et al. (2019a).  The enhanced X-ray Timing and Polarimetry mission—eXTP.  _Science China Physics, Mechanics, and Astronomy_ 62, 29502.  10.1007/s11433-018-9309-2
  * Zhang et al. (2019b) Zhang, W., Dovčiak, M., and Bursa, M. (2019b).  Constraining the Size of the Corona with Fully Relativistic Calculations of Spectra of Extended Coronae. I. The Monte Carlo Radiative Transfer Code.  _ApJ_ 875, 148.  10.3847/1538-4357/ab1261
  * Zhang and Lu (2017) Zhang, X. and Lu, Y. (2017).  On the mean radiative efficiency of accreting massive black holes in AGNs and QSOs.  _Science China Physics, Mechanics, and Astronomy_ 60, 109511.  10.1007/s11433-017-9062-1
  * Zoghbi et al. (2017) Zoghbi, A., Matt, G., Miller, J. M., Lohfink, A. M., Walton, D. J., Ballantyne, D. R., et al. (2017).  A Long Look at MCG-5-23-16 with NuSTAR. I. Relativistic Reflection and Coronal Properties.  _ApJ_ 836, 2.  10.3847/1538-4357/aa582c


# Multivariate Differential Association Analysis

Hoseung Song and Michael C. Wu Hoseung Song is with the Public Health Sciences
Division at Fred Hutchinson Cancer Center, Seattle, WA 98109 USA (e-mail:
hsong3@fredhutch.org).
Michael C. Wu is with the Public Health Sciences Division at Fred Hutchinson
Cancer Center, Seattle, WA 98109 USA (e-mail: mcwu@fredhutch.org).

###### Abstract

Identifying how dependence relationships vary across different conditions
plays a significant role in many scientific investigations. For example, it is
important for the comparison of biological systems to see if relationships
between genomic features differ between cases and controls. In this paper, we
seek to evaluate whether the relationships between two sets of variables is
different across two conditions. Specifically, we assess: do two sets of high-
dimensional variables have similar dependence relationships across two
conditions?. We propose a new kernel-based test to capture the differential
dependence. Specifically, the new test determines whether two measures that
detect dependence relationships are similar or not under two conditions. We
introduce the asymptotic permutation null distribution of the test statistic
and it is shown to work well under finite samples such that the test is
computationally efficient, making it easily applicable to analyze large data
sets. We demonstrate through numerical studies that our proposed test has high
power for detecting differential linear and non-linear relationships. The
proposed method is implemented in an R package kerDAA.

###### Index Terms:

Kernel methods; Permutation null distribution; Nonparametrics; High-
dimensional data; Non-linear dependence; Correlation; Co-expression.

##  I Introduction

Evaluating dissimilarities in dependence relationships across conditions is a
powerful strategy for understanding regulatory mechanisms and offers
significant insights into the scientific mechanism underlying differences
across groups. For instance, [1] studied differential association of left and
right hippocampal volumes with verbal episodic and spatial memory and revealed
that episodic verbal memory heavily relies on the left hippocampus, whereas
spatial memory processing appears to be predominantly governed by the right
hippocampus in non-demented older adults. [2] investigated the association of
viral load dynamics with patient’s age and severity of COVID-19 and found that
a positive correlation was observed between increased viral burden and
inflammatory responses, particularly in the younger cohort and mild cases
across all age groups, whereas elderly patients with critical disease
exhibited minimal indications of such inflammatory responses. These examples
and countless others demonstrate that identifying disparities in co-
expression/co-occurrence patterns in features (e.g. genes) between case and
control groups is crucial to comprehending intricate human diseases. Such
analyses, often referred to as differential co-expression (DCE) analysis, has
been extensively studied [3, 4]. DCE analysis detects pairs or sets of
features that are differentially associated or regulated in different groups.

Although DCE analysis has achieved considerable success, more general
application is stymied by the complex nature of the problem and the absence of
robust statistical tests capable of comparing multi-dimensional patterns.
Presently, DCE analyses predominantly rely on the Pearson’s correlation
coefficient [5, 6, 7, 8], which is susceptible to outliers and solely
evaluates the degree of linear correlation. Pearson’s correlation coefficient
also only works for pairs of univariate data making it inappropriate to
examine complicated associations with high-dimensional variables. Furthermore,
most existing DCE analysis methods quantify the score or pattern of a pair of
variables that are differentially co-expressed. This makes DCE analysis
complicated and limited: testing procedures for univariate pairs are
recursively applied to construct differentially co-expressed multi-variables
(e.g., clusters or networks) and corresponding measures (e.g., correlation
coefficients) are not effective for assessing non-linear dependence
relationships or associations among high-dimensional variables.

To bypass the limitations of extant strategies, we propose to directly perform
differential association analysis by testing whether two high-dimensional
variables have similar dependence relationships (beyond Pearson’s correlation)
or not across different conditions or groups. To measure changes in complex
associations on multivariate data, we base we base our approach on the
Hilbert-Schmidt Independence Criterion (HSIC) proposed by [9]. The HSIC is a
powerful kernel-based statistic for assessing the generalized dependence
between two multivariate variables and this measure makes no assumption on the
distributions of the variables or the nature of the dependence.

In this work, we develop an efficient and effective kernel-based test that
achieves high power in detecting changes in associations on two multivariate
variables across different conditions. The main contributions of this paper
are:

  * •

The new approach builds upon the HSIC and this allows the new test to work for
generalized dependence relationships between two high-dimensional variables.

  * •

We introduce the asymptotic permutation null distribution of the test
statistic, offering an easy off-the-shelf tool for large data sets. Moreover,
we apply an omnibus test with different kernels, making the new test perform
well for a wide range of alternatives.

  * •

The new method is implemented in an R package kerDAA.

The organization of the paper is as follows. In the next subsection, we first
introduce the permutation test, assumptions, and notations. In Section II, we
provide an overview of the HSIC statistic. The new test with its asymptotic
distribution and testing procedure is provided in Section III. Section IV
examines the performance of the new tests under various simulation settings.
We conclude with a brief discussion in Section V.

###  I-A Permutation tests, assumptions, and notations

In this paper, we propose an asymptotic distribution-free test that avoids any
parametric modeling assumptions. It is generally difficult to derive the true
null distribution of the test statistic and this also applies to HSIC [10]. To
overcome this, we work under the permutation null distribution, which places
$1/N!$ probability on each of the $N!$ permutations of pooled observations
$\\{Z_{i}\\}_{1,\ldots,N}$. Permutation tests are easy to implement and
provide an exact control of the type I error rate for finite samples for all
test statistics under the null hypothesis [11, 12]. Hence, throughout this
paper, the exchangeability is assumed that the underlying distributions of
samples are identical across conditions under the null hypothesis.

With no further specification, we use pr, E, var, and cov to denote the
probability, expectation, variance, and covariance, respectively, under the
permutation null distribution. In addition, we write $a_{n}=O(b_{n})$ when
$a_{n}$ has the same order as $b_{n}$ and $a_{n}=o(b_{n})$ when $a_{n}$ is
dominated by $b_{n}$ asymptotically, i.e.,
$\lim_{n\rightarrow\infty}(a_{n}/b_{n})=0$.

##  II Hilbert-Schmidt Independence Criterion

Given two random vectors $X$ and $Y$, let $f_{X}$ and $f_{Y}$ be the marginal
distributions of $X$ and $Y$, respectively. We say the variables $X$ and $Y$
are statistically independent if $f_{XY}=f_{X}f_{Y}$ where $f_{XY}$ is a joint
probability measure defined on $(X,Y)$.

To detect the potential associations between two sample data, the Hilbert-
Schmidt independence criterion (HSIC) is widely used in many applications,
such as clustering [13, 14, 15], time series [16, 17, 18], and feature
screening [19, 20, 21].

The HSIC was first proposed by [9]. The authors map the observations into a
reproducing kernel Hilbert space $\mathcal{H}$ (RKHS) generated by a given
kernel $k(\cdot,\cdot)$. For each point $x\in X$, there corresponds an element
(feature map) $\phi(x)\in\mathcal{H}$ such that
$<\phi(x),\phi(x^{\prime})>_{\mathcal{H}}=k(x,x^{\prime})$, where $k:X\times
X\rightarrow\mathcal{R}$ is a unique positive definite kernel. With this
mapping, the authors consider a cross-covariance operator between feature maps
and the squared Hilbert-Schmidt norm of the cross-covariance operator, which
can be expressed as

$\displaystyle\textrm{HSIC}(X,Y)$
$\displaystyle=E_{XX^{\prime}YY^{\prime}}[k_{X}(X,X^{\prime})k_{Y}(Y,Y^{\prime})]$
$\displaystyle+E_{XX^{\prime}}[k_{X}(X,X^{\prime})]E_{YY^{\prime}}[k_{Y}(Y,Y^{\prime})]$
$\displaystyle-2E_{XY}\left[E_{X^{\prime}}[k_{X}(X,X^{\prime})]E_{Y^{\prime}}[k_{Y}(Y,Y^{\prime})]\right],$

where $X^{\prime}$ and $Y^{\prime}$ are independent copies of $X$ and $Y$,
respectively. When characteristic kernels, such as the Gaussian kernel or
Laplacian kernel, are used for $k_{X}$ and $k_{Y}$, it is known that
$\textrm{HSIC}(f_{XY})=0$ if and only if $f_{XY}=f_{X}f_{Y}$. In other words,
HSIC is zero if and only if two random variables are independent.

Given $N$ pairs of observations $(X_{i},Y_{i})$ from
$(X,Y)\in\mathcal{R}^{p\times q}$ $(i=1,\ldots,N)$, an empirical estimate of
HSIC was also proposed as follows:

$\displaystyle\textrm{HSIC}_{N}$
$\displaystyle=\frac{1}{N^{2}}\sum_{i,j}^{N}k_{X}(X_{i},X_{j})k_{Y}(Y_{i},Y_{j})$
$\displaystyle+\frac{1}{N^{4}}\sum_{i,j,u,v}^{N}k_{X}(X_{i},X_{j})k_{Y}(Y_{u},Y_{v})$
$\displaystyle-\frac{2}{N^{3}}\sum_{i,j,u}^{N}k_{X}(X_{i},X_{j})k_{Y}(Y_{i},Y_{u}).$

Let $K_{X}$ and $K_{Y}$ be kernel matrices with entries $k_{X}(X_{i},X_{j})$
and $k_{Y}(Y_{i},Y_{j})$, respectively. Then, HSIC can be rewritten as

$\displaystyle\textrm{HSIC}_{N}=\frac{trace(\tilde{K}_{X}\tilde{K}_{Y})}{N^{2}},$

where $\tilde{K}_{X}=H_{N}K_{X}H_{N}$ and $\tilde{K}_{Y}=H_{N}K_{Y}H_{N}$ are
centered kernel matrices of $K_{X}$ and $K_{Y}$, respectively, and
$H_{N}=I_{N}-1_{N}1_{N}^{t}/N$ is a centering matrix with $I_{N}$ being an
identity matrix of order $N$ and $1_{N}$ being a $N\times 1$ vector of all
ones.

##  III New tests

We seek to evaluate the assumption that there is underlying similar
relationship between two high-dimensional sets of variables across two
conditions or groups, by asking the question: do two sets of high-dimensional
variables have similar dependence relationships across two conditions?

Formally speaking, given $m$ pairs of observations
$(X_{i}^{A},Y_{i}^{A})\stackrel{{\scriptstyle iid}}{{\sim}}f_{XY}^{A}$ on
condition A where $(X_{i}^{A},Y_{i}^{A})\in\mathcal{R}^{p\times q}$
$(i=1,\ldots,m)$ and $n$ pairs of observations
$(X_{i}^{B},Y_{i}^{B})\stackrel{{\scriptstyle iid}}{{\sim}}f_{XY}^{B}$ on
condition B where $(X_{i}^{A},Y_{i}^{B})\in\mathcal{R}^{p\times q}$
$(i=1,\ldots,n)$, we concern the following hypothesis testing:

$\displaystyle H_{0}:\textrm{HSIC}^{A}=\textrm{HSIC}^{B}\ \ vs.\ \
H_{1}:\textrm{HSIC}^{A}\neq\textrm{HSIC}^{B}.$ (1)

The goal of a new test is to determine whether two high-dimensional variables
have similar dependence relationships or not. For example, if two variables
are independent in one condition, but not in the other, one of the two HSICs
would be zero and the other would be greater than zero. On the other hand, if
two variables are not independent in both conditions, but have different
dependence relationships, two HSICs would be different.

To assess the difference of associations across conditions, the following
measure is naturally considered:

$\displaystyle T$ $\displaystyle=\textrm{HSIC}^{A}-\textrm{HSIC}^{B}$
$\displaystyle=\textrm{HSIC}(X^{A},Y^{A})-\textrm{HSIC}(X^{B},Y^{B}).$ (2)

If $X$ and $Y$ are independent in both conditions, we would expect both
$\textrm{HSIC}^{A}$ and $\textrm{HSIC}^{B}$ to be close to zero, then $T$
would be close to zero. If $X$ and $Y$ are not independent in both conditions,
but have similar dependence relationships, we would expect $\textrm{HSIC}^{A}$
and $\textrm{HSIC}^{B}$ to be similar, then $T$ would also be close to zero.
Hence, the test defined in this way is sensitive to different dependence
relationships, and a value of $T$ that is far from zero would be the evidence
against the null hypothesis.

Since the empirical estimate of HSIC can be written in terms of kernel
matrices, we can use the following empirical measure

$\displaystyle T_{N}$
$\displaystyle=\textrm{HSIC}_{m}^{A}-\textrm{HSIC}_{n}^{B}$
$\displaystyle=\frac{trace(\tilde{K}_{X}^{A}\tilde{K}_{Y}^{A})}{m^{2}}-\frac{trace(\tilde{K}_{X}^{B}\tilde{K}_{Y}^{B})}{n^{2}},$

where $N=m+n$, $\tilde{K}_{X}^{A}=H_{m}K_{X}^{A}H_{m}$,
$\tilde{K}_{Y}^{A}=H_{m}K_{Y}^{A}H_{m}$,
$\tilde{K}_{X}^{B}=H_{n}K_{X}^{B}H_{n}$, and
$\tilde{K}_{Y}^{B}=H_{n}K_{Y}^{B}H_{n}$. Here, we consider

$\displaystyle\tilde{T}_{N}$
$\displaystyle=\frac{trace\left(\left(\tilde{K}_{X}^{A}-diag(\tilde{K}_{X}^{A})\right)\left(\tilde{K}_{Y}^{A}-diag(\tilde{K}_{Y}^{A})\right)\right)}{m(m-1)}$
$\displaystyle-\frac{trace\left(\left(\tilde{K}_{X}^{B}-diag(\tilde{K}_{X}^{B})\right)\left(\tilde{K}_{Y}^{B}-diag(\tilde{K}_{Y}^{B})\right)\right)}{n(n-1)},$
(3)

where $diag(M)$ represents a matrix containing only the diagonal matrix of $M$
on its diagonal, and zero’s elsewhere. Compared to $T_{N}$, $\tilde{T}_{N}$
has a tractable asymptotic permutation null distribution, while losing very
little information (see below). It is also known that eliminating the diagonal
terms of the cross-product matrix can reduce the bias by focusing on the
dependence structure while removing the marginal effects [22, 23, 24].

The analytical expressions for the expectation and variance of $\tilde{T}_{N}$
and the asymptotic permutation null distribution of $\tilde{T}_{N}$ can be
derived using an alternative expression for $\tilde{T}_{N}$. We first pool
pairs of observations from the two conditions together and denote them by
$(X_{i}^{U},Y_{i}^{U})=(X^{A},Y^{A})\cup(X^{B},Y^{B})$ $(i=1,\ldots,N)$. Let
$K_{X}^{U}$ and $K_{Y}^{U}$ be kernel matrices for the pooled observations
$X_{i}^{U}$ and $Y_{i}^{U}$, respectively $(i=1,\ldots,N)$. Let

$\displaystyle H=\begin{pmatrix}H_{m}&\textbf{0}_{m\times n}\\\
\textbf{0}_{n\times m}&H_{n}\end{pmatrix},$

where $\textbf{0}_{m\times n}$ and $\textbf{0}_{n\times m}$ are $m\times n$
and $n\times m$ matrices of all zeros, respectively. For centered matrices
$\tilde{K}_{X}^{U}=HK_{X}^{U}H$ and $\tilde{K}_{Y}^{U}=HK_{Y}^{U}H$, let
$\tilde{K}^{U}$ be a kernel matrix with entries
$\tilde{k}_{X}^{U}(X_{i},X_{j})\tilde{k}_{Y}^{U}(Y_{i},Y_{j})$
$(i,j=1,\ldots,N)$, namely the element-wise products between
$\tilde{K}_{X}^{U}$ and $\tilde{K}_{Y}^{U}$, with diagonal elements set to
zero. We denote the $(i,j)$-th element of $\tilde{K}^{U}$ by
$\tilde{k}_{ij,U}$. Then, $\tilde{T}_{N}$ can be rewritten as:

$\displaystyle\tilde{T}_{N}=\frac{1}{m(m-1)}\sum_{i,j=1}^{m}\tilde{k}_{ij,U}-\frac{1}{n(n-1)}\sum_{i,j=m+1}^{N}\tilde{k}_{ij,U}.$
(4)

The analytical expressions for the expectation and variance of $\tilde{T}_{N}$
and its asymptotic permutation null distribution can be obtained in a similar
way to that in [25] and are provided in Lemma 1 and 2.

###### Lemma 1

Under the permutation null distribution, we have

$\displaystyle\textsf{E}(\tilde{T}_{N})$ $\displaystyle=0,$
$\displaystyle\textsf{var}(\tilde{T}_{N})$
$\displaystyle=\frac{2Af_{1}(m)+4Bf_{2}(m)+Cf_{3}(m)}{m^{2}(m-1)^{2}}$
$\displaystyle+\frac{2Af_{1}(n)+4Bf_{2}(n)+Cf_{3}(n)}{n^{2}(n-1)^{2}}$
$\displaystyle-\frac{2C}{N(N-1)(N-2)(N-3)},$

where

$\displaystyle f_{1}(x)$ $\displaystyle=\frac{x(x-1)}{N(N-1)},\ \
f_{2}(x)=\frac{x(x-1)(x-2)}{N(N-1)(N-2)},$ $\displaystyle f_{3}(x)$
$\displaystyle=\frac{x(x-1)(x-2)(x-3)}{N(N-1)(N-2)(N-3)},$ $\displaystyle A$
$\displaystyle=\sum_{i,j=1}^{N}\tilde{k}^{2}_{ij,U},\ \
B=\sum_{i,j,r=1}^{N}\tilde{k}_{ij,U}\tilde{k}_{ir,U},$ $\displaystyle C$
$\displaystyle=\sum_{i,j,r,s=1}^{N}\tilde{k}_{ij,U}\tilde{k}_{rs,U}.$

We then define the test statistic as

$\displaystyle
Z_{\tilde{T}}=\frac{\tilde{T}_{N}-\textsf{E}(\tilde{T}_{N})}{\sqrt{\textsf{var}(\tilde{T}_{N})}}.$
(5)

###### Lemma 2

Let $\bar{k}=\sum_{i,j=1}^{N}\tilde{k}_{ij,U}/(N^{2}-N)$ and
$\bar{k}_{i\cdot}=\sum_{j=1,j\neq i}^{N}(\tilde{k}_{ij,U}-\bar{k})$ for
$i=1,\ldots,N$. Under the permutation null distribution, as
$N\rightarrow\infty$, $m/N\rightarrow p\in(0,1)$, if
$\sum_{i=1}^{N}|\bar{k}_{i\cdot}|^{\epsilon}=o(\\{\sum_{i=1}^{N}\bar{k}_{i\cdot}^{2}\\}^{\epsilon/2})$
for all integers $\epsilon>2$,

$\displaystyle
Z_{\tilde{T}}\stackrel{{\scriptstyle\mathcal{D}}}{{\rightarrow}}\mathcal{N}(0,1).$

###### Remark 1

The condition for Lemma 2 can be satisfied when
$|\bar{k}_{i\cdot}|=O(N^{\delta})$ for a constant $\delta$, $\forall i$. When
there is no big outlier in the data, it is not hard to have this condition
satisfied.

Figure 1 shows the normal quantile-quantile plots for $Z_{\tilde{T}}$ from
10,000 permutations under different choices of kernels and $p,q$ for Gaussian
data with $N=200$. We see that, when $N$ is in the hundreds, the permutation
distributions can already be well approximated by the standard normal
distribution for $Z_{\tilde{T}}$.

Figure 1: Quantile-quantile plots of $Z_{\tilde{T}}$ using the Gaussian kernel
and linear kernel when $N=200$.

To assess the dependence relationship, we use the Gaussian kernel and linear
kernel as each kernel is suitable for a different type of dependence
relationship [10, 26, 27]. To accommodate both effects based on the Gaussian
kernel and linear kernel, we apply the Cauchy combination test to obtain the
omnibus $p$-value [28]. The detailed testing procedure is summarized in
Algorithm 1.

Algorithm 1 Testing procedure

0: Observations $\\{(X_{i}^{U},Y_{i}^{U})\\}_{i=1,\dots,N}$ and the
significance level $\alpha$.

0: Reject the null hypothesis $H_{0}$ if $p$-value $p\leq\alpha$.

1: Compute $Z_{\tilde{T}}$ using the Gaussian kernel and linear kernel by
Lemma 1.

2: Calculate $p$-values of $Z_{\tilde{T}}$ by the standard normal distribution
based on Lemma 2 using the Gaussian kernel and linear kernel ($p_{G}$ and
$p_{L}$).

3: Obtain the omnibus $p$-value:  $\displaystyle p$
$\displaystyle=\frac{\tan\\{(0.5-\min(p_{G},0.99))\pi\\}}{2}$ $\displaystyle\
\ \ \ +\frac{\tan\\{(0.5-\min(p_{L},0.99))\pi\\}}{2}.$

##  IV Experiments

###  IV-A Simulation studies

In this section, we examine the performance of the new test under various
simulation examples. We compare the new test (NEW) with the existing
correlation-based test (dCoxS) proposed by [29]. The authors utilize the
Pearson’s correlation coefficient between two pairwise Euclidean distances of
samples. dCoxS compares the Pearson’s correlation coefficient between two
conditions and calculates a z-score by applying the Fisher’s transformation to
the Pearson’s correlation coefficients. Similarly, many methods are based on
the correlation coefficient, which focuses on the linear dependence
relationships [5, 6, 30, 31]. Here, we apply the permutation test to dCoxS for
fair comparison.

We first consider the following settings (Setting 1): given
$\\{(X_{i}^{A},Y_{i}^{A})\\}\stackrel{{\scriptstyle iid}}{{\sim}}f_{XY}^{A}$
vs. $\\{(X_{i}^{B},Y_{i}^{B})\\}\stackrel{{\scriptstyle
iid}}{{\sim}}f_{XY}^{B}$,

  * •

Multivariate normal: $f_{XY}^{A}\sim N_{100}(\textbf{0}_{100},I_{100})$ and
$f_{XY}^{B}\sim N_{100}(\textbf{0}_{100},\Sigma)$.

  * •

Multivariate log-normal:
$f_{XY}^{A}\sim\exp\left(N_{100}(\textbf{0}_{100},I_{100})\right)$ and
$f_{XY}^{B}\sim\exp\left(N_{100}(\textbf{0}_{100},\Sigma)\right)$.

We use $m=n=100$, $p=q=50$, and $\Sigma=\rho^{|i-j|}$. $\textbf{0}_{d}$
represents $d$ dimensional vector of zeros. When $\rho=0$, both
$\\{(X_{i}^{A},Y_{i}^{A})\\}$ and $\\{(X_{i}^{B},Y_{i}^{B})\\}$ are
independent, and we would expect the new test does not reject the null
hypothesis. On the other hand, when $\rho>0$, $\\{(X_{i}^{B},Y_{i}^{B})\\}$
are not independent each other, while $\\{(X_{i}^{A},Y_{i}^{A})\\}$ are still
independent each other, we thus expect the new test rejects the null
hypothesis. We simulate 1000 data sets to estimate the power of the tests and
the significance level is set to be 0.05 for all tests.

Table I: Empirical size of the tests at 0.05 significance level | NEW | dCoxS
---|---|---
Normal | 0.047 | 0.038
Log-normal | 0.045 | 0.047

Table I shows the empirical size of the tests at 0.05 significance level for
the multivariate normal and log-normal data. We see that both the new test and
dCoxS control the type I error rate well.

Figure 2: Estimated power of the tests under different $\rho$ when $m=n=100$
and $p=q=50$.

Figure 2 shows the estimated power of the tests under different $\rho$ when
$m=n=100$ and $p=q=50$. Since the dependence relationship between $X_{i}$ and
$Y_{i}$ gets stronger as $\rho$ increase, we would expect the power of tests
increases as $\rho$ increases. We see that the performance of the new test
indeed increases as $\rho$ increases, but dCoxS performs poorly.

We also consider different types of settings (Setting 2):

  * •

Case 1: $\\{(X_{i}^{A},Y_{i}^{A})\\}\stackrel{{\scriptstyle
iid}}{{\sim}}N(\textbf{0}_{100},\Sigma_{1})$ vs.
$\\{(X_{i}^{B},Y_{i}^{B})\\}\stackrel{{\scriptstyle
iid}}{{\sim}}N(\textbf{0}_{100},\Sigma_{2})$, where $\Sigma_{1}=0.4^{|i-j|}$
and $\Sigma_{2}=(0.4+\rho)^{|i-j|}$.

  * •

Case 2: $\\{(X_{i}^{A},Y_{i}^{A})\\}\stackrel{{\scriptstyle
iid}}{{\sim}}\exp(N(\textbf{0}_{100},\Sigma_{1}))$ vs.
$\\{(X_{i}^{B},Y_{i}^{B})\\}\stackrel{{\scriptstyle
iid}}{{\sim}}\exp(N(\textbf{0}_{100},\Sigma_{2}))$, where
$\Sigma_{1}=0.4^{|i-j|}$ and $\Sigma_{2}=(0.4+\rho)^{|i-j|}$.

  * •

Case 3: $\\{X_{i}^{A}\\}\stackrel{{\scriptstyle
iid}}{{\sim}}N_{100}(\textbf{0}_{100},I_{100})$ and $\\{Y_{i}^{A}\\}=\sin(2\pi
X_{i,1:50}^{A}/3)$ vs. $\\{X_{i}^{B}\\}\stackrel{{\scriptstyle
iid}}{{\sim}}N_{100}(\textbf{0}_{100},I_{100})$ and
$\\{Y_{i}^{B}\\}=\sin((2+\rho)\pi X_{i,1:50}^{B}/3)$, where $X_{i,1:50}^{A}$
and $X_{i,1:50}^{B}$ are the first 50 variables of $X_{i}^{A}$ and
$X_{i}^{B}$, respectively.

  * •

Case 4: $\\{X_{i}^{U}\\}\stackrel{{\scriptstyle
iid}}{{\sim}}\log|N_{100}(\textbf{0}_{100},I_{100})|$ and
$\\{Y_{i}^{U}\\}\stackrel{{\scriptstyle
iid}}{{\sim}}\sin(N_{100}(\textbf{0}_{100},I_{100}))$, where the first 10
variables of $\\{Y_{i}^{A}\\}$ depend on the first variable of
$\\{X_{i}^{A}\\}$ and the first $10+\rho$ variables of $\\{Y_{i}^{B}\\}$
depend on the first variable of $\\{X_{i}^{B}\\}$.

We use $m=n=100$ and $p=q=50$. Setting 2 is somewhat different from Setting 1:
in Setting 1, two variables are independent in one condition, but not
independent in the other, while in Setting 2, two variables are not
independent in both conditions, but dependence relationships could be
different. In Setting 2, when $\rho=0$, two variables have the same dependence
relationships in both conditions, so we would expect the new test does not
reject the null hypothesis. On the other hand, when $\rho>0$, two variables
have different dependence relationships, so we would expect the new test
rejects the null hypothesis. We simulate 1000 data sets to estimate the power
of the tests and the significance level is set to be 0.05 for all tests.

Table II: Empirical size of the tests at 0.05 significance level | NEW | dCoxS
---|---|---
Case 1 | 0.048 | 0.040
Case 2 | 0.054 | 0.043
Case 3 | 0.046 | 0.047
Case 4 | 0.052 | 0.064

The empirical size of the tests at 0.05 significance level for Setting 2 is
presented in Table II. We see that both the new test and dCoxS control the
type I error rate well in all cases.

Figure 3: Estimated power of the tests under different $\rho$ when $m=n=100$
and $p=q=50$.

Figure 3 shows the performance of the tests under different $\rho$ for all
cases. Similar to Setting 1, a large value of $\rho$ indicates strong
dependence relationships. Following this, the power of the new test increases
as $\rho$ increases and the new test exhibits high power. However, dCoxS does
not capture the difference in dependence relationships.

###  IV-B Application to real data

We apply the new test to the data from Menopause Studies - Finding Lasting
Answers for Symptoms and Health (MsFLASH) Vaginal Health Trial [32]. The
MsFLASH trial was a randomized clinical trial to evaluate the treatment effect
of vaginal estradiol vs. placebo on vaginal discomfort in postmenopausal
women. In an endeavor to explore the mechanisms underlying postmenopausal
vaginal discomfort, a longitudinal investigation was conducted to study the
vaginal microbiota and vaginal fluid metabolites of 141 participants. The
vaginal microbiome profiles include abundance data for 381 taxa and the
metabolome profiles include abundance data for 171 metabolites.

We applied the proposed test to assess differential dependency between
metabolites and the overall vaginal microbiome compositions across the placebo
(45 participants) and estradiol treatment arms (50 participants).
Specifically, we applied the new test to assess the differential dependence
relationships at baseline and week 4. We obtained $p$-values 0.937 and 0.038
for participants at baseline and week 4, respectively. This result shows that
the dependence relationship between metabolites and vaginal microbiome
compositions was not different by arm at baseline. This makes sense as none of
the women had undergone treatment yet. However, after four weeks of treatment,
the dependency between metabolites and microbiome composition was
significantly different ($\alpha=0.05$). Hence, this result shows that the
association between metabolites and vaginal microbiome compositions is
perturbed over time.

Table III: HSIC values using the Gaussian kernel | HSIC: Placebo | HSIC: Estradiol
---|---|---
Baseline | 0.0051 | 0.0046
Week 4 | 0.0056 | 0.0047
Table IV: HSIC values using the linear kernel | HSIC: Placebo | HSIC: Estradiol
---|---|---
Baseline | 6883.3 | 5474.8
Week 4 | 3762.3 | 7127.6

We further investigated HSIC values for treatment and control groups over
time. Table III and IV present HSIC values for the control (placebo) and
treatment (Estradiol) groups over time (baseline vs. week 4) using the
Gaussian kernel and linear kernel, respectively. We see that, though HSIC
values using the Gaussian kernel are difficult to distinguish from each other,
the HSIC value using the linear kernel for the treatment group is
significantly larger than the value for the control group at week 4. This
result indicates that the new test detects the perturbation in the dependence
relationship between the control and treatment groups over time.

##  V Conclusion and discussion

In this paper, we proposed a new kernel-based test for evaluating whether
pairs of high-dimensional variables have similar dependence relationships or
not across two conditions. Using the previously developed HSIC statistic for
our test statistic, the new test works for high-dimensional data and performs
well for assessing complicated dependence relationships. The asymptotic
distribution of the test statistic facilitates its application to large data
sets and the omnibus test enables the proposed test to work for a wide range
of alternatives. The new test often exhibits superior power without particular
model assumptions or specifications.

This paper mainly handles the generalized dependence relationships caused by
multivariate data. Compared to the existing DCE analysis that quantifies
differential correlated patterns across conditions, the focus of this paper is
to study whether the new test captures the change in dependence relationships
across different conditions. Hence, the new test does not seek to measure how
much or in what way the dependence relationships of two variables differ
depending on their conditions. A canonical correlation may be used to quantify
the relationship between two multivariate data, though it is still difficult
to gauge the generalized dependence between two high-dimensional variables.

The proposed method works under the permutation null distribution and the
testing procedure is based on the permutation test. The permutation test is
easy to implement and avoids any parametric modeling assumptions. Moreover, it
can ensure exact control of the type I error rate for all test statistics
under the null hypothesis. These attractive properties have led to the
permutation approach being used across a wide range of settings [33, 25, 27].
However, the permutation test has the requirement for exchangeability under
the null hypothesis and the permutation approach could be problematic if the
assumption of exchangeability is violated [34, 35, 36]. To address this issue,
one could derive the asymptotic true null distribution of $\tilde{T}_{N}$ as
it does not require the assumption of exchangeability. However, this is beyond
the scope of the current endeavor and requires further investigation.
Nonetheless, our proposed approach represents a critical step towards more
effective differential dependency analysis.

## References

  * [1] A. Ezzati, M. J. Katz, A. R. Zammit, M. L. Lipton, M. E. Zimmerman, M. J. Sliwinski, and R. B. Lipton, “Differential association of left and right hippocampal volumes with verbal episodic and spatial memory in older adults,” _Neuropsychologia_ , vol. 93, pp. 380–385, 2016.
  * [2] Y. Kim, S. Cheon, H. Jeong, U. Park, N.-Y. Ha, J. Lee, K. M. Sohn, Y.-S. Kim, and N.-H. Cho, “Differential association of viral dynamics with disease severity depending on patients’ age group in covid-19,” _Frontiers in Microbiology_ , vol. 12, p. 712260, 2021.
  * [3] D. D. Bhuva, J. Cursons, G. K. Smyth, and M. J. Davis, “Differential co-expression-based detection of conditional relationships in transcriptional data: comparative analysis and application to breast cancer,” _Genome biology_ , vol. 20, no. 1, pp. 1–21, 2019.
  * [4] H. A. Chowdhury, D. K. Bhattacharyya, and J. K. Kalita, “(differential) co-expression analysis of gene expression: a survey of best practices,” _IEEE/ACM transactions on computational biology and bioinformatics_ , vol. 17, no. 4, pp. 1154–1173, 2019.
  * [5] Y. Choi and C. Kendziorski, “Statistical methods for gene set co-expression analysis,” _Bioinformatics_ , vol. 25, no. 21, pp. 2780–2786, 2009.
  * [6] B. M. Tesson, R. Breitling, and R. C. Jansen, “Diffcoex: a simple and sensitive method to find differentially coexpressed gene modules,” _BMC bioinformatics_ , vol. 11, no. 1, pp. 1–9, 2010.
  * [7] Y. Rahmatallah, F. Emmert-Streib, and G. Glazko, “Gene sets net correlations analysis (gsnca): a multivariate differential coexpression test for gene sets,” _Bioinformatics_ , vol. 30, no. 3, pp. 360–368, 2014.
  * [8] A. T. McKenzie, I. Katsyv, W.-M. Song, M. Wang, and B. Zhang, “Dgca: a comprehensive r package for differential gene correlation analysis,” _BMC systems biology_ , vol. 10, pp. 1–25, 2016.
  * [9] A. Gretton, O. Bousquet, A. Smola, and B. Schölkopf, “Measuring statistical dependence with Hilbert-Schmidt norms,” in _International conference on algorithmic learning theory_. Springer, 2005, pp. 63–77.
  * [10] A. Gretton, K. Fukumizu, C. H. Teo, L. Song, B. Schölkopf, and A. J. Smola, “A kernel statistical test of independence.” in _Nips_ , vol. 20. Citeseer, 2007, pp. 585–592.
  * [11] W. Hoeffding, “The large-sample power of tests based on permutations of observations,” _The Annals of Mathematical Statistics_ , pp. 169–192, 1952\.
  * [12] E. L. Lehmann, J. P. Romano, and G. Casella, _Testing statistical hypotheses_. Springer, 2005, vol. 3.
  * [13] L. Song, A. Smola, A. Gretton, and K. M. Borgwardt, “A dependence maximization view of clustering,” in _Proceedings of the 24th international conference on Machine learning_ , 2007, pp. 815–822.
  * [14] D. Niu, J. G. Dy, and M. I. Jordan, “Iterative discovery of multiple alternativeclustering views,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 36, no. 7, pp. 1340–1353, 2013.
  * [15] X. He, T. Gumbsch, D. Roqueiro, and K. Borgwardt, “Kernel conditional clustering,” in _2017 IEEE International Conference on Data Mining (ICDM)_. IEEE, 2017, pp. 157–166.
  * [16] J. Peters, D. Janzing, A. Gretton, and B. Schölkopf, “Kernel methods for detecting the direction of time series,” in _Advances in Data Analysis, Data Handling and Business Intelligence: Proceedings of the 32nd Annual Conference of the Gesellschaft für Klassifikation eV, Joint Conference with the British Classification Society (BCS) and the Dutch/Flemish Classification Society (VOC), Helmut-Schmidt-University, Hamburg, July 16-18, 2008_. Springer, 2010, pp. 57–66.
  * [17] M. Yamada, A. Kimura, F. Naya, and H. Sawada, “Change-point detection with feature selection in high-dimensional time-series data,” in _Twenty-Third International Joint Conference on Artificial Intelligence_ , 2013.
  * [18] G. Wang, W. K. Li, and K. Zhu, “New hsic-based tests for independence between two stationary multivariate time series,” _Statistica Sinica_ , vol. 31, no. 1, pp. 269–300, 2021.
  * [19] K. Balasubramanian, B. Sriperumbudur, and G. Lebanon, “Ultrahigh dimensional feature screening via rkhs embeddings,” in _Artificial Intelligence and Statistics_. PMLR, 2013, pp. 126–134.
  * [20] T. Freidling, B. Poignard, H. Climente-González, and M. Yamada, “Post-selection inference with hsic-lasso,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 3439–3448.
  * [21] D. He, J. Cheng, and K. Xu, “High-dimensional variable screening through kernel-based conditional mean dependence,” _Journal of Statistical Planning and Inference_ , vol. 224, pp. 27–41, 2023.
  * [22] M. J. Greenacre, “Correspondence analysis of multivariate categorical data by weighted least-squares,” _Biometrika_ , vol. 75, no. 3, pp. 457–467, 1988\.
  * [23] R. B. Nelsen, _An introduction to copulas_. Springer science & business media, 2007.
  * [24] A. K. Smilde, H. A. Kiers, S. Bijlsma, C. Rubingh, and M. Van Erk, “Matrix correlations for high-dimensional data: the modified rv-coefficient,” _Bioinformatics_ , vol. 25, no. 3, pp. 401–405, 2009.
  * [25] H. Song and H. Chen, “Generalized kernel two-sample tests,” _arXiv preprint arXiv:2011.06127_ , 2020.
  * [26] H. Liu, A. Plantinga, Y. Xiang, and M. Wu, “A kernel-based test of independence for cluster-correlated data,” _Advances in neural information processing systems_ , vol. 34, pp. 9869–9881, 2021.
  * [27] H. Liu, W. Ling, X. Hua, J.-Y. Moon, J. S. Williams-Nguyen, X. Zhan, A. M. Plantinga, N. Zhao, A. Zhang, R. Knight _et al._ , “Kernel-based genetic association analysis for microbiome phenotypes identifies host genetic drivers of beta-diversity,” _Microbiome_ , vol. 11, no. 1, pp. 1–19, 2023\.
  * [28] Y. Liu and J. Xie, “Cauchy combination test: a powerful test with analytic p-value calculation under arbitrary dependency structures,” _Journal of the American Statistical Association_ , vol. 115, no. 529, pp. 393–402, 2020.
  * [29] S. B. Cho, J. Kim, and J. H. Kim, “Identifying set-wise differential co-expression in gene expression microarray data,” _BMC bioinformatics_ , vol. 10, pp. 1–13, 2009.
  * [30] D. Amar, H. Safer, and R. Shamir, “Dissection of regulatory networks that are altered in disease via differential co-expression,” _PLoS computational biology_ , vol. 9, no. 3, p. e1002955, 2013.
  * [31] C. Siska, R. Bowler, and K. Kechris, “The discordant method: a novel approach for differential correlation,” _Bioinformatics_ , vol. 32, no. 5, pp. 690–696, 2016.
  * [32] C. M. Mitchell, S. D. Reed, S. Diem, J. C. Larson, K. M. Newton, K. E. Ensrud, A. Z. LaCroix, B. Caan, and K. A. Guthrie, “Efficacy of vaginal estradiol or vaginal moisturizer vs placebo for treating postmenopausal vulvovaginal symptoms: a randomized clinical trial,” _JAMA internal medicine_ , vol. 178, no. 5, pp. 681–690, 2018.
  * [33] X. Zhan, A. Plantinga, N. Zhao, and M. C. Wu, “A fast small-sample kernel independence test for microbiome community-level association analysis,” _Biometrics_ , vol. 73, no. 4, pp. 1453–1463, 2017.
  * [34] E. Chung and J. P. Romano, “Exact and asymptotically robust permutation tests,” _The Annals of Statistics_ , vol. 41, no. 2, pp. 484–507, 2013.
  * [35] ——, “Multivariate and multiple permutation tests,” _Journal of Econometrics_ , vol. 193, no. 1, pp. 76–91, 2016.
  * [36] C. J. DiCiccio and J. P. Romano, “Robust permutation tests for correlation and regression coefficients,” _Journal of the American Statistical Association_ , vol. 112, no. 519, pp. 1211–1220, 2017.