# Unsupervised Disentanglement without Autoencoding:
Pitfalls and Future Directions

Andrea Burns  Aaron Sarna  Dilip Krishnan  Aaron Maschinot

###### Abstract

Disentangled visual representations have largely been studied with generative
models such as Variational AutoEncoders (VAEs). While prior work has focused
on generative methods for disentangled representation learning, these
approaches do not scale to large datasets due to current limitations of
generative models. Instead, we explore regularization methods with contrastive
learning, which could result in disentangled representations that are powerful
enough for large scale datasets and downstream applications. However, we find
that unsupervised disentanglement is difficult to achieve due to optimization
and initialization sensitivity, with trade-offs in task performance. We
evaluate disentanglement with downstream tasks, analyze the benefits and
disadvantages of each regularization used, and discuss future directions.

Machine Learning, ICML

##  1 Introduction

Learning semantically interpretable image representations has many benefits,
such as better generalization (Eastwood & Williams, 2018), robustness to noise
(Lopez et al., 2018), and increased transfer performance (van Steenkiste et
al., 2019). Disentanglement can add interpretability if each representation
dimension maps to one factor of variation present in the images while being
relatively invariant to changes in the other factors (Bengio et al., 2013);
this is the definition of disentanglement that we adopt in this work.

Most unsupervised disentanglement approaches are built upon Variational
AutoEncoders (VAEs) (Kingma & Welling, 2014), which are a type of generative
model trained with a reconstruction and regularization loss. The
regularization pushes the posterior distribution of the generative factors
toward an isotropic Gaussian, encouraging its representations toward a
factorized prior with independent dimensions, under the hope that this
independence is close to the desired semantic disentanglement. However,
because the reconstruction loss limits the amount of disentanglement that can
be imposed in a vanilla VAE, these approaches have largely focused on
augmenting the regularization loss to increase disentanglement without
degrading reconstruction quality (Higgins et al., 2016; Kim & Mnih, 2018).
Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) have also
been used for disentanglement: the generator inputs are divided into latent
and noise components, and a regularization loss is added to maximize the
mutual information (MI) between the observations and latent components to the
discriminator loss (Chen et al., 2016; Gulrajani et al., 2017).

However, the losses that such models use to generate realistic images compete
with the disentanglement regularization loss (Kim & Mnih, 2018) and the latent
representations resulting from VAEs and GANs (disentangled or not) have
significantly worse performance on downstream tasks such as retrieval and
transfer learning (Pathak et al., 2016; Brock et al., 2019). We therefore
consider disentanglement through contrastive representation learning (Chen et
al., 2020; Tian et al., 2020a; van den Oord et al., 2018), with the hope of
overcoming these limitations. Self-supervised contrastive learning frameworks
achieve state-of-the-art performance for many tasks such as classification,
detection and segmentation (Chen et al., 2020; Xiong et al., 2020; Chaitanya
et al., 2020), so a disentangled contrastive representation that maintains
task performance would be of great interest.

In the self-supervised setting, the contrastive loss is applied to the outputs
of a shallow MLP (the ‘projection network’) that takes the encoder output as
input (Chen et al., 2020). Our initial hypothesis is that it may be feasible
to disentangle the encoder outputs while training the projection network. We
explore three methods for disentanglement with an unsupervised contrastive
loss: minimizing MI, orthogonalizing to encourage independence, and reducing
dependencies via Hessian regularization on the contrastive loss. While each
regularizer has supporting intuition, in practice we find that issues arise
due to disentanglement on undesired axes, initialization sensitivity, or
optimization. This prevents the regularizers from reliably scaling to real-
world datasets. We hope our results can be used to guide the community at
large and spur new research directions.

##  2 Contrastive Setup

We use the SimCLR setup (Chen et al., 2020) to study unsupervised
disentanglement: encoder outputs $r_{i}$ are fed into a projection network and
the contrastive loss is computed on projection outputs $z_{i}$ for $i$ in a
batch of size $N$. We evaluate disentanglement on subembeddings $r_{i,k}$ or
$z_{i,k}$, where $k\in[K]$ is a slice of $r_{i}$ or $z_{i}$. The InfoNCE loss
(van den Oord et al., 2018), which we refer to as InfoMax, is:

$\mathcal{L_{\text{InfoMax}}}=-\sum_{i\in I}\log{\frac{\exp({z_{i}\cdot
z_{j}/\tau})}{\sum\limits_{a\in A}\exp({z_{i}\cdot z_{a}/\tau})}}$ (1)

where we have anchor $z_{i}$ and want to maximize MI with its positive
$z_{j}$. Positives of the image $x_{i}$ are augmentations (i.e., ‘views’)
obtained via scaling, flipping, and color distortion transformations. The set
$A$ includes the positive as well as all negatives in the batch with respect
to the anchor.

We aim to disentangle subembeddings (slices of the representation) such that
each subembedding encodes a distinct factor of variation. To do this, we
maximize MI between views of the same subembedding. We adapt Eq. (1) to:

$\mathcal{L_{\text{SubInfoMax}}}=\sum_{k\in K}\left(-\sum_{i\in
I}\log{\frac{\exp({z_{i,k}\cdot z_{j,k}/\tau})}{\sum\limits_{a\in
A}\exp({z_{i,k}\cdot z_{a,k}/\tau})}}\right)$ (2)

where $z_{j,k}$ is the $k$th subembedding of the positive $z_{j}$ for anchor
$z_{i}$. This results in $K$ InfoMax terms, which we sum over. Our experiments
use the base case of two subembeddings (i.e., $K=2$). The regularization
methods are added to the InfoMax objective as such for a regularizer $R$:

$\mathcal{L_{\text{Disentanglement}}}=\mathcal{L_{\text{SubInfoMax}}}+\lambda
R$ (3)

where $\lambda$ weighs the regularization in the final objective.

##  3 Experiment Setup

###  3.1 Dataset

We form a more realistic dataset for disentanglement, which we name
MNIST/STL-10. It is similar to Colorful-Moving-MNIST (Tian et al., 2020b), but
designed for disentanglement. MNIST (LeCun et al., 2010) contains handwritten
digits, and STL-10 is an image classification dataset (Coates et al., 2011).
We create MNIST/STL-10 instances by overlaying a MNIST digit on a STL-10
image. We can vary the following factors: digit class (DC), digit location
(DL), or background class (BC). A view pair holds two of three factors
constant while the third is varied uniformly at random111Examples can be found
in the Appendix with additional dataset details. The dataset has been made
publicly available at https://github.com/aburns4/UnsupDisent.

We run experiments with the DC-BC dataset. The dataset has two disjoint
downstream tasks; digit (DC) and background (BC) classification. Upstream
training fixes digit and background class in a view pair, but varies digit
location. We train the encoder and projection network with joint labels, where
each digit (ten classes) and background (ten classes) combination is one
class, creating 100 classes. Separate downstream DC and BC linear classifiers
are trained.

###  3.2 Model Architecture

We use a ResNet34 (He et al., 2016) encoder, which outputs $r$, a 512D
representation. As compactness is a prerequisite for disentanglement (i.e., a
factor of variation is represented with minimal elements) and our analysis
finds our regularizers have a local effect on the representation (see Section
4.1), we also try using the projection output $z$ as input to the linear
classifier $C$ used for our downstream tasks.

###  3.3 Evaluation

We compare accuracy when the full representation ($r$ or $z$) or subembeddings
($r_{k}$ or $z_{k}$) are used as classification input; subembeddings are
halves of the full representation. Table 1 shows the classification accuracy
reflecting ideal disentanglement (achieved in a supervised setting, see
details in Appendix). Accuracy of $97.3\%$ and $64.5\%$ is achieved with $r$
for DC and BC, respectively and $r$ performs no better than either
subembedding, showing no loss of information. There also is a large gap
$|C(r_{0})-C(r_{1})|$, where $r_{1}$ has near random performance on DC, and
likewise for $r_{0}$ on BC. This diagonal accuracy trend suggests maximal
disentanglement.

Table 1: Accuracy for digit (DC) and background (BC) classification tasks using a fully supervised pipeline with the full representation $r$ or its subembeddings $r_{0}$, $r_{1}$ as classification input. Classification Input | DC | BC
---|---|---
$r$ | 97.3 | 64.5
$r_{0}$ | 97.3 | 10.1
$r_{1}$ | 11.7 | 64.5
$|C(r_{0})-C(r_{1})|$ | 85.6 | 54.4

##  4 Method

###  4.1 InfoMin

Our first regularizer follows the intuition of wanting to minimize MI between
subembeddings, similar to reducing total correlation of latent codes used by
Kim & Mnih (2018). We follow the approach of Tian et al. (2020b) for view
selection and define the first regularizer as
$R_{\text{InfoMin}}=-\mathcal{L}_{\text{InfoMax}}$, but suspect a priori that
it may cause optimization issues since a lower bound is now being minimized
instead of maximized.

$R_{\text{InfoMin}}=\sum_{k\neq k^{\prime}}\sum_{i\in
I}\log{\frac{\exp({z_{i,k}\cdot z_{i,k^{\prime}}/\tau})}{\sum\limits_{a\in
A}\exp({z_{i,k}\cdot z_{a,k}/\tau})}}$ (4)

We let $k$ and $k^{\prime}$ map to subembeddings of sample $z_{i}$ and sum
over all such pairs. Eq. (4) is computed over both views, as we compute the
term $R_{\text{InfoMin}}$ within a single view. The regularization is applied
to $z$ so that the InfoMin and InfoMax terms compete during the optimization
of Eq. (3).

####  4.1.1 InfoMin Experiments

We try using $r$ or $z$ and their subembeddings as input to the downstream
classifier. In the InfoMin pipeline, we use two projection heads, each with
output dimension eight: $r_{0}$ and $r_{1}$ are passed to each network and
result in $z_{0}$ and $z_{1}$. We define the full projection output $z$ as
$z:=[z_{0},z_{1}]$.

Table 2 summarizes disentanglement performance as measured by classification
accuracy for InfoMin regularization. For perfect disentanglement, one would
expect the DC accuracy for one of the two subembeddings (e.g., $r_{0}$ or
$r_{1}$) to be equal to that achieved when using the full embedding ($r$), and
the DC accuracy for the other subembedding would be equal to chance ($10\%$).
For BC classification, the same would be true except that $r_{0}$ and $r_{1}$
would switch roles.

The results in Table 2 are consistent with InfoMin not inducing
disentanglement. The subembeddings have nearly the same downstream performance
on each task; e.g., $z$, $z_{0}$, and $z_{1}$ have $\sim$74% accuracy on DC
when $\lambda=0.001$. We also see that InfoMin primarily affects the
projection head output, demonstrating the regularization effects are local to
the layer where it is applied. This can be seen as $r$, $r_{0}$, and $r_{1}$
perform nearly the same with different $\lambda$, while $z$, $z_{0}$, and
$z_{1}$ performance drops heavily ($\sim$25%) with $\lambda=0.1$.

We find $R_{\text{InfoMin}}$ has a direct trade-off with
$\mathcal{L}_{\text{SubInfoMax}}$, potentially canceling out any
disentanglement (see Appendix). As this approach causes optimization issues
and hurts performance, we next explore approaches that enforce disentanglement
without directly fighting the InfoMax objective.

Table 2: Accuracy for digit (DC) and background (BC) classification tasks varying the weight $\lambda$ of the InfoMin term. Classification Input | $\lambda=0.001$ | $\lambda=0.1$
---|---|---
DC | BC | DC | BC
$r$ | 97.0 | 63.1 | 96.4 | 61.1
$r_{0}$ | 97.0 | 62.8 | 96.3 | 60.8
$r_{1}$ | 97.0 | 63.0 | 96.3 | 60.8
$z$ | 74.3 | 60.7 | 47.6 | 35.7
$z_{0}$ | 74.4 | 60.6 | 47.5 | 35.7
$z_{1}$ | 74.2 | 60.8 | 47.5 | 35.8

###  4.2 Orthogonality Constraint

Due to pitfalls experienced with $R_{\text{InfoMin}}$, we next tried
constraining subembeddings to be orthogonal to enforce approximate (linear)
independence. The resulting regularizer is an unsigned cosine distance between
the subembeddings. Unlike $R_{\text{InfoMin}}$, computed between subembeddings
from a single sample, we compute $R_{\text{Ortho}}$ between a subembedding
$z_{i,k}$ of representation $z_{i}$ with subembeddings $z_{i,k^{\prime}}$
_and_ $z_{j,k^{\prime}}$

$R_{\text{Ortho}}=\sum_{k\neq k^{\prime}}\sum_{i}\sum_{j}\frac{|z_{i,k}\cdot
z_{j,k^{\prime}}|}{\|z_{i,k}\|\|z_{j,k^{\prime}}\|}$ (5)

$R_{\text{Ortho}}$ is applied to $z$ due to the locality seen with
$R_{\text{InfoMin}}$ and to have a lower dimension representation.

Degenerate Solutions Several degenerate solutions may arise when minimizing
$R_{\text{Ortho}}$. E.g., two subembeddings could be rotations of one another,
or the regularization could push one subembedding to a random vector. Both
would satisfy Eq. (5) without enforcing meaningful disentanglement. We found
evidence of degenerate solutions experimentally as well, which are included in
the Appendix.

To address these issues, we also try adding a permutation matrix $P$ which
randomly permutes one subembedding before computing the constraint; this
should prevent degenerate solutions which rely on $z$’s element ordering to
avoid penalty. Without loss of generality we permute subembedding $k$:

$R_{P(\text{Ortho})}=\sum_{k\neq
k^{\prime}}\sum_{i}\sum_{j}\frac{|P(z_{i,k})\cdot
z_{j,k^{\prime}}|}{\|z_{i,k}\|\|z_{j,k^{\prime}}\|}$ (6)

####  4.2.1 Orthogonality Experiments

Table 3: Accuracy for digit (DC) and background (BC) classification tasks with orthogonality and Hessian regularization, varying the weight $\lambda$. For orthogonality regularization, ablations with the permutation mechanism ($P$) are also included.  Classification Input | Orthogonality w/o $P$ | Orthogonality w/ $P$ | Hessian
---|---|---|---
$\lambda=0.001$ | $\lambda=0.1$ | $\lambda=0.001$ | $\lambda=0.1$ | $\lambda=0.001$ | $\lambda=0.1$
DC | BC | DC | BC | DC | BC | DC | BC | DC | BC | DC | BC
$z$ | 97.0 | 62.4 | 69.3 | 59.6 | 97.0 | 62.2 | 97.2 | 61.6 | 92.4 | 57.2 | 93.6 | 56.3
$z_{0}$ | 76.6 | 48.4 | 51.2 | 39.9 | 77.1 | 52.4 | 88.0 | 56.9 | 60.7 | 46.7 | 74.7 | 42.4
$z_{1}$ | 73.2 | 54.0 | 46.3 | 42.7 | 71.8 | 57.0 | 82.2 | 60.0 | 47.9 | 34.6 | 57.5 | 50.9
$|C(z_{0})-C(z_{1})|$ | 3.4 | 5.6 | 4.9 | 2.8 | 5.3 | 4.6 | 5.8 | 3.1 | 12.8 | 12.1 | 17.2 | 8.5

The orthogonality contrastive pipeline also uses two projection heads (as
described in Section 4.1.1). In Table 3, we now see some disentanglement with
$R_{\text{Ortho}}$, as nonzero differences (e.g.,
$|C_{DC}(z_{0})-C_{DC}(z_{1})|$, $|C_{BC}(z_{0})-C_{BC}(z_{1})|$, where
$C(\cdot)$ denotes classification accuracy) are achieved. Without permutation
matrix $P$, the higher weight $\lambda=0.1$ hurts absolute performance using
$z$ and its subembeddings. With $P$, performance is no longer hurt, and
downstream subembedding performance increases with both $\lambda$ weights.
This may be due to $P$ preventing subembedding slices from becoming random or
rotations of other elements.

While the higher $\lambda$ does not increase $|C(z_{0})-C(z_{1})|$ values when
$P$ is used, it does result in the highest subembedding performance over all
experiments (88.0% and 60.0% for DC and BC tasks, respectively).
Interestingly, the permutation $P$ results in a larger average subembedding
difference for both tasks, having more impact than $\lambda$.

Using $R_{P(\text{Ortho})}$ with $\lambda=0.1$ shows the desired diagonal
trend between $z_{0}$ and $z_{1}$, where the former performs better on DC
(88.0 vs. 82.2) and the latter better on BC (60.0 vs. 56.9). However, the
performance difference $|C(z_{0})-C(z_{1})|$ varied over multiple runs, with
some having smaller values. Disentanglement also has higher variance on the BC
task, and we include additional analysis in the Appendix.

Ultimately, $R_{P(\text{Ortho})}$ resulted in some disentanglement, unlike
$R_{\text{InfoMin}}$, but $|C(z_{0})-C(z_{1})|$ was not as large as the
difference demonstrated with supervision in Table 1. The permutation matrix
also prevented loss of information, as seen by the improvement in performance
across $z$ and its subembeddings. We’d like to retain downstream task
performance while creating larger degrees of disentanglement.

###  4.3 Hessian Regularization

Although $R_{\text{Ortho}}$ avoids direct optimization trade-off with
$\mathcal{L}_{\text{SubInfoMax}}$, it can still result in trivial solutions
with no disentanglement. Moreover, $R_{\text{Ortho}}$ did not create a
significant gap in subembedding accuracy. We next explore constraining the
Hessian of $\mathcal{L}_{\text{InfoMax}}$ with respect to $z$, as it gives a
measure of dependence between individual subembedding elements. We expect the
Hessian of $z_{i}$ and $z_{j}$ to be zero if they are disentangled; this
element-wise constraint may reduce trivial solutions. Computing the Hessian is
expensive, but we can use the Gauss-Newton approximation:

$H_{ij}=\frac{\partial^{2}\mathcal{L}_{\text{InfoMax}}}{\partial z_{i}\partial
z_{j}}\approx\frac{\partial\mathcal{L}_{\text{InfoMax}}}{\partial
z_{i}}\cdot\frac{\partial\mathcal{L}_{\text{InfoMax}}}{\partial z_{j}}$ (7)

This is similar to the approach used by Peebles et al. (2020), but they
constrain all off-diagonal terms to disentangle individual latent codes.
Instead, we disentangle subembeddings with $\frac{d}{2}$ elements (for a $d$
element representation), allowing for intra-subembedding dependencies; i.e.,
we minimize the sum of the off-diagonal block terms.

One InfoMax and one projection network are used, as the Hessian is computed on
a unified representation $z$; $z_{0}$ and $z_{1}$ are now its halves. The
final regularizer is the Frobenius norm of the off-diagonal Hessian blocks:

$R_{\text{Hess}}=\sqrt{\sum_{i\in[k]}\sum_{j\in[k^{\prime}]}\left|\frac{\partial\mathcal{L}_{\text{InfoMax}}}{\partial
z_{i}}\cdot\frac{\partial\mathcal{L}_{\text{InfoMax}}}{\partial
z_{j}}\right|^{2}}$ (8)

####  4.3.1 Hessian Experiments

The Hessian regularizer only creates the desired diagonal accuracy (i.e.,
where one subembedding performs better on DC, and the other better on BC) with
the higher $\lambda=0.1$. Yet both $\lambda$ values suggest that
$R_{\text{Hess}}$ is a stronger regularizer than $R_{\text{InfoMin}}$ or
$R_{\text{Ortho}}$, as $|C(z_{0})-C(z_{1})|$ grows larger.

The diagonal performance trend found with the larger $\lambda$ was consistent
in four out of five randomized experiments, significantly better than the one
of out five experiments in the case of the orthogonality constraint. While
$|C(z_{0})-C(z_{1})|$ is higher on average using $R_{\text{Hess}}$, it also
has higher variance. We include reproducibility analysis in the Appendix.

The downstream accuracy is similar using $z$ with either $\lambda$ weight, but
$\lambda=0.1$ results in subembeddings $z_{0}$ and $z_{1}$ with higher
performance (74.7 and 50.9 vs. 60.7 and 46.7 for the DC and BC tasks). Despite
this, the strength of the Hessian constraint costs downstream accuracy, with
lower accuracy using $z$, $z_{0}$, or $z_{1}$ compared to the
$R_{\text{Ortho}}$ results.

In the end, $R_{\text{Hess}}$ is a strong enough constraint to enforce larger
$|C(z_{0})-C(z_{1})|$, suggesting greater disentanglement, but it comes at the
cost of downstream performance. The InfoMax loss may not balance this
regularization enough, and the final objective may require additional losses
to prevent loss of information in the representation. Using the Hessian and
orthogonality terms together may also better balance the goal of
disentanglement and high downstream accuracy.

##  5 Discussion

Learning disentangled representations without autoencoding remains a difficult
task with trade-offs in downstream accuracy, relative subembedding
performance, and optimization. Classification accuracy suggests that enforcing
orthogonality or reducing Hessian dependencies cannot achieve both high
downstream performance and significant disentanglement. Visualizing the
representations learned via each regularizer may provide insight on how to
balance both desired qualities. We also find a loss of information, as neither
subembedding performs as well as the full representation; additional
constraints enforcing the full representation performs no better than either
subembedding may help to prevent this. Lastly, additional inductive biases may
be needed for greater disentanglement without hurting task performance.

## References

  * Bengio et al. (2013) Bengio, Y., Courville, A. C., and Vincent, P.  Representation learning: A review and new perspectives.  In _IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)_ , 2013.
  * Brock et al. (2019) Brock, A., Donahue, J., and Simonyan, K.  Large scale GAN training for high fidelity natural image synthesis.  In _International Conference on Learning Representations (ICLR)_ , 2019.
  * Chaitanya et al. (2020) Chaitanya, K., Erdil, E., Karani, N., and Konukoglu, E.  Contrastive learning of global and local features for medical image segmentation with limited annotations.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2020.
  * Chen et al. (2020) Chen, T., Kornblith, S., Norouzi, M., and Hinton, G.  A simple framework for contrastive learning of visual representations.  In _International Conference on Learning Representations (ICLR)_ , 2020.
  * Chen et al. (2016) Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P.  Infogan: Interpretable representation learning by information maximizing generative adversarial nets.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2016.
  * Coates et al. (2011) Coates, A., Ng, A., and Lee, H.  An analysis of single-layer networks in unsupervised feature learning.  In _International Conference on Artificial Intelligence and Statistics (AISTATS)_ , 2011.
  * Eastwood & Williams (2018) Eastwood, C. and Williams, C. K. I.  A framework for the quantitative evaluation of disentangled representations.  In _International Conference on Learning Representations (ICLR)_ , 2018.
  * Goodfellow et al. (2014) Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.  Generative adversarial nets.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2014.
  * Gulrajani et al. (2017) Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.  Improved training of wasserstein gans.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2017.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _IEEE conference on computer vision and pattern recognition (CVPR)_ , 2016.
  * Higgins et al. (2016) Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A.  Beta-vae: Learning basic visual concepts with a constrained variational framework.  In _International Conference on Learning Representations (ICLR)_ , 2016.
  * Kim & Mnih (2018) Kim, H. and Mnih, A.  Disentangling by factorising.  In _International Conference on Machine Learning (ICML)_ , 2018.
  * Kingma & Welling (2014) Kingma, D. P. and Welling, M.  Auto-encoding variational bayes.  In _International Conference on Learning Representations (ICLR)_ , 2014.
  * LeCun et al. (2010) LeCun, Y., Cortes, C., and Burges, C.  MNIST handwritten digit database.  2010\.
  * Lopez et al. (2018) Lopez, R., Regier, J., Jordan, M. I., and Yosef, N.  Information constraints on auto-encoding variational bayes.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2018.
  * Pathak et al. (2016) Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A. A.  Context encoders: Feature learning by inpainting.  In _IEEE conference on computer vision and pattern recognition (CVPR)_ , 2016.
  * Peebles et al. (2020) Peebles, W., Peebles, J., Zhu, J.-Y., Efros, A. A., and Torralba, A.  The hessian penalty: A weak prior for unsupervised disentanglement.  In _European Conference on Computer Vision (ECCV)_ , 2020.
  * Tian et al. (2020a) Tian, Y., Krishnan, D., and Isola, P.  Contrastive multiview coding.  In _European Conference on Computer Vision (ECCV)_ , 2020a.
  * Tian et al. (2020b) Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., and Isola, P.  What makes for good views for contrastive learning?  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2020b.
  * van den Oord et al. (2018) van den Oord, A., Li, Y., and Vinyals, O.  Representation learning with contrastive predictive coding.  _CoRR_ , abs/1807.03748, 2018.
  * van Steenkiste et al. (2019) van Steenkiste, S., Locatello, F., Schmidhuber, J., and Bachem, O.  Are disentangled representations helpful for abstract visual reasoning?  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2019.
  * Xiong et al. (2020) Xiong, Y., Ren, M., and Urtasun, R.  Loco: Local contrastive representation learning.  In _International Conference on Neural Information Processing Systems (NeurIPS)_ , 2020.

##  Appendix A Appendix

###  A.1 MNIST/STL-10 Dataset

####  A.1.1 Dataset Creation

We create MNIST/STL-10 by overlaying MNIST (LeCun et al., 2010) handwritten
digits on STL-10 (Coates et al., 2011) images. Each overlayed MNIST image is
centered at one of eight possible pixel locations in the STL-10 image (Figure
1). Since MNIST digits tend to saturate each color channel, a simple per-pixel
maximum of the MNIST digit image and the STL-10 image was used. An
MNIST/STL-10 instance is generated by specifying three properties: 1) the
MNIST digit instance belonging to one of 10 classes (0 - 9); 2) the STL-10
instance belonging to one of 10 classes (airplane, bird, car, cat, deer, dog,
horse, monkey, ship, truck); and 3) the pixel location at which to center the
MNIST instance on the STL-10 instance (8 possible values).

Figure 1: Possible digit locations (DLs) at which to center the MNIST instance
on a STL-10 image. Each MNIST instance is 28x28, and each STL-10 instance is
64x64. Digit locations were chosen to be uniformly distributed in 2 rows and 4
columns such that the entire MNIST instance was always overlaid. MNIST
bounding box examples are shown for location 3 (in blue) and 5 (in green).

As we have three factors of variation (digit class (DC), background class
(BC), and digit location (DL)), we can create three dataset variants: DC-BC,
DC-DL, and BC-DL. In each of these MNIST/STL-10 variants, we hold two of the
three factors of variation constant in a view pair. So, DC-BC holds digit
class and background class constant in view pairs while digit location is
varied, DC-DL holds digit class and digit location constant in view pairs
while background class is varied, and so on. Each dataset variant has a total
of 100k train samples and 10k test samples.

Note that when we fix a factor of variation, we do not fix a particular
instance in that view pair. E.g., if a view pair of the DC-BC dataset has DC
set to the digit one, each view can contain a different instance of the one
class, as seen in Figure 2.

####  A.1.2 Dataset Examples

We include view pairs from all three dataset variants to illustrate what the
MNIST/STL-10 samples look like. In Figure 2, we show a view pair from the DC-
BC dataset. In this dataset, the digit location is varied between views. View
0 lays the digit in the upper middle-left position (location 2), while view 1
lays the digit in the lower middle-right position (location 7). Fixed between
views is both the digit class (one) and background class (deer). As seen in
the view pair, the instances of the written digit and image class are varied.

Figure 3 and Figure 4 include view pair examples from the DC-DL and BC-DL
datasets. In a DC-DL view pair the digit class and digit location are fixed,
as can be seen by the digit one and lower right digit position (location 8)
held constant in the view pair of Figure 3. The background class in this case
is varied, with view 0 containing the plane class, while view 1 contains the
monkey class. Lastly, Figure 4 illustrates a view pair from the BC-DL
MNIST/STL-10 dataset variant, in which digit class is varied between views and
background class and digit location are held constant. This can be seen as
view 0 contains the digit one, while view 1 contains the digit eight, but both
are located in the upper right most position (location 4) atop images of
ships.

(a) View 0 in view pair

(b) View 1 in view pair

Figure 2: Dataset examples from the MNIST/STL-10 DC-BC variant.

(a) View 0 in view pair

(b) View 1 in view pair

Figure 3: Dataset examples from the MNIST/STL-10 DC-DL variant.

(a) View 0 in view pair

(b) View 1 in view pair

Figure 4: Dataset examples from the MNIST/STL-10 BC-DL variant.

###  A.2 Fully Supervised Pipeline

While we are interested in a fully unsupervised contrastive learning pipeline
for learning disentangled representations, we verify the feasibility of our
general approach via a fully supervised experiment setting.

We split $r$ into its halves $r_{0}$ and $r_{1}$, and apply the InfoMax loss
to each half using disjoint labels. Instead of using joint labels as described
in Section 3.1, we separate the task labels for each InfoMax. E.g., for the
DC-BC dataset, $\mathcal{L}_{\text{SubInfoMax}}$ is applied to $r_{0}$ using
the DC labels (ten MNIST digit classes) to define positive and negative
samples, and $L_{\text{SubInfoMax}}$ is applied to $r_{1}$ using the BC labels
(ten STL-10 image classes) to define positive and negative samples.

###  A.3 InfoMin-Max Trade-Off

(a) The InfoMax of subembedding 0

(b) The InfoMax of subembedding 1

(c) The InfoMin between subembeddings

Figure 5: Losses during upstream training of the DC-BC dataset with InfoMin.
We want the InfoMax loss to decrease (maximizing MI between views) and the
InfoMin to increase (minimizing MI between subembeddings). Near 55k steps a
trade-off occurs, in which minimizing the InfoMax is prioritized. This results
in greater subembedding entanglement, as seen by the undesired drop in the
InfoMin.

We suspected that there is a direct trade-off between the InfoMin and InfoMax
loss terms, which is confirmed by the loss visualization provided in Figure 5.
The optimization is stagnant until nearly 55k steps, at which point the
InfoMax terms ‘win’ and the InfoMin values drop. Unfortunately, the higher
weighted InfoMin term with $\lambda=0.1$ does not resolve this trade-off and
only hurts the optimization of the InfoMax, reducing the representation
quality.

###  A.4 DC-DL Experiments

The main paper includes experiments on the DC-BC dataset, and we report
additional results on the DC-DL dataset here due to space.

####  A.4.1 Ideal Disentanglement

Similar to the accuracies reported in Table 1, we report the results on DC-DL
when training the contrastive pipeline using the fully supervised setup
described in Section A.2. Table 4 shows that $r_{0}$ performs well on the DC
task, specifically just as well as $r$; the same holds for $r_{1}$ and the DL
downstream task. The resulting performance differences between subembeddings
show maximal accuracy differences, with each subembedding performing near
random for the other task.

Table 4: Accuracy for digit class (DC) and digit location (DL) classification tasks using a fully supervised pipeline with the full representation $r$ or its subembeddings $r_{0}$, $r_{1}$ as classification input. Classification Input | DC | DL
---|---|---
$r$ | 98.2 | 99.9
$r_{0}$ | 98.2 | 13.4
$r_{1}$ | 11.7 | 99.9
$|C(r_{0})-C(r_{1})|$ | 86.5 | 86.5

####  A.4.2 InfoMin

Table 5: Accuracy for digit class (DC) and digit location (DL) classification tasks using the InfoMin term. Classification Input | DC | DL
---|---|---
$r$ | 11.7 | 12.6

Using the InfoMin regularization is ineffective with the DC-DL MNIST/STL-10
dataset variant. As seen in Table 5, performance on both downstream tasks is
near random. This is due to the InfoMin directly combatting the InfoMax terms
in the final objective, which prevents optimization. As a result, no
meaningful information is encoded in the representation.

####  A.4.3 Orthogonality

Table 6 includes experiment results using the orthogonality constraint with
and without the permutation matrix $P$ and using various $\lambda$ weights.
While there are subembedding performance gaps ($|C(z_{0})-C(z_{1})|$) on the
DC task without the permutation $P$ being used, there is little difference
between the subembeddings’ performance for the DL task. As the orthogonality
regularizer is not an element-wise constraint, digit location information may
be easily repeated within a subembedding and may not be penalized by this
regularizer. The $|C_{DL}(z_{0})-C_{DL}(z_{1})|$ difference does increase with
the higher $\lambda=0.1$ (0.1 vs. 2.8).

Surprisingly, the performance differences are smaller when the permutation
matrix is used. Additionally, the accuracy achieved when $z$ is used as
classification input is lower with $R_{P(\text{Ortho})}$, which may be due to
disentanglement being enforced on axes that are not related to the downstream
tasks.

Table 6: Accuracy for digit class (DC) and digit location (DL) classification tasks with orthogonality and Hessian regularization, varying the weight $\lambda$. For orthogonality regularization, ablations with the permutation matrix ($P$) are also included.  Classification Input | Orthogonality w/o P | Orthogonality w/P | Hessian
---|---|---|---
$\lambda=0.001$ | $\lambda=0.1$ | $\lambda=0.001$ | $\lambda=0.1$ | $\lambda=0.001$ | $\lambda=0.1$
DC | DL | DC | DL | DC | DL | DC | DL | DC | DL | DC | DL
$z$ | 82.6 | 99.6 | 97.8 | 99.6 | 96.6 | 99.8 | 68.0 | 99.8 | 86.6 | 99.1 | 91.2 | 99.1
$z_{0}$ | 44.3 | 99.4 | 46.5 | 83.9 | 45.5 | 99.6 | 41.3 | 99.7 | 46.7 | 41.5 | 39.9 | 48.1
$z_{1}$ | 33.6 | 99.5 | 38.4 | 86.7 | 44.5 | 99.7 | 34.5 | 98.6 | 41.7 | 67.9 | 34.8 | 64.7
$|C(z_{0})-C(z_{1})|$ | 10.7 | 0.1 | 8.1 | 2.8 | 1.0 | 0.1 | 6.8 | 1.1 | 5.0 | 26.4 | 5.1 | 16.6

####  A.4.4 Hessian

Similar to the DC-BC results in Table 3, the Hessian constraint can result in
larger performance differences between the two subembeddings. Notably, there
are much larger differences for the digit location task,
$|C_{DL}(z_{0})-C_{DL}(z_{1})|$. The Hessian regularizer is the first to break
the same performance of $z_{0}$ and $z_{1}$ on the digit location task (i.e.,
all other regularization ablations in Table 6 have about $99\%$ accuracy using
$z$ or its subembeddings as classification input).

####  A.4.5 Degenerate Solutions

We found the DC-DL dataset variant is more prone to extreme degenerate
solutions, as illustrated in Table 7. This may in part be due to a task
difficulty imbalance: digit location information may be significantly more
easily learned and encoded than digit class. Table 7 includes experiment
results from using $R_{\text{Ortho}}$ with $\lambda=0.1$, which only provides
a whole representation constraint instead of an element-wise constraint such
as the Hessian. The digit location information may be repeated multiple times
within the full representation; this can be seen in the Task Difficulty
columns of Table 7, in which DL accuracy is constant across $z$, $z_{0}$,
$z_{1}$.

Another example of a trivial solution is shown in the first two columns of
Table 7, in which performance on both downstream tasks is near random. The
orthogonality constraint is too strong and prevents optimization during
training, resulting in random representations. The DC-DL dataset may be more
prone to such optimization issues because of task difficulty imbalance, too.

Table 7: Degenerate solutions on digit class (DC) and digit location (DL) classification tasks using the orthogonality term without the permutation matrix $P$ and a weight of $\lambda=0.1$. Classification Input | Random | Task Difficulty
---|---|---
DC | DL | DC | DL
$z$ | 11.7 | 12.6 | 76.7 | 99.8
$z_{0}$ | 11.7 | 12.6 | 42.0 | 99.7
$z_{1}$ | 11.7 | 12.6 | 32.1 | 99.7

###  A.5 Reproducibility

Table 8: Reproducibility results on downstream digit (DC) and background (BC) classification tasks using the orthogonality regularizer with the permutation matrix $P$ and a weight of $\lambda=0.1$. Run 0 is the original result reported in Table 3, runs 1-5 are reruns that have different random initialization. Classification Input | Run 0 | Run 1 | Run 2 | Run 3 | Run 4 | Run 5
---|---|---|---|---|---|---
DC | BC | DC | BC | DC | BC | DC | BC | DC | BC | DC | BC
$z$ | 97.2 | 61.6 | 97.0 | 62.3 | 97.0 | 61.3 | 97.2 | 61.6 | 97.2 | 61.4 | 97.0 | 58.3
$z_{0}$ | 88.0 | 56.9 | 86.8 | 59.3 | 85.4 | 57.8 | 83.0 | 59.7 | 80.8 | 57.7 | 71.6 | 56.5
$z_{1}$ | 82.2 | 60.0 | 80.8 | 59.6 | 79.6 | 57.6 | 81.2 | 56.2 | 75.6 | 54.7 | 69.7 | 48.1
$|C(z_{0})-C(z_{1})|$ | 5.8 | 3.1 | 6.0 | 0.3 | 5.8 | 0.2 | 1.8 | 3.5 | 5.2 | 3.0 | 1.9 | 8.4
Table 9: Reproducibility results on downstream digit (DC) and background (BC) classification tasks using the Hessian regularizer and a weight of $\lambda=0.1$. Run 0 is the original result reported in Table 3, runs 1-5 are reruns that have different random initialization. Classification Input | Run 0 | Run 1 | Run 2 | Run 3 | Run 4 | Run 5
---|---|---|---|---|---|---
DC | BC | DC | BC | DC | BC | DC | BC | DC | BC | DC | BC
$z$ | 93.6 | 56.3 | 73.0 | 56.0 | 96.2 | 56.6 | 93.9 | 56.6 | 73.1 | 58.1 | 92.2 | 56.9
$z_{0}$ | 74.7 | 42.4 | 67.9 | 47.9 | 73.7 | 40.4 | 58.4 | 36.6 | 42.2 | 34.8 | 61.1 | 36.4
$z_{1}$ | 57.5 | 50.9 | 64.2 | 49.4 | 66.5 | 43.2 | 51.0 | 37.5 | 40.0 | 30.2 | 53.5 | 37.2
$|C(z_{0})-C(z_{1})|$ | 17.2 | 8.5 | 3.7 | 1.5 | 7.2 | 2.8 | 7.4 | 0.9 | 2.2 | 4.6 | 7.6 | 0.8

We include five additional DC-BC experiments for both the orthogonality and
Hessian constraints to see how performance varies over different random
initializations. Experiments are rerun under the best settings for each
regularizer: $R_{P(\text{Ortho})}$ and $R_{\text{Hess}}$, both with a weight
of $\lambda=0.1$ in the final training objective.

####  A.5.1 Orthogonality Constraint

The first run in Table 8, run 0, is the original experiment included in the
results of Table 3. The diagonal trend of one subembedding performing better
on DC and one subembedding performing better on BC only occurs once in the
additional five runs, with a barely nonzero difference $|C(z_{0})-C(z_{1})|$
for the background classification task (0.3).

In the four experiments with no diagonal performance trend, the subembeddings
$z_{0}$ and $z_{1}$ either perform nearly the same on background
classification (e.g., run 2), or one subembeddings performs better on both
tasks (e.g., run 3, 4, 5).

The mean difference in subembedding performance for the digit classification
task, e.g., $\frac{1}{R}|C_{\text{DC}}(z_{0})-C_{\text{DC}}(z_{1})|$ over $R$
runs, is 4.4, and the differences
$|C_{\text{DC}}(z_{0})-C_{\text{DC}}(z_{1})|$ have a variance of 3.4. For
background classification the mean difference is 3.1, with a higher variance
of 7.4. This demonstrates that the digit classification task is prioritized
during optimization as it has larger performance gaps with lower variance.

####  A.5.2 Hessian Regularization

The first run in Table 9, run 0, is the original experiment included in the
results of Table 3. Among the additional five experiments, four satisfied the
desired diagonal performance trend, with one subembedding performing best on
the DC task, and the other performing best on the BC task.

The mean difference in subembedding performance for the digit classification
task, e.g., $\frac{1}{R}|C_{\text{DC}}(z_{0})-C_{\text{DC}}(z_{1})|$ over $R$
runs, is 7.6, which is larger than the 4.4 mean difference obtained using the
orthogonality constraint. However, the variance of performance differences
($|C_{\text{DC}}(z_{0})-C_{\text{DC}}(z_{1})|$) is significantly larger using
the Hessian (22.8 variance of subembedding performance differences on the DC
task vs. 3.4 variance when $R_{P(\text{Ortho})}$ is used).

On the other hand, the mean subembedding performance difference increases and
the variance of these differences decreases on the background classification
task, showing different performance trends for each downstream task. The mean
BC difference over all experiments,
$\frac{1}{R}|C_{\text{BC}}(z_{0})-C_{\text{BC}}(z_{1})|$, is 3.2 and variance
of $|C_{\text{BC}}(z_{0})-C_{\text{BC}}(z_{1})|$ is 7.4. This is negligibly
better than the mean of 3.1 and variance of 7.4 obtained from the
orthogonality results.


# Measurement and completely decoherence

Weijing Li<EMAIL_ADDRESS>School of Science,Beijing University of
Posts and Telecommunications, People’s Republic of China

###### Abstract

By virtue of quantum coherence resource measure, we show that the dephasing
measurement on a coherence basis can transfer the coherence contained in
system into environment totally, which gives a quantification of decoherence.

††preprint: APS/123-QED

##  I Introduction

Quantum decoherence is the loss of quantum coherence. As long as there exists
a definite phase relation between different quantum states of a quantum
system, the system is said to be coherent. Decoherence was first introduced in
1970 by the German physicist H. Dieter Zeh [1] and has been a subject of
active research since the 1980s [2]. Decoherence has been developed into a
complete framework and can be viewed as the loss of information from a system
into the environment, since every system is loosely coupled with the energetic
state of its surroundings.

Quantum coherence underlies phenomena such as quantum interference and
multipartite entanglement that play a central role in the applications of
quantum physics and quantum information science. Despite the fundamental
importance of quantum coherence, the development of a rigorous theory of
quantum coherence as a physical resource has been initiated only recently.
Indeed, T. Baumgratz, M. Cramer, M. B. Plenio introduced a rigorous framework
for the quantification of coherence and identify intuitive and easily
computable measures of coherence in 2014 [3], and since then plethora of
coherence measures had been formulated. Each of these measures captures some
physical traits of quantum coherence with specific operational
interpretations, see Ref. [4]for a comprehensive review.

Decoherence is intimately related to quantum measurement process. Given a
quantum system, after a measurement, the essential quantum superposition
property disappears-decoherence occurs. It is desirable to exploit the
quantitative characterization of decoherence process via the coherence
measures within the quantum resource framework.

In this note, we show that after a measurement in the coherence basis, the
original coherence contained in principal system can be transposed into the
environmental completely, which echos the issue of decoherence.

##  II Main results

Given a quantum state $\rho$ who lives in a finite-dimensional Hilbert space
$\mathcal{H}$, which can be spanned by the coherence basis
$\mathcal{H}=span\\{|i\rangle\\}$. The quantum state $\rho$ can be written as

$\rho=\sum_{ij}\langle i|\rho|j\rangle|i\rangle\langle j|.$

Define the total dephasing operation $\Phi$ by, which completely destroys
coherence of states,

$\displaystyle\Phi(\rho)=\sum_{i}\langle i|\rho|i\rangle|i\rangle\langle i|.$

After performing a total dephasing operation $\Phi$, the post-measurement
state can be represented as

$\sigma=\Phi(\rho)=\sum_{i}p_{i}|i\rangle\langle i|,$

where $p_{i}=\langle i|\rho|i\rangle$.

Since the purification of a quantum state contains all of the information
encoded in quantum state, it is natural to consider the purification of
$\rho$,

$|\Psi\rangle=\sqrt{\rho}\otimes I|\Sigma\rangle,$

where $|\Sigma\rangle=\sum_{i}|ii\rangle$ is the maximally entangled state, up
to a constant.

For pure state $|\Psi\rangle$, after performing the total dephasing operation
on principal system, the post-measurement state reduces to

$\displaystyle\sum_{i}|i\rangle\langle i|\otimes
I|\Psi\rangle\langle\Psi|i\rangle\langle i|\otimes I.$

The probability of outcome $i$ can be written as

$\displaystyle q_{i}=$ $\displaystyle\langle\Psi|i\rangle\langle i|\otimes
I|\Psi\rangle$ $\displaystyle=$ $\displaystyle\sum_{j}|\langle
i|\sqrt{\rho}|j\rangle|^{2}$ $\displaystyle=$ $\displaystyle\sum_{j}\langle
i|\sqrt{\rho}|j\rangle\langle j|\sqrt{\rho}|i\rangle$ $\displaystyle=$
$\displaystyle\langle i|\rho|i\rangle=p_{i}.$

Since after measurement the state for principal system contains zero
coherence, it is desirable to consider the reduced environment state. After
tracing out the principal system, the reduced environment state is

$\displaystyle\tau=$
$\displaystyle\text{Tr}_{\mathcal{H}}\\{\sum_{i}|i\rangle\langle i|\otimes
I|\Psi\rangle\langle\Psi|i\rangle\langle i|\otimes I\\}$ $\displaystyle=$
$\displaystyle\sum_{ij}\langle j|\rho|i\rangle|i\rangle\langle j|.$

One can see that $\tau$ is just the entry-wise complex conjugate of $\rho$ in
coherence basis representation. It is directly to observe that for any
coherence measure involved the the modular of the entries of quantum state,
$\rho$ and $\tau$ have equal coherence. For example, the $l_{1}$ norm
coherence measure:

$C_{l_{1}}(\rho)=\sum_{i\neq j}|\rho_{ij}|,$

we have

$C_{l_{1}}(\rho)=C_{l_{1}}(\tau).$ (1)

This interesting fact can be interpreted as that the measurement process
transfers the coherence contained in original quantum state into the
environment completely, and $l_{1}$ norm coherence measure can be used to
quantify both coherence and decoherence.

Besides of $l_{1}$ norm coherence measure $C_{l_{1}}(\rho)$, other typical
coherence measures, relative entropy for example, are also of same property.

Recall that the quantum relative entropy is written as

$\displaystyle
S(\rho||\sigma)=\text{Tr}\left(\rho\log\rho\right)-\text{Tr}\left(\rho\log\sigma\right)$

and the relative entropy coherence measure is defined as

$\displaystyle C_{R}(\rho)=\min_{\sigma\in\mathcal{I}}S(\rho||\sigma),$

where $\mathcal{I}$ is the set of incoherent states, that is, all incoherent
sates $\sigma\in\mathcal{I}$ are of the form
$\sigma=\sum_{i}p_{i}|i\rangle\langle i|$ with probabilities $p_{i}$. Relative
entropy coherence measure $C_{R}(\rho)$ has the operational interpretation of
distillable coherence and admits a closed expression

$\displaystyle C_{R}(\rho)=S(\Phi(\rho))-S(\rho)$

in terms of the von Neumann entropy

$\displaystyle S(\rho)=-\text{Tr}\left(\rho\log\rho\right).$

One can observe that

$\displaystyle C_{R}(\rho)=C_{R}(\tau).$ (2)

In fact, for the spectral decomposition of
$\rho=\sum_{k}\lambda_{k}|\psi_{k}\rangle\langle\psi_{k}|$, it is easy to
verify that $\tau$ admits a spectral decomposition

$\displaystyle\tau=\sum_{k}\lambda_{k}|\psi_{k}^{*}\rangle\langle\psi_{k}*|$

where $|\psi_{k}^{*}\rangle$ is the entry-wise complex-conjugate of
$|\psi_{k}\rangle$. Due to the positivity of $\rho$, it must hold that

$\displaystyle S(\Phi(\rho))=S(\Phi(\tau))\quad\text{and}\quad
S(\rho)=S(\tau),$

Hence we obtain

$\displaystyle C_{R}(\rho)=C_{R}(\tau).$

Another interesting example is the coherence measure induced by Wigner-Yanase
skew information $C_{WY}(\rho)$ [5]. Note that the Wigner-Yanase skew
information is defined as [6]

$\displaystyle I(\rho,H)=\text{Tr}(\rho
H^{2})-\text{Tr}(\sqrt{\rho}H\sqrt{\rho}H)$

where $H=H^{\dagger}$ plays the role of conversed quantity. $C_{WY}(\rho)$ is
constructed by

$\displaystyle C_{WY}(\rho)=$ $\displaystyle\sum_{i}I(\rho,|i\rangle\langle
i|)$ $\displaystyle=$ $\displaystyle 1-\sum_{i}\langle
i|\sqrt{\rho}|i\rangle^{2}.$

By direct calculation it is easy to find that

$\displaystyle C_{WY}(\rho)=C_{WY}(\tau).$ (3)

Indeed, one only need to verify

$\displaystyle\langle i|\sqrt{\rho}|i\rangle$ $\displaystyle=$
$\displaystyle\sum_{k}\sqrt{\lambda_{k}}|\langle i|\psi_{k}\rangle|^{2}$
$\displaystyle=$ $\displaystyle\sum_{k}\sqrt{\lambda_{k}}|\langle
i|\psi_{k}^{*}\rangle|^{2}$ $\displaystyle=$ $\displaystyle\langle
i|\sqrt{\tau}|i\rangle.$

In the contrast, robustness of coherence introduced in [7] seems challenging
to verify equation relations similar to equations (1-3), though it admit some
different operational interpretations. Nevertheless, by focusing on the
specific dephasing operation, we can use coherence measures to quantify
quantum decoherence quantitatively.

##  III Discussions

Quantum decoherence has been a subject of active research over forty years and
various methods had been proposed to interpret its mechanisms quantitatively,
for example, einselection [8] and semigroup approach [9]. In this note we
quantify the decoherence process by virtue of the quantum coherence resource
measures quantitatively. Our results is in agreement with decoherence issue
and quantum coherence resource theory.

## References

  * Zeh [1970] H. D. Zeh, On the interpretation of measurement in quantum theory, Foundations of Physics 1, 69 (1970).
  * Schlosshauer [2005] M. Schlosshauer, Decoherence, the measurement problem, and interpretations of quantum mechanics, Reviews of Modern physics 76, 1267 (2005).
  * Baumgratz  _et al._ [2014] T. Baumgratz, M. Cramer, and M. B. Plenio, Quantifying coherence, Physical review letters 113, 140401 (2014).
  * Streltsov  _et al._ [2017] A. Streltsov, G. Adesso, and M. B. Plenio, Colloquium: Quantum coherence as a resource, Reviews of Modern Physics 89, 041003 (2017).
  * Yu [2017] C.-s. Yu, Quantum coherence via skew information and its polygamy, Physical Review A 95, 042337 (2017).
  * Wigner and Yanase [1963] E. P. Wigner and M. M. Yanase, Information contents of distributions, Proceedings of the National Academy of Sciences 49, 910 (1963).
  * Napoli  _et al._ [2016] C. Napoli, T. R. Bromley, M. Cianciaruso, M. Piani, N. Johnston, and G. Adesso, Robustness of coherence: an operational and observable measure of quantum coherence, Physical review letters 116, 150502 (2016).
  * Zurek [2003] W. H. Zurek, Decoherence, einselection, and the quantum origins of the classical, Reviews of modern physics 75, 715 (2003).
  * Bauer  _et al._ [2012] M. Bauer, D. Bernard, and T. Benoist, Iterated stochastic measurements, Journal of Physics A: Mathematical and Theoretical 45, 494020 (2012).


# The weakness of soft X-ray intensity: possible physical reason for weak line
quasars

Jiancheng Wu Department of Astronomy, School of Physics, Huazhong University
of Science and Technology, Luoyu Road 1037, Wuhan, China  Qingwen Wu∗
Department of Astronomy, School of Physics, Huazhong University of Science and
Technology, Luoyu Road 1037, Wuhan, China  Chichuan Jin National Astronomical
Observatories, Chinese Academy of Sciences, 20A Datun Road, Beijing 100101,
China  School of Astronomy and Space Sciences, University of Chinese Academy
of Sciences, 19A Yuquan Road, Beijing 100049, China  Jianfeng Wu Department of
Astronomy, Xiamen University, Xiamen, Fujian 361005, People’s Republic of
China  Weihua Lei Department of Astronomy, School of Physics, Huazhong
University of Science and Technology, Luoyu Road 1037, Wuhan, China  Xinwu Cao
Institute for Astronomy, School of Physics, Zhejiang University, 866
Yuhangtang Road, Hangzhou 310058, People’s Republic of China  Xiao Fan
Department of Astronomy, School of Physics, Huazhong University of Science and
Technology, Luoyu Road 1037, Wuhan, China  Xiangli Lei Department of
Astronomy, School of Physics, Huazhong University of Science and Technology,
Luoyu Road 1037, Wuhan, China  Mengye Wang Department of Astronomy, School of
Physics, Huazhong University of Science and Technology, Luoyu Road 1037,
Wuhan, China  Hanrui Xue Department of Astronomy, School of Physics, Huazhong
University of Science and Technology, Luoyu Road 1037, Wuhan, China  Bing Lyu
Kavli Institute for Astronomy and Astrophysics, Peking University, Beijing
100871, Peoples Republic of China

###### Abstract

Weak-line quasars (WLQs) are a notable group of active galactic nuclei (AGNs)
that show unusually weak UV lines even though their optical-UV continuum
shapes are similar to those of typical quasars. The physical mechanism for
WLQs is an unsolved puzzle in the AGN unified model. We explore the properties
of UV emission lines by performing extensive photoionization calculations
based on Cloudy simulation with different spectral energy distributions (SEDs)
of AGNs. The AGN continua are built from several observational empirical
correlations, where the black-body emission from the cold disk, the power-law
emission from the hot corona, and a soft X-ray excess component are
considered. We find that the equivalent width (EW) of C IV from our models is
systematically lower than observational values if the component of soft X-ray
excess is neglected. The EW will increase several times and is roughly
consistent with the observations after considering the soft X-ray excess
component as constrained from normal type I AGNs. We find that the UV lines
are weak for QSOs with quite large BH mass (e.g., $M_{\rm
BH}>10^{9}M_{\odot}$) and weak soft X-ray emission due to the deficit of
ionizing photons. As an example, we present the strength of C IV based on the
multi-band SEDs for three nearby weak-line AGNs, where the weaker soft X-ray
emission normally predicts the weaker lines.

Active galactic nuclei (16), Supermassive black holes (1663), Quasars (1319),
X-ray active galactic nuclei (2035), Accretion (14), Line intensities (2084)

††software: Cloudy (Chatzikos et al., 2023)

##  1 Introduction

Active Galactic Nuclei (AGNs) are the most luminous persistent sources of
electromagnetic radiation in the Universe, which is believed to be powered by
accretion onto supermassive black holes (SMBHs, e.g., Shakura & Sunyaev,
1973). One of most important features of AGNs is that their optical spectra
exhibit fruitful broad and narrow emission lines. The type I AGNs show both
broad lines (i.e., Full Width at Half Maximum (FWHM)$\rm{\gtrsim
1000km\,s^{-1}}$) and narrow lines and type II AGNs show only narrow lines,
which are unified based on orientation effect (Antonucci, 1993; Urry &
Padovani, 1995). The Sloan Digital Sky Survey (SDSS) project has discovered
millions of quasars in the past few decades. Diamond-Stanic et al. (2009)
studied the equivalent widths (EW) of UV emission lines for a sample quasars
and found that the average equivalent widths are $63.6\text{\AA}$ and
$41.9\text{\AA}$ for Ly$\alpha$+N V and C IV respectively. However, a small
fraction of quasars show extremely weak ultraviolet high-ionization lines,
where EW(C IV) $<10\text{\AA}$ and ${\rm EW(Ly\alpha}$+N V)$<15.4\text{\AA}$
even their properties of optical continuum and H$\alpha$ lines are similar to
normal quasars (e.g., Diamond-Stanic et al., 2009; Liu et al., 2013). These
sources are 3$\sigma$ outliers at the low-end of log-normal EW distribution
(e.g., Diamond-Stanic et al., 2009), which are called as weak line quasars
(WLQs, e.g., Fan et al., 1999; Anderson et al., 2001; Collinge et al., 2005;
Shemmer et al., 2010; Wu et al., 2011, 2012; Luo et al., 2015). Most of WLQs
lie at high-redshifts (e.g., Fan et al., 1999; Diamond-Stanic et al., 2009).
However, more and more intermediate- and low-redshift WLQ are also found
(e.g., Plotkin et al., 2010b, a; Paul et al., 2022), even though the fraction
of WLQs seems to increase with increasing redshift (Diamond-Stanic et al.,
2009; Wu et al., 2012).

The physical mechanism for the weak or absent emission lines is still quite
unclear so far. One possible physical reason is the underdeveloped broad line
region in these AGNs with a lower covering factor of BLR clouds, where WLQs
may belong to an early stage in AGN evolution and the BLR was not yet fully
formed (e.g., Shemmer et al., 2010; Hryniewicz et al., 2010; Zhang & Liu,
2016; Kumar et al., 2023). If this is the case, most emission lines, including
the low-ionization lines, in WLQs should be weaker than those of normal
quasars. Some WLQs indeed show weaker H$\beta$ lines (Shemmer et al., 2010;
Plotkin et al., 2015). However, many WLQs show quite strong H$\alpha$ and
H$\beta$ lines (McDowell et al., 1995; Liu et al., 2013), which suggests that
the underdeveloped BLR should be not the dominant or the only reason for all
WLQs. The second possible reason is the inefficient photo-ionization of
photons. Wu et al. (2011) and Luo et al. (2015) proposed that the “shielding”
gas exists between the X-ray corona and BLR, which may be related to the inner
geometrically thick super-Eddington slim disk (e.g., Abramowicz et al., 1988;
Czerny, 2019; Feng et al., 2019) and/or the wide-angle optically thick winds
(e.g., Murray et al., 1995; Giustini & Proga, 2019; Jiang et al., 2019). The
ionization photons from the inner disk/corona will be absorbed by the
“shielding” gas (Wu et al., 2011; Luo et al., 2015; Ni et al., 2018). The slim
disk will be geometrical thick for the case of super Eddington accretion
(e.g., $H/R\sim 0.5$ for $\dot{M}\sim 5\dot{M}_{\rm Edd}$). The more careful
estimations for the extinction from the possible “shielding” gas, the mass of
SMBHs and bolometric luminosities will help to confirm whether these WLQs stay
in a slim state or not. The absence of UV ionization photons can also be
caused by the colder accretion disk surrounding very massive non-spinning
SMBHs (Laor & Davis, 2011). The weaker hard X-ray emission may be also one of
the reasons for a deficit of ionization photons. About half of the WLQs have
evident lower X-ray luminosities compared to the expectation from the
$L_{\nu}(2500{\text\AA})-\alpha_{\rm ox}$ relation, where $\alpha_{\rm
ox}=-0.3838\,{\rm log}(F_{\rm 2500\text{\AA}}/F_{2{\rm keV}})$ (Luo et al.,
2015; Ni et al., 2018, 2022). The weaker X-ray can either be intrinsic or
caused by the shielding gas (e.g., Luo et al., 2015), where the hard X-ray
observations (e.g., NuSTAR) can discriminate them. It should be noted that the
WLQs are not the same with beamed BL Lacs, where they have intrinsically weak
emission lines or their emission lines are diluted by the emission from the
relativistic beaming jets (e.g., Londish et al., 2004; Shemmer et al., 2006;
Lane et al., 2011; Wu et al., 2012; Kumar et al., 2016). Kumar et al. (2016)
explored the intranight optical variability and polarization property for a
couple of radio quiet WLQs, which are different from the typical BL Lacs (see
also, Gopal-Krishna et al., 2013; Chand et al., 2014; Kumar & Gopal-Krishna,
2015). Therefore, the WLQs should be not caused by the diluted emission from
the relativistic beaming jets.

The SEDs of UV bump and power-law component have been well built for many
AGNs, which dominantly come from cold disk and hot corona respectively. It
should be noted that the soft X-ray excess below 2 keV concerning the
extrapolation of the hard X-ray spectrum is also a very common feature among
type I AGNs even though its radiative mechanism is unclear. It should be noted
that this component is always neglected in exploring the emission lines in the
photon ionization model even though we know that the high-ionization UV lines
are very sensitive to the spectrum from extreme UV to soft X-ray bands. We aim
to explore the properties of UV emission lines by performing extensive
photoionization calculations based on SEDs and covering factors as constrained
from several observational empirical correlations, where we also consider the
soft X-ray excess as that constrained from X-ray observations for type I AGNs.
This paper is structured as follows: Sec. 2 presents the model of the incident
AGN SEDs and the photon ionization model for broad emission lines. Sec. 3
describes our simulation results. Conclusion and discussion are presented in
Sec. 4.

Figure 1: The typical SEDs with different parameters are adopted in Cloudy
simulations. The blue and red lines represent the SEDs for $\dot{m}=1$ and
$\dot{m}=0.05$ respectively with $10^{7.5}M_{\odot}$ (left panel) and
$10^{9.5}M_{\odot}$(right panel), respectively. In both panels, the dashed
line represents the SEDs without soft X-ray excess (i.e., $q=0$). The orange
and green dot-dashed represent the ionization energy for H and C IV,
respectively.

##  2 Photoionization Model

The photoionization code is widely adopted to investigate the optical emission
lines in AGNs (Ferland et al., 1998). In this work, we generate various SEDs
as the incident continuum for Cloudy simulation, where the incident SEDs and
photoionization model will be described in more detail as follows.

###  2.1 Incident AGN continuum

The optical-UV emission of bright AGNs is believed mainly from the cold
optically thick accretion disk, where the thermal emission is the origin for
the so-called “big blue bump”. We calculate the optical-UV spectrum based on
slim disk models, which can cover a wide range of accretion rates from sub-
Eddington to super-Eddington accretion(e.g., Abramowicz et al., 1988). We
calculate the hard X-ray spectrum based on two observational correlations,
where the 2 keV emission is calculated from $\alpha_{\rm ox}=-0.14\times{\rm
log}(L(2500{\text\AA}))+2.64$ (Steffen et al., 2006) and the spectral index is
derived from $\Gamma_{2-10{\rm keV}}=0.32\times\lambda_{\rm Edd}+2.27$ from
Brightman et al. (2013), where $\lambda_{\rm Edd}$ is the bolometric Eddington
ratio. The X-ray spectra of most AGNs also exhibit a “soft excess” at energies
below $\sim$2 keV, which sits above what would be expected from the hard X-ray
power-law. To mimic this soft X-ray component, we simply adopt a cool,
optically thick Comptonization component (COMPTT model in XSPEC; Titarchuk,
1994), where the input soft photon temperature tied to the blackbody is 0.01
keV, the plasma temperature is 0.2 keV and the plasma optical depth is 15
(e.g., Done et al., 2012; Jin et al., 2022). The observations suggest that the
intensity of soft X-ray excess is normally correlated with the bolometric
Eddington ratios (Boissay et al., 2016). We set the intensity ratio of soft
excess to power-law component with a parameter of $q=F^{\rm SE}/F^{\rm PL}$ in
0.5–2 keV bands, where we also use the empirical correlation of
$q=0.28\times{\rm log}\lambda_{\rm Edd}+0.72$ as constrained from the
observations on type I AGNs (Boissay et al., 2016).

In building the incident AGN SEDs, we adopt uniform random distributions of BH
mass from $10^{7}-10^{10}M_{\odot}$ and dimensionless Eddington-scaled
accretion rate $\dot{m}=0.02-2$ in log space. The broader parameter space will
help to generate various ionization SEDs for incident continuum in Cloudy
simulation, which will help us to understand the dependence of the intensities
of emission lines on the model parameters. For parameters $\alpha_{\rm ox}$
and $q$, we adopt above observed empirical correlations with standard
scatters. For statistical study, we generate 200 incident AGN SEDs for Cloudy
simulations, where the distribution of the parameters is given in Fig. 2. In
Fig. 1, we present several SEDs for the accretion rate of $\dot{m}=0.05$ and 1
with BH mass of $10^{7.5}M_{\odot}$ and $10^{9.5}M_{\odot}$ respectively,
where the parameters of $q$, $\alpha_{\rm ox}$ are derived from above
observational empirical correlations.

###  2.2 BLR model and broad-line simulation

The photoionization models are calculated using version 23.00 of Cloudy, which
is described in Chatzikos et al. (2023). The Locally Optimally Emitting Cloud
(LOC) was adopted in calculating the line intensities, where for any given
line significant emission occurs only in a narrow range of photoionization
parameters (the optimally emitting clouds, Baldwin et al., 1995; Ferguson et
al., 1997; Korista & Goad, 2000; Ferland, 2003). The total line luminosity
emitted by the BLR is

$L_{\rm line}\propto\int{\int{r^{2}F(r,n)f(r)g(n)dndr}},$ (1)

(see Bottorff et al., 2002, for more details). $F(r,n)$ is the line flux from
a single cloud surface with hydrogen density $n$ and distance from the
continuum source $r$. The radial differential covering factor is $f(r)\propto
r^{\Gamma}$, and the fraction of the clouds with density $n$ is $g(n)\propto
n^{\beta}$ (density covering factor). In this work, we adopt the typical
values of $\Gamma=-1$ and $\beta=-1$ as suggested in Baldwin et al. (1995) and
Korista & Goad (2000). It should be noted that the line intensities are also
affected by the BLR geometry or covering factor, where the cloud covering
factor normally decreases as the Eddington ratio increases as constrained from
observations (Ferland et al., 2020; Ezhikode et al., 2017). In this work, the
covering factor is adopted to be ${\rm CF}=-0.53\,{\rm log}\dot{m}+0.17$,
which is constrained from the H$\beta$ lines in Ferland et al. (2020). In our
simulations, we also introduce a Gaussian random factor for the CF with upper
and lower limits of CF = 1 and 0.001, respectively (see Fig. 2).

For BLR size, we adopt $r_{\rm in}=0.1r_{\rm BLR}$ and $r_{\rm out}=10r_{\rm
BLR}$, where $r_{\rm BLR}$ is calculated from the reverberation mapping
methods ${\rm log}(r_{\rm BLR}/{\rm lt-days})=1.53+0.53\times{\rm
log}(L_{5100}/10^{44}{\rm erg\,s^{-1}})$ (Bentz et al., 2013). The number
density of BLR cloud is roughly in the range of $10^{8}{\rm cm^{-3}}$ and
$10^{12}{\rm cm^{-3}}$(e.g., Guo et al., 2020; Wu et al., 2023), where the
density can not be too low ($\gtrsim 10^{7}{\rm cm^{-3}}$) due to the absence
of forbidden broad lines (e.g. [O III] $\lambda 4363$) or too high due to BLR
will emit more continuum thermal emission than emission lines(e.g., Korista &
Goad, 2000). In all Cloudy simulations, the grid steps in hydrogen density
$(n)$ and distance to the continuum source $(r)$ are set to 0.2 dex. We take
the abundance to be solar abundance and the cloud column density to be $N_{\rm
H}=10^{23}{\rm cm^{-2}}$ which is consistent with the general assumption in
BLR photoionization simulations (Dumont et al., 1998).

Figure 2: The parameter distributions are adopted in our simulations, where
$\Delta\alpha_{\rm ox}$, $\Delta q$ and $\Delta{\rm CF}$ are the deviations
from the adopted empirical correlations.

##  3 Result

In Fig. 3, we present the relations of EW-$\alpha_{\rm ox}$ (left panel) and
EW-$q$ (right panel) respectively for several typical emission lines
(H$\alpha$, H$\beta$, Ly$\alpha$, Mg II and C IV). From the left panel, it can
be found that the EW of all lines increases with increases of $\alpha_{\rm
ox}$, where C IV lines are much more sensitive to the emission from soft to
hard X-ray bands. For the case of varying $q$ from 0 to 1.2, the EW of
H$\alpha$, H$\beta$, Ly$\alpha$ and Mg II increase less than 2.5 times, while
EW of C IV increases more than 8 times (see right panel). The high ionization
lines are more sensitive to these two parameters compared to the low
ionization lines. Both low values of $\alpha_{\rm ox}$ and $q$, will lead to
the weaker C IV lines (e.g., EW $<$ 10Å). EW of Mg II, Ly$\alpha$ and H$\beta$
will also be less than $\sim 20-30\text{\AA}$ for the case of steep optical to
X-ray spectrum (e.g., $\alpha_{\rm ox}<-1.6$). Therefore, both high-ionization
and low-ionization lines will become weaker with the decrease of soft X-ray
intensity ($\alpha_{\rm ox}$ and $q$). It should be noted that the BH mass
${\rm log}M_{\rm BH}/M_{\odot}=9$ is adopted as an example, which is a typical
value for WLQ (see Fig. 3). For lower BH mass of ${\rm log}M_{\rm
BH}/M_{\odot}=7-8$, this trend is roughly unchanged, however, all EWs are
several tens larger (or no WLQs).

Figure 3: The relations of EW–$\alpha_{\rm ox}$ (left panel) and EW–$q$(right
panel) for several emission lines. The typical parameters ${\rm log}M_{\rm
BH}/M_{\odot}=9$, $\dot{m}=0.1$, CF = 0.3, and $q=0.5$ (left panel) and
$\alpha_{\rm ox}=-1.6$ (right panel) are adopted.

In the left panel of Fig. 4, we present the results of 200 simulation points
in $L(2500\text{\AA})$ \- EW(C IV) relation without considering the soft X-ray
excess (i.e., $q=0$). EW(C IV) follows an anti-correlation with
$L(2500\text{\AA})$, which is consistent with the famous Baldwin effect.
However, the EW(C IV) is systematically lower than those of observational
results for normal quasars (grey shades and red lines, Timlin et al., 2020).
After including the soft X-ray component, the simulation results of
$L(2500\text{\AA})$ \- EW(C IV) correlation roughly consistent with that of
the observational result (right panel in Fig. 4). The slope of the simulation
points in the right panel is -0.38 which is roughly consistent but slightly
steeper than -0.23 from the observed $L(2500\text{\AA})$ \- EW(C IV) relation.
The sources with larger BH masses normally have smaller EW(C IV) and,
therefore, the weak lines are more likely to occur in AGNs with heavier black
holes (e.g., $10^{9}-10^{10}M_{\odot}$).

In Fig. 5, the circles represent the simulation results in $L(2500\text{\AA})$
\- $\alpha_{\rm ox}$ diagram, where the solid circle enclosed by the blue
circle represents that the equivalent width of C IV is less than
$10\text{\AA}$ (i.e. WLQ). It can be found that the WLQs mostly stay in the
lower right corner. The red diamond and downward arrow represent the
observation WLQs sample and WLQs with X-ray upper limits from Ni et al. (2018,
2022), where some WLQs have much lower $\alpha_{\rm ox}$ compared to our
simulation parameter space. Therefore, we further simulate the line properties
for $\alpha_{\rm ox}$ smaller than typical values by 0.5. We find that most of
the simulated data belong to WLQs for $M_{\rm BH}>10^{8.5}M_{\odot}$, and
there is also one WLQ with $M_{\rm BH}<10^{8}M_{\odot}$. This result suggests
that AGNs with $M_{\rm BH}\sim 10^{8}M_{\odot}$ can also show weak high-
ionization lines when their soft/hard X-ray emission is extremely weak.

Figure 4: The relation between $L(2500\text{\AA})$ and EW in our simulations
for C IV (upper panels) and Ly$\alpha$ (lower panels). The left and right two
panels represent the simulations without and with including the X-ray soft
excess respectively. The dashed lines and grey region represent the
observational results from Timlin et al. (2020) and Green et al. (2001) for
EW(C IV) and EW(Ly$\alpha$), respectively. The colorbar is BH mass.  Figure 5:
The weak-line sources in $\alpha_{\rm ox}-L(2500\text{\AA})$ plane. In the
left panel, the solid circles show our simulation result and the solid circles
surrounded by hollow circles represent the weak-line AGNs in our simulations.
For comparison, the observational data of some weak-line AGNs are also
presented with the red hollow diamonds, which are selected from Ni et al.
(2018, 2022). In right panel, we also present 60 simulations with extremely
weak X-ray emissions (solid squares), where $\alpha_{\rm ox}$ is smaller by
0.5 compared to that from relation $\alpha_{\rm ox}-L(2500\text{\AA})$, where
solid squares surrounded by hollow squares also represent the weak-line AGNs
in simulations.

To further test our model prediction, we also compare our model results with
three WLQs as an example, where the multi-waveband observations and, in
particular, the soft excess emission are detected. PHL 1811 and PHL 1092 are
both well-known classic WLQs. RX J0134.2-4258 is currently the only known
example of a Seyfert with similar weak UV lines. These three sources have
excellent multi-band observational data available to reconstruct their SEDs,
providing valuable inputs for our Cloudy simulations. We fit the spectral
energy distribution (SED) for three weak-line AGNs and then explore the
properties of their broad emission lines, where the multiband SEDs are adopted
from Jin et al. (2023). The continua are fitted with the above slim disk
model, a soft X-ray component, and a power-law hard X-ray component. The
fitting results and model parameters are presented in Fig. 6 and Table 1
respectively, where the SEDs of high state and low state are fitted
respectively for PHL 1092 and RX J0134.2-4258 (hereafter, RX J0134). For PHL
1811, we also test the different strengths of soft X-ray excess since that its
observed X-ray is very weak. In modeling the emission lines, we firstly
constrain the covering factor with the strength of H$\beta$ lines (Ferland et
al., 2020), and then give the predicted value of EW(C IV), where the results
are presented in Table 1. For PHL 1092 and RX J0134, the predicted values of
EW(C IV) are 36.5 and 24.5 respectively, which do not belong to weak-line
AGNs. However, the values of EW(C IV) are 12.1 and 7.4 in their low states,
which are roughly consistent with the observed values. For PHL 1811, the
predicted value of EW(C IV) is 4.8, which may stay in a low state since its
X-ray emission is very weak. A slightly higher value of $q$ or stronger soft
X-ray emission will lead to a larger value of EW(C IV).

Figure 6: The SED modeling for three weak-line AGNs, where the optical-UV
emission is fitted with slim disk model while the X-ray emission is fitted by
a soft X-ray component and a pow-law component. The high state and low state
in PHL 1092 and RX J0134.2-4258 are fitted respectively. To test the model, we
also consider two different strengths of soft X-ray components in PHL 1811.
Table 1:  Fitting parameters for a single object.

Object Name/SED | Observed REW(H$\beta$)/Å | Observed REW(C IV)/Å | CF | $M_{\rm BH}/M_{\odot}$ | $\dot{m}$ | $q$ | $\alpha_{\rm ox}$ | EW(C IV)/Å
---|---|---|---|---|---|---|---|---
PHL 1092/A | $26.1\pm 5.4$ | $12.1\pm 2.5$ | 0.1 | $3\times 10^{8}$ | 0.6 | 10.7 | -1.7 | 36.5
PHL 1092/B | … | … | 0.2 | … | … | 4.2 | -2.4 | 12.1
RX J0134/C | $23.8\pm 1.2$ | $7.6\pm 1.5$ | 0.1 | $4\times 10^{8}$ | 0.2 | $0.8^{a}$ | -1.5 | 24.5
RX J0134/D | … | … | 0.2 | … | … | 0.4 | -1.7 | 7.4
PHL 1811/E | $36.1\pm 3.9$ | $7.9\pm 2.5$ | 0.3 | $7\times 10^{8}$ | 0.7 | 20.0 | -2.3 | 8.1
PHL 1811/F | … | … | 0.3 | … | … | 0.8 | … | 4.8
PHL 1811/G | … | … | 0.3 | … | … | 0.2 | … | 4.7

Note. — The rest frame equivalent width (REW) of observed lines. The last
column is the EW(C IV) predicted by our model. All of the observational data
come from Jin et al. (2023). The BH mass $M_{\rm BH}$ and $\dot{m}$ are
determined by SED fitting. aThe value $q$ of RX J0134.2-4258 in high state (C)
is calculated based on 0.2-1 keV due to the the absence of soft excess above 1
keV.

##  4 Conclusion and Discussion

To understand the possible physical mechanism for the weak lines in a fraction
of quasars, we investigate the properties of emission lines based on Cloudy
code with AGN SEDs built from several observational empirical correlations,
where we particularly consider the possible soft X-ray excess component. We
find that the EW of high excitation lines (e.g., C IV, Ly$\alpha$, etc.) is
normally smaller than the observational results if just considering the
optical-UV emission from cold disk and power-law X-ray emission from corona.
However, the theoretical predictions on EW(C IV) and EW(Ly$\alpha$) are
consistent with the observations very well after considering the soft X-ray
excess component as constrained from observations, which suggests that the
soft X-ray emission is necessary in explaining the strength of high excitation
lines. Several other parameters will also lead to weaker high excitation
lines, which include steeper $\alpha_{\rm ox}$ (weaker hard/soft X-ray
emission), unusual supermassive BHs (e.g., $M_{\rm BH}\gtrsim
10^{9}M_{\odot}$) and/or low covering factor of BLR.

It has been found that many AGNs show strong variability in the soft X-ray
bands (Miniutti et al., 2009; Jin et al., 2022, 2023), even though their
optical-UV emission is roughly unchanged (e.g., PHL 1092 and RX J0134 in this
work). The variation in UV and soft X-ray wavebands will affect the
ultraviolet high-ionization lines based on the photonionization model.
Firstly, we explore the properties of the emission lines with different X-ray
emissions that may stay in low or high states. We find that the C IV emission
lines are strongly correlated with the strength of soft X-ray emission, where
the weaker soft X-ray emission normally predicts much lower strength of C IV
lines (see Table 1). Both PHL 1811 and the low X-ray state of PHL 1092 and RX
J0134 have a very small value of $\alpha_{\rm ox}$ (see Fig. 5), which
suggests that their soft X-ray emission is very weak. Half of weak-line AGNs
shown lower X-ray emission support this scenario. It should be noted that the
observations of the C IV lines and X-ray emission are not simultaneous in most
WLQ sources. Future quasi-simultaneous observations on the optical spectrum
and X-ray emission are wished to further test this issue, where the time delay
between X-ray and emission line should be considered.

The power-law X-ray emission is widely believed to be produced via
Comptonization of optical/UV seed photons from a standard accretion disk in an
optically thin hot corona, where the intensity of power-law emission and
power-law slope are determined by parameters like BH mass and accretion rate
(Cao, 2009; Netzer, 2013). It is found that the soft X-ray emission below 2
keV is much higher than that extrapolated from the hard X-ray power-law
emission, which is very common in type I AGN (Pravdo et al., 1981; Singh et
al., 1985; Walter & Fink, 1993; Piconcelli et al., 2005; Gliozzi & Williams,
2020). The nature of the soft X-ray excess is poorly understood, where the
most promising explanations include the warm corona scenario (e.g., Done et
al., 2012) and relativistically smeared reflection scenario (see e.g.,
Ballantyne et al., 2002; Gierliński & Done, 2004; Crummy et al., 2006). In
this work, we adopt a phenomenological model to mimic the soft X-ray emission
in building the AGN SEDs. The physical origin of soft X-ray excess may differ
from our model, but this will not affect our main conclusion. The soft excess
may be very common in AGNs as suggested in Boissay et al. (2016). The strength
of the soft excess is positively correlated with the strength of the Eddington
ratio, which is significantly larger in the narrow-line Seyfert I sources
compared to the broad-line Seyfert I sources (e.g., Boissay et al., 2016;
Gliozzi & Williams, 2020). The shape of the AGN SEDs plays a central role in
determining both the ionization states and electron excitation in BLR clouds.
In particular, the strength of UV lines strongly depends on the photons from
UV to X-ray wavebands. Leighly et al. (2007a) studied the emission lines in a
WLQ of PHL 1811 by calculating the various SEDs and proposed that the weak
collisionally excited high ionization line (e.g., C IV) can be attributed to
the absence of EUV photons. In Fig. 4, it can be found that EW of UV lines of
both Ly$\alpha$ and C IV in our simulations are statistically lower than the
observational results if just considering the optical/UV component from the
cold accretion disk and power-law component from the corona. However, the
simulation results are roughly consistent with the observations very well
after we include the component of the soft X-ray excess. It should be noted
that the strength of the soft X-ray excess in WLQs is considered to be
correlated with Eddington ratios as that constrained from local type I AGNs.
More X-ray observations on the local WLQs and other observational constraints
are expected to further test this issue. The adopted empirical correlations in
the disk-corona model are found from normal AGNs, and it is still unclear
whether WLQs have peculiar SED or not. For optical-UV emission, WLQs can well
be described by standard disk model (Marculewicz & Nikolajuk, 2020) and the
relation between the hard X-ray index $\Gamma$ and accretion rate $\dot{m}$
still follow the relationship from normal AGNs (Shemmer et al., 2010).

Besides the soft X-ray excess component, several other factors may also lead
to the weak lines in AGNs. The first one is the BH mass. The maximum effective
temperature of the accretion disk is correlated with both BH mass and
accretion rate. The exceptionally massive BHs will lead to lower disk
temperature and a softer ionizing spectrum. Laor & Davis (2011) proposed that
the high-excitation line should be weak or even absent for non-spinning very
massive BH AGNs, where the low-ionization lines may still be present. In Fig.
4, it can be found that most sources with EW(C IV) and EW(Ly$\alpha$) less
than 10 normally have BH mass in range of $10^{9}-10^{10}M_{\odot}$ (right
panels), which is consistent with results of Laor & Davis (2011). We note that
the BH mass estimations in WLQs are still controversial, since their broad
lines may be peculiar. Based on the continuum fitting method, Marculewicz &
Nikolajuk (2020) found that most of the WLQs have BH mass larger than
$10^{9}M_{\odot}$ and pointed out that the BH mass estimated from
FWHM(H$\beta$) is underestimated. However, Ha et al. (2023) proposed that the
BH mass estimated from the traditional method of broad-emission-line region
size–luminosity relation may be overestimated up to an order of magnitude,
where the corrections based on the strength of the optical Fe II emission and
possible blue-shift of C IV lines are applied. It should be noted that the
larger value of $R_{\rm Fe\,{\footnotesize II}}=$Fe II(4434-4684Å)/H$\beta$ in
WLQs may not be an indicator of high Eddington ratios as found in narrow-line
Seyferts, but caused by the weaker H$\beta$ emission since both low-ionization
and high-ionization lines can become weaker in WLQs (see Fig. 3 and Shemmer et
al. (2010)). Therefore, further investigations on the BH mass of WLQs will
help understand their intrinsic physics.

The second parameter is $\alpha_{\rm ox}$, which controls the strength of
X-ray emission. It is well known that $\alpha_{\rm ox}$ is anti-correlated
with the UV luminosity (e.g., $L(2500\text{\AA})$ \- $\alpha_{\rm ox}$), which
may be triggered by the stronger corona cooling for sources with higher disk
luminosity. If this is the case, it suggests that high BH mass and high
Eddington ratios will lead to strong UV luminosity and weak X-ray emission.
From Fig. 3, it can be found that the steeper $\alpha_{\rm ox}$ will lead to
smaller EW, where we have assumed the soft X-ray emission is a certain
fraction of hard X-ray emission (i.e., a fixed $q$ parameter). It was found
that half of the WLQs show weaker X-ray emission, where they deviate from the
$L(2500\text{\AA})$ \- $\alpha_{\rm ox}$ relation (e.g., Luo et al., 2015; Ni
et al., 2018, 2022; Paul et al., 2022). The special weak X-ray emission in
these WLQs is not so clear, which can be intrinsic or obscured by the
shielding gas (e.g., torus-like slim disk or optically thick winds). We
further do a couple of simulations with weaker X-ray emission, where
$\alpha_{\rm ox}$ is reduced by 0.5. We find that roughly all simulations with
this extremely weak X-ray emission are weak line AGNs for $M_{\rm
BH}>10^{9}M_{\odot}$, where one source with $M_{\rm BH}<10^{8}M_{\odot}$ also
belong to weak line AGNs. It should be noted that the strength of soft X-ray
emission is controlled by the parameters of both $\alpha_{\rm ox}$ and $q$.
The soft X-ray emission may still be weak if parameter $q$, which is
determined by the correlation with large intrinsic scatter, is small at given
$\alpha_{\rm ox}$, which may be the possible physical reason for some WLQs
still follow $L(2500\text{\AA})$ \- $\alpha_{\rm ox}$ relation.

The third parameter is the covering factor of BLR, which will affect the line
intensities. Nikołajuk & Walter (2012) proposed that the covering factor of
WLQs may be around one order of magnitude lower than that of normal AGNs based
on the ratios of high-ionization line and low-ionization line regions. Kumar
et al. (2023) found that the BLR in WLQs is underdeveloped, where the covering
factor of WLQs is about two times lower than that of normal AGNs based on the
assumption that the covering factors of the BLR and the dusty torus have to be
the same (e.g., Gaskell et al., 2007). The fraction of infrared emission is
used to infer the torus covering factors in WLQs, which is reasonable if the
emission from the other band is the same. But the UV to soft X-ray emission of
WLQs is lower than that of normal quasars(see Figure 2 in Kumar et al., 2023),
which roughly supports our conclusion for weak UV-soft X-ray emissions in
WLQs. Therefore, the lower infrared emission in WLQs may be not fully caused
by the low covering factor of the torus, where their intrinsic bolometric
luminosity may be low. Furthermore, the difference of two times on the
covering factor is still not enough to explain the weak lines in most WLQs. In
this work, we adopt the cloud covering factor decrease as the Eddington ratio
increases, where the covering factor is around 80% at low-luminosity AGNs and
decreases to several percent at Eddington ratios as that constrained from the
H$\beta$ and He II lines. Therefore, the fraction of WLQs is quite high in our
simulations, which is mainly caused by the low covering factor for
$\dot{M}>\dot{M}_{\rm Edd}$. Better constraints on the covering factor from
observations for wide parameter space of BH mass and Eddington ratio are
wished to further explore this issue.

We explore the UV line properties in AGNs based on Cloudy simulations after
considering AGN SEDs with wide parameter space of BH mass, accretion rate, and
soft X-ray excess, where the UV to soft X-ray spectrum is important for these
UV lines. Most of the works assumed that one or two parameters may dominate
the physical reasons for the weak lines. Based on our simulation, a large
fraction of quasars with BH mass $M_{\rm BH}>10^{9}M_{\odot}$ might be WLQs if
do not consider the possible contribution from the soft X-ray excess. Based on
observations, the quasars with very massive BHs (e.g., $M_{\rm
BH}>10^{9}M_{\odot}$) normally have weaker extreme UV emission and weaker soft
X-ray emission (or steeper $\alpha_{\rm ox}$), which normally cannot excite
strong UV lines. Therefore, we propose that the intensity of soft X-ray excess
plays an important role in the intensity of UV lines. The AGNs with very
heavier BHs, or extreme X-ray emission in both soft and hard X-ray wavebands,
or very low covering factor of BLR can also lead to weak UV lines. The broad-
band observations on AGN SED and emission lines (both high ionization lines
and low ionization lines) can help to distinguish the above properties. We
adopt several observed correlations from normal quasars in exploring the line
properties, where we assume that most properties of WLQs may be similar to
those of normal quasars. We have tested several possible parameters (even not
all), where more massive BH mass, lower accretion rate, weaker soft X-ray
emission (steeper $\alpha_{\rm ox}$ or weaker hard X-ray) will possibly lead
to weaker high-ionization lines. It is possible that WLQs may stay in a
special stage of AGNs, where the empirical correlations as constrained from
the normal quasars cannot applied to WLQs. Most of the WLQs are found in high
redshift (z$>$2) objects, which may be an artifact of the SDSS optical
spectral survey. Paul et al. (2022) explored the WLQs in intermediate redshift
(0.9 $<$ z $<$ 1.5) based on the ultraviolet spectroscopy of HST, where the
weak high ionization lines are more or less similar to those of high-redshift
WLQs (e.g. see also Leighly et al., 2007b; Wu et al., 2012; Plotkin et al.,
2010a). UV spectroscopy surveys can find more nearby WLQs, which will help to
explore the possible evolution of WLQs. Though hundreds of WLQs have been
found in SDSS, the WLQs sample may be still incomplete due to observation
bias. In our model, we just investigate the possible affection of AGN SEDs on
the line intensities, where we cannot rule out the possibility that WLQs may
stay in a special stage of AGNs (e.g., BLR is not well-formed).

The work is supported by the National Natural Science Foundation of China
(grants 12233007 and U1931203) and the science research grants from the China
Manned Space Project (No. CMS-CSST-2021-A06). The authors acknowledge Beijng
PARATERA Tech CO., Ltd. for providing HPC resources that have contributed to
the results reported within this paper.

## References

  * Abramowicz et al. (1988) Abramowicz, M. A., Czerny, B., Lasota, J. P., & Szuszkiewicz, E. 1988, ApJ, 332, 646, doi: 10.1086/166683
  * Anderson et al. (2001) Anderson, S. F., Fan, X., Richards, G. T., et al. 2001, AJ, 122, 503, doi: 10.1086/321168
  * Antonucci (1993) Antonucci, R. 1993, ARA&A, 31, 473, doi: 10.1146/annurev.aa.31.090193.002353
  * Baldwin et al. (1995) Baldwin, J., Ferland, G., Korista, K., & Verner, D. 1995, ApJ, 455, L119, doi: 10.1086/309827
  * Ballantyne et al. (2002) Ballantyne, D. R., Ross, R. R., & Fabian, A. C. 2002, in X-ray Spectroscopy of AGN with Chandra and XMM-Newton, ed. T. Boller, S. Komossa, S. Kahn, H. Kunieda, & L. Gallo, 73, doi: 10.48550/arXiv.astro-ph/0204260
  * Bentz et al. (2013) Bentz, M. C., Denney, K. D., Grier, C. J., et al. 2013, ApJ, 767, 149, doi: 10.1088/0004-637X/767/2/149
  * Boissay et al. (2016) Boissay, R., Ricci, C., & Paltani, S. 2016, A&A, 588, A70, doi: 10.1051/0004-6361/201526982
  * Bottorff et al. (2002) Bottorff, M. C., Baldwin, J. A., Ferland, G. J., Ferguson, J. W., & Korista, K. T. 2002, ApJ, 581, 932, doi: 10.1086/344408
  * Brightman et al. (2013) Brightman, M., Silverman, J. D., Mainieri, V., et al. 2013, MNRAS, 433, 2485, doi: 10.1093/mnras/stt920
  * Cao (2009) Cao, X. 2009, MNRAS, 394, 207, doi: 10.1111/j.1365-2966.2008.14347.x
  * Chand et al. (2014) Chand, H., Kumar, P., & Gopal-Krishna. 2014, MNRAS, 441, 726, doi: 10.1093/mnras/stu624
  * Chatzikos et al. (2023) Chatzikos, M., Bianchi, S., Camilloni, F., et al. 2023, arXiv e-prints, arXiv:2308.06396, doi: 10.48550/arXiv.2308.06396
  * Collinge et al. (2005) Collinge, M. J., Strauss, M. A., Hall, P. B., et al. 2005, AJ, 129, 2542, doi: 10.1086/430216
  * Crummy et al. (2006) Crummy, J., Fabian, A. C., Gallo, L., & Ross, R. R. 2006, MNRAS, 365, 1067, doi: 10.1111/j.1365-2966.2005.09844.x
  * Czerny (2019) Czerny, B. 2019, Universe, 5, 131, doi: 10.3390/universe5050131
  * Diamond-Stanic et al. (2009) Diamond-Stanic, A. M., Fan, X., Brandt, W. N., et al. 2009, ApJ, 699, 782, doi: 10.1088/0004-637X/699/1/782
  * Done et al. (2012) Done, C., Davis, S. W., Jin, C., Blaes, O., & Ward, M. 2012, MNRAS, 420, 1848, doi: 10.1111/j.1365-2966.2011.19779.x
  * Dumont et al. (1998) Dumont, A.-M., Collin-Souffrin, S., & Nazarova, L. 1998, A&A, 331, 11
  * Ezhikode et al. (2017) Ezhikode, S. H., Gandhi, P., Done, C., et al. 2017, MNRAS, 472, 3492, doi: 10.1093/mnras/stx2160
  * Fan et al. (1999) Fan, X., Strauss, M. A., Gunn, J. E., et al. 1999, ApJ, 526, L57, doi: 10.1086/312382
  * Feng et al. (2019) Feng, J., Cao, X., Gu, W.-M., & Ma, R.-Y. 2019, ApJ, 885, 93, doi: 10.3847/1538-4357/ab4592
  * Ferguson et al. (1997) Ferguson, J. W., Korista, K. T., Baldwin, J. A., & Ferland, G. J. 1997, ApJ, 487, 122, doi: 10.1086/304611
  * Ferland (2003) Ferland, G. J. 2003, ARA&A, 41, 517, doi: 10.1146/annurev.astro.41.011802.094836
  * Ferland et al. (2020) Ferland, G. J., Done, C., Jin, C., Landt, H., & Ward, M. J. 2020, MNRAS, 494, 5917, doi: 10.1093/mnras/staa1207
  * Ferland et al. (1998) Ferland, G. J., Korista, K. T., Verner, D. A., et al. 1998, PASP, 110, 761, doi: 10.1086/316190
  * Gaskell et al. (2007) Gaskell, C. M., Klimek, E. S., & Nazarova, L. S. 2007, arXiv e-prints, arXiv:0711.1025, doi: 10.48550/arXiv.0711.1025
  * Gierliński & Done (2004) Gierliński, M., & Done, C. 2004, MNRAS, 349, L7, doi: 10.1111/j.1365-2966.2004.07687.x
  * Giustini & Proga (2019) Giustini, M., & Proga, D. 2019, A&A, 630, A94, doi: 10.1051/0004-6361/201833810
  * Gliozzi & Williams (2020) Gliozzi, M., & Williams, J. K. 2020, MNRAS, 491, 532, doi: 10.1093/mnras/stz3005
  * Gopal-Krishna et al. (2013) Gopal-Krishna, Joshi, R., & Chand, H. 2013, MNRAS, 430, 1302, doi: 10.1093/mnras/sts706
  * Green et al. (2001) Green, P. J., Forster, K., & Kuraszkiewicz, J. 2001, ApJ, 556, 727, doi: 10.1086/321600
  * Guo et al. (2020) Guo, H., Shen, Y., He, Z., et al. 2020, ApJ, 888, 58, doi: 10.3847/1538-4357/ab5db0
  * Ha et al. (2023) Ha, T., Dix, C., Matthews, B. M., et al. 2023, ApJ, 950, 97, doi: 10.3847/1538-4357/acd04d
  * Hryniewicz et al. (2010) Hryniewicz, K., Czerny, B., Nikołajuk, M., & Kuraszkiewicz, J. 2010, MNRAS, 404, 2028, doi: 10.1111/j.1365-2966.2010.16418.x
  * Jiang et al. (2019) Jiang, Y.-F., Blaes, O., Stone, J. M., & Davis, S. W. 2019, ApJ, 885, 144, doi: 10.3847/1538-4357/ab4a00
  * Jin et al. (2022) Jin, C., Done, C., Ward, M., et al. 2022, MNRAS, 512, 5642, doi: 10.1093/mnras/stac827
  * Jin et al. (2023) —. 2023, MNRAS, 518, 6065, doi: 10.1093/mnras/stac3513
  * Korista & Goad (2000) Korista, K. T., & Goad, M. R. 2000, ApJ, 536, 284, doi: 10.1086/308930
  * Kumar et al. (2016) Kumar, P., Chand, H., & Gopal-Krishna. 2016, MNRAS, 461, 666, doi: 10.1093/mnras/stw1374
  * Kumar & Gopal-Krishna (2015) Kumar, P., & Gopal-Krishna, Chand, H. 2015, MNRAS, 448, 1463, doi: 10.1093/mnras/stv060
  * Kumar et al. (2023) Kumar, R., Chand, H., & Joshi, R. 2023, MNRAS, 519, 3656, doi: 10.1093/mnras/stac3689
  * Lane et al. (2011) Lane, R. A., Shemmer, O., Diamond-Stanic, A. M., et al. 2011, ApJ, 743, 163, doi: 10.1088/0004-637X/743/2/163
  * Laor & Davis (2011) Laor, A., & Davis, S. W. 2011, MNRAS, 417, 681, doi: 10.1111/j.1365-2966.2011.19310.x
  * Leighly et al. (2007a) Leighly, K. M., Halpern, J. P., Jenkins, E. B., & Casebeer, D. 2007a, ApJS, 173, 1, doi: 10.1086/519768
  * Leighly et al. (2007b) Leighly, K. M., Halpern, J. P., Jenkins, E. B., et al. 2007b, ApJ, 663, 103, doi: 10.1086/518017
  * Liu et al. (2013) Liu, Y., Zhang, J., & Zhang, S.-N. 2013, in Feeding Compact Objects: Accretion on All Scales, ed. C. M. Zhang, T. Belloni, M. Méndez, & S. N. Zhang, Vol. 290, 267–268, doi: 10.1017/S1743921312019941
  * Londish et al. (2004) Londish, D., Heidt, J., Boyle, B. J., Croom, S. M., & Kedziora-Chudczer, L. 2004, MNRAS, 352, 903, doi: 10.1111/j.1365-2966.2004.07980.x
  * Luo et al. (2015) Luo, B., Brandt, W. N., Hall, P. B., et al. 2015, ApJ, 805, 122, doi: 10.1088/0004-637X/805/2/122
  * Marculewicz & Nikolajuk (2020) Marculewicz, M., & Nikolajuk, M. 2020, ApJ, 897, 117, doi: 10.3847/1538-4357/ab9597
  * McDowell et al. (1995) McDowell, J. C., Canizares, C., Elvis, M., et al. 1995, ApJ, 450, 585, doi: 10.1086/176168
  * Miniutti et al. (2009) Miniutti, G., Fabian, A. C., Brandt, W. N., Gallo, L. C., & Boller, T. 2009, MNRAS, 396, L85, doi: 10.1111/j.1745-3933.2009.00669.x
  * Murray et al. (1995) Murray, N., Chiang, J., Grossman, S. A., & Voit, G. M. 1995, ApJ, 451, 498, doi: 10.1086/176238
  * Netzer (2013) Netzer, H. 2013, The Physics and Evolution of Active Galactic Nuclei
  * Ni et al. (2018) Ni, Q., Brandt, W. N., Luo, B., et al. 2018, MNRAS, 480, 5184, doi: 10.1093/mnras/sty1989
  * Ni et al. (2022) —. 2022, MNRAS, 511, 5251, doi: 10.1093/mnras/stac394
  * Nikołajuk & Walter (2012) Nikołajuk, M., & Walter, R. 2012, MNRAS, 420, 2518, doi: 10.1111/j.1365-2966.2011.20216.x
  * Paul et al. (2022) Paul, J. D., Plotkin, R. M., Shemmer, O., et al. 2022, ApJ, 929, 78, doi: 10.3847/1538-4357/ac5bd6
  * Piconcelli et al. (2005) Piconcelli, E., Jimenez-Bailón, E., Guainazzi, M., et al. 2005, A&A, 432, 15, doi: 10.1051/0004-6361:20041621
  * Plotkin et al. (2010a) Plotkin, R. M., Anderson, S. F., Brandt, W. N., et al. 2010a, ApJ, 721, 562, doi: 10.1088/0004-637X/721/1/562
  * Plotkin et al. (2010b) —. 2010b, AJ, 139, 390, doi: 10.1088/0004-6256/139/2/390
  * Plotkin et al. (2015) Plotkin, R. M., Shemmer, O., Trakhtenbrot, B., et al. 2015, ApJ, 805, 123, doi: 10.1088/0004-637X/805/2/123
  * Pravdo et al. (1981) Pravdo, S. H., Nugent, J. J., Nousek, J. A., et al. 1981, ApJ, 251, 501, doi: 10.1086/159489
  * Shakura & Sunyaev (1973) Shakura, N. I., & Sunyaev, R. A. 1973, A&A, 24, 337
  * Shemmer et al. (2006) Shemmer, O., Brandt, W. N., Schneider, D. P., et al. 2006, ApJ, 644, 86, doi: 10.1086/503543
  * Shemmer et al. (2010) Shemmer, O., Trakhtenbrot, B., Anderson, S. F., et al. 2010, ApJ, 722, L152, doi: 10.1088/2041-8205/722/2/L152
  * Singh et al. (1985) Singh, K. P., Garmire, G. P., & Nousek, J. 1985, ApJ, 297, 633, doi: 10.1086/163560
  * Steffen et al. (2006) Steffen, A. T., Strateva, I., Brandt, W. N., et al. 2006, AJ, 131, 2826, doi: 10.1086/503627
  * Timlin et al. (2020) Timlin, J. D., Brandt, W. N., Ni, Q., et al. 2020, MNRAS, 492, 719, doi: 10.1093/mnras/stz3433
  * Titarchuk (1994) Titarchuk, L. 1994, ApJ, 434, 570, doi: 10.1086/174760
  * Urry & Padovani (1995) Urry, C. M., & Padovani, P. 1995, PASP, 107, 803, doi: 10.1086/133630
  * Walter & Fink (1993) Walter, R., & Fink, H. H. 1993, A&A, 274, 105
  * Wu et al. (2012) Wu, J., Brandt, W. N., Anderson, S. F., et al. 2012, ApJ, 747, 10, doi: 10.1088/0004-637X/747/1/10
  * Wu et al. (2023) Wu, J., Wu, Q., Xue, H., Lei, W., & Lyu, B. 2023, ApJ, 950, 106, doi: 10.3847/1538-4357/acce9e
  * Wu et al. (2011) Wu, J., Brandt, W. N., Hall, P. B., et al. 2011, ApJ, 736, 28, doi: 10.1088/0004-637X/736/1/28
  * Zhang & Liu (2016) Zhang, X., & Liu, Y. 2016, ApJ, 830, 69, doi: 10.3847/0004-637X/830/2/69