# Comparative Review of Cloud Computing Platforms for Data Science Workflows

Mohammad Rehman  School of Computer Science and Applied Mathematics
University of the Witwatersrand
Johannesburg, South Africa
<EMAIL_ADDRESS>Hairong Wang  School of Computer Science and
Applied Mathematics
University of the Witwatersrand
Johannesburg, South Africa
<EMAIL_ADDRESS>
###### Abstract

With the advantages that cloud computing offers in terms of platform as a
service, software as a service, and infrastructure as a service, data
engineers and data scientists are able to leverage cloud computing for their
ETL/ELT (extract, transform and load) and ML (machine learning) requirements
and deployments. The proposed framework for the comparative review of cloud
computing platforms for data science workflows uses an amalgamation of the
analytical hierarchy process, Saaty’s fundamental scale of absolute numbers,
and a selection of relevant evaluation criteria (namely: automation, error
handling, fault tolerance, performance quality, unit testing, data encryption,
monitoring, role based access, security, availability, ease of use,
integration and interoperability). The framework enables users to evaluate
criteria pertaining to cloud platforms for data science workflows, and
additionally is able to recommend which cloud platform would be suitable for
the user based on the relative importance of the above criteria. Evaluations
of the criteria are shown to be consistent and thus the weighting of criteria
against the goal of cloud service provider or cloud platform selection are
sensible. The proposed framework is robust enough to accommodate for changes
in criteria and alternatives, depending on user cloud platform requirements
and scope of cloud platform selection.

###### Index Terms:

Data science workflows, analytical hierarchy process (AHP), evaluation of
criteria, cloud platform, cloud services

##  I Introduction

With an increase in migration to cloud-based extract, transform and load (ETL)
operations, analytics, machine learning (ML) and reporting solutions, data
scientists and analysts need to become comfortable with moving away from tools
such as Microsoft Excel and on-premises databases for their data storage and
compute requirements, and gain confidence in storage and compute technology
offered by cloud service providers. Although there are the giants among the
cloud computing service providers (CSPs), for example
Microsoft111https://portal.azure.com, Amazon222https://aws.amazon.com/console
and Google333https://console.cloud.google.com, consumers can still opt to
partner with many other CSPs such as Alibaba444https://home-
intl.console.aliyun.com, Oracle555https://cloud.oracle.com and
IBM666https://cloud.ibm.com.
As the migration to cloud platforms is a journey that could span many months
to years, consumers need to be sure that the services offered by their desired
CSP are a good fit for their requirements, and thus need to know the
advantages and disadvantages of the different CSPs based on factors such as
compute, security, governance, scalability, pricing, availability, etc.
Therefore, this research has investigated, evaluated and reviewed the ETL
capabilities and the ML service integration of the cloud platforms offered by
three popular cloud service providers: Microsoft Azure, Amazon Web Services,
and Google Cloud Platform. The analytical hierarchy process was used to
provide a framework for evaluating criteria pertaining to cloud based
platforms specifically for data science workflows, for the use case of cloud
service provider selection. The research shows that consumers can use multi-
criteria decision making methodologies to determine which cloud provider would
be most beneficial for them based on the relative importance of their
requirements.
The following sections in this report delve into the available literature,
followed by an overview of the methodology used for experimentation and
evaluation, and subsequently elaborate on the results obtained from the
experiments while providing a concise discussion on the implications of the
results. Finally, a conclusion is provided in terms of the overall outcomes of
the research.

##  II Related Work

In as early as 2008, Vouk had considered the use of cloud computing
infrastructure for building out effective workflows [1].The importance and
need of on-demand as well as reserve-based provisioning of computational
services, automation and security was highlighted as a measure of
effectiveness, and this is true to-date. Integration of services, appropriate
governance and security, and automation are all critical to having efficient
and effective workflows within any cloud computing platform.

C. Chen, J. Chen, J. Liu and Y. Wen have presented the need for workflow
scheduling on cloud platforms and how it can be implemented [2], however, this
was based on using external workflow scheduling platforms. Zhou et al. have
investigated this further by incorporating parallelism into cloud-based
workflows [3] and have shown that external workflow engines can be used
(together with a run-time service) to leverage parallelism in order to achieve
improvements when scaling workflows.

The generally accepted criteria for evaluating cloud platforms and services
are: security, performance, accessibility and useability, scalability, and
adaptability [4] which can be further explored thanks to the consolidation of
past evaluations by Abdel-Basset, Mohamed and Chang [4]. Reference [4] has
additionally provided individuals with a ”neutrosophic multi-criteria decision
analysis” methodology to determine the quality of service that a cloud
provider offers. A methodology that has been used in many frameworks that aim
to evaluate various aspects of cloud computing is the analytical hierarchy
process, which aims to decompose multi-criteria decision problems into
hierarchical segments for evaluation (both qualitatively and quantitatively)
[5] [6] [7]. For security specifically, Ghazizadeh and Cusack have proposed a
”Trust Framework” for evaluation of cloud computing technologies [8].
Additionally, the National Institute of Standards and Technology (U.S.
Department of Commerce) have developed the ”Cloud Usability Framework” that
enables users to be informed of what cloud service providers need to adhere to
and ensure is available within their platforms and services [9].

Costa, Santos and Mira da Silva have looked into a significant number of
criteria for the evaluation of cloud services in general, broadly within the
realm of accountability, agility, management, performance, security and
usability [10]. Metrics for evaluating cloud-based machine learning services
were proposed and reviewed in 2017 [11]. The paper included, but was not
limited to, metrics such as availability, data capacity, compatibility,
documentation, functionality, integration, performance, pricing and usability.
In a recent study looking at the selection of cloud services adequate for big
data, the proposed important criteria were identified as: architecture, cost,
functionality, performance, usability and vendor reputation [12]. In a quality
of service evaluation approach study pertaining to managing cloud service
evaluation and selection, a similar range of criteria were considered, namely:
accountability, agility, assurance, cost, performance and security [13].

##  III Methodology

A qualitative review of criteria pertaining to the top cloud computing
platforms required an in-depth understanding of the various ETL processes
available across the cloud platforms and thus necessitated the exploration of
the services related to data science workflows prior to building out the
workflows to gain familiarity with their functionality, governance and
accessibility related capabilities, thus making these the criteria that were
used within the evaluation framework. As such, each of the cloud computing
platforms that were utilised explored the following capabilities, i.e. their
sub-criteria:

Functionality: Automation, error handling, fault tolerance, performance
quality and unit testing
Governance Data encryption, monitoring, security and role-based access
Accessibility Availability, ease of use, integration, interoperability

A focus was placed on services that enabled data engineering and analytics in
order to remain in scope of the proposed research. The accessibility criteria
were included in order to supplement the evaluation as a major driver of cloud
adoption is the ability to learn and use the platform relatively easily to
promote its adoption, as well as to have certainty that it is able to
integrate well with applications and tools external to the platform, thus
mitigating vendor lock-in. A similar workflow was developed using the various
platforms and the experience gained while building out the end-to-end
workflows was used as a basis for reviewing each of the criteria and sub-
criteria from an entry level data scientist perspective, thus informing the
valuation of each of the attributes within the framework.

The analytical hierarchy process (AHP) was subsequently used to develop a
framework [14] that evaluated the criteria pertaining to cloud platforms based
on the valuations of the criteria and sub-criteria previously mentioned. In
the evaluation process, Saaty’s fundamental scale of absolute numbers [14] was
used to attribute numerical values to linguistic associations of importance to
a pair of relevant attributes. Although Microsoft Azure, Amazon Web Services
and Google Cloud Platform were used as the cloud platforms in this research,
the actual development of the framework used anonymised platform providers,
i.e. CSP1, CSP2 and CSP3, as the intention of the research is to ultimately
propose a feasible, dynamic evaluation framework to identify suitable cloud
computing platforms for data science workflows rather than to compare Azure,
AWS and GCP in an official and categorical manner.

The context of this research is within the domain of a multi-criteria decision
making problem, where a selection of alternatives need to be evaluated based
on a set of criteria in order to decide on which alternative is the most
favourable for a requirement. Therefore, the analytical hierarchy process can
be used to develop a framework for decision making pertaining to the
comparative review of a multitude of criteria that pertain to a set of cloud
platforms for data science workflows, from the perspective of an entry level
data scientist who has the goal of deciding which platform would be most
suitable for the user’s data science requirements. The hierarchy tree used for
this research is illustrated in Fig. 1

The analytical hierarchy process was selected as it provides decision makers
with a framework to attribute mathematical valuations to subjective
opinionations pertaining to a set of criteria in an overall evaluation,
thereby enabling users to mathematically deduce which alternative is most
suitable for the user against a defined goal. It does this by associating
relative measures among a pair of criteria that are relevant to the user, thus
propagating a pairwise matrix of relative importance for each of the criteria.
This can be developed further by means of using sub-criteria, which provide a
granular numeric representation of importance among the hierarchy of
attributes relevant to the user. As the process is based on subjective
beliefs, aggregated inputs from a myriad of users is beneficial, especially
for important decision making.

Figure 1: AHP model for determining cloud platform based on data science
workflow requirements

##  IV Experimentation and Results

The analytical hierarchy process was used to develop a framework for reviewing
criteria pertaining to cloud computing platforms for data science workflows.
The experimental process entailed the construction of ETL pipelines using the
cloud-based ETL platforms of various (anonymised) cloud service providers to
investigate attributes such as automation, error handling, fault tolerance,
performance quality, unit testing, data encryption, monitoring, security, role
based access, availability, ease of use, integration and interoperability. The
attributes were subsequently used to review and evaluate the ETL platforms
relative to one another based on the experience of an entry level data
scientist, to determine which cloud platform would be the most suitable in
terms of selection for carrying out data science workflows. In the evaluation
process, Saaty’s fundamental scale of absolute numbers was used to attribute
numerical values to linguistic associations of importance to a pair of
relevant attributes. This conversion of linguistic valuation to numeric
valuation ensured that matrices containing the relative importance of criteria
and sub-criteria were able to be constructed, which aided in assigning global
weights to sub-criteria in terms of their overall importance. Subsequently, a
similar approach made it possible to evaluate each of the cloud platforms
relative to one another in order to determine the platform with the highest
overall weighting in terms of the chosen sub-criteria, thus serving as a
recommendation for the platform that would likely be most suitable for the
data scientist.

Once all criteria and sub-criteria evaluations were validated to be
consistent, the weightings of the sub-criteria were converted to global
weights. This was done by multiplying the weight of each sub-criteria with the
weight of its parent criteria. Remembering that the sum of weights for
criteria, as well as sub-criteria sets, was equal to $1$, the weights can be
treated as percentages, thus the multiplication of sub-criteria weight and
parent weight yields the weighting of a sub-criteria in terms of the goal of
the analytical hierarchy process. As such, the sum of the global weights is
also equal to $1$. Table I illustrates the global weights of each of the sub-
criteria:

TABLE I: Global sub-criteria weightings for the analytical hierarchy process Criteria | Sub-criteria | Global Weight
---|---|---
Functionality | Automation | 0.14
Functionality | Error Handling | 0.07
Functionality | Fault Tolerance | 0.02
Functionality | Performance Quality | 0.04
Functionality | Unit Testing | 0.28
Governance | Data Encryption | 0.02
Governance | Monitoring | 0.07
Governance | Security | 0.01
Governance | Role Based Access | 0.03
Accessibility | Availability | 0.02
Accessibility | Ease of Use | 0.17
Accessibility | Integration | 0.04
Accessibility | Interoperability | 0.08

Having obtained the weights of each alternative against each sub-criteria
ultimately allowed for the calculation of the overall weighting of the
alternatives against the goal, similar to how the sub-criteria were weighed
against the goal, as shown in Table I. Specifically, this was done by
multiplying the weight of each alternative per sub-criteria to the global
weight of the sub-criteria. As a result, the final weighting of the
alternatives were calculated and are demonstrated in Table II:

TABLE II: Overall weightings of alternatives against the goal | Weights | Implications
---|---|---
CSP1 | 0.28 | Suitable for user
CSP2 | 0.19 | Least suitable for the user
CSP3 | 0.53 | Most suitable for the user

The results indicate that based on the relative importance of criteria and
sub-criteria to the user, the experience obtained by making use of the cloud
platform provided by CSP3 would be most suitable for the data scientist as it
has the highest weighting based on the evaluation of the criteria and sub-
criteria relevant to the tasks needed to be performed on said platform.

##  V Discussion

Within the scope of the research, the attributes selected were what the data
scientist would look for when using the cloud platforms for data science
workflows, thus ensuring that the criteria used to review and evaluate each
platform were suitable. Additionally, as a comparative review was desired, and
not a review to determine the absolute best platform, the analytical hierarchy
process was perfect for achieving this. Pairwise comparisons were done between
criteria and sub-scriteria to determine the numeric representation of the
relative levels of importance among attributes. As the analytical hierarchy
process has a means of evaluating consistency, the evaluations made when
reviewing the attributes were able to be tested for any inconsistency and the
outcomes for the evaluations pertaining to the criteria and sub-criteria were
indeed consistent. Finally, the alternatives were reviewed against each of the
sub-criteria in order to weigh the various alternatives (cloud platforms
provided by the cloud service providers) against the goal. Ultimately, this
allowed the framework to recommend a cloud provider to the user by answering
the question present in the goal of the analytical hierarchy process with the
alternative that yielded the highest weight, which in the experiment conducted
for this research was CSP3.

Evaluation of multi-criteria systems can become quite complex, moreso when the
criteria are of a linguistic nature and are dependent on subjective beliefs.
This was an additional motivation of using the analytical hierarchy process,
along with Saaty’s fundamental scale of absolute numbers, so that linguistic
associations of relative importance between each of the attributes could be
converted to numeric values upon which mathematical operations could be
conducted to produce weightings of sub-criteria and alternatives against the
goal. It enabled the framework to be both replicable and robust, as the
analytical hierarchy process is a widely known methodology for multi-criteria
decision analysis and thus can be followed to reproduce the experiments and
results obtained in this report. Additionally, based on how the framework is
set up, users can apply the framework to systems with varying criteria, sub-
criteria and alternatives, depending on their requirements and accessibility
to platforms.

As such, having shown the consistency in evaluation as well as the ability for
the framework to recommend a cloud service provider, the outcomes of the
research efforts have been positive: a successful framework was developed
using the analytical hierarchy process in order to review, compare and
evaluate criteria pertaining to cloud platforms based on the required
attributes, and to recommend a suitable platform for a user based on the
review and subsequent evaluation of the attributes, while maintaining
confidence in the evaluation process by means of determining that the
evaluations of the criteria and sub-criteria were consistent.

##  VI Conclusion

Although a significant amount of research has been done regarding scheduling
of data science workflows, cloud computing theory, and the comparison of cloud
computing platforms in terms of concepts such as performance, pricing and
storage, not much has been done with regards to evaluating or reviewing cloud
based platforms for data science workflows specifically. Thus research was
conducted to understand various data science workflow platforms, the
requirements that a data scientist would have when using such platforms and
the availability of existing frameworks or methodologies that would enable
some one to conduct a comparative review of cloud platforms for data science
workflows. Since no such framework was identified, a novel framework using the
analytical hierarchy process, Saaty’s fundamental scale of absolute numbers,
and relevant attributes to a data science workflow was created. It was applied
to a scenario where a data scientist needed to select a cloud platform that
would be suitable for the user’s requirements and the framework was successful
in recommending a suitable cloud service provider to the data scientist.

The benefits of using the analytical hierarchy process were twofold: the first
was that a measure, namely consistency, could be used to ensure that
evaluations were done correctly, and the second was that it enabled the review
and evaluations of criteria pertaining to the cloud platforms (alternatives)
to be done in a relative manner depending on the experience of an entry level
data scientist. In this way, decisive or categorical proposals of one cloud
platform being objectively better than another could be avoided, with the
additional benefit on users of the framework being able to have their own
alternatives depending on which set of platforms would be feasible for them
(depending on budget, vendor agreements, brand affiliation, etc.).

Overall, a successful framework was created and demonstrated for the
comparative review of cloud computing platforms for data science workflows.
The evaluations were deemed consistent and the framework is dynamic enough to
be tailored to user requirements (attributes can be changed as needed) and
platform scope (relevant alternatives to the user’s circumstances can be
chosen).

## References

  * [1] M.A. Vouk, “Cloud Computing - issues, research and implementations,” Journal of Computing and Information Technology, vol. 4, 2008, pp. 235–246.
  * [2] C. Chen and J. Chen and J. Liu and Y. Wen, “Research on workflow scheduling algorithms the cloud,” , in Process-Aware Systems, 2014, pp.35–48.
  * [3] J. Zhou and C. Sun and W. Fu and J. Liu and L. Jia and H. Tan, “Modeling, design, and implementation of a cloud workflow engine based on Aneka,” Journal of Applied Mathematics, pp. 1–9, 2014.
  * [4] M. Abdel-Basset and M. Mohamed and V. Chang, “NMCDA: A framework for evaluating cloud computing services,” Future Generation Computer Systems, vol. 86, pp. 12–29, 2018.
  * [5] W. Liao and W. Qiu, “Applying analytic hierarchy process to assess healthcare-oriented cloud computing service systems,” SpringerPlus, vol. 5, pp. 1–9, 2016.
  * [6] B. M. R. Wilson and B. Khazaei and L. Hirsch, “Cloud adoption decision support for SMEs using analytical hierarchy process (AHP),” SpringerPlus, pp. 1–5, 2016.
  * [7] N. Chawla and D. Kumar, “Decision-making model to evaluate cloud computing service nodel using analytic hierarchy process (AHP) and benefits, costs, opportunities, and risks (BCOR) analysis,” International Journal of Sensors, Wireless Communications and Control, vol. 10, pp. 846–856, 2020.
  * [8] E. Ghazizadeh and B. Cusack, “Evaluation theory for characteristics of cloud identity trust framework,” Cloud Computing - Technology and Practices, 2019.
  * [9] B. Stanton and M. Theofanos and K. P. Joshi, “Framework for cloud usability,” NIST Special Publication, vol. 500, pp. 1–18, 2015.
  * [10] P. Costa and J. Santos and M. Mira da Silva, “Evaluation criteria for cloud services,” IEEE International Conference on Cloud Computing, CLOUD, pp. 598–605, June 2013.
  * [11] A. Tataru, “Metrics for evaluating machine learning cloud services,” pp. 1–103, 2017.
  * [12] N. Afifi and H. Belhadaoui and I. Hilal and F.E. Mdarbi, “An approach for selecting cloud service adequate to big data,” International Journal of Computer Science and Information Security, vol. 16, pp. 20–32, 2018.
  * [13] N. Uphadyay, “Managing cloud service evaluation and selection,” Procedia Computer Science, vol. 122, pp. 1061–1068, 2008.
  * [14] T.L. Saaty, “Decision making with the analytical hierarchy process,” International Journal of Services Sciences, vol. 1, pp. 83–98, 2008.


# CITE: A Corpus of Image–Text Discourse Relations

Malihe Alikhani
Computer Science
Rutgers University
malihe.alikhani
@rutgers.edu
Sreyasi Nag Chowdhury
Max Planck Institute
for Informatics
sreyasi
@mpi-inf.mpg.de
Gerard de Melo
Computer Science
Rutgers University
gerard.demelo
@rutgers.edu
Matthew Stone
Computer Science
Rutgers University
matthew.stone
@rutgers.edu

###### Abstract

This paper presents a novel crowd-sourced resource for multimodal discourse:
our resource characterizes inferences in image–text contexts in the domain of
cooking recipes in the form of coherence relations. Like previous corpora
annotating discourse structure between text arguments, such as the Penn
Discourse Treebank, our new corpus aids in establishing a better understanding
of natural communication and common-sense reasoning, while our findings have
implications for a wide range of applications, such as understanding and
generation of multimodal documents.

_K_ eywords Discourse Coherence $\cdot$ Multimodal Documents

##  1 Introduction

“ _Sometimes a picture is worth the proverbial thousand words; sometimes a few
well-chosen words are far more effective than a picture_ ” – [1]. Modeling how
visual and linguistic information can jointly contribute to coherent and
effective communication is a longstanding open problem with implications
across cognitive science. As [1] already observe, it is particularly important
for automating the understanding and generation of text–image presentations.
Theoretical models have suggested that images and text fit together into
integrated presentations via _coherence relations_ that are analogous to those
that connect text spans in discourse; see [2] and Section 3. This paper
follows up this theoretical perspective through systematic corpus
investigation. We are inspired by research on text discourse, which has led to
large-scale corpora with information about discourse structure and discourse
semantics. The Penn Discourse Treebank (PDTB) is one of the most well-known
examples [3, 4]. However, although multimodal corpora increasingly include
discourse relations between linguistic and non-linguistic contributions,
particularly for utterances and other events in dialogue [5, 6], to date there
has existed no dataset describing the coherence of text–image presentations.
In this paper, we describe the construction of an annotated corpus that fills
this gap, and report initial analyses of the communicative inferences that
connect text and accompanying images in this corpus. As we describe in Section
3, our approach asks annotators to identify the presence of specific
inferences linking text and images, rather than to use a taxonomy of coherence
relations. This enables us to deal with the distinctive discourse
contributions of photographic imagery. We describe our data collection process
in Section 4, showing that our annotation scheme allows us to get reliable
labels by crowdsourcing. We present analyses in Section 5 that show that our
annotation highlights a range of cases where text and images work together in
distinctive and theoretically challenging ways, and discuss the implications
of our work for the understanding and generation of multimodal documents. We
conclude in Section 6 with a number of problems for future research.

##  2 Introduction

“ _Sometimes a picture is worth the proverbial thousand words; sometimes a few
well-chosen words are far more effective than a picture_ ” –[1]. Modeling how
visual and linguistic information can jointly contribute to coherent and
effective communication is a longstanding open problem with implications
across cognitive science. As [1] already observe, it is particularly important
for automating the understanding and generation of text–image presentations.
Theoretical models have suggested that images and text fit together into
integrated presentations via _coherence relations_ that are analogous to those
that connect text spans in discourse; see [2] and Section 3.
This paper follows up this theoretical perspective through systematic corpus
investigation.

We are inspired by research on text discourse, which has led to large-scale
corpora with information about discourse structure and discourse semantics.
The Penn Discourse Treebank (PDTB) is one of the most well-known examples [3,
4]. However, although multimodal corpora increasingly include discourse
relations between linguistic and non-linguistic contributions, particularly
for utterances and other events in dialogue [5, 6], to date there has existed
no dataset describing the coherence of text–image presentations. In this
paper, we describe the construction of an annotated corpus that fills this
gap, and report initial analyses of the communicative inferences that connect
text and accompanying images in this corpus.

As we describe in Section 3, our approach asks annotators to identify the
presence of specific inferences linking text and images, rather than to use a
taxonomy of coherence relations. This enables us to deal with the distinctive
discourse contributions of photographic imagery. We describe our data
collection process in Section 4, showing that our annotation scheme allows us
to get reliable labels by crowdsourcing. We present analyses in Section 5 that
show that our annotation highlights a range of cases where text and images
work together in distinctive and theoretically challenging ways, and discuss
the implications of our work for the understanding and generation of
multimodal documents. We conclude in Section 6 with a number of problems for
future research.

##  3 Discourse Coherence and Text–Image Presentations

We begin with an example to motivate our approach and clarify its relationship
to previous work. Figure 1 shows two steps in an online recipe for a ravioli
casserole from the RecipeQA data set [7]. The image of Figure 1a shows a
moment towards the end of carrying out the “covering” action of the
accompanying text; that of Figure 1b shows one instance of the result of the
“spooning” actions of the text.

(a) Text: Cover with a single layer of ravioli.

(b) Text: Let cool 5 minutes before spooning onto individual plates.

Figure 1: Two steps in a recipe from [7] illustrating diverse inferential
relationships between text and accompanying imagery in instructions. The
recipe is from Autodesk Inc. www.instructables.com and is contributed by
www.RealSimple.com.

Cognitive scientists have argued that such images are much like text
contributions in the way their interpretation connects to the broader
discourse. In particular, inferences analogous to those used to interpret text
seem to be necessary with such images to recognize their spatio-temporal
perspective [8], the objects they depict [9], and their place in the arc of
narrative progression [10, 11]. In fact, such inferences seem to be a general
feature of multimodal communication, applying also in the coherent
relationships of utterance to co-speech gesture [12] or the coherent
relationships of elements in diagrams [13, 14].

In empirical analyses of text corpora, researchers in projects such as the
Penn Discourse Treebank [3, 4] have been successful at documenting such
effects by annotating discourse structure and discourse semantics via
coherence relations. We would like to apply a similar strategy to text–image
documents like that shown in Figure 1. However, existing discourse annotation
guidelines depend on the distinctive ways that coherence is signaled in text.
In text, we find syntactic devices such as structural parallelism, semantic
devices such as negation, and pragmatic elements such as discourse
connectives, all of which can help annotators to recognize coherence relations
in text. Images lack such features. At the same time, characterizing the
communicative role of imagery, particularly photographic imagery, involves a
special problem: distinguishing the content that the author specifically aimed
to depict from merely incidental details that happen to appear in the scene
[15].

Thus, rather than start from a taxonomy of discourse relations like that used
in PDTB, we characterize the different kinds of inferential relationships
involved in interpreting imagery separately.

  * •

To characterize temporal relationships between imagery and text, we ask if the
image gives information about the preparation, execution or results of the
accompanying step.

  * •

To characterize the logical relationship of imagery to text, we ask if the
image shows one of several actions described in the text, and if it depicts an
action that needs to be repeated.

  * •

To characterize the significance of incidental detail, we ask a range of
further questions (some relevant specifically to our domain of instructions),
asking about what the image depicts from the text, what it leaves out from the
text, and what it adds to the text.

Our approach is designed to elicit judgments that crowd workers can provide
quickly and reliably.

This approach allows us to highlight a number of common patterns that we can
think of as prototypical coherence relations between images and text. Figure
1a, for example, instantiates a natural Depiction relation: the image shows
the action described in the text in progress; the mechanics of the action are
fully visible in the image, but the significant details in the imagery are all
reported in the text as well. Our approach also lets us recognize more
sophisticated inferential relationships, like the fact that Figure 1b shows an
Example:Result of the accompanying instruction. Many of the relationships that
emerge from our annotation effort involve newly-identified features of
text–image presentations that deserve further investigation: particularly, the
use of loosely-related imagery to provide background and motivation for a
multimodal presentation as a whole, and depictions of action that seem
simultaneously to give key information about the context, manner and result of
an action.

##  4 Annotation Effort

Work on text has found that text genre heavily influences both the kinds of
discourse relations one finds in a corpus and the way those relations are
signalled [16]. Since our focus is on developing methodology for consistent
annotation, we therefore choose to work within a single genre. We selected
instructional text because of its concrete, practical subject matter and
because of its step-by-step organization, which makes it possible to
automatically group together short segments of related text and imagery.

### Text–Image Pairs.

We base our data collection on an existing instructional dataset, RecipeQA
[7]. This is the only publicly available large-scale dataset of multimodal
instructions. It consists of multimodal recipes—textual instructions
accompanied by one or more images.

We excluded documents that either have multiple steps without images or that
have multiple images per set. This was so that we could more easily study the
direct relationship between an image and the associated text. There are 1,690
documents with this characteristic in the RecipeQA train set. To avoid
overwhelming crowd workers, we further filtered those to retain only recipes
with 70 or fewer words per step, for a final count of 516 documents (2,047
image–text pairs).

### Protocol.

We recruit participants through Amazon Mechanical Turk. All subjects were US
citizens, agreed to a consent form approved by Rutgers’s institutional review
board, and were compensated at an estimated rate of USD 15 an hour.

### Experiment Interface.

Given an image and the corresponding textual instruction from the dataset,
participants were requested to answer the following 10 questions.

For Question 1, participants were asked to highlight the relevant part of the
text. For the others, we solicited True/False responses.

  1. 1.

Highlight the part of the text that is most related to the image.

  2. 2.

The image gives visual information about the step described in the text.

  3. 3.

You need to see the image in order to be able to carry out the step properly.

  4. 4.

The text provides specific quantities (amounts, measurements, etc.) that you
would not know just by looking at the picture.

  5. 5.

The image shows a tool used in the step but _not_ mentioned in the text.

  6. 6.

The image shows how to prepare before carrying out the step.

  7. 7.

The image shows the results of the action that is described in the text.

  8. 8.

The image depicts an action in progress that is described in the text.

  9. 9.

The text describes several different actions but the image only depicts one.

  10. 10.

One would have to repeat the action shown in the image many times in order to
complete this step.

The interface is designed such that if the answer to Question 8 is True, the
subject will be prompted with Question 9 and 10. Otherwise, Question 8 is the
last question in the list.111The dataset and the code for the machine learning
experiments are available at https://github.com/malihealikhani/CITE

### Agreement.

To assess the inter-rater agreement, we determine Cohen’s $\kappa$ and
Fleiss’s $\kappa$ values. For Cohen’s $\kappa$, we randomly selected 150
image–text pairs and assigned each to two participants, obtaining a Cohen’s
$\kappa$ of 0.844, which indicates almost perfect agreement. For Fleiss’s
$\kappa$ [17, 18, 19], we randomly selected 50 text–image pairs, assigned them
to five subjects, and computed the average $\kappa$. We obtain a score of
0.736, which indicates substantial agreement [20].

##  5 Analysis

### Overall Statistics.

Table 1 shows the rates of true answers for questions Q2–Q10.

Subjects reported that in 17% of cases the images did not give any information
about the step described in the accompanying text. Such images deserve further
investigation to characterize their interpretive relationship to the document
as a whole. Our anecdotal experience is that such images sometimes provide
context for the recipe, which may suggest that imagery, like real-world events
[6], creates more flexible discourse structures than linguistic segments on
their own.

|  Q2 |  Q3 |  Q4 |  Q5 |  Q6 |  Q7 |  Q8 |  Q9 |  Q10
---|---|---|---|---|---|---|---|---|---
True |  0.829 |  0.058 |  0.211 |  0.131 |  0.056 |  0.491 |  0.209 |  0.289 |  0.133
Table 1: Rate of true answers for annotation questions Q2–Q10 across the corpus. |  Q1 |  Q2** |  Q3** |  Q4** |  Q5 |  Q6** |  Q7** |  Q8** |  Q9* |  Q10**
---|---|---|---|---|---|---|---|---|---|---
F1 |  0.74 |  0.86 |  0.76 |  0.85 |  0.88 |  0.92 |  0.64 |  0.83 |  0.77 |  0.92
Table 2: SVM classification accuracy: bag-of-words features; 80-20 train-test
split; 5-fold cross validation. For the first question, this distinguishes
highlighted text vs. its complement (excluded vs. included). For the rest of
the questions, this distinguishes text of true instances from text of false
instances, and is different from majority class baseline ∗ at $p<0.04$,
$t=-3.5$ and ∗∗ at $p<0.01$, $t>|2.49|$.

Subjects reported that the image was required in order to carry out the
instruction only for 6% of cases. This suggests that subjects construe imagery
as backgrounded or peripheral to the document, much as speakers regard co-
speech iconic gesture as peripheral to speech [21]. Note, by contrast, that
subjects characterized 12.7% of images as introducing a new tool: this
includes many cases where the same subjects say the image is not required. In
other words, subjects’ intuitions suggest that coherent imagery typically does
not contribute instruction content, but rather serves as a visual signal that
facilitates inferences that have to be made to carry out the instruction
regardless. Our annotated examples, where imagery is linked to specific kinds
of inferences, provide materials to test this idea.

(a) Text: Top with another layer of ravioli and the remaining sauce not all
the ravioli may be needed. Sprinkle with the Parmesan. Figure 2: The image
depicts both the action and the result of the action. The recipe is from
Autodesk Inc. www.instructables.com and was contributed by www.RealSimple.com.

### The Complex Coherence of Imagery.

Our annotation reveals cases where a single image does include more
information than could be packaged into a single textual discourse unit (the
proverbial thousand words). In particular, such imagery participates in more
complex coherence relationships than we find between text segments. Multiple
temporal relationships show this most clearly: 12% of images that have any
temporal relation have more than one. For example, many images depict the
action that is described in the text, while also showing preparations that
have already been made by displaying the scene in which the action is
performed. Figure 2 depicts the action and the result of the action. It also
shows how to prepare before carrying out the action. Other images show an
action in progress but nearing completion and thereby depict the result. For
instance, the image that accompanies “mix well until blended” can show both
late-stage mixing and the blended result. Looking at a few such cases closely,
the circumstances and composition of the photos seem staged to invite such
overlapping inferences.

Such cases testify to the richness of multimodal discourse, and help to
justify our research methodology. The True/False questions characterize the
relevant features of interpretation without necessarily mapping to single
discourse relations. For instance, Q4 and Q5 indicate inferences in line with
an Elaboration relation; Q9 and Q10 indicate inferences in line with an
Exemplification relation, as information presented in images show just one
case of a generalization presented in accompanying text. However, our data
shows that these inferences can be combined in productive ways, in keeping
with the potentially complex relevant content of images.

Q4. Text has quantities not in image
---
True | False
1 |  -4.1 |  add |  -4.5
cup |  -4.4 |  place |  -4.9
minutes |  -4.7 |  put |  -5.0
2 |  -4.7 |  make |  -5.1
1/2 |  -4.9 |  mix |  -5.1
Q8. Image depicts action in progress
True | False
add |  -5.0 |  1 |  -4.6
mix |  -5.2 |  cup |  -4.7
place |  -5.3 |  minutes |  -4.9
bread |  -5.5 |  160 |  -5.1
make |  -5.6 |  put |  -5.2
Table 3: Top five features of Multimodal Naive Bayes classifier for two
classification problems and their corresponding log–probability estimates.

### Information across modalities.

We carried out machine learning experiments to assess what information images
provide and what textual cues can guide image interpretation. We use SVM
classifiers for performance, and Multinomial Naive Bayes classifiers to
explain classifier decision making, both with bag-of-words features.

Q1. Information in text
---
1 | do it clearly | on which
2 | let cool for | favorite toppings
3 | recipe with directions | after an
4 | how slowly the | lightly season
5 | 7 minutes on | the 2

Q1. Information in images
---
1 | added a beautiful | cover with
2 | put as much | scrapping the
3 | skin off of | finally fold
4 | cut side toward | after an
5 | blend and blend | add a

Table 4: Top five bigram and trigram features of NBSVM for the first question.
The highlighted text that is most relevant to the image describes depicted
actions, while the complement descriptions describe quantities or
modifications of the actions that are described in the highlighted segments.

Table 2 reports the F1 measure for instance classification with SVMs (with
5-fold cross validation). In many cases, machine learning is able to find cues
that reliably help guess inferential patterns. Table 3 looks at two effective
Naive Bayes classifiers, for Q4 (text has quantities) and Q8 (image depicts
action in progress). It shows the features most correlated with the
classification decision and their log probability estimates. For Q4, not
surprisingly, numbers and units are positive instances.

More interestingly, verbs of movement and combination are negative instances,
perhaps because such steps normally involve material that has already been
measured. For Q8, a range of physical action verbs are associated with actions
in progress; negative features correlate with steps involved in actions that
don’t require ongoing attention (e.g., baking). Table 4 reports top SVM with
NB (NBSVM) [22] features for Q1 that asks subjects to highlight the part of
the text that is most related to the image. Action verbs are part of
highlighted text, whereas adverbs and quantitative information that cannot be
easily depicted in images are part of the remaining segments of the text. Such
correlations set a direction for designing or learning strategies to select
when to include imagery.

##  6 Conclusions

In this paper, we have presented the first dataset describing discourse
relations across text and imagery. This data affords theoretical insights into
the connection between images and instructional text, and can be used to train
classifiers to support automated discourse analysis. Another important
contribution of this study is that it presents a discourse annotation scheme
for cross-modal data, and establishes that annotations for this scheme can be
procured from non-expert contributors via crowd-sourcing.

Our paper sets the agenda for a range of future research. One obvious example
is to extend the approach to other genres of communication with other
coherence relations, such as the distinctive coherence of images and caption
text [23]. Another is to link coherence relations to the structure of
multimodal discourse. For example, our methods have not yet addressed whether
image–text relations have the same kinds of subordinating or coordinating
roles that comparable relations have in structuring text discourse [24].
Ultimately, of course, we hope to leverage such corpora to build and apply
better models of multimodal communication.

##  7 Acknowledgement

The research presented here is supported by NSF Award IIS-1526723 and through
a fellowship from the Rutgers Discovery Informatics Institute. Thanks to
Gabriel Greenberg, Hristiyan Kourtev and the anonymous reviewers for helpful
comments. We would also like to thank the Mechanical Turk annotators for their
contributions.

## References

  * [1] Steven K Feiner and Kathleen R McKeown.  Automating the generation of coordinated multimedia explanations.  Computer, 24(10):33–41, 1991.
  * [2] M. Alikhani and M. Stone.  Exploring coherence in visual explanations.  In 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pages 272–277, April 2018.
  * [3] Eleni Miltsakaki, Rashmi Prasad, Aravind K. Joshi, and Bonnie L. Webber.  The Penn Discourse Treebank.  In LREC. European Language Resources Association, 2004.
  * [4] Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K. Joshi, and Bonnie L. Webber.  The Penn Discourse TreeBank 2.0.  In LREC. European Language Resources Association, 2008.
  * [5] Heriberto Cuayáhuitl, Simon Keizer, and Oliver Lemon.  Strategic dialogue management via deep reinforcement learning.  In NIPS Workshop on Deep Reinforcement Learning, 2015.  arXiv:1511.08099.
  * [6] J. Hunter, N. Asher, and A. Lascarides.  Integrating non-linguistic events into discourse structure.  In Proceedings of the 11th International Conference on Computational Semantics (IWCS), pages 184–194, London, 2015.
  * [7] Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis.  RecipeQA: A challenge dataset for multimodal comprehension of cooking recipes.  In EMNLP, pages 1358–1368. Association for Computational Linguistics, 2018.
  * [8] Samuel Cumming, Gabriel Greenberg, and Rory Kelly.  Conventions of viewpoint coherence in film.  Philosophers’ Imprint, 17(1):1–29, 2017.
  * [9] Dorit Abusch.  Applying discourse semantics and pragmatics to co-reference in picture sequences.  In Emmanuel Chemla, Vincent Homer, and Grégoire Winterstein, editors, Proceedings of Sinn und Bedeutung 17, pages 9–25, Paris, 2013\.
  * [10] Scott McCloud.  Understanding comics: The invisible art.  William Morrow, 1993.
  * [11] Neil Cohn.  Visual narrative structure.  Cognitive science, 37(3):413–452, 2013.
  * [12] Alex Lascarides and Matthew Stone.  Discourse coherence and gesture interpretation.  Gesture, 9(2):147–180, 2009.
  * [13] Malihe Alikhani and Matthew Stone.  Arrows are the verbs of diagrams.  In Proceedings of the 27th International Conference on Computational Linguistics, pages 3552–3563, 2018.
  * [14] Tuomo Hiippala and Serafina Orekhova.  Enhancing the ai2 diagrams dataset using rhetorical structure theory.  05 2018.
  * [15] Matthew Stone and Una Stojnic.  Meaning and demonstration.  Review of Philosophy and Psychology, 6(1):69–97, 2015.
  * [16] Bonnie Webber.  Genre distinctions for discourse in the penn treebank.  In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, pages 674–682. Association for Computational Linguistics, 2009.
  * [17] Joseph L Fleiss and Jacob Cohen.  The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability.  Educational and psychological measurement, 33(3):613–619, 1973\.
  * [18] Anne Cocos, Aaron Masino, Ting Qian, Ellie Pavlick, and Chris Callison-Burch.  Effectively crowdsourcing radiology report annotations.  In Proceedings of the Sixth International Workshop on Health Text Mining and Information Analysis, pages 109–114, 2015.
  * [19] Mousumi Banerjee, Michelle Capozzoli, Laura McSweeney, and Debajyoti Sinha.  Beyond kappa: A review of interrater agreement measures.  Canadian journal of statistics, 27(1):3–23, 1999.
  * [20] Anthony J Viera, Joanne M Garrett, et al.  Understanding interobserver agreement: the kappa statistic.  Fam Med, 37(5):360–363, 2005.
  * [21] Philippe Schlenker and Emmanuel Chemla.  Gestural agreement.  Natural Language & Linguistic Theory, pages 1–39, 2017.
  * [22] Sida Wang and Christopher D Manning.  Baselines and bigrams: Simple, good sentiment and topic classification.  In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, pages 90–94. Association for Computational Linguistics, 2012.
  * [23] Malihe Alikhani and Matthew Stone.  “caption” as a coherence relation: Evidence and implications.  In Second Workshop on Shortcomings in Vision and Language (SiVL), 2019.
  * [24] Nicholas Asher and Alex Lascarides.  Logics of conversation.  Cambridge University Press, 2003.


# The Fundamental Planes of Black Hole Activity for Radio-Loud and Radio-Quiet
Quasars

Luis Gabriel C. Bariuan,1,2 Bradford Snios,2 Małgosia Sobolewska,2 Aneta
Siemiginowska,2 Daniel A. Schwartz2
1Massachusetts Institute of Technology, Cambridge, MA 02139, USA
2Center for Astrophysics $|$ Harvard & Smithsonian, Cambridge, MA 02138, USA
E-mail<EMAIL_ADDRESS>
(Accepted 2022 April 21. Received 2022 April 1;; in original form 2021
November 4)

###### Abstract

We examine the fundamental plane of black hole activity for correlations with
redshift and radio loudness in both radio-loud and radio-quiet quasar
populations. Sources are compiled from archival data of both radio-loud and
radio-quiet quasars over redshifts $0.1<z<5.0$ to produce a sample of 353
sources with known X-ray, radio, and black hole mass measurements. A
fundamental plane of accretion activity is fit to a sample of radio-loud and
radio-quiet quasars, and we find a dichotomy between radio-loud and radio-
quiet sources. The set of best-fit equations that best describe the two
samples are $\log{L_{R}}=(1.12\pm 0.06)\log{L_{X}}-(0.20\pm
0.07)\log{M}-(5.64\pm 2.99)$ for our radio-loud sample and
$\log{L_{R}}=(0.48\pm 0.06)\log{L_{X}}+(0.50\pm 0.08)\log{M}+(15.26\pm 2.66)$
for our radio-quiet sample. Our results suggest that the average radio-quiet
quasar emission is consistent with advection dominated accretion, while a
combination of jet and disc emission dominates in radio-loud quasars. We
additionally examine redshift trends amongst the radio-loud and radio-quiet
samples, and we observe a redshift dependence for the fundamental plane of
radio-loud quasars. Lastly, we utilize the fundamental plane as a black hole
mass estimation method and determine it useful in studying systems where
standard spectral modeling techniques are not viable.

###### keywords:

Accretion – Active Galaxies – Black Hole Physics – Galaxy Nuclei – High-
Redshift Galaxies

††pubyear: 2022††pagerange: The Fundamental Planes of Black Hole Activity for
Radio-Loud and Radio-Quiet Quasars–References

##  1 Introduction

Black holes are exotic astrophysical objects with masses ranging from stellar
mass up to $10^{10}\,M_{\odot}$, whose influence can be observed due to
accretion processes that produce emissions across a broad range of different
wavelengths. In particular, relativistic jets generated from the black hole
will produce synchrotron radiation which is detectable at radio wavelengths
(Begelman et al., 1984), while adiabatic compression within the inner
accretion disc heats the surrounding material and produces X-ray emission
(Heinz et al., 1998; Stawarz et al., 2008). Black hole luminosities can be as
low as $10^{30}$ – $10^{33}\rm\,erg\,s^{-1}$ in quiescent stellar mass black
holes (e.g., Gallo et al., 2003), and as high as $10^{47}\rm\,erg\,s^{-1}$ in
the most luminous quasars. By deriving correlations between the observables,
black hole properties such accretion rate $\dot{M}$, disc-jet coupling, and
jet models can be extrapolated (e.g., Corbel et al., 2003; Gallo et al.,
2003). Thus, multiwavelength observations of black holes are critical for
studies of their unique physical properties.

The observational study by Merloni et al. (2003) examined physical properties
of low-redshift black holes ($z<0.3$) over broad mass and luminosity ranges in
X-ray and radio wavelengths for any evidence of correlations between those
parameters. By studying $116$ black holes with known masses and luminosities,
a “fundamental plane of black hole accretion activity" characterized by
$\log{L_{R}}=\xi_{RX}\log{L_{X}}+\xi_{RM}\log{M}+b_{R}$ was discovered. The
result demonstrated that the radio luminosity of a black hole has a direct
dependence on its mass and X-ray luminosity, regardless of local environmental
effects. Furthermore, the fundamental plane provides an empirical method of
characterizing black hole behavior and properties that is scale invariant
(Heinz & Sunyaev, 2003; Merloni et al., 2003). More recent works refined the
fundamental plane parameters through the use of more precise radio and X-ray
luminosities together with known masses obtained from direct dynamical
measurements (Körding et al., 2006; Gültekin et al., 2009). The tighter
constraints led to a relation between the luminosities and mass that was
consistent with previous fundamental plane measurements but with $33\%$ less
scatter, further reinforcing the presence of a fundamental accretion process
that is consistent across black holes in our nearby Universe.

Despite the merit of previous results, the majority of fundamental plane
studies prioritized black holes at low redshifts ($z<0.5$) since mass
measurements and luminosities are well-constrained for those sources (e.g.,
Merloni et al., 2003; Falcke et al., 2004; Körding et al., 2006; Gültekin et
al., 2009). Increased cosmic densities within the early Universe may have
driven periods of rapid formation due to increased merger rates and gas
consumption, giving rise to different accretion physics than what is presently
observed (Volonteri, 2012). In more recent timescales, the Universe has since
undergone cosmological expansion which has also affected black hole formation
and possibly accretion physics (Merloni et al., 2003; Merloni, 2004; Micic et
al., 2007). Thus, the validity of the fundamental plane is currently not well
understood across a broad redshift range, resulting in a limited understanding
of how black hole accretion evolved over large timescale.

In addition to an unknown redshift dependence, several fundamental plane
studies utilized a combination of radio-loud and radio-quiet sources in order
to increase sample sizes and improve statistical errors (e.g., Merloni et al.,
2003; Falcke et al., 2004). However, radio loudness has been shown to
correlate with jet outflows and accretion activity in a black hole system
(Sikora et al., 2007; Baloković et al., 2012). Thus, it is probable that the
initial sample of black hole sources utilized in a fundamental plane study
will significantly impact the best-fit results. Examination of a sample with a
broad range of radio loudness is therefore important in understanding if a
singular fundamental plane can be derived for all black holes, or if there is
evidence that the fundamental plane evolves with radio loudness.

Motivated by these facts, we explored the observable properties of black holes
across a broad redshift range in order to develop a fundamental plane of black
hole activity. This derived relationship may then be compared with the
relationship found amongst low-redshift sources to determine the presence, if
any, of a redshift dependence in the fundamental plane. We additionally
assemble a sample of radio-loud and radio-quiet quasars and examine
differences between their respective fundamental plane models. This work
complements recent studies of quasar structure and spectra performed across
different wavelengths (e.g., Just et al., 2007; Lusso & Risaliti, 2016; Vito
et al., 2019; Snios et al., 2020), by providing insights into the properties
of black holes that formed at the early epochs of the Universe as well as the
evolution of their accretion properties.

The structure of the paper is as follows. In § 2, we describe the sources we
obtained from independent surveys. In § 3, we describe the fitting method for
our fundamental plane analysis and present our best-fit results. § 4 discusses
observed trends in both redshift and radio loudness in our fundamental plane
results as well as comment on utilizing the fundamental plane as method of
estimating black hole mass. Our concluding remarks are discussed in § 5.

For this paper, we adopted the cosmological parameters
$H_{0}=70\rm\,km\,s^{-1}\,Mpc^{-1}$, $\Omega_{\Lambda}=0.7$, and
$\Omega_{M}=0.3$ (Hinshaw et al., 2013). Estimated uncertainties are reported
at a 1$\sigma$ confidence level, unless otherwise specified.

Table 1: Properties of the Quasar Sample

Name | $z$ | log($R)$ | log($L_{R}$) |  $\Delta$log($L_{R}$) | Ref | log($L_{X}$) |  $\Delta$log($L_{X}$) | Ref | log($M$) |  $\Delta$log($M$) | Ref | log($L_{X}$/$L_{\rm Edd}$)
---|---|---|---|---|---|---|---|---|---|---|---|---
150436$-$024404 | 0.200 | 1.024 | 39.169 | 0.065 | 5 | 42.16 | 0.09 | 4 | 7.44 | 0.86 | 6 | $-3.39$
140051$+$025905 | 0.256 | 1.023 | 39.407 | 0.065 | 5 | 42.12 | 0.09 | 4 | 8.37 | 0.42 | 6 | $-4.36$
162901$+$400759 | 0.272 | 1.430 | 40.777 | 0.065 | 4 | 44.44 | 0.09 | 4 | 7.36 | 0.23 | 6 | $-1.03$
172255$+$320307 | 0.275 | 1.031 | 39.484 | 0.065 | 5 | 43.77 | 0.09 | 4 | 6.75 | 1.48 | 6 | $-1.09$
103336$+$573106 | 0.295 | 0.981 | 39.538 | 0.065 | 5 | 42.30 | 0.09 | 4 | 8.82 | 0.30 | 6 | $-4.63$

The first five sources from our compiled catalog of quasars are shown. The
complete table is available online in machine-readable format.

Figure 1: Distributions of the radio-loud quasar (blue) and radio-quiet quasar
(orange) samples used in our study of the fundamental plane for black hole
activity. The histograms illustrate distributions across several different
physical properties, including: (a) redshift $z$, (b) 2–10 keV rest-frame
X-ray luminosity log($L_{X}$), (c) spectroscopically measured black hole mass
log($M$), (d) 5 GHz rest-frame radio luminosity log($L_{R}$), (e) the ratio of
X-ray luminosity to Eddington luminosity log($L_{x}/L_{\text{Edd}}$), and (f)
radio loudness log($R$).

##  2 Quasar Sample

Previous investigations of quasars have indicated that radio loudness is
inversely dependent to the accretion rate of the black hole system (Sikora et
al., 2007; Baloković et al., 2012). It is therefore plausible that the derived
fundamental plane of black hole activity for a sample will be impacted by the
radio loudness distribution of its sources. We therefore require samples of
both radio-loud and radio-quiet quasars with comparable physical properties in
order to investigate dependencies on radio loudness in our fundamental plane
analysis.

For our study, we define radio-loudness according to the criterion described
by Kellermann et al. (1989) where radio-loud objects are defined as any source
with a radio loudness111Radio loudness is defined as $R=f_{\rm 5\,GHz}/f_{\rm
4400\,\textup{\AA}}$, where $f_{\rm 5\,GHz}$ and $f_{\rm 4400\,\textup{\AA}}$
correspond to the rest-frame flux density at 5 GHz and 4400 Å, respectively.
parameter $R>30$. Furthermore, we require that each sample must have a
comparable sample size to previous works on the fundamental plane ($>$ 100
sources, Merloni et al., 2003; Falcke et al., 2004) to ensure comparable
statistical precision. We therefore searched the literature for archival
datasets that would supply us with a sufficient number of quasars spanning our
desired physical parameter space.

###  2.1 Radio-Loud Quasars

For our radio-loud sample of quasars, we gathered data across several
different works that contained complete descriptions of radio and X-ray
luminosities. Since a primary goal of our analysis is to also investigate
redshift dependence in accretion activity, we included samples spanning a
broad redshift range. Sources with X-ray and radio luminosity measurements
were selected from the radio-loud quasar catalogs of Zhu et al. (2019), Snios
et al. (2020), Snios et al. (2021), and Zhu et al. (2021), where the
luminosity derivation methods utilized in those works are briefly summarized
below.

Radio fluxes were taken from the VLA-FIRST radio survey (Becker et al., 1995),
or from independent VLA observations when FIRST data of the source was not
available. A region was defined to isolate the AGN core in the radio
observation, and the radio flux was measured from the region. The rest-frame 5
GHz radio luminosity of each source was extrapolated from the flux
measurements assuming a spectral index of $\alpha_{R}=-0.5$. Radio loudness
parameters for the quasars from Zhu et al. (2019), Snios et al. (2020), and
Zhu et al. (2021) were provided from the original sources of the data. For the
Snios et al. (2021) sample, radio loudness values were derived by
extrapolating the provided $f_{2500\,\text{\AA}}$ values to 4400 Å assuming a
spectral index of $\alpha_{o}=-0.5$.

At the 5″ resolution of FIRST, the survey resolved radio features up to a
separation distance of 31 kpc at $z=5$, the maximum redshift of our quasar
sample. We note that radio emission in nearby sources studied in the context
of the fundamental plane reflects activity of the nucleus and generally does
not include the extended radio emission from large scale jets (e.g., Merloni
et al., 2003; Falcke et al., 2004). Since a priority of our sample is to
investigate high-redshift sources, we anticipate a possible bias in our sample
due to the inclusion of extended radio emission in our measurements. We
explore such possible bias in the discussion of our redshift analysis (§ 4.2).

X-ray luminosities were derived using observations from Chandra, XMM-Newton,
and Swift. A region was defined around the AGN core, and an emission spectrum
was extracted from the source. Extended X-ray features potentially present in
the regions may be ignored as they generally provide a negligible contribution
to the total quasar luminosity (e.g., Marshall et al., 2018; Worrall et al.,
2020). Furthermore, the majority of quasars in our sample were observed in
short exposures, which reduces the signal-to-noise of diffuse X-ray features
and subsequently limits their impact on the extracted spectral profile of the
AGN. The observed X-ray flux was measured from a spectral model of the 0.5–7.0
keV energy band, and the rest-frame 2–10 keV luminosity was extrapolated from
the best-fit model. We specifically selected quasars with radio and X-ray flux
measurements and omitted any sources with upper limit estimates. Our quasar
catalog was cross-referenced with the SDSS-DR14 catalog (Rakshit et al.,
2020), where sources with confirmed spectroscopic mass measurements via SDSS
spectra were included in our fundamental plane analysis.

Ultimately, our radio-loud quasar sample included 11 sources from Zhu et al.
(2019), 6 sources from Snios et al. (2020), 12 sources from Snios et al.
(2021), and 196 sources from Zhu et al. (2021). Radio luminosities for the
sample ranged between $10^{40}$ – $10^{46}$ erg $\text{s}^{-1}$, while X-ray
luminosities ranged between $10^{43}$ – $10^{47}$ erg $\text{s}^{-1}$. The
masses of the sources ranged between $10^{6}$ – $10^{11}\,M_{\odot}$ with an
average uncertainty of $0.342\,M$. Consistent with Merloni et al. (2003), we
also calculated for each source the ratio of the X-ray luminosity to the
Eddington luminosity, which may be used as a proxy for the Eddington ratio. We
defined the ratio as $L_{X}/L_{\rm Edd}$ where $L_{\rm Edd}=1.3\times
10^{38}M/M_{\odot}$. The range of log($L_{X}/L_{\rm Edd}$) for our radio-loud
sample is [$-3.31$,$-0.31$] with a mean of $-1.92$. In total, we obtained
$225$ unique radio-loud quasars at $0.1<z<5.0$ with complete measurements over
a broad range of observables, which is sufficient for our modeling analysis.
We note that our sample includes quasars with known X-ray luminosity, radio
luminosity, and mass. We found that inclusion of sources with luminosity upper
limits added only 3 sources to our total, so we discarded upper limit sources
from our sample for consistency with previous studies. Details on our radio-
loud catalog are provided in Table 1, and Figure 1 shows the source
distribution of our radio-loud sample across several different physical
properties.

###  2.2 Radio-Quiet Quasars

Table 2: Best-fit Parameters of the Fundamental Plane of Black Hole Activity
Analysis

| Number of Sources | $\xi_{X}$ | $\xi_{M}$ | $b$ | $\sigma^{2}_{\rm res}$ | $R_{\rm adj}^{2}$ | $\sigma_{R}$
---|---|---|---|---|---|---|---
Total Dataset | $353$ | $1.44\pm 0.08$ | $0.16\pm 0.10$ | $-23.85\pm 3.49$ | $54.97$ | $0.5207$ | $0.85$
Radio Loudness Dependence |  |  |  |  |  |  |
Radio-Loud Quasars | $225$ | $1.12\pm 0.06$ | $-0.20\pm 0.07$ | $-5.64\pm 2.99$ | $20.09$ | $0.4629$ | $0.62$
Radio-Quiet Quasars | $128$ | $0.48\pm 0.06$ | $0.50\pm 0.08$ | $15.26\pm 2.66$ | $17.77$ | $0.5866$ | $0.39$
Redshift-dependence (Radio-Loud Quasars) |  |  |  |  |  |  |
$z<1.5$ | $87$ | $1.32\pm 0.15$ | $-0.06\pm 0.13$ | $-16.20\pm 6.59$ | $18.58$ | $0.3390$ | $0.55$
$1.5<z<3$ | $94$ | $1.06\pm 0.13$ | $-0.17\pm 0.10$ | $-3.27\pm 6.00$ | $21.66$ | $0.3564$ | $0.52$
$z>3$ | $44$ | $0.63\pm 0.15$ | $-0.27\pm 0.15$ | $17.55\pm 6.64$ | $22.63$ | $0.2779$ | $0.48$
Redshift-dependence (Radio-Quiet Quasars) |  |  |  |  |  |  |
$z<1.5$ | $66$ | $0.38\pm 0.08$ | $0.40\pm 0.11$ | $20.20\pm 3.32$ | $22.68$ | $0.5035$ | $0.42$
$z>1.5$ | $62$ | $0.18\pm 0.09$ | $0.22\pm 0.08$ | $31.49\pm 4.12$ | $11.17$ | $0.4460$ | $0.26$

note: The reported parameters for the fundamental plane best-fits correspond
to the X-ray luminosity slope $\xi_{X}$, the black hole mass slope $\xi_{M}$,
the y-intercept $b$, the best-fit residual variance $\sigma^{2}_{\rm res}$,
the adjusted R-squared of the best-fit model $R_{\rm adj}^{2}$, and the
scatter of the log$(L_{R})$ parameter relative to the predicted value from the
fundamental plane model $\sigma_{R}$.

Radio-quiet quasars comprise the majority of the known quasar population and
are therefore commonly utilized in fundamental plane analyses (e.g., Merloni
et al., 2003; Falcke et al., 2004; Körding et al., 2006; Gültekin et al.,
2009; Plotkin et al., 2012). To obtain our radio-quiet quasar sample, we
cross-referenced the radio-quiet quasar catalog from Timlin et al. (2020) with
the X-ray fluxes of Zhu et al. (2021). Consistent with our sample selection
method for the radio-loud catalog, we selected radio-quiet quasars with known
radio and X-ray flux measurements. Radio and X-ray luminosities were provided
from Timlin et al. (2020) and Zhu et al. (2021), where the derivation methods
are the same as those described in § 2.1. Our quasar catalog was cross-
referenced with the SDSS-DR14 catalog for spectroscopic mass measurements
(Rakshit et al., 2020). The radio loudness parameters for our sample were
provided from Timlin et al. (2020).

In total, we found $128$ radio-quiet quasars with complete measurements which
we included in our fundamental plane analysis. Similar to the radio-loud
sample, inclusion of radio-quiet quasars with luminosity upper limits added a
negligible amount of sources to the catalog, 9 sources in total. Our radio-
quiet sample of $128$ sources therefore omits sources with luminosity upper
limits. We note that although radio-quiet quasars comprise the majority of the
known quasar population, our radio-loud sample is larger than our radio-quiet
sample because relatively few radio-quiet sources have radio fluxes above
current detector sensitivity limits. The sources span a redshift range of
$0.1<z<4.0$, X-ray luminosities that ranged between $10^{42}$ – $10^{46}$ erg
$\text{s}^{-1}$, radio luminosities that ranged between $10^{39}$ – $10^{43}$
erg $\text{s}^{-1}$, and masses that ranged between $10^{6}$ –
$10^{10}\,M_{\odot}$ with an average uncertainty of $0.341\,M$. The ratio
log($L_{X}/L_{\rm Edd}$) was also calculated for our radio-quiet sample,
finding a range of [$-4.87$,$-1.03$] with a mean of $-2.43$. The radio-quiet
sources possess similar physical parameters as our radio-loud sources, and
both samples are comparable in size. This ensures that radio loudness is the
primary difference between our two samples. Details on our radio-quiet catalog
are provided in Table 1, Figure 1 shows the source distribution of our radio-
quiet sample across different physical properties.

##  3 Modeling the Fundamental Activity Plane

With our accumulated dataset from § 2, we constructed a fundamental plane of
black hole activity. Uncertainties for sources from Snios et al. (2020); Snios
et al. (2021) were provided in their work. Since sources found within Zhu et
al. (2019); Timlin et al. (2020); Zhu et al. (2021) did not include quoted
errors, we adopted a $20\%$ uncertainty for their X-ray luminosities. For all
of our radio luminosities, we adopted a $15\%$ uncertainty. We note that our
adopted errors exceed those from similar studies (e.g., Gültekin et al., 2009;
Gültekin et al., 2019), which reassures us that our error estimates are
conservative. For the mass measurements obtained from Rakshit et al. (2020),
we utilized the quoted uncertainties. Due to our model fitting method
(described below), we adopted conservative symmetric uncertainties for all
measurements.

Consistent with Merloni et al. (2003), we defined the fundamental plane of
activity as

$\log{L_{R}}=\xi_{X}\log{L_{X}}+\xi_{M}\log{M}+b,$ (1)

where $L_{R}$ is the rest-frame 5 GHz luminosity, $L_{X}$ is the rest-frame
2–10 keV luminosity, and $M$ is the black hole mass. The dataset was fit using
the multivariate orthogonal distance regression (ODR) method to estimate the
best-fit regression coefficients (Boggs & Rogers, 1990). ODR is a fitting
scheme that modifies the ordinary least squares by accounting for
uncertainties within both the dependent and independent variables, and the
data is minimized by the sum of squared perpendicular distances from the data
points to the best-fit line weighted by the uncertainty. Given the presence of
measurement uncertainties for our three input parameters, ODR provided a more
accurate best-fit than the commonly used ordinary least square regression
method.

Despite the robustness of the ODR method, a single run is not guaranteed to
produce well-constrained best-fit coefficients with our data set as the
maximally strict convergence conditions of the sums of squares and relative
changes in the estimated parameters yield fits that are not necessarily
located at the global minimum. We therefore ran ODR with $10^{4}$ iterations,
which produced a symmetric distribution of fitting coefficients. We used the
mean of the resultant parameters as our best-fit coefficients, while their
errors were calculated from the mean covariance matrices. We additionally
implemented randomized initial conditions for each run to reduce input bias on
the best-fit parameters.

Figure 2: An edge-on view of the fundamental planes of black hole activity,
where the sample is separated into radio-loud and radio-quiet quasars. The
x-axis is defined in accordance with the best-fit parameters to the total
quasar dataset. Best-fit lines for the radio-quiet (blue) and radio-loud (red)
are shown. The data demonstrates a clear dichotomy between the fundamental
plane best-fit for the two populations.

For our analysis, we utilized our accumulated samples to construct a
fundamental plane of black hole activity. Performing the ODR fit on our radio-
loud and radio-quiet sources, we found a fundamental plane best-fit of
$\log{L_{R}}=(1.12\pm 0.06)\log{L_{X}}+(-0.20\pm 0.07)\log{M}-(5.64\pm 2.99)$
for our radio-loud sample and $\log{L_{R}}=(0.48\pm 0.06)\log{L_{X}}+(0.50\pm
0.08)\log{M}+(15.26\pm 2.66)$ for our radio-quiet sample. Figure 2 shows the
result of fitting across the two samples, and Table 2 displays the best-fit
coefficients. For completeness, a best-fit was obtained for the entire dataset
of radio-loud and radio-quiet quasars, which is also provided in Table 2. The
residual variance $\sigma^{2}_{\rm res}$ and the adjusted coefficient of
determination222We define the coefficient of determination as
$R^{2}=\frac{r_{xz}^{2}+r_{yz}^{2}-2r_{xy}r_{xz}r_{yz}}{1-r_{xy}^{2}}$, where
$r_{xy}$, $r_{xz}$, and $r_{yz}$ are the correlation coefficients. Here $x$
and $y$ are the independent variables of the fit, while $z$ is the dependent
variable. To account for the number of model parameters in the fit, we
calculate the adjusted coefficient of determination $R_{\rm
adj}^{2}=1-\frac{(1-R^{2})(N-1)}{N-p-1}$, where $N$ is the total sample size
and $p$ is the number of independent variables. $R_{\rm adj}^{2}$ were
measured for each best-fit model, and the results are shown in Table 2. A
positive correlation is observed for each best-fit, while the fit statistics
demonstrate a preference towards utilizing separate models for the radio-loud
and radio-quiet samples.

We compared our best-fit parameters of each sample to the result from Merloni
et al. (2003). The measured scatters of the fit in the log$(L_{R})$ parameter
relative for radio-loud and radio-quiet quasar samples are $\sigma_{R}=0.62$
and $\sigma_{R}=0.39$, which is at least $68\%$ less than the scatter found by
Merloni et al. (2003) and at least $75\%$ less than the scatter found by
Gültekin et al. (2009), though this difference may be attributable to our
larger adopted uncertainties than those from similar studies. Overall, we
found that the radio-quiet and Merloni et al. (2003) best-fit parameters agree
with one another within $1\sigma$ despite different data selection criteria
(e.g., redshift and mass). This agreement is unsurprising given that the
sample from Merloni et al. (2003) was primarily comprised of radio-quiet
sources. In comparison, the radio-loud best-fit parameters each diverge from
the Merloni et al. (2003) result by $>3\sigma$. Our results suggest that the
dichotomy between radio-loud and radio-quiet quasars is also found within the
fundamental plane, hence, two fundamental planes are better descriptors of
quasar populations. In § 4, we discuss the physical impacts of trends observed
within the fundamental planes.

##  4 Discussion

Figure 3: A comparison of the correlation coefficients $\xi_{\rm X}$ and
$\xi_{\rm M}$ for the fundamental plane best-fits of the radio-loud and radio-
quiet samples (left) as well as for redshift subsamples (right). Each figure
also shows the theoretically predicted correlation coefficients from Merloni
et al. (2003), where circles, diamonds, and squares correspond to Advection
Dominated Accretion Flows (ADAF), jet, and disc models respectively. Empty
symbols are for a radio spectral index $\alpha_{\rm R}=0$ and filled symbols
for $\alpha_{\rm R}=0.5$. The dotted lines represent different possible
correlation coefficients due to variations in radio emission.

###  4.1 Radio-Loud Dependence on Fundamental Plane

The fundamental plane comparisons performed in § 3 reveal several key
properties within our quasar samples. First, the fundamental plane from
Merloni et al. (2003) with their predominantly radio-quiet heterogeneous
sample is consistent with our more homogeneous radio-quiet quasar sample.
Second, the Merloni et al. (2003) fundamental plane underpredicts the radio
luminosity of the radio-loud quasar sample, resulting in a divergence between
the two best-fits over our examined physical parameter space. Radio loudness
has previously been shown to impact accretion rates (Sikora et al., 2007;
Baloković et al., 2012), so it follows that it should also impact the
fundamental plane of black hole activity. Furthermore, studies of radio
loudness could help constrain jet formation, acceleration, and collimation
(Sikora et al., 2007). Motivated by our observation and its physical
significance, we explored the fundamental plane within the context of radio
loudness.

The fundamental plane best-fit parameters for the two radio loudness samples
are shown in Table 2, and Figure 2 displays our results. Comparing the two
best-fits, we find that their difference exceeds the measured scatter
$\sigma_{R}$ of 0.62 and 0.39 for the radio-loud and radio-quiet fits,
respectively. Thus, we observe a dichotomy between the black hole activity of
radio-loud and radio-quiet sources. The observed separation between the radio-
loud and radio-quiet samples is consistent with previous studies that have
demonstrated a bimodal distribution of radio loudness amongst the quasar
population (e.g., Kellermann et al., 1989; Miller et al., 1990; Ivezić et al.,
2004; White et al., 2007; Zamfir et al., 2008; Baloković et al., 2012).

The measured correlation coefficients from our fundamental plane best-fits
provide insight into the different emission mechanisms within the quasar
samples, where the ratio of these observables can correspond to predicted
values from different emission models (Merloni et al., 2003; Plotkin et al.,
2012). We therefore plotted the correlation coefficients from our radio-loud
and radio-quiet best-fits, which is shown in Figure 3, left. Joint errors for
the $\xi_{x}$–$\xi_{M}$ parameters of each best-fit function were calculated
from their respective mean covariance matrix, and found to be consistent with
their marginalized uncertainties for both the radio-loud and radio-quiet best-
fits. The resulting figure further illustrates the dichotomy between our
radio-loud and radio-quiet quasar samples as the two best-fits are separated
by $>$ $3\sigma$ for both $\xi_{\rm X}$ and $\xi_{\rm M}$ parameters.
Furthermore, these results indicate that the dominant emission mechanism
differs between the two samples.

To assess the probable emission mechanism for our quasar samples, we compared
our correlation coefficients to those predicted by different emission models
from Merloni et al. (2003). The predicted coefficients are also overlaid on
Figure 3. We stress that the plotted model parameters represent a small
subsample of the possible quasar emission models, so any emission mechanism
predictions and subsequent comparisons should be regarded as approximate.
Based on the different model parameterizations from Merloni et al. (2003) that
were tested, we found that the radio-quiet sample, on average, favors the
advection dominated accretion flows (ADAF) instead of the standard jet or disc
models, which agrees with previous fundamental plane studies for radio-quiet
black holes (e.g., Merloni et al., 2003; Gültekin et al., 2009; Plotkin et
al., 2012). The distribution of $L_{X}/L_{\rm Edd}$ values for the radio-quiet
sample, shown in Figure 1, is also broadly consistent with radiatively
inefficient emission based on the model results of Merloni et al. (2003). In
comparison, the radio-loud quasar sample is, on average, more consistent with
the standard jet emission model, or a disc model, or a combination of the two.
Our finding agrees with previous quasar studies where jet contributions are
invoked to explain the radio-loud phenomenon (Worrall et al., 1987; Kellermann
et al., 1989; Urry & Padovani, 1995).

Overall, our results demonstrate that two fundamental planes are necessary to
accurately reflect the different physical mechanisms that govern radio-loud
and radio-quiet quasars. These two separate models will help better constrain
and predict physical properties of quasar populations such as the nature of
jets, accretion processes, and their local environments.

Figure 4: An edge-on view of the fundamental planes of black hole activity
across different redshift bins for the radio-loud quasars (left) and radio-
quiet quasars (right). The best-fit lines for each redshift bin (colored) and
the total sample (black) are plotted. The fundamental plane best-fit parameter
are found to be broadly consistent across redshifts for both the radio-loud
and radio-quiet samples.

###  4.2 Redshift Dependence on Fundamental Plane

Previous works, due to observational constraints, focused on the fundamental
plane within the context of low-redshift sources $z<0.5$ (e.g., Merloni et
al., 2003; Falcke et al., 2004; Körding et al., 2006; Gültekin et al., 2009).
As a result, the fundamental plane at high-redshifts is not well-understood.
Furthermore, whether the fundamental plane model is constant over the age of
the Universe is not well-constrained. We therefore investigated the
fundamental plane model across varying redshift.

Due to the observed differences between the population of our radio-loud and
radio-quiet sample, we analyzed redshift dependencies of each sample
separately. We binned the radio-loud sources across three redshift ranges
($z<1.5$, $1.5<z<3.0$, and $z>3$), where each redshift bin was selected to
ensure similar sample statistics for our analysis. In contrast, we binned the
radio-quiet sources across two redshift ranges ($z<1.5$, $z>1.5$), due to its
smaller sample size. The best-fit results from these additional tests are
shown in Table 2. We note that the measured scatter $\sigma_{R}$ demonstrates
consistency between the different models over the parameter space investigated
with our sample (Table 2). Figure 4, left displays the fundamental plane best-
fit for the radio-loud quasars, while Figure 4, right displays the best-fit
for the radio-quiet quasars.

From our analysis, the radio-quiet best-fits demonstrate broad consistency
with one another across the examined redshift range, where the correlation
coefficients are in agreement to within 2$\sigma$ in all scenarios.
Additionally, Figure 3, right illustrates the consistency between the two
radio-quiet subsamples while also indicating that each subsample is most
consistent with ADAF from the emission models tested. Our result agrees with
optical–X-ray studies of quasars that have shown constant trends in power
output and accretion rates within radio-quiet quasars up to redshifts $z\sim
6$ (Just et al., 2007; Nanni et al., 2017; Vito et al., 2019). Ultimately, our
current data which spans a redshift range of $0.1<z<5.0$ suggests that the
radio-quiet fundamental plane model is independent of redshift. Hence, low-
redshift fundamental plane models from previous works (e.g., Merloni et al.,
2003; Falcke et al., 2004; Körding et al., 2006; Gültekin et al., 2009;
Plotkin et al., 2012) may be valid for radio-quiet black holes extending all
the way to $9\%$ of the present age of the Universe.

In contrast to the radio-quiet sample, examination of the radio-loud quasar
subsamples indicates a decreasing X-ray luminosity dependence for increasing
redshift at a $>$ $3\sigma$ significance. This result is also illustrated in
Figure 3, right, where the radio-loud samples clearly demonstrate a decreasing
X-ray luminosity dependence versus redshift while mass dependence remains
constant to within $1\sigma$. As noted in § 2, the radio luminosity
measurements of the high-redshift radio-loud quasar cores may include
contributions from extended features due to instrument resolution limits,
which would elevate $L_{R}$ with increasing redshift. Comparatively, extended
emission, on average, accounts for $<2\%$ the total X-ray luminosity of a
quasar (e.g., Marshall et al., 2018; Worrall et al., 2020; Snios et al.,
2021), which is below other sources of measurement error. If extended emission
contamination is present in our radio-loud sample, the $L_{R}$ values of the
high-redshift quasars should be regarded as upper limits on the true AGN core
luminosity. Reducing $L_{R}$ to account for such effects would also reduce the
measured slope of the fundamental plane for sources at $z>3$, increasing the
statistical significance of the inverse relationship between X-ray luminosity
and redshift for the fundamental plane. Thus, the presence of contamination
would serve to enhance the observed redshift trend for the fundamental plane
of the radio-loud quasar population.

Comparing the samples to different emission models from Merloni et al. (2003),
we found that the low-redshift radio-loud sample agrees well with the disc
model. However, the high-redshift sample is more consistent with a jet model,
indicating a possible evolution of the primary emission mechanism for radio-
loud quasars. Such an effect would be consistent with the recent theory that
inverse Compton upscattering of the cosmic microwave background radiation
(IC/CMB) will increase the observed jet intensity for quasars at $z\gtrsim 3$
as the energy density of the CMB exceeds the magnetic energy density for the
quasar jets (Schwartz et al., 2020; Hodges-Kluck et al., 2021).

We stress that the observed redshift dependence in the radio-loud sample is
indicative given the limited sample size and coarse redshift binning utilized
in our analysis, and a greater sample size across all redshifts is necessary
to accurately constrain the existence of a redshift dependence on the primary
emission mechanism. Nonetheless, our analysis does suggest a redshift
dependence for the fundamental plane of radio-loud quasars.

###  4.3 Using the Fundamental Plane as a Mass Estimator

Table 3:  Best-Fit Parameters of Black Hole Mass Fundamental Plane Analysis

| Number of Sources | $\xi_{MX}$ | $\xi_{MR}$ | $b_{M}$ | $\sigma^{2}_{\rm res}$ | $R_{\rm adj}^{2}$ | $\sigma_{M}$
---|---|---|---|---|---|---|---
Radio-Loud Quasars | 225 | $0.63\pm 0.05$ | $-0.21\pm 0.04$ | $-10.47\pm 2.09$ | $15.56$ | $0.1474$ | $0.43$
Radio-Quiet Quasars | 128 | $0.08\pm 0.07$ | $0.41\pm 0.06$ | $-11.42\pm 3.61$ | $32.00$ | $0.3281$ | $0.45$

note: The reported parameters for the fundamental plane best-fits correspond
to the X-ray luminosity slope $\xi_{MX}$, the radio luminosity slope
$\xi_{MR}$, the y-intercept $b_{M}$, the best-fit residual variance
$\sigma^{2}_{\rm res}$, the adjusted R-squared of the best-fit model $R_{\rm
adj}^{2}$, and the scatter of the log$(L_{R})$ parameter relative to the
predicted value from the fundamental plane models $\sigma_{M}$.

Figure 5: An edge-on view of the fundamental planes of black hole activity
where $\log{M_{BH}}$ is the predicted variable, where the sample is separated
into radio-loud (left) and radio-quiet quasars (right). Best-fit lines for the
radio-quiet and radio-loud are shown. The derived mass measurements are found
to be consistent with spectroscopic mass estimation techniques for black
holes, where the accuracy of the fundamental plane mass estimates is
comparable to those obtained from spectroscopic mass measurements of high-
redshift quasars.

Historically, fundamental plane studies have focused on investigating the
accretion properties of black holes across broad mass and luminosity ranges.
However, the fundamental plane may also be used as an estimator of black hole
masses due to the scale-invariant nature of the model (e,g., Merloni et al.,
2003; Gültekin et al., 2019). Motivated by this fact, we examined the
fundamental plane in the context of mass prediction for our quasar sample.

Using the ODR fitting method described in § 3, we defined our model as the
expression

$\log{M}=\xi_{MX}\log{L_{X}}+\xi_{MR}\log{L_{R}}+b_{M}.$ (2)

Due to the observed dichotomy between radio-loud and radio-quiet quasars
described in § 4.1, we fit the model to each sub-sample separately. The
results for all our fits are shown in Table 3, and Figure 5 displays the best-
fit results of our analysis. The results indicate a dichotomy between the
radio-loud and radio-quiet sample when fitting the fundamental plane defined
in Equation 2, where the best-fit slope parameters $\xi_{MX}$ and $\xi_{MR}$
differed by $>3\sigma$ between the two best-fits. The measured scatter of the
log$(M)$ parameter between the measured and predicted values $\sigma_{M}$ are
determined to be 0.43 and 0.45 for the radio-loud and radio-quiet samples,
respectively. The scatters are consistent between the two datasets, and both
samples cover a comparable physical parameter range. Altogether, we find that
the fundamental plane model for mass measurements is dependent on radio
loudness.

To assess the accuracy of the fundamental plane as a black hole mass
predictor, we compared the predicted masses determined by our fundamental
plane fits with the spectroscopically measured values from the SDSS-DR14
emission line analysis compiled by Rakshit et al. (2020). We therefore
calculated the root mean square error333We define the root-mean square error
as $[\sum_{i}^{N}(O_{i}-P_{i})^{2}/N]^{1/2}$, where $O$ is the measured value
from the fundamental mass plane, $P$ is the mass value from Rakshit et al.
(2020), and $N$ is the sample size. (RMSE), also known as the standard
deviation of the residuals, between the predicted black hole mass from the
fundamental plane model and the spectrum-measured mass from Rakshit et al.
(2020). We found that the radio-loud and radio-quiet quasars have an RMSE of
$0.43$ log($M_{\odot})$ (5.0%) and $0.45$ log$(M_{\odot})$ (5.5%),
respectively. Our method of black hole mass estimation is broadly consistent
with previous studies that utilized the fundamental plane as a mass estimator
(e.g., Gültekin et al., 2019), despite our study focusing on high-mass quasars
ranging between $10^{6}$ –$10^{10}M_{\odot}$ while previous works prioritized
low-mass, low-redshift sources.

The accuracy of black hole mass estimates from spectroscopic modeling is known
to worsen with increasing redshift due to a reduction in the available
emission lines in the optical band as well as a greater reliance on low
signal-to-noise emission lines, such as Mg II and C IV (Shen, 2013). We
theorize that the mass estimates from the fundamental plane would not suffer
from a similar degradation in accuracy with increasing redshift given the
comparable radio and X-ray flux errors across our sample. To test this theory,
we divided our total quasar sample into low-redshift ($z<3$) and high-redshift
($z>3$) subsamples. For the fundamental plane mass measurements, we found an
average RMSE of 5.1% for quasars at $z<3$ and 5.4% for quasars at $z>3$. In
contrast, the spectroscopic mass measurements taken from Rakshit et al. (2020)
have an average error percentage of 1.6% for quasars at $z<3$ and 2.5% for
quasars at $z>3$.

Given our selection of conservative errors for the radio and X-ray fluxes (§
3), the precision of the fundamental plane method could improve if more
accurate uncertainties are utilized. Nonetheless, our results demonstrate that
the accuracy of the fundamental plane mass model is predominantly redshift
invariant, while the accuracy of spectrum-measured masses worsens with
redshift. Thus, our derived fundamental plane for black hole mass may prove
useful in studying high-redshift quasars as well as situations where more
standard mass estimation techniques, such as spectral modeling, are not
possible.

##  5 Conclusion

We examined the fundamental plane of black hole activity for correlations with
redshift and radio loudness in both radio-loud and radio-quiet quasar
populations. Utilizing archival observations, we compiled quasar data that
varied across radio loudness, mass, and redshift to obtain a sample of 353
sources. We constructed a fundamental plane of black hole activity, and our
best-fits are $\log{L_{R}}=(1.12\pm 0.06)\log{L_{X}}-(0.20\pm
0.07)\log{M}-(5.64\pm 2.99)$ for our radio-loud sample and
$\log{L_{R}}=(0.48\pm 0.06)\log{L_{X}}+(0.50\pm 0.08)\log{M}+(15.26\pm 2.66)$
for our radio-quiet sample.

We examined the impact of radio loudness on the fundamental plane best-fit for
within our quasar sample. Our results demonstrate that two different
fundamental planes are required to accurately model the dichotomy of radio-
loud and radio-quiet quasar populations. Thus, we concluded that the radio-
loud and radio-quiet quasar populations are governed by different fundamental
plane models. Comparing the best-fit correlation coefficients to values
predicted from different emission models, we found that a possible emission
mechanism for the radio-quiet sample is advection dominated accretion flows
while the radio-loud sample is likely a combination of jet and disc emission.

We also explored the dependencies of redshift on the fundamental plane model
for our samples of radio-loud and radio-quiet quasars. Our derived best-fit
results show no redshift dependence for radio-quiet quasars $z\leq 5$, where
all correlation coefficients agreed within $2\sigma$. In contrast, the radio-
loud quasars exhibit a decreasing X-ray luminosity dependence for increasing
redshift at a $>3\sigma$ level, which we theorize may be due to the primary
emission mechanism of radio-loud quasars evolving over redshift. However, a
greater sample size is necessary to conclusively verify the existence of, or
lack thereof, an evolving emission mechanism for the radio-loud quasar
population.

Leveraging our fundamental plane results, we derived a best-fit expression for
black hole mass estimation that used only the radio and the X-ray
luminosities. We found a best-fit of $\log{M}=(0.63\pm
0.05)\log{L_{X}}-(0.21\pm 0.04)\log{L_{R}}-(10.47\pm 2.09)$ for the radio-loud
quasars and $\log{M}=(0.08\pm 0.07)\log{L_{X}}-(0.41\pm
0.06)\log{L_{R}}-(11.42\pm 3.61)$ for the radio-quiet quasars. The accuracy of
our fundamental plane mass estimation method was assessed for each sample, and
we found an average $4.0\%$ difference between it and standard estimates from
optical spectrum emission lines. Thus, this method may prove useful in
studying the high-redshift quasar population as well as situations where high
signal-to-noise emission lines utilized for spectral modeling are not present
at standard observing wavelengths.

Follow-up studies with more robust statistics may be used to better constrain
model uncertainties. Furthermore, additional sources from future large-scale
quasar surveys, such as the VLA and eROSITA all-sky surveys that are currently
in progress, will increase the number of available targets and improve the the
statistical significance of future fundamental plane studies.

## Acknowledgements

L.B. was supported by the SAO REU program, which is funded in part by the
National Science Foundation REU and Department of Defense ASSURE programs
under NSF Grants no. AST 1852268 and 2050813, and by the Smithsonian
Institution. B.S., M.S., A.S., and D.A.S. were supported by NASA contract
NAS8-03060 (Chandra X-ray Center). B.S. was also supported in part by CXC
grants GO8-19093X and GO0-21101X.

L.B. also thanks Matthew Ashby and Jonathan McDowell for their support and
mentorship throughout the SAO REU program.

## Data Availability

The data used in this investigation are available with the article in a
machine-readable format.

## References

  * Baloković et al. (2012) Baloković M., Smolčić V., Ivezić Ž., Zamorani G., Schinnerer E., Kelly B. C., 2012, ApJ, 759, 30
  * Becker et al. (1995) Becker R. H., White R. L., Helfand D. J., 1995, ApJ, 450, 559
  * Begelman et al. (1984) Begelman M. C., Blandford R. D., Rees M. J., 1984, RvMP, 56, 255
  * Boggs & Rogers (1990) Boggs P. T., Rogers J. E., 1990, in Statistical analysis of measurement error models and applications: proceedings of the AMS-IMS-SIAM joint summer research conference held June 10-16, 1989.
  * Corbel et al. (2003) Corbel S., Nowak M. A., Fender R. P., Tzioumis A. K., Markoff S., 2003, A&A, 400, 1007
  * Falcke et al. (2004) Falcke H., Körding E., Markoff S., 2004, A&A, 414, 895
  * Gallo et al. (2003) Gallo E., Fender R. P., Pooley G. G., 2003, MNRAS, 344, 60
  * Gültekin et al. (2019) Gültekin K., King A. L., Cackett E. M., Nyland K., Miller J. M., Di Matteo T., Markoff S., Rupen M. P., 2019, ApJ, 871, 80
  * Gültekin et al. (2009) Gültekin K., Cackett E. M., Miller J. M., Matteo T. D., Markoff S., Richstone D. O., 2009, ApJ, 706, 404
  * Heinz & Sunyaev (2003) Heinz S., Sunyaev R. A., 2003, MNRAS, 343, L59
  * Heinz et al. (1998) Heinz S., Reynolds C. S., Begelman M. C., 1998, ApJ, 501, 126
  * Hinshaw et al. (2013) Hinshaw G., et al., 2013, ApJS, 208, 19
  * Hodges-Kluck et al. (2021) Hodges-Kluck E., Gallo E., Ghisellini G., Haardt F., Wu J., Ciardi B., 2021, MNRAS, 505, 1543
  * Ivezić et al. (2004) Ivezić v., et al., 2004, Proc Int Astron Union, 2004, 525–526
  * Just et al. (2007) Just D. W., Brandt W. N., Shemmer O., Steffen A. T., Schneider D. P., Chartas G., Garmire G. P., 2007, ApJ, 665, 1004
  * Kellermann et al. (1989) Kellermann K. I., Sramek R., Schmidt M., Shaffer D. B., Green R., 1989, AJ, 98, 1195
  * Körding et al. (2006) Körding E., Falcke H., Corbel S., 2006, A&A, 456, 439
  * Lusso & Risaliti (2016) Lusso E., Risaliti G., 2016, ApJ, 819, 154
  * Marshall et al. (2018) Marshall H. L., et al., 2018, ApJ, 856, 66
  * Merloni (2004) Merloni A., 2004, MNRAS, 353, 1035
  * Merloni et al. (2003) Merloni A., Heinz S., Di Matteo T., 2003, MNRAS, 345, 1057
  * Micic et al. (2007) Micic M., Holley-Bockelmann K., Sigurdsson S., Abel T., 2007, MNRAS, 380, 1533
  * Miller et al. (1990) Miller L., Peacock J. A., Mead A. R. G., 1990, MNRAS, 244, 207
  * Nanni et al. (2017) Nanni R., Vignali C., Gilli R., Moretti A., Brandt W. N., 2017, A&A, 603, A128
  * Plotkin et al. (2012) Plotkin R. M., Markoff S., Kelly B. C., Körding E., Anderson S. F., 2012, MNRAS, 419, 267
  * Rakshit et al. (2020) Rakshit S., Stalin C. S., Kotilainen J., 2020, ApJS, 249, 17
  * Schwartz et al. (2020) Schwartz D. A., et al., 2020, ApJ, 904, 57
  * Shen (2013) Shen Y., 2013, BASI, 41, 61
  * Sikora et al. (2007) Sikora M., Stawarz Ł., Lasota J.-P., 2007, ApJ, 658, 815
  * Snios et al. (2020) Snios B., et al., 2020, ApJ, 899, 127
  * Snios et al. (2021) Snios B., et al., 2021, ApJ, 914, 130
  * Stawarz et al. (2008) Stawarz Ł., Ostorero L., Begelman M. C., Moderski R., Kataoka J., Wagner S., 2008, ApJ, 680, 911
  * Timlin et al. (2020) Timlin J. D., Brandt W. N., Zhu S., Liu H., Luo B., Ni Q., 2020, MNRAS, 498, 4033
  * Urry & Padovani (1995) Urry C. M., Padovani P., 1995, PASP, 107, 803
  * Vito et al. (2019) Vito F., et al., 2019, A&A, 630, A118
  * Volonteri (2012) Volonteri M., 2012, Science, 337, 544
  * White et al. (2007) White R. L., Helfand D. J., Becker R. H., Glikman E., de Vries W., 2007, ApJ, 654, 99
  * Worrall et al. (1987) Worrall D. M., Giommi P., Tananbaum H., Zamorani G., 1987, ApJ, 313, 596
  * Worrall et al. (2020) Worrall D. M., Birkinshaw M., Marshall H. L., Schwartz D. A., Siemiginowska A., Wardle J. F. C., 2020, MNRAS, 497, 988
  * Zamfir et al. (2008) Zamfir S., Sulentic J. W., Marziani P., 2008, MNRAS, 387, 856
  * Zhu et al. (2019) Zhu S. F., Brandt W. N., Wu J., Garmire G. P., Miller B. P., 2019, MNRAS, 482, 2016
  * Zhu et al. (2021) Zhu S. F., Timlin J. D., Brandt W. N., 2021, MNRAS, 505, 1954