∎

11institutetext: S. Höppner 22institutetext: Ulm University
James-Franck-Ring 1, 89081 Ulm
Phone: +49-0731-5024208
22email<EMAIL_ADDRESS>33institutetext: M. Tichy 44institutetext:
Ulm University
James-Franck-Ring 1, 80901 Ulm
44email<EMAIL_ADDRESS>
# Traceability and Reuse Mechanisms, the most important Properties of Model
Transformation Languages

Stefan Höppner  Matthias Tichy

(Received: date / Accepted: date)

###### Abstract

Context
Dedicated model transformation languages are claimed to provide many benefits
over the use of general purpose languages for developing model
transformations. However, the actual advantages and disadvantages associated
with the use of model transformation languages are poorly understood
empirically. There is little knowledge and even less empirical assessment
about what advantages and disadvantages hold in which cases and where they
originate from. In a prior interview study, we elicited expert opinions on
what advantages result from what factors surrounding model transformation
languages as well as a number of moderating factors that moderate the
influence.

Objective
We aim to quantitatively asses the interview results to confirm or reject the
influences and moderation effects posed by different factors. We further
intend to gain insights into how valuable different factors are to the
discussion so that future studies can draw on these data for designing
targeted and relevant studies.

Method
We gather data on the factors and quality attributes using an online survey.
To analyse the data and examine the hypothesised influences and moderations,
we use universal structure modelling based on a structural equation model.
Universal structure modelling produces significance values and path
coefficients for each hypothesised and modelled interdependence between
factors and quality attributes that can be used to confirm or reject
correlation and to weigh the strength of influence present.

Results
We analyzed 113 responses. The results show that the MTL capabilities Tracing
and Reuse Mechanisms are most important overall. Though the observed effects
were generally 10 times lower than anticipated. Additionally, we found that a
more nuanced view of moderation effects is warranted. Their moderating
influence differed significantly between the different influences, with the
strongest effects being 1000 times higher than the weakest.

Conclusion
The empirical assessment of MTLs is a complex topic that cannot be solved by
looking at a single stand-alone factor. Our results provide clear indication
that evaluation should consider transformations of different sizes and use-
cases that go beyond mapping one elements attributes to another. Language
development on the other hand should focus on providing practical,
transformation specific reuse mechanisms that allow MTLs to excel in areas
such as maintainability and productivity compared to GPLs.

###### Keywords:

Survey Universal Structure Modeling Model Transformation Language DSL Model
Transformation MDSE advantages disadvantages Quantitative Analysis

††journal: JOURNALNAME

##  1 Introduction

Model driven engineering (MDE) envisions the use of model transformations as a
main activity during development Sendall2003 (50). When practising MDE, model
transformations are used for a wide array of tasks such as manipulating and
evolving models metzger2005systematic (39), deriving artefacts like source
code or documentation, simulating system behaviour or analysing system aspects
Schmidt2006 (48).

Numerous dedicated model transformation languages (MTLs) of different form,
aim and syntax kahani2019survey (30) have been developed to aid with model
transformations. Using MTLs is associated with many benefits compared to using
general purpose languages (GPLs), though little evidence for this has been
brought forth Goetz2020 (19). The number of claimed benefits is enormous and
includes, but is not limited to, better Comprehensibility, Productivity and
Maintainability as well as easier development in general Goetz2020 (19). The
existence of such claims can partially be attributed to the advantages that
are ascribed to domain specific languages (DSLs) Hermans2009 (21, 28).

In a prior systematic literature review, we have shown that it is still
uncertain whether these advantages exist and where they arise from Goetz2020
(19). Due to this uncertainty it is hard to convincingly argue the use of MTLs
over GPLs for transformation development. This problem is exacerbated when
considering recent GPL advancements, like Java Streams, LINQ in C# or advanced
pattern matching syntax, that help reduce boilerplate code Hoeppner2021 (26)
and have put them back into the discussion for transformation development.
Even a community discussion held at the 12th edition for the International
Conference on Model Transformations (ICMT’19) acknowledges GPLs as suitable
contenders Burgueno2019 (9). Moreover, the few existing empirical studies on
this topic provide mixed and limited results. Hebig et al. found no direct
advantage for the development of transformations, but did find an advantage
for the comprehensibility of transformation code in their limited setup
Hebig2018 (20). A study conducted by us, found that certain use cases favour
the use of MTLs, while in others the versatility of GPLs prevails Hoeppner2021
(26). Overall there exists a gap in knowledge in what the exact benefits of
MTLs are, how strong their impact really is and what parts of the language
they originate from.

To bridge this gap, we conducted an interview study with 56 experts from
research and industry to discuss the topic of advantages and disadvantages of
model transformation languages Hoeppner2022 (25). Participants were queried
about their views on the advantages and disadvantages of model transformation
languages and the origins thereof. The results point towards three main-areas
that are relevant to the discussion, namely General Purpose Languages
Capabilities, Model Transformation Languages Capabilities and Tooling. From
the responses of the interviewees we identified which claimed MTL properties
are influenced by which sub-areas and why. They also provided us with insights
on moderation effects on these interdependencies caused by different Use-
Cases, Skill & Experience levels of users and Choice of Transformation
Language.

All results of the interview study are qualitative and therefore limited in
their informative value as they do not provide indication on the strength of
influence between the involved variables. It is also not clear whether the
influence model is complete and whether the views pretended by the interview
participants withstand community scrutiny. Therefore they only represent an
initial data set that requires a quantitative and detailed analysis.

In this paper, we report on the results of a study to confirm or deny the
interdependencies hypothesised from our interview results. We provide
quantification of the influence strengths and moderation effects. To ensure a
more complete theory of interactions, we also present the results of exploring
interdependencies between factors and quality properties not hypothesised in
the interviews.

Due to limited resources, this study focuses on the effects of MTL
capabilities (namely Bidirectionality, Incrementality, Mappings, Model
Management, Model Navigation, Model Traversal, Pattern Matching, Reuse
Mechanisms and Traceability) on MTL properties (namely Comprehensibility, Ease
of Writing, Expressiveness, Productivity, Maintainability and Reusability and
Tool Support) in the context of their uses-case (namely bidirectional or
unidirectional, incremental or non-incremental, meta-model sanity, meta-model,
model and transformation size and semantic gap between input and output), the
skills & experience of users and language choice. Further studies can follow
the same approach and focus on different areas. Descriptions for all MTL
capabilities and MTL properties can be found in Section 2 and thorough
explanations can be found in our previous works Goetz2020 (19, 25).

The goal of our study is to provide quantitative results on the influence
strengths of interdependences between model transformation language
Capabilities and claimed Quality Properties as perceived by users.
Additionally we provide data on the strength of moderation expressed by
contextual properties. The study is structured around the hypothesised
interdependencies between these variables, and their more detailed breakdown,
extracted from our previous interview study. Each presumed influence of a MTL
capability on a MTL property forms one hypothesis which is to be examined in
this study. All hypotheses are extended with an assumption of moderation by
the context variables. The system of hypotheses that arises from these
deliberations is visualised in a structure model, which forms the basis for
our study. The structure model is depicted in Figure 1. The model shows
exogenous variables on the left and right and endogenous variables at the
centre. Exogenous variables depicted in a ellipse with a dashed outline
constitute the hypothesised moderating variables.

All hypotheses investigated in our study are of the form: “<MTL Property> is
(positively or negatively) influenced by <MTL Capability>”. They are
represented by arrows from exogenous variables on the left of Figure 1 to
endogenous variable at the centre. A moderation on the hypothesised influence
is assumed from all exogenous variables on the right of the figure connected
to the considered endogenous variable. In total we investigate 31 hypothesised
influences, i.e. the number of outgoing arrows from the exogenous variables on
the left of Figure 1.

Bidirectionality $\xi_{1}$ Incrementality $\xi_{2}$ Mappings $\xi_{3}$ Model
Management $\xi_{4}$ Model Navigation $\xi_{5}$ Model Traversal $\xi_{6}$
Pattern Matching $\xi_{7}$ Reuse Mechanisms $\xi_{8}$ Traceability $\xi_{9}$
Meta-model Size $\xi_{13}$ Model Size $\xi_{14}$ Transformation Size$\xi_{15}$
Bidirectional Use$\xi_{18}$ Language Choice $\xi_{10}$ Language Skills
$\xi_{11}$ Experience $\xi_{12}$ I/O Semantic gap $\xi_{16}$ Meta-model sanity
$\xi_{17}$ Incremental Use $\xi_{19}$ Comprehensi-bility $\eta_{1}$
$\zeta_{1}$ Ease of Writing $\eta_{2}$ $\zeta_{2}$ Expressiveness $\eta_{3}$
$\zeta_{3}$ Tool Support $\eta_{4}$ $\zeta_{4}$ Maintainability $\eta_{5}$
$\zeta_{5}$ Productivity $\eta_{6}$ $\zeta_{6}$ Reusability $\eta_{7}$
$\zeta_{1}$ Figure 1: Structure model depicting the hypothesised influence and
moderation effects of factors on MTL properties.

Our study is guided by the following research questions:

  * RQ1

Which of the hypothesised interdependencies withstands a test of significance?

  * RQ2

How strong are the influences of model transformation language capabilities on
the properties thereof?

  * RQ3

How strong are moderation effects expressed by the contextual factors use-
case, skills & experience and MTL choice?

  * RQ4

What additional interdependencies arise from the analysis that were not
initially hypothesised?

As the first study on this subject it contains confirmatory and exploratory
elements. We intend to confirm which of the interdependencies between MTL
capabilities, MTL properties and contextual properties withstand quantitative
scrutiny (RQ1). We explore how strong the influence and moderation effects
between variables are (RQ2 & RQ3), to gain new insights and to confirm their
significance and relevance (minor influence strengths might suggest
irrelevance even if goodness of fit tests confirm a correlation that is not
purely accidental). Lastly, we utilise the exploratory elements of USM to
identify interdependencies not hypothesised by the experts in our interviews
(RQ4).

We use an online survey to gather data on language use and perceived quality
of researchers and practitioners. The responses are analysed using universal
structure modelling (USM) buckler2008identifying (8) based on the structure
model developed from the interview responses. This results in a quantified
structure model with influence weights, significance values and effect
strengths.

Based on the responses from 113 participants, the key contributions of this
paper are:

  * $\bullet$

An adjusted structure model with newly discovered interdependencies;

  * $\bullet$

Quantitative data on the influence weight and effect strength of all factors
as well as significant values for the influences;

  * $\bullet$

Quantitative data on the moderation strength of context factors;

  * $\bullet$

An analysis of the implications of the results for further empirical studies
and language development;

  * $\bullet$

Reflections on the use of USM for investigating large hypotheses systems in
software engineering research;

The method used in the reported study has been reviewed and published as part
of the Registered Reports track at ESEM’22 Hoeppner2022a (27).

The structure of this paper is as follows: Section 2 provides an extensive
overview of model-driven engineering, domain-specific languages, model
transformation languages and structural equation modelling as well as
universal structure modelling. Afterwards, in Section 3 the methodology is
outlined. Demographic data of the responses is reported in Section 4 and the
results of analysis is presented in Section 5. In Section 6 we discuss
implications of the results and report our reflections on the use of USM.
Section 7 discusses threats to validity of our study and how we met them.
Lastly, in Section 8 we present related work before giving concluding remarks
on our study in Section 9.

##  2 Background

In this section we provide the necessary background for our study. Since it is
a follow up study to our interview study Hoeppner2022 (25) much of the
background is the same and is therefore taken from those descriptions. To stay
self contained we still provide these descriptions. This concerns Sections
2.1, 2.2 and 2.3. Sections 2.4 and 2.5 contains an extension of our
descriptions from the registered report Hoeppner2022a (27).

###  2.1 Model-driven engineering

The Model-Driven Architecture (MDA) paradigm was first introduced by the
Object Management Group in 2001 OMG2001 (41). It forms the basis for an
approach commonly referred to as Model-driven development (MDD) Brown2005 (7),
introduced as means to cope with the ever growing complexity associated with
software development. At the core of it lies the notion of using models as the
central artefact for development. In essence this means, that models are used
both to describe and reason about the problem domain as well as to develop
solutions Brown2005 (7). An advantage ascribed to this approach that arises
from the use of models in this way, is that they can be expressed with
concepts closer to the related domain than when using regular programming
languages Selic2003 (49).

When fully utilized, MDD envisions automatic generation of executable
solutions specialized from abstract models Selic2003 (49, 48). To be able to
achieve this, the structure of models needs to be known. This is achieved
through so called meta-models which define the structure of models. The
structure of meta-models themselves is then defined through meta-models of
their own. For this setup, the OMG developed a modelling standard called Meta-
object Facility (MOF) OMG2016 (42) on the basis of which a number of modelling
frameworks such as the Eclipse Modelling Framework (EMF) steinberg2008emf (52)
and the .NET Modelling Framework hinkel2016nmf (22) have been developed.

###  2.2 Domain-specific languages

Domain-specific languages (DSLs) are languages designed with a notation that
is tailored for a specific domain by focusing on relevant features of the
domain vanDeuersen2002 (54). In doing so DSLs aim to provide domain specific
language constructs, that let developers feel like working directly with
domain concepts thus increasing speed and ease of development Sprinkle2009
(51). Because of these potential advantages, a well defined DSL can provide a
promising alternative to using general purpose tools for solving problems in a
specific domain. Examples of this include languages such as shell scripts in
Unix operating systems Kernighan1984 (31), HTML Raggett1999 (45) for designing
web pages or AADL an architecture design language SAEMobilus2004 (46).

###  2.3 Model transformation languages

The process of (automatically) transforming one model into another model of
the same or different meta-model is called model transformation (MT). They are
regarded as being at the heart of Model Driven Software Development
Sendall2003 (50, 39), thus making the process of developing them an integral
part of MDD. Since the introduction of MDE at the beginning of the century, a
plethora of domain specific languages for developing model transformations, so
called model transformation languages (MTLs), have been developed Arendt2010
(3, 4, 29, 33, 24, 14, 23). Model transformation languages are DSLs designed
to support developers in writing model transformations. For this purpose, they
provide explicit language constructs for tasks involved in model
transformations such as model matching. There are various features, such as
directionality or rule organization Czarnecki2006 (12), by which model
transformation languages can be distinguished. For the purpose of this paper,
we will only be explaining those features that are relevant to our study and
discussion in Sections 2.3.1, 2.3.2, 2.3.4, 2.3.5, 2.3.6, 2.3.3 and 2.3.7.
Table 1 provides an overview over the presented features.

Please refer to Czarnecki2006 (12, 30, 38) for complete classification.

Table 1: MTL feature overview Feature | Characteristic |  Representative Language
---|---|---
Embeddedness | Internal |  FunnyQT (Clojure), RubyTL (Ruby), NMF Synchronizations (C#)
External |  ATL, Henshin, QVT
Rules | Explicit Syntax Construct |  ATL, Henshin, QVT
Repurposed Syntax Construct |  NMF Synchronizations (Classes), FunnyQT (Macros)
Location Determination | Automatic Traversal |  ATL, QVT
Pattern Matching |  Henshin
Directionality | Unidirectional |  ATL, QVT-O
Bidirectional |  QVT-R, NMF Synchronisations
Incrementality | Yes |  NMF Synchronizations
No |  QVT-O
Tracing | Automatic |  ATL, QVT
Manual |  NMF Synchronizations
Dedicated Model Navigation Syntax | Yes |  ATL (OCL), QVT (OCL), Henshin (implicit in rules)
No |  NMF Synchronizations, FunnyQT, RubyTL

####  2.3.1 External and Internal transformation languages

Domain specific languages, and MTLs by extension, can be distinguished on
whether they are embedded into another language, the so called host language,
or whether they are fully independent languages that come with their own
compiler or virtual machine.

Languages embedded in a host language are called internal languages. Prominent
representatives among model transformation languages are FunnyQT
10.1007/978-3-642-38883-5_7 (24) a language embedded in Clojure, NMF
Synchronizations and the .NET transformation language P23 (23) embedded in C#,
and RubyTL Cuadrado2006 (11) embedded in Ruby.

Fully independent languages are called external languages. Examples of
external model transformation languages include one of the most widely known
languages such as the Atlas transformation language (ATL) Jouault2006 (29),
the graphical transformation language Henshin Arendt2010 (3) as well as a
complete model transformation framework called VIATRA Balogh2006 (4).

####  2.3.2 Transformation Rules

Czarnecki2006 (12) describe rules as being “understood as a broad term that
describes the smallest units of [a] transformation [definition]”. Examples for
transformation rules are the rules that make up transformation modules in ATL,
but also functions, methods or procedures that implement a transformation from
input elements to output elements.

The fundamental difference between model transformation languages and general-
purpose languages that originates in this definition, lies in dedicated
constructs that represent rules. The difference between a transformation rule
and any other function, method or procedure is not clear cut when looking at
GPLs. It can only be made based on the contents thereof. An example of this
can be seen in LABEL:lst:Java:example, which contains exemplary Java methods.
Without detailed inspection of the two methods it is not apparent which method
does some form of transformation and which does not.

In a MTL on the other hand transformation rules tend to be dedicated
constructs within the language that allow a definition of a mapping between
input and output (elements). The example rules written in the model
transformation language ATL in LABEL:lst:ATL:example make this apparent. They
define mappings between model elements of type Member and model elements of
type Male as well as between Member and Female using rules, a dedicated
language construct for defining transformation mappings. The transformation is
a modified version of the well known Families2Persons transformation case
fam2per (2).

⬇

1public void methodExample(Member m) {

2 System.out.println(m.getFirstName());

3}

4public void methodExample2(Member m) {

5 Male target = new Male();

6 target.setFullName(m.getFirstName() + " Smith");

7 REGISTRY.register(target);

8}

List. 1: Example Java methods

⬇

1rule Member2Male {

2 from

3 s : Member (not s.isFemale())

4 to

5 t : Male (

6 fullName <- s.firstName + ’ Smith’

7 )

8}

9

10rule Member2Female {

11 from

12 s : Member (s.isFemale())

13 to

14 t : Female (

15 fullName = s.firstName + ’ Smith’

16 partner = s.companion

17 )

18}

List. 2: Example ATL rules

####  2.3.3 Rule Application Control: Location Determination

Location determination describes the strategy that is applied for determining
the elements within a model onto which a transformation rule should be applied
Czarnecki2006 (12). Most model transformation languages such as ATL, Henshin,
VIATRA or QVT QvT2016 (44), rely on some form of automatic traversal strategy
to determine where to apply rules.

We differentiate two forms of location determination, based on the kind of
matching that takes place during traversal. There is the basic automatic
traversal in languages such as ATL or QVT, where single elements are matched
to which transformation rules are applied. The other form of location
determination, used in languages like Henshin, is based on pattern matching,
meaning a model- or graph-pattern is matched to which rules are applied. This
does allow developers to define sub-graphs consisting of several model
elements and references between them which are then manipulated by a rule.

The automatic traversal of ATL applied to the example from
LABEL:lst:ATL:example will result in the transformation engine automatically
executing the Member2Male on all model elements of type Member where the
function isFemale() returns false and the Member2Female on all other model
elements of type Member.

The pattern matching of Henshin can be demonstrated using Figure 2, a modified
version of the transformation examples by Krause2014 (35). It describes a
transformation that creates a couple connection between two actors that play
in two films together. When the transformation is executed the transformation
engine will try and find instances of the defined graph pattern and apply the
changes on the found matches.

This highlights the main difference between automatic traversal and pattern
matching as the engine will search for a sub graph within the model instead of
applying a rule to single elements within the model.

Figure 2: Example Henshin transformation

####  2.3.4 Directionality

The directionality of a model transformation describes whether it can be
executed in one direction, called a unidirectional transformation or in
multiple directions, called a multidirectional transformation Czarnecki2006
(12).

For the purpose of our study the distinction between unidirectional and
bidirectional transformations is relevant. Some languages allow dedicated
support for executing a transformation both ways based on only one
transformation definition, while other require users to define transformation
rules for both directions. General-purpose languages can not provide
bidirectional support and also require both directions to be implemented
explicitly.

The ATL transformation from LABEL:lst:ATL:example defines a unidirectional
transformation. Input and output are defined and the transformation can only
be executed in that direction.

The QVT-R relation defined in LABEL:lst:QVTR:example is an example of a
bidirectional transformation definition (For simplicity reasons the
transformation omits the condition that males are only created from members
that are not female). Instead of a declaration of input and output, it defines
how two elements from different domains relate to one another. As a result
given a Member element its corresponding Male elements can be inferred, and
vice versa.

⬇

1top relation Member2Male {

2 n, fullName : String;

3 domain Families s:Member {

4 firstName = n };

5 domain Persons t:Male {

6 fullName = fullName};

7 where {

8 fullName = n + ’ Smith’; };

9}

List. 3: Example QVT-R relation

####  2.3.5 Incrementality

Incrementality of a transformation describes whether existing models can be
updated based on changes in the source models without rerunning the complete
transformation Czarnecki2006 (12). This feature is sometimes also called model
synchronisation.

Providing incrementality for transformations requires active monitoring of
input and/or output models as well as information which rules affect what
parts of the models. When a change is detected the corresponding rules can
then be executed. It can also require additional management tasks to be
executed to keep models valid and consistent.

####  2.3.6 Tracing

According to Czarnecki2006 (12) tracing “is concerned with the mechanisms for
recording different aspects of transformation execution, such as creating and
maintaining trace links between source and target model elements”.

Several model transformation languages, such as ATL and QVT have automated
mechanisms for trace management. This means that traces are automatically
created during runtime. Some of the trace information can be accessed through
special syntax constructs while some of it is automatically resolved to
provide seamless access to the target elements based on their sources.

An example of tracing in action can be seen in line 16 of
LABEL:lst:ATL:example. Here the partner attribute of a Female element that is
being created, is assigned to s.companion. The s.companion reference points
towards a element of type Member within the input model. When creating a
Female or Male element from a Member element, the ATL engine will resolve this
reference into the corresponding element, that was created from the referred
Member element via either the Member2Male or Member2Female rule. ATL achieves
this by automatically tracing which target model elements are created from
which source model elements.

####  2.3.7 Dedicated Model Navigation Syntax

Languages or syntax constructs for navigating models is not part of any
feature classification for model transformation languages. However, it was
often discussed in our interviews and thus requires an explanation as to what
interviewees refer to.

Languages such as OCL OMG2014 (43), which is used in transformation languages
like ATL, provide dedicated syntax for querying and navigating models. As such
they provide syntactical constructs that aid users in navigation tasks.
Different model transformation languages provide different syntax for this
purpose. The aim is to provide specific syntax so users do not have to
manually implement queries using loops or other general purpose constructs.
OCL provides a functional approach for accumulating and querying data based on
collections while Henshin uses graph patterns for expressing the relationship
of sought-after model elements.

###  2.4 Structural equation modelling and (Universal) Structural Equation
Modelling

Figure 3: The makeup of a structural equation model.

Structural equation modelling (SEM) is an approach used for confirmatory
factor analysis 10.1145/3469888 (17). It defines a set of methods used to
“investigate complex relationship structures between variables and allows for
quantitative estimates of interdependencies thereof. Its goal is to map the
a-priori formulated cause-effect relationships into a linear system of
equations and to estimate the model parameters in such a way that the initial
data, collected for the variables, are reproduced as well as possible”
Weiber2021 (55).

Structural equation modelling distinguishes between two sets of variables
manifest and latent. Manifest variables are variables that are empirically
measured and latent variables describe theoretical constructs that are
hypothesised to interact with each other. Latent variables are further divided
into exogenous or independent and endogenous or dependent variables.

So called structural equation models, a sample of which can be seen in Figure
3, comprised of manifest and latent variables, form the heart of analysis.
They are made up of three connected sub-models. The structure model, the
measurement model of the exogenous latent variables and the measurement model
of the endogenous latent variables.

The structure model defines all hypothesised interactions between exogenous
($\xi_{exID}$) and endogenous ($\eta_{endID}$) latent variables. Each
exogenous variable is linked, by arrow, to all endogenous variables that are
presumed to be influenced by it. Each of these connections is given a variable
($\gamma_{exID\\_endID}$) that measures the influence strength. If an
exogenous variable moderates the influences on a endogenous variable, the
exogenous variable is depicted with a dashed outline and connected to all
endogenous variables that are moderated by it111To illustrate moderation,
arrows are usually shown from the moderating exogenous variable to the arrow
representing the moderated influence , i.e., an arrow between an exogenous
variable and an endogenous variable. However our illustration deviates from
this due to the size and makeup of our hypothesis system. Standard
representations can be found in the basic literature such as Weiber2021 (55)..
For each moderated influence a separate variable of the form
$\gamma_{exID\\_endID\\_modEndID}$ is assigned. In addition, an residual (or
error) variable is appended to each endogenous latent variable to represent
the influence of variables not represented in the model.

Figure 3 shows an example structure equation model model for the hypothesis
that “Mappings help with the comprehensibility of transformations, depending
on the developers experience.”. The structure model seen at the centre of the
figure, is comprised of the exogenous latent variable $\xi_{1}$ (Mappings),
the moderating exogenous variable $\xi_{2}$ (Experience), the endogenous
latent variable $\eta_{1}$ (Comprehensibility), a presumed influence of
Mappings on Comprehensibility via $\gamma_{11}$ and the error variable
$\zeta_{1}$. Lastly the model also contains a moderation of Experience on all
influences of Comprehensibility. As described earlier, this moderation effect
is assigned the variable $\gamma_{11\\_2}$. The moderation variables are not
depicted in our graphical representation of the structure model because of
their high number and associated visibility issues.

The measurement model of the exogenous latent variables reflects the
relationships between all exogenous latent variables and their associated
manifest variables. Each manifest variable is linked, by arrow, to all
exogenous latent variables that are measured through it. Each of these
connections is given a variable that measures the indication strength of the
manifest variable for the latent variable. Additionally, an error variable for
each manifest variable is introduced that represents measurement errors. In
Figure 3, the measurement model for exogenous latent variables, seen at the
left of the figure, is comprised of the exogenous latent variables $\xi_{1}$
(Mappings) and $\xi_{2}$ (Experience), the manifest variables $x_{1}$ (% of
code using Mappings), $x_{2}$ (number of years a person has been a programmer)
and $x_{3}$ (number of hours per month spent developing transformations) their
measurement accuracy for Mapping usage $\lambda_{11}$ and their measurement
accuracy for Experience $\lambda_{22}$ and and $\lambda_{32}$ and the
associated measurement error $\delta_{1}$ and $\delta_{2}$ and $\delta_{3}$.

The measurement model of the endogenous latent variables reflects the
relationships between all endogenous latent variables and their associated
manifest variables. It is structured the same way as the measurement model of
the exogenous latent variables. In Figure 3, it is shown on the right of the
figure.

Given a structural equation model and measurements for manifest variables, the
SEM approach calls for estimating the influence weights and latent variables
within the models. This is done in alternation for the measurement models and
the structure model until a predefined quality criterion is reached.
Traditional methods (covariance-based structural equation modeling & partial
least squares) use different mathematical approaches such as maximum-
likelihood estimation or least squares Weiber2021 (55) to estimate influence
weights.

Universal Structure Modeling (USM) is an exploratory approach that complements
the traditional confirmatory SEM methods buckler2008identifying (8). It
combines the iterative methodology of partial least squares with a Bayesian
neural network approach using multilayer perceptron architecture. USM derives
a starting value for latent variables in the model via principal component
analysis and then applies the Bayesian neural network to discover an optimal
system of linear, nonlinear and interactive paths between the variables. This
enables USM to identify complex relationships that may not be detected using
traditional SEM approaches including hidden structures within the data and
highlights unproposed model paths, nonlinear relations among model variables,
and moderation effects.

The primary measures calculated in USM are the ‘Average Simulated Effect’
(ASE), ‘Overall Explained Absolute Deviation’ (OEAD), ‘interaction effect’
(IE) and ‘parameter significance’. ASE measures the average change in the
endogenous variable resulting from a one-unit change in the exogenous variable
across all simulations. OEAD assesses the degree of fit between the observed
and simulated values of the endogenous variable, capturing the overall
explanatory power of the model. IE evaluates the extent to which the effect of
one exogenous variable on the endogenous variable depends on the level of
another variable. Parameter Significance determines whether the estimated
coefficients for each exogenous variable in the model are statistically
significant at a predetermined level of confidence which indicated if the
exogenous variable has a meaningful impact on the endogenous variable and is
calculated through a bootstrapping routine mooney1993bootstrapping (40). These
metrics together provide a comprehensive assessment of the performance and
explanatory power of a USM model.

USM is recommended for use in situations where traditional SEM approaches may
not be sufficient to fully explore the relationships between variables. Using
USM instead of traditional structural equation modelling approaches is
suggested for studies where there are still uncertainties about the
completeness of the underlying hypotheses system and for exploring non-
linearity in the influences Weiber2021 (55, 8). Moreover its use of a neural
network also reduces the requirements for the scale levels of data thus
allowing the introduction of categorical variables in addition to metric
variables Weiber2021 (55).

At present, the tool NEUSREL222https://www.neusrel.com is the only tool
available for conducting USM.

###  2.5 MTL Quality Properties

There exists a large body of quality properties that get associated with model
transformation languages. In literature many claims are made about advantages
or disadvantages of MTLs in these different properties. We categorised these
properties in a previous work of ours Goetz2020 (19). This study focuses on a
subset of all the identified quality properties of MTLs which requires them to
be properly explained. In this section, we give a brief description of our
definitions of each of the quality properties of MTLs relevant to the study.

Comprehensibility describes the ease of understanding the purpose and
functionality of a transformation based on reading code.

Ease of Writing describes the ease at which a developer can produce a
transformation for a specific purpose.

Expressiveness describes the amount of useful dedicated transformation
concepts in a language.

Productivity describes the degree of effectiveness and efficiency with which
transformations can be developed and used.

Maintainability describes the degree of effectiveness and efficiency with
which a transformation can be modified.

Reusability describes the ease of reusing transformations or parts of
transformations to create new transformations (with different purposes).

Tool Support describes the amount of quality tools that exist to support
developers in their efforts.

##  3 Methodology

The methodology used in this study has been reviewed and published as part of
the Registered Reports track at ESEM’22 Hoeppner2022a (27). In the following,
we provide a more detailed description and highlight all deviations from the
reported method as well as justification for the changes.

The study itself is comprised of the following steps which were executed
sequentially and are reported on in this section.

  1. 1.

Development of survey methodology.

  2. 2.

Submission to the Registered Reports track at EMSE’22.

  3. 3.

Methodology revision based on feedback.

  4. 4.

Development of online survey using an on premise version of the survey tool
LimeSurvey333https://www.limesurvey.org/.

  5. 5.

Survey review and pilot test by co-authors.

  6. 6.

Reworking survey based on pilot test.

  7. 7.

Opening online survey to public.

  8. 8.

Reaching out to potential survey subjects per mail and social media.

  9. 9.

Closing of online survey (9 weeks after opening).

  10. 10.

Data extraction.

  11. 11.

Data analysis using the USM tool NEUSREL.

The steps executed differ in two ways from those reported in the registered
report. First, we do not contact potential participants for a second time
after two weeks. This was deemed unnecessary based on the number of
participants at that point in time. Moreover we did not want to bother those
that participated already and had no way of knowing their identity. Second, we
kept the survey open 3 weeks longer than intended due to receiving several
requests to do so.

###  3.1 Survey Design

In this section we detail the design of the used questionnaire and methodology
used to develop and distribute it.

####  3.1.1 Questionnaire

The questions in the questionnaire are designed to query data for measuring
the latent variables from the structure model in Figure 1. The complete
questionnaire can be found in LABEL:apdx:questions. In the following, we
describe each latent variable and explain how we measure it through questions
in the questionnaire.

There are 26 latent variables relevant to our study. Variables $\xi_{1..19}$
describe exogenous variables and $\eta_{1..7}$ describe endogenous variables.
Each latent variable is measured through one or more manifest variables.
Extending the structure model from Figure 1 with the manifest variables
produces the complete structural equation model evaluated in this study. Note
that USM reduces the requirements for the scale levels of data thus allowing
the use of categorical variables in addition to metric variables Weiber2021
(55).

All latent variables related to MTL capabilities ($\xi_{1..9}$) are associated
with a single manifest variable $x_{1..9}$, which measures how frequently the
participants utilized the MTL capabilities in their transformations. This
measurement is represented as a ratio ranging from 0% to 100%. The higher the
value of $x_{1..9}$, the more frequently the participants used the MTL
capabilities in their transformations. Similarly, latent variables related to
MTL properties ($\eta_{1..7}$) are associated with a single manifest variable
$y_{1..7}$ which measures the perceived quality of the property on a 5-point
likert scale (e.g., very good, good, neither good nor bad, bad, very bad).

The use of single-item scales is a debated topic. We justify their usage for
the described latent variables on multiple grounds. First, the latent
variables are of high complexity due to the abstract concepts they represent.
Second, our study aims to produce first results that need to be investigated
in more detail in follow up studies, more focused on single aspects of the
model. And third, due to the size of our structural equation model multi-item
scales for all latent variables would increase the size of the survey,
potentially putting off many subjects. The validity of these deliberations for
using single-item scales is supported by fuchs2009using (13).

The latent variable language choice ($\xi_{10}$) is measured by means of
querying participants to list their 5 most recently used transformation
languages. In our registered report we planned to also request participants to
give an estimate on the percentage of their respective use % ($x_{10})$. This
was discarded during pilot testing as it was seen as unnecessarily prolonging
the questionnaire. Pilot testers had difficulties providing accurate data and
questioned whether this data was actually used in analysis.

Language skills ($\xi_{11}$) is measured through $x_{11}$ and $x_{12}$ for
which participants are asked to give the amount of years they have been using
each language ($x_{11}$) and the amount of hours they use the language per
month ($x_{12}$).

Similarly, experience ($\xi_{12}$) is associated with the amount of years
subjects have been involved in defining model transformations ($x_{13}$) and
the amount of hours they spend on developing transformations each month
($x_{14}$).

Meta-model size ($\xi_{13}$) and model size ($\xi_{14}$) both require
participants to state the range between which their (meta-) models vary
($x_{15}$, $x_{16}$). This is measured by offering participants a number of
ranges of (meta-) model objects. For each range participants should give an
estimate on how much percent of the (meta-) models they work fall within that
size range. For models the ranges are: #objects $\leq 10$, $10\leq$ #objects
$\leq 100$, $100\leq$ #objects $\leq 1000$, $1000\leq$ #objects $\leq 10000$,
$10000\leq$ #objects $\leq 100000$, $100000\leq$ #objects. For meta-model the
ranges are: #objects $\leq 10$, $10\leq$ #objects $\leq 20$, $20\leq$ #objects
$\leq 50$, $50\leq$ #objects $\leq 100$, $100\leq$ #objects $\leq 1000$,
$1000\leq$ #objects. Similarly, Transformation size ($\xi_{15}$) is measured
on a range of lines of code ($x_{17}$). The options being: LOC $\leq 100$,
$100\leq$ LOC $\leq 500$, $500\leq$ LOC $\leq 1000$, $1000\leq$ LOC $\leq
5000$, $5000\leq$ LOC $\leq 10000$, $10000\leq$ LOC. Querying size data in
this manner and the associated ranges have been successfully applied in a
prior work the authors were involved in JOT:issue_2021_02/article5 (18).

To formulate the semantic gap between input and output ($\xi_{16}$) we elicit
the similarity of the structure ($x_{18}$) and data types ($x_{19}$) on a
5-point likert scale (very similar, similar, neither similar nor dissimilar,
dissimilar, very dissimilar). Participants are asked to give the percentage of
all their meta-models that fall within each of the five assessments.

The meta-model sanity ($\xi_{17}$) is measured through means of how well
participants perceive their structure ($x_{20}$) and their documentation
($x_{21}$) to be on a 5-point scale (very well, well, neither well nor bad,
bad, very bad). Participants are asked to give the percentage of all their
meta-models that fall within each of the five assessments.

Lastly, for both bidirectional uses ($\xi_{18}$) and incremental uses
($\xi_{19}$) we query participants on the ratio of bidirectional ($x_{22}$)
and incremental ($x_{23}$) transformations compared to simple uni-directional
transformations they have written.

####  3.1.2 Pilot Study

We pilot tested the study with three researchers from the institute. All pilot
testers are researchers in the field of model driven engineering with more
than 5 years of experience. Based on their feedback, we reworded some
questions questions, removed the usage percentage part of the question for
language choice and added more precise descriptions of the queried concepts.
We then made the questionnaire publicly available and distributed a link to it
via emails.

####  3.1.3 Target Subjects & Distribution

The target subjects are both researchers and professionals from industry that
have used dedicated model transformation languages to develop model
transformations in the last five years. We use voluntary and convenience
sampling to select our study participants. Both authors reached out to
researchers and professionals they knew personally via mail and request them
to fill out the online survey. We further reach out, via mail, to all authors
of publications listed in ACM Digital Library, IEEE Xplore, Springer Link and
Web of Science that contain the key word model transformation from the last
five years. A third source of subjects is drawn from social media. The authors
use their available social media channels to recruit further subjects by
posting about the online-survey on the platforms. The social media platform
used for distribution was MDE-Net444https://mde-network.com/, a community
platform dedicated to model driven engineering.

The sampling method differs from the intended method by not including
snowballing sampling as a secondary sampling method. We decided on this to
have more control over the subjects receiving a link to the study as we
believe secondary and tertiary contacts might be too far secluded from our
target subjects.

Participation was voluntary and we did not incentivise participation through
offering rewards. This decision is rooted in our experience in previous
studies one other survey with 83 subjects JOT:issue_2021_02/article5 (18) and
the interview study we are basing this study on with 56 subjects Hoeppner2022
(25).

It is suggested in literature to have between 5 to 10 times as many
participants as the largest number of parameters to be estimated in each
structural equation (i.e., the largest number of incoming paths for a latent
model variable) buckler2008identifying (8). Thus, the minimal number of
subjects for our study to achieve stable results is 80. To gain any meaningful
results a sample size of 30 must not be undercut buckler2008identifying (8).

In total we contacted 2383 potential participants and got 113555This
constitutes a response rate of $\leavevmode\nobreak\ 4.8\%$. We do however not
know how many responses are a result of our social media posting. responses
exceeding the minimum requirement for stable results.

###  3.2 Data Analysis

We use USM to examine the hypotheses system modelled by the structure model
shown in Figure 1. USM is chosen over its structural equation modelling
alternatives due to it being able to better handle uncertainty about the
completeness of the hypothesis system under investigation, it having more
capabilities to analyse moderation effects and the ability to investigate non
linear correlations Weiber2021 (55).

USM requires a declaration of an initial likelihood of an interdependence
between two variables. This is used as a starting point for calculating
influence weights but can change over the course of calculation. For this,
buckler2008identifying (8) suggest to only assign a value of 0 to those
relationships that are known to be wrong. We use the results of our interview
study Hoeppner2022 (25), shown in the structure model, to assign these values.
For each path that is present in the model, we assume a likelihood of 100%. To
check for interdependencies that might have been missed by interview
participants, we also use a likelihood of 100% for all missing paths between
$\xi_{1..19}$ and $\eta_{1..7}$. Our plan was to use a likelihood of 50% for
these interdependencies but the tool available to us only allowed for either
100% or 0% to be put as input.

The tool NEUSREL is used on the extracted empirical data and the described
additional input to estimate path weights and moderation weights within the
extended structure model, i.e., the structure model where each exogenous
latent variable is connected to all endogenous latent variables. It also runs
significance tests via a bootstrapping routine buckler2008identifying (8, 40)
and produces the significance value estimates for each influence. The
following procedures are then followed to answer the research questions from
Section 1.

RQ1. We reject all hypothesised influences, i.e., those present in our
structure model in Figure 1, that do not pass the statistical significance
test. The threshold we set for this is $0.01$. Moreover, we discard
hypothesised influences with minimal effects strengths that are several
magnitudes lower than the median influence of all coefficients. If, for
example, the median of all path coefficients is 0.03 all influences with a
coefficient lower or equal to 0.0009 are discarded. We do so because such low
influences suggest that the influence is negligible.

RQ2 & RQ3. All path coefficients produced that were not rejected in RQ1 will
then provide direct values for the influence and moderation strengths to
answer RQ2 & RQ3.

RQ4. The same significance criteria we applied to all hypothesised influences
for RQ1, we also apply to the extended influences, i.e., those not present in
the structure model from Figure 1. Those influences that pass the significance
test are added to the initial structural model as newly discovered influences.

###  3.3 Privacy and Ethical concerns

All participants were informed of the data collection procedure, handling of
the data and their rights, prior to filling out the questionnaire.
Participation was completely voluntary and not incentivised through rewards.

During selection of potential participants the following data was collected
and processed.

  * $\bullet$

First & last name.

  * $\bullet$

E-Mail address.

The questionnaire did not collect any sensitive or identifiable data.

All data collected during the study was not shared with any person outside of
the group of authors.

The complete information and consent form can be found in Appendix D.

The study design was not presented to an ethical board. The basis for this
decision are the rules of the German Research Foundation (DFG) on when to use
a ethical board in humanities and social
sciences666https://www.dfg.de/foerderung/faq/geistes_sozialwissenschaften/. We
refer to these guidelines because there are none specifically for software
engineering research and humanities and social sciences are the closest
related branch of science for our research.

##  4 Demographics

We detail the background and experience of the participants in our study in
the following sections.

###  4.1 Experience in developing model transformations ($\xi_{12}$)

Our survey captured model transformation developers with wide range of
experience. The experience span ($x_{13}$) ranges from the least experience
participant with half a year of experience up to the one with most experience
of 30 years. Figure 4 shows a histogram of the experience stated by
participants. Over half of all participants have between 1 to ten years of
experience in writing model transformations. Three stated to have more than 20
years in total. On average our participants have 9 years of experience.

Figure 4: Histogram of participants total experience in years

How much time participants spend developing transformations each month
($x_{14}$) also greatly varies. Some participants have not developed
transformations in recent time whereas others stated to spend 70 or more hours
each month on transformation development. Figure 5 shows an overview over the
hours participants spend each month in developing transformations. The vast
majority spends around 1 to 10 hours each month on transformation development.
Nine stated that they did not develop any transformation in recent times. On
average our participants spend about 14 hours per month developing model
transformations.

Figure 5: Histogram of participants recent experience in hours per month

###  4.2 Languages used for developing model transformations ($\xi_{10}$) and
experience therein ($\xi_{11}$)

To develop their transformations, participants use a wide array of languages.
In total 43 languages ($x_{10}$) have been named 24 of which are unique
languages used only by a single participant.

Surprisingly the language that has been used by the most participants is Java,
a general purpose language. Java has been used by 70 of the 113 participants.
The most used MTL is ATL with 58 users closely followed by another GPL, namely
Xtend with 52 users. Table 2 shows how many participants use one of the ten
most used languages for developing transformations.

Table 2: Overview of languages used by participants Language | # number of participants
---|---
Java | 70
ATL | 58
Xtend | 52
ETL | 29
QVTo | 22
Henshin | 14
JavaScript | 12
eMoflon | 7
Fujaba | 5
Python | 4

Overall the prevalence of general purpose programming languages is higher than
expected. This might be explained by the large number of existing MTLs which
reduce the amount of total users per language while only four different GPLs
are used.

###  4.3 Sizes ($\xi_{12}$,$\xi_{14}$)

The size distribution of meta-models ($x_{15}$) transformed by participants is
shown in Figure 6. On the x-axis the given intervals of meta-model sizes are
shown and on the y-axis the distribution for each participant is shown. For
example, the first ridge line at the bottom of Figure 6 shows the answers of a
participant who has stated that 100% of their transformations revolve around
meta-models with 10 or less meta-model elements.

Figure 6: Distribution of meta-model sizes per participant

The figure illustrates that most transformations involve meta-models with 20
to 100 meta-model elements. Moreover, most participants have some experience
with small meta-models while only a handful of them has experience with
transformations involving large meta models of more than 1.000 elements.

The size distribution of model transformations ($x_{17}$) written by
participants is shown in Figure 7. Similarly to the meta-model sizes, the
figure illustrates that most participants have some experience with small
transformations of sizes up to 100 lines of code. Most also have experience
with large transformations up to 1.000 lines of code. More than 25% of all
participants also have experience with large and very large transformations
ranging from 5.000 up to more than 10.000 lines of transformation code.

Figure 7: Distribution of transformation sizes per participant

Overall the experience of our participants includes many moderately large to
large transformations. This strengthens us in the assumption that their
answers are meaningful for our study.

###  4.4 Conceptual distance between meta-models ($\xi_{16}$)

The similarity distribution of meta-models involved in the transformations of
our participants is shown in Figure 8 for the similarity of meta-model
structures ($x_{18}$) and Figure 9 for the similarity of data types
($x_{19}$). Both show a even mix between structurally similar and distant
meta-models as well as similar and dissimilar attribute types within the
elements that are transformed into each other.

Figure 8: Distribution of input output meta-model structure similarity Figure
9: Distribution of input output meta-model attribute types similarity

###  4.5 Meta-model quality ($\xi_{17}$)

Participants agreed that the vast majority of meta-models they transform are
well structured ($x_{20}$). This means there is little to no additional burden
put onto development solely due to unfavourably structured meta-models. The
distribution of structure assessment per participant is shown in Figure 10.

Figure 10: Distribution of structure quality of meta-models per participant

The situation is different with documentation ($x_{21}$). Most participants
stated that they have experience with badly or even very baldy documented
meta-models Figure 11. For many participants, this constitutes the majority of
meta-models they work with.

Figure 11: Distribution of meta-model documentation quality per participant

##  5 Results

In this section, we present the results of our analysis of the questionnaire
responses using universal structure modelling structured around the research
questions RQ1-4. The quantitative results for all influences between MTL
capabilities and MTL properties are shown in Table 3 in Appendix A. On the
x-axis the different MTL Properties are shown. On the y-axis the MTL
Capabilities are shown. The first number in a cell describes the average
simulated effect. The second number describes the overall explained absolute
deviation. The third number shows the significance value. A significance value
lower or equal to 0.01* (the chosen significance level) is indicated with one
asterisks. The effect strengths of moderation effects can be found in Tables
7, 8, 13, 11, 9, 12, 4, 10, 5 and 6 in Appendix A. Each table describes the
moderation effect of one of the moderating factors on all influences between
MTL Capabilities and MTL Properties.

The rest of this section presents our results in context of the four research
questions. We focus on the most salient influences that we deem interesting
for the respective research question. Detailed interpretation and discussion
of the implications of the presented results are done in Section 6.

###  5.1 RQ1: Which of the hypothesised interdependencies withstands a test of
significance? & RQ4: What additional interdependencies arise from the analysis
that were not initially hypothesised?

Our first research question is aimed at evaluating the accuracy of the
structure model developed in the previous study Hoeppner2022 (25). We do so by
subjecting all hypothesised influences to a significance test during analysis.
The significance test can also be used to directly gain insights into
interdependencies missed in the initial model. Thus we discuss both the
rejection of previously hypothesised influences as well as the extension of
the model through newly discovered significant interdependencies in this
section.

Most initially hypothesised influences withstand the test of significance but
there are several exceptions. Most notably all but one(Maintainability) of the
hypothesised influences of Bidirectionality functionality of MTLs have to be
rejected. This means that from our results we can not conclude that the
presence of Bidirectionality functionality in a language changes how people
perceive the Comprehensibility, Ease of Writing Expressiveness, Tool Support,
Productivity and Reusability of the language.

Similarly, half of the influences on Ease of Writing and Expressiveness are
also rejected. This means that the presence of Bidirectionality,
Incrementality, Model Management and Model Traversal functionality do not
change how people perceive the Ease of Writing transformations with a
language. And that the presence of Bidirectionality, Incrementality, Model
Navigation, Model Traversal and Reuse functionality do not change how people
perceive the Expressiveness of a language.

On a more positive note. The hypothesised influences on Comprehensibility are
confirmed (apart from the one exerted by Bidirectionality). The same goes for
Productivity and Reusability.

We also found that the perceived quality in Tool Support and Maintainability
are influenced by most of the MTL Properties. A result that was not apparent
from previous interview study. Tool support was hypothesised to be influenced
only by the chosen language and Maintainability only by Bidirectionality and
Mapping functionality. Our results however show, that the perceived
Maintainability of transformations written in a language is influenced by all
MTL functionality considered in this study with the exception of
Incrementality.

Moreover, several additional influences on Productivity and Reusability were
also discovered. The perceived Productivity and Reusability of transformations
in a language are influenced by Mappings, Model Management, Model Navigation,
Model Traversal, Pattern Matching, Reuse Mechanisms and Tracability
functionality.

Regarding the moderating effects, our findings suggest that a nuanced view is
warranted. The hypothesis that context moderates all influences on an MTL
Property still holds but the strength of the moderation effects varies
greatly.

As hypothesised, we are able to observe that Comprehensibility and Ease of
Writing are the two properties moderated by the most context variables. But
the moderation is only significant for a hand full of influences on these
properties. This can be seen e.g. in the moderation effects of Meta-Model Size
on influences on Comprehensibility depicted in Table 5 in Appendix A. Changes
in the Meta-model sizes participants worked with had next to no effect on how
their usage of Bidirectionality functionality affected their view on the
Comprehensibility of transformations. The impact on the influence of Model
Management on Comprehensibility is orders of magnitudes higher.

Another observation that stands out is the impact of Language Choice and
Language Experience. The moderation effects of both variables are negligible
or even 0 for all influences. We believe this is due to the large number of
languages considered in this study. It makes analysing the effects of choosing
one of the languages difficult.

Overall the results for research questions RQ1 & 4 suggest that our initial
structure model contains many relevant interdependencies but several more have
to be considered as well. We do have to reject several direct influences due
to low significance and moderation effects have to be considered on a per
influence basis instead of being generalised for each MTL Property.

###  5.2 RQ2: How strong are the influences of model transformation language
capabilities on the properties thereof?

Our second research question is intended to provide numbers that can help to
identify the most important factors to consider when evaluating the advantages
and disadvantages of model transformation languages empirically. We do this by
considering both the average simulated effect of influences calculated by
NEUSREL as well as the overall explained absolute deviation of influences
compared to each other. As explained earlier in this section all numbers can
be found in Table 3.

Table 3: Average simulated effect, overall explained absolute deviation and
significance of direct influences

|  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  -3.2e-07/ 3.3e-07/ 1 |  6.6e-08/ 9.2e-05/ 1 |  1.5e-09/ 4.2e-06/ 1 |  -1.5e-05/ 2.0e-05/ 1 |  -0.06/ 0.05/ 0.1* |  -8.3e-06/ 8.9e-07/ 1 |  1.4e-06/ 5.5e-06/ 1
Incrementality |  0.02/ 0.01/ 0.01* |  -3.5e-07/ 0.0003/ 1 |  7.4e-07/ 0.002/ 1 |  0.0002/ 0.0002/ 1 |  0.0004/ 1.7e-05/ 1 |  -0.0001/ 6.5e-07/ 1 |  0.02/ 0.005/ 0.05
Mappings |  0.03/ 0.01/ 0.01* |  -1.8e-05/ 0.01/ 0.01* |  -6.8e-06/ 0.01/ 0.01* |  -0.0116/ 0.01/ 0.01* |  -0.000793/ 0.003/ 1 |  -0.005/ 0.04/ 0.01* |  -0.02/ 0.01/ 0.01*
Model Management |  0.03/ 0.1/ 0.01* |  4.4e-05/ 0.003/ 1 |  0.0006/ 0.006/ 0.01* |  0.02/ 0.07/ 0.01* |  0.03/ 0.04/ 0.01* |  -0.0005/ 0.05/ 0.01* |  0.06/ 0.03/ 0.01*
Model Navigation |  -0.01/ 0.03/ 0.01* |  -2.3e-05/ 0.01/ 0.01* |  5.9e-05/ 0.005/ 0.05 |  0.06/ 0.2/ 0.01* |  -0.004/ 0.05/ 0.01* |  0.05/ 0.07/ 0.01* |  -0.08/ 0.08/ 0.01*
Model Traversal |  0.008/ 0.009/ 0.01* |  2.1e-07/ 0.002/ 1 |  -9.4e-05/ 0.0005/ 1 |  -0.09/ 0.2/ 0.01* |  0.07/ 0.1/ 0.01* |  -0.003/ 0.03/ 0.01* |  0.007/ 0.01/ 0.01*
Pattern Matching |  0.05/ 0.07/ 0.01* |  -8.4e-05/ 0.01/ 0.01* |  -0.0001/ 0.006/ 0.01* |  0.05/ 0.06/ 0.01* |  -0.04/ 0.08/ 0.01* |  0.005/ 0.1/ 0.01* |  -0.06/ 0.06/ 0.01*
Reuse Mechanisms |  0.1/ 0.08/ 0.01* |  -7.8e-05/ 0.008/ 0.01* |  -0.0002/ 0.002/ 1 |  0.1/ 0.1/ 0.01* |  0.1/ 0.2/ 0.01* |  0.1/ 0.2/ 0.01* |  0.2/ 0.1/ 0.01*
Traceability |  0.29/ 0.12/ 0.01* |  0.002/ 0.02/ 0.01* |  -2.1e-05/ 0.007/ 0.01* |  -0.05/ 0.2/ 0.01* |  -0.02/ 0.1/ 0.01* |  0.04/ 0.05/ 0.01* |  0.08/ 0.09/ 0.01*

Please note that the significance values obtained through the NEUSREL tool may
exhibit reduced accuracy compared to standard approaches due to the
bootstrapping method used for their estimation.

Overall the effects identified in our analysis are lower than anticipated.
They range from 0.29 down till 6.5e-8. We expected some effects to be low,
mainly those from non significant interdependencies, but the fact that even
significant effects are in the order of 0.01 is surprising. We assume this
stems from the large number of variables that are involved and the overall
complexity of the matter under investigation. Nonetheless we believe there are
meaningful insights that can be drawn when comparing the influences for each
MTL Property with each other.

Of the influences hypothesised from our previous interview study Traceability
is the most impactful MTL Capability. Its usage exerts the highest influence
on perceived Comprehensibility with $0.29$. Similarly it has the highest
influence for Ease of Writing though with a value of $0.0021$ the effect is
small. We were, however, already able to show empirical evidence that MTLs
utilising automatic trace handling provide clear advantages for writing
transformations compared to GPLs Hoeppner2021 (26).

For the properties Tool Support, Maintainability and Productivity the
availability of Reuse Mechanisms seems to be the strongest driving factor with
an average simulated effect of $0.1$, $0.1$, $0.1$ and $0.2$, respectively. No
other factor has an ASE or effect strength as high as Reuse Mechanisms for
these properties. This result is surprising as the influences were not raised
even once during our interview study.

Overall, automatic tracing and reuse mechanisms appear to be the most
influential factors for MTL properties. This suggests to us two main pathways
for further research. First, to improve model transformation languages more
research should be devoted to developing effective ways to reuse
transformations or parts of transformations. From our experience, current
mechanism are hard to use and are especially unsuited for different use-cases.
Secondly, the first area to address for improved adoption of model
transformation concepts in general purpose languages should be the development
of mechanisms for automatic trace handling.

###  5.3 RQ3: How strong are moderation effects expressed by the contextual
factors use-case, skills & experience and MTL choice?

As expressed in Section 5.1 the results of our analysis suggest that a more
nuanced view of moderation effects is warranted. In this section we go into
detail on these nuances.

As hypothesised the size of meta-models moderates the influences on
Comprehensibility. The moderation strength differs greatly between the
different causing factors though. For example, Meta-model size exerts the
strongest moderation on the influence of Model Management onto
Comprehensibility with $0.14$. All other moderation effects are far lower. The
second highest moderation effect, the moderation of Meta-model size on the
influence of Traceability on Comprehensibility, is about half es strong
($0.0778$) and the lowest, the moderation of Meta-model size on the influence
of Bidirectionality functionality on Comprehensibility, is only $0.0009$. The
moderations make sense intuitively as larger meta-models would make
implementing these tasks manually more labour intensive and thus clutter the
code unnecessarily.

Model size exerts similar moderation effects as meta-model size. Its strongest
moderation effect is also on the influence of Model Management on
Comprehensibility ($0.36$). Moreover, Model size also strongly moderates the
influence of Traceability functionality on the Ease of Writing transformations
($0.17$). Most other moderation effects of Model size are far lower than
$0.1$.

Transformation size seems to be the most relevant moderating factors across
the board. It has many noteworthy moderation effects on all influences of MTL
Capabilities on Tool Support, none being less than $0.16$, and Productivity,
most being above $0.12$. We assume this is because the larger transformations
get, the more reliant developers are on tooling and abstractions that reduce
the development effort.

Another interesting effect we found is, that developer experience moderates
the influence of many of the domain specific abstractions, e.g. Mappings and
Model Traversal, on Productivity. This makes sense because these specific
features often break with how developers are used to develop programs and thus
need practice to use them effectively.

The semantic gap between input and output meta-models exerts its moderation
strongest on the influences on Maintainability. Most notable are the
moderations on the influences of Model Traversal ($0.194$), Pattern Matching
($0.239$) and Reuse Mechanisms ($0.237$).

Lastly, there is a strong moderation effect of the meta-model sanity onto the
influence of Model Management facilities and Bidirectionality on
Comprehensibility. Both being about $0.2$. This makes sense as badly
structured or poorly documented meta-models are harder to handle and thus the
tasks revolving around working with the structure are most influenced by that.

Overall the size of transformations is in our opinion the most relevant
moderating variable. The assumption on the relevance of language choice could
however not be confirmed. This is most likely due to the large amount of
languages each participant has had experience with which weakens the ability
to elicit the effect of differences of language choice between participants.

##  6 Discussion

The results of our analysis provide useful insights for research on model
transformation languages. In this section, we discuss the implications of our
results for evaluation and development of MTLs. Additionally, we provide a
critical evaluation of our methodology with regards to the goals of this
study.

###  6.1 Implications of results

The topic of influences on the quality properties of model transformation
language is vastly complex, as reflected in the already large structure model
which we set out to analyse. While we were able to reject some of the
hypothesised influences, our analysis also identified several new influences.
As a result, the structure model depicting the influences grew in complexity,
further highlighting the need for comprehensive studies of the factors that
influence MTL quality properties. The updated structure model can be seen in
Figure 12. It contains 36 more interdependencies than the one we started our
analysis with.

Our analysis produced a number of interesting observations that have important
implications for further research. In particular, we now discuss the
implications for empirical evaluations. Additionally, we highlight the
implications of our results for further development of MTLs and domain-
specific features thereof.

Bidirectionality $\xi_{1}$ Incrementality $\xi_{2}$ Mappings $\xi_{3}$ Model
Management $\xi_{4}$ Model Navigation $\xi_{5}$ Model Traversal $\xi_{6}$
Pattern Matching $\xi_{7}$ Reuse Mechanisms $\xi_{8}$ Traceability $\xi_{9}$
Meta-model Size $\xi_{13}$ Model Size $\xi_{14}$ Transformation Size$\xi_{15}$
Bidirectional Use$\xi_{18}$ Language Choice $\xi_{10}$ Language Skills
$\xi_{11}$ Experience $\xi_{12}$ I/O Semantic gap $\xi_{16}$ Meta-model sanity
$\xi_{17}$ Incremental Use $\xi_{19}$ Comprehensi-bility $\eta_{1}$
$\zeta_{1}$ Ease of Writing $\eta_{2}$ $\zeta_{2}$ Expressiveness $\eta_{3}$
$\zeta_{3}$ Tool Support $\eta_{4}$ $\zeta_{4}$ Maintainability $\eta_{5}$
$\zeta_{5}$ Productivity $\eta_{6}$ $\zeta_{6}$ Reusability $\eta_{7}$
$\zeta_{1}$ Figure 12: Structure model depicting the confirmed influence and
moderation effects of factors on MTL properties.

####  6.1.1 Suggestions for further empirical evaluation studies

Traceability is one of the most important factors to consider when it comes to
the development of model transformations. This is because it has the strongest
influence on the perceived quality of both the ease of writing and the
comprehensibility of the resulting code. It is crucial to consider scenarios
where tracing is involved in order to properly evaluate the value of MTL
abstractions for writing and comprehending transformations. Additionally, it
is important to evaluate scenarios where tracing is not necessary to
understand the difference that MTL abstractions can make. To truly understand
the relevance of this feature, it is also important to assess how many real-
world use cases require it. By taking all of these factors into account, it is
possible to gain a comprehensive understanding of the value of MTL
abstractions for writing and comprehending transformations.

For evaluation of Maintainability, Reuse Mechanisms as well as Model Traversal
functionality are important capabilities to consider. We therefore believe
that researchers focusing on such an evaluation must make sure to use
transformations that utilise these capabilities. Moreover, the most important
context to consider is the semantic gap between input and output meta-models.
Empirical evaluations focusing on maintainability should therefore make sure
to evaluate transformation cases with varying degrees of differences between
input and output meta-models. These studies should then analyse how much the
effectiveness of MTLs and GPLs changes in light of the semantic gap between
input and output.

When selecting transformations for evaluation, it is essential to consider
their size. Our results have shown that size has the most significant impact
on the influence of other factors on properties. Put differently, the larger
the transformation, the more noticeable the effect of all capabilities will
be. As such, it is imperative to focus on large transformation use-cases when
designing a study to evaluate MTLs.

####  6.1.2 Suggestions on language development

For us, the most surprising finding of this study is the importance of reuse
functionality. The quality attributes tool support, maintainability,
productivity and reusability are all most influenced by it. This is especially
surprising because there was no indication of this in our interviews
Hoeppner2022 (25). We suppose this influence stems from the fact that reuse
mechanisms allow for more abstraction and thus less code that can be developed
and maintained more efficiently.

As a result we believe that more focus should be put on developing
transformation specific reuse mechanisms. We are aware that some languages,
e.g. ATL, already provide general reuse mechanisms through concepts like
inheritance. However, these concepts are limited by the fact that they rely on
the object-oriented nature of the involved models. This means that they can
only be used to define reusable code within transformations of a single meta-
model. Defining transformation behaviour that can be reused between different
meta-models is not possible. But this would be important to further reduce
redundancy in transformation development.

As result, we believe, that development of reuse mechanisms tailored to MTs is
important to focus on. In order to stand out compared to the reuse mechanisms
of GPLs, it may be valuable to explore ways to define and reuse common
transformation patterns independently of meta-models. Higher order
transformations are sometimes used to allow reuse too Kusel2015 (36), but from
our experience current implementations are too cumbersome to be used
productively. Chechik2016 (10) provide a number of suggestions for
transformation specific reuse mechanisms but to the best of our knowledge
there exist no implementations of their concepts.

###  6.2 Interesting observations outside of USM

When discussing model transformation languages, it is often stated that they
are only demonstrated on ‘toy examples’ that have little to no real world
value. This argumentation has for example been raised several times in our
previous interview study Hoeppner2022 (25). However, the demographic data
collected in our study disputes this.

There are several participants that stated to have worked solely on small
transformations with small meta-and input models. But this group is opposed by
a similarly large group of participants that have worked with huge
transformations, dissimilar and large meta-models as well as large inputs.
From this we conclude, that there are large use-cases where model
transformations and MTLs are applied but they rarely get described in
publications. It seems likely that such examples are not used for highlighting
important aspects authors want to discuss due to the space describing such
cases would take up. However, we argue that it is paramount that such case-
studies are published to diminish the cynicism that MTLs are only useful for
small examples.

Another noteworthy observation based on the demographic data of our
participants is that documentation pertaining meta-models is predominantly
perceived as inadequate. We believe that this is primarily due to the fact
that many of meta-models stem from research projects that prioritize
expeditious prototyping over the long-term viability of the artefacts.
Nonetheless, we are convinced that there is an urgent need to enhance the
documentation surrounding model transformations. This issue is not limited
solely to the meta-models, but also extends to the languages that are known
for their challenging learning curve because of lack of tutorials Hoeppner2022
(25).

###  6.3 Critical Assessment of the used methodology

The appeal of using structural equation modelling for analysing the responses
to our survey was to have a method of analysis that can be used to investigate
a complex hypothesis system in its entirety. Moreover, analysis is straight
forward after an initial setup due to the sophisticated tooling for this
methodology. Instead of presenting participants with a case that they should
assess we also opted for querying them on their overall assessment of MTL
quality attributes. These design decisions have implications and ramifications
that we discuss in this section.

First, the effects observed in our study are small. We assume this stems from
the intricate and large structure model and the comparatively small sample
size. As explained in Section 3 it is suggested to have between 5 to 10 times
as many participants as the largest number of parameters to be estimated in
each structural equation. In light of the newly discovered paths in our
structure model, the 113 total participants are close to the minimum sample
size required. Moreover, because of the large number of influences we do
expect the influence of a single factor to be much smaller than in structure
models where only 2-3 factors are relevant. The results therefore reinforce
our assessment that it is a very complex topic.

We also ran into some difficulties when using NEUSREL to analyse our data. The
structure model was so large that sometimes the tool crashed during
calculations. The online tooling to set everything up was also painfully
inefficient leading to more problems during setup like browser crashes. It
took us some trial and error to find a way to get everything set up and run
the analysis without crashes.

We chose to execute a study based on our study design in hopes of producing a
complete theory independent of the use case under consideration. The results
exhibit less effect strength but we believe them to be more externally valid.
Nonetheless, we think that several additional studies need to be conducted to
confirm our results for different use-cases.

##  7 Threats to validity

Our study is carefully designed and follows standard procedures for this type
of study. There are, however still threats to validity that stem from design
decisions and limitations. In this section we discuss these threats.

###  7.1 Internal Validity

Internal validity is threatened by manual errors and biases of the involved
researchers throughout the process.

The two activities where such errors and biases can be introduced are the
subject selection and question creation. The selection criteria for study
subjects is designed in such a way, that no ambiguities exist during
selection. This prevents researcher bias.

The survey questions and answers to the questions pose another threat to
internal validity. We used neutral questions to prevent subconsciously
influencing the opinions of research subjects. We also provide explanations
for ambiguous terms used in the survey. However, there are several instances
where we can not fully ensure that each participant interprets terms the same
way. The questions on quality properties of model transformation languages
allow room for interpretation in that we do not provide a clear metric what
terms such as ‘Very Comprehensible’ or ‘Very Hard to write’ mean. Similarly,
the questions on meta-model quality leave room for interpretation on the side
of participants. We opted for this limitation because there are no universal
ways to quantify such estimates and because the subjective assessment is what
we want to collect. The reason for this is, that subjective experiences are
the main driving factor for all discussions on development when people are the
main subject.

To ensure overall understandability and prevent errors in the setup of the
survey we used a pilot study.

###  7.2 External Validity

External validity is threatened by our subject sampling strategy and the
limitations on the survey questions imposed by the complexity of the subject
matter.

We utilise convenience sampling. Convenience sampling can limit how
representative the final group of interviewees is. Since we do not know the
target populations makeup, it is difficult to asses the extend of this
problem.

Using research articles as a starting point introduces a bias towards
researchers. There is little potential to mitigate this problem during the
study design, because there exists no systematic way to find industry users.

Due to the complexity and abstractness of the concepts under investigation, a
measurement via reflective of formative indicators is not possible. Instead we
use single item questions. We further assume that positive and negative
effects of a feature are more prominent if the feature is used more
frequently. This can have a negative effect on the external validity of our
results. However, we consciously decided for these limitations to be able to
create a study that concerns itself with all factors and influences at once.

###  7.3 Construct Validity

Construct validity is threatened by inappropriate methods used for the study.

Using the results of online surveys as input for structural equation modelling
techniques is common practice in market research Weiber2021 (55). It is less
common in computer science. However, we argue that for the purpose of our
study it is an appropriate methodology. This is because the goal of extracting
influence strengths and moderation effects of factors on different properties
aligns with the goals of market research studies that employ structural
equation modelling.

###  7.4 Conclusion Validity

Conclusion validity is mainly threatened by biases of our survey participants.

It is possible that people who do research on model transformation languages
or use them for a long time are more likely to see them in a positive light.
As such there is the risk that too little experiences will be reported on in
our survey. However, this problem did not present itself in a previous study
by us on the subject matter Hoeppner2022 (25). In fact researchers were far
more critical in dealing with the subject. As a result, there might be a
slight positive bias in the survey responses, but we believe this to be
negligible.

##  8 Related Work

There are numerous works that explore the possibilities gained through the
usage of MTLs such as automatic parallelisation 9146715 (47, 6, 5),
verification lano2015framework (37, 32) or simply the application of difficult
transformations 10.1007/978-3-540-75209-7_30 (1). There is, however, only a
small amount of works trying to evaluate the languages to gain insights into
where specific advantages or disadvantages associated with the use of MTLs
originate from. Several other works that can be related to our study also
exist. The related work is divided into studies focused on the investigation
of properties of model transformation languages and empirical studies on model
transformation languages.

###  8.1 Studies on the Properties of Model Transformation Languages

Burgueno2019 (9) conducted on a online survey and open discussion at the 12th
edition of the International Conference on Model Transformations (ICMT’2019).
The goal of the survey was to identify reasons why developers decided to use
or dismiss MTLs for writing transformations. They also tried to gauge the
communities sentiment on the future of model transformation languages. At
ICMT’2019, where the results of the survey were presented, they then held an
open discussion on this topic and collected the responses of participants.
Their results show that MTLs have fallen in popularity. They attribute this to
3 types of issues, technical issues, tooling issues and social issues, as well
as the fact that GPLs have assimilated many ideas from MTLs. The results of
their study are a major driver in the motivation of our work. While they
identified issues and potential avenues for future research, their results are
qualitative and broad which we try to improve upon with our study.

In a prior study of ours Goetz2020 (19), we conducted a structured literature
review which forms the basis of much of our work since then. The literature
review aimed at extracting and categorising claims about the advantages and
disadvantages of model transformation languages as well as the state of
empirical evaluation thereof. We searched over 4000 publication for this
purpose and extracted 58 that directly claim properties of MTLs. In total 137
claims were found and categorised into 15 quality properties of model
transformation languages. The results of the study show that little to no
empirical studies to evaluate MTLs exist and that there is a severe lack of
context and background information that further hinders their evaluation.

Lastly, there is our interview study Hoeppner2022 (25) the data of which forms
the basis for the reported study. We interviewed 56 people on what they
believe the most relevant factors are that facilitate or hamper their
advantages for different quality properties identified in the prior literature
review. The interviews brought forth insights into factors from which the
advantages and disadvantages of MTLs originate from as well as suggested a
number of moderation effects on the effects of these factors. These results
for the data basis for this study.

###  8.2 Empirical Studies on Model Transformation Languages

Hebig2018 (20) report on a controlled experiment to evaluate how the use of
different languages, namely ATL, QVT-O and Xtend affects the outcome of
students solving several transformation tasks. During the study student
participants had to complete a series of three model transformation tasks. One
task was focused on comprehension, one task focused on modifying an existing
transformation and one task required participants to develop a transformation
from scratch. The authors compared how the use of ATL, QVTo and Xtend affected
the outcome of each of the tasks. Unfortunately their results show no clear
evidence of an advantage when using a model transformation language compared
to Xtend. However, they concede that the conditions under which the
observations are made, were narrow.

We published a study on how much complexity stems from what parts of ATL
transformations gotz2020investigating (16) and compared these results with
data for transformations written in Java Hoeppner2021 (26) to elicit
advantageous features in ATL and to explore what use-cases justify the use of
a general purpose language over a model transformation language. In the study,
the complexity of transformations written in ATL were compared to the same
transformations written in Java SE5 and Java SE14 allowing for a comparison
and historical perspective. The Java transformations were translated from the
ATL transformations using a predefined translation schema. The results show
that new language features in Java, like the Streams API, allow for
significant improvement over older Java code, the relative amount of
complexity aspects that ATL can hide stays the same between the two versions.

Gerpheide2016 (15) use a mixed method study consisting of expert interviews, a
literature review and introspection, to formalize a quality model for the QVTo
model transformation standard. The quality model is validated using a survey
and used to identify the necessity of quality tool support for developers.

We know of two study templates for evaluating model transformation languages
that have been proposed but not yet used. Kramer2016 (34) propose a template
for a controlled experiment to evaluate comprehensibility of MTLs. The
template envisages using a questionnaire to evaluate the ability of
participants to understand what presented transformation code does. The
influence of the language used for the transformation should then be measured
by comparing the average number of correct answers and average time spent to
fill out the questionnaire. Strueber2016 (53) also propose a template for a
controlled experiment. The aim of the study is to evaluate the benefits and
drawbacks of rule refinement and variability-based rules for reuse. The
quality of reusability is measured through measuring the comprehensibility as
well as the changeability collected in bug-fixing and modification tasks.

##  9 Conclusion

Our study provides the first quantification of the importance of model
transformation language capabilities for the perception of quality attributes
by developers. It once again highlight the complexity of the subject matter as
the effect sizes of the influences are small and the final structure model
grew in size.

As demonstrated by the amount of influences contained in the structure model
many language capabilities need to be considered when designing empirical
studies on MTLs. The results however point towards Traceability and Reuse
Mechanisms as the two most important MTL capabilities. Moreover, the size of
the transformations provides the strongest moderation effects to many of
influences and is thus the most important context factor to consider.

Apart from implications for further empirical studies our results also point a
clear picture for further language development. Transformation specific reuse
mechanisms should be the main focus as shown by their relevance for many
development lifecycle focused quality attributes such as Maintainability and
Productivity.

## Declarations

Conflict of Interests The authors have no competing interests to declare that
are relevant to the content of this article.

## References

  * (1) Kyriakos Anastasakis, Behzad Bordbar, Geri Georg and Indrakshi Ray  “UML2Alloy: A Challenging Model Transformation”  In _Model Driven Engineering Languages and Systems_ Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 436–450
  * (2) Anthony Anjorin, Thomas Buchmann and Bernhard Westfechtel  “The Families to Persons Case”, TTC’17, 2017
  * (3) Thorsten Arendt, Enrico Biermann, Stefan Jurack, Christian Krause and Gabriele Taentzer  “Henshin: Advanced Concepts and Tools for In-Place EMF Model Transformations”  In _Model Driven Engineering Languages and Systems_ , MODELS 2010, 2010  DOI: 10.1007/978-3-642-16145-2_9
  * (4) András Balogh and Dániel Varró  “Advanced Model Transformation Language Constructs in the VIATRA2 Framework”  In _Proceedings of the 2006 ACM Symposium on Applied Computing_ , SAC ’06, 2006  DOI: 10.1145/1141277.1141575
  * (5) Amine Benelallam, Abel Gómez, Massimo Tisi and Jordi Cabot  “Distributed Model-to-model Transformation with ATL on MapReduce”  In _Proceedings of the 2015 ACM SIGPLAN International Conference on Software Language Engineering_ , 2015, pp. 37–48
  * (6) Enrico Biermann, Claudia Ermel and Gabriele Taentzer  “Lifting parallel graph transformation concepts to model transformation based on the eclipse modeling framework”  In _Electronic Communications of the EASST_ 26, 2010
  * (7) Alan W Brown, Jim Conallen and Dave Tropeano  “Introduction: Models, modeling, and model-driven architecture (mda)”  In _Model-Driven Software Development_ Springer, 2005, pp. 1–16  DOI: 10.1007/3-540-28554-7_1
  * (8) Frank Buckler and Thorsten Hennig-Thurau  “Identifying hidden structures in marketing’s structural models through universal structure modeling”  In _Marketing ZFP_ 30.JRM 2 Verlag Franz Vahlen, 2008, pp. 47–66
  * (9) Loli Burgueño; Jordi Cabot and Sébastien Gérard  “The Future of Model Transformation Languages: An Open Community Discussion”  In _Journal of Object Technology_ , 2019  DOI: 10.5381/jot.2019.18.3.a7
  * (10) Marsha Chechik, Michalis Famelis, Rick Salay and Daniel Strüber  “Perspectives of Model Transformation Reuse”  In _Integrated Formal Methods_ Cham: Springer International Publishing, 2016, pp. 28–44
  * (11) Jesús Sánchez Cuadrado, Jesús García Molina and Marcos Menarguez Tortosa  “RubyTL: A Practical, Extensible Transformation Language”  In _Model Driven Architecture – Foundations and Applications_ , ECMDA-FA 2006, 2006  DOI: 10.1007/11787044_13
  * (12) K. Czarnecki and S. Helsen  “Feature-based survey of model transformation approaches”  In _IBM Systems Journal_ , 2006  DOI: 10.1147/sj.453.0621
  * (13) Christoph Fuchs and Adamantios Diamantopoulos  “Using single-item measures for construct measurement in management research: Conceptual issues and application guidelines”  In _Die Betriebswirtschaft_ 69.2 Schaeffer Poeschel Verlag, 2009, pp. 195
  * (14) Lars George, Arif Wider and Markus Scheidgen  “Type-Safe Model Transformation Languages as Internal DSLs in Scala”  In _Theory and Practice of Model Transformations_ , ICMT 2012, 2012  DOI: 10.1007/978-3-642-30476-7_11
  * (15) Christine M. Gerpheide, Ramon R. H. Schiffelers and Alexander Serebrenik  “Assessing and improving quality of QVTo model transformations”  In _Software Quality Journal_ 24.3, 2016, pp. 797–834  DOI: 10.1007/s11219-015-9280-8
  * (16) Stefan Götz and Matthias Tichy  “Investigating the Origins of Complexity and Expressiveness in ATL Transformations.”  In _J. Object Technol._ 19.2, 2020, pp. 12–1
  * (17) Daniel Graziotin, Per Lenberg, Robert Feldt and Stefan Wagner  “Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines”  In _ACM Trans. Softw. Eng. Methodol._ 31.1 New York, NY, USA: Association for Computing Machinery, 2021  DOI: 10.1145/3469888
  * (18) Raffaela Groner, Katharina Juhnke, Stefan Götz, Matthias Tichy, Steffen Becker†, Vijayshree Vijayshree and Sebastian Frank  “A Survey on the Relevance of the Performance of Model Transformations” OPEN REGULAR ISSUE  In _Journal of Object Technology_ 20.2, 2021, pp. 2:1–27  DOI: 10.5381/jot.2021.20.2.a5
  * (19) Stefan Götz, Matthias Tichy and Raffaela Groner  “Claimed advantages and disadvantages of (dedicated) model transformation languages: a systematic literature review”  In _Software and Systems Modeling_ 20.2, 2021, pp. 469–503  DOI: 10.1007/s10270-020-00815-4
  * (20) Regina Hebig, Christoph Seidl, Thorsten Berger, John Kook Pedersen and Andrzej Wąsowski  “Model Transformation Languages Under a Magnifying Glass: A Controlled Experiment with Xtend, ATL, and QVT”  In _Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ESEC/FSE 2018, 2018  DOI: 10.1145/3236024.3236046
  * (21) Felienne Hermans, Martin Pinzger and Arie Deursen  “Domain-Specific Languages in Practice: A User Study on the Success Factors”  In _Model Driven Engineering Languages and Systems_ , MODELS 2009, 2009  DOI: 10.1007/978-3-642-04425-0_33
  * (22) Georg Hinkel  “NMF: A Modeling Framework for the. NET Platform”  KIT, 2016
  * (23) Georg Hinkel and Erik Burger  “Change propagation and bidirectionality in internal transformation DSLs”  In _Software & Systems Modeling_, 2019  DOI: 10.1007/s10270-017-0617-6
  * (24) Tassilo Horn  “Model Querying with FunnyQT”  In _Theory and Practice of Model Transformations_ Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 56–57
  * (25) Stefan Höppner, Yves Haas, Matthias Tichy and Katharina Juhnke  “Advantages and disadvantages of (dedicated) model transformation languages”  In _Empirical Software Engineering_ 27.6, 2022, pp. 159  DOI: 10.1007/s10664-022-10194-7
  * (26) Stefan Höppner, Timo Kehrer and Matthias Tichy  “Contrasting dedicated model transformation languages versus general purpose languages: a historical perspective on ATL versus Java based on complexity and size”  In _Software and Systems Modeling_ , 2021  DOI: 10.1007/s10270-021-00937-3
  * (27) Stefan Höppner and Matthias Tichy  “The Impact of Model Transformation Language Features on Quality Properties of MTLs: A Study Protocol”  arXiv, 2022  DOI: 10.48550/ARXIV.2209.06570
  * (28) Jendrik Johannes, Steffen Zschaler, Miguel A. Fernández, Antonio Castillo, Dimitrios S. Kolovos and Richard F. Paige  “Abstracting Complex Languages through Transformation and Composition”  In _Model Driven Engineering Languages and Systems_ , MODELS 2009, 2009  DOI: 10.1007/978-3-642-04425-0_41
  * (29) Frédéric Jouault, Freddy Allilaire, Jean Bézivin, Ivan Kurtev and Patrick Valduriez  “ATL: A QVT-like Transformation Language”  In _Companion to the 21st ACM SIGPLAN Symposium on Object-oriented Programming Systems, Languages, and Applications_ , OOPSLA ’06, 2006  DOI: 10.1145/1176617.1176691
  * (30) Nafiseh Kahani, Mojtaba Bagherzadeh, James R Cordy, Juergen Dingel and Daniel Varró  “Survey and classification of model transformation tools”  In _Software & Systems Modeling_, 2019  DOI: 10.1007/s10270-018-0665-6
  * (31) Brian W. Kernighan and Rob Pike  “The Unix Programming Environment”  Prentice Hall, Inc., 1984
  * (32) Jong-Won Ko, Kyung-Yong Chung and Jung-Soo Han  “Model transformation verification using similarity and graph comparison algorithm”  In _Multimedia Tools and Applications_ 74.20, 2015, pp. 8907–8920  DOI: 10.1007/s11042-013-1581-y
  * (33) Dimitrios S. Kolovos, Richard F. Paige and Fiona A. C. Polack  “The Epsilon Transformation Language”  In _Theory and Practice of Model Transformations_ , ICMT 2008, 2008  DOI: 10.1007/978-3-540-69927-9_4
  * (34) M. E. Kramer, G. Hinkel, H. Klare, M. Langhammer and E. Burger  “A controlled experiment template for evaluating the understandability of model transformation languages”  In _2nd International Workshop on Human Factors in Modeling, HuFaMo 2016; Saint Malo; France; 4 October 2016 through. Ed. : M. Goulao_ 1805, CEUR Workshop Proceedings  CEUR Workshop Proceedings, 2016, pp. 11–18
  * (35) Christian Krause, Matthias Tichy and Holger Giese  “Implementing Graph Transformations in the Bulk Synchronous Parallel Model”  In _Fundamental Approaches to Software Engineering_ Berlin, Heidelberg: Springer Berlin Heidelberg, 2014, pp. 325–339
  * (36) A. Kusel, J. Schönböck, M. Wimmer, G. Kappel, W. Retschitzegger and W. Schwinger  “Reuse in model-to-model transformation languages: are we there yet?”  In _Software & Systems Modeling_ 14.2, 2015, pp. 537–572  DOI: 10.1007/s10270-013-0343-7
  * (37) Kevin Lano, Tony Clark and S Kolahdouz-Rahimi  “A framework for model transformation verification”  In _Formal Aspects of Computing_ 27.1 Springer, 2015, pp. 193–235
  * (38) Tom Mens and Pieter Van Gorp  “A Taxonomy of Model Transformation”  In _Electronic Notes in Theoretical Computer Science (GraMoT 2005)_ , 2006  DOI: 10.1016/j.entcs.2005.10.021
  * (39) Andreas Metzger  “A systematic look at model transformations”  In _Model-driven Software Development_ Springer, 2005, pp. 19–33  DOI: 10.1007/3-540-28554-7_2
  * (40) Christopher Z Mooney, Christopher F Mooney, Christopher L Mooney, Robert D Duval and Robert Duvall  “Bootstrapping: A nonparametric approach to statistical inference”  sage, 1993
  * (41) OMG  “Model Driven Architecture (MDA), ormsc/2001-07-01”, 2001
  * (42) OMG  “Meta Object Facility(MOF)”, 2002
  * (43) OMG  “Object Constraint Language (OCL)”, 2014  URL: https://www.omg.org/spec/OCL/2.4/PDF
  * (44) OMG  “Meta Object Facility (MOF) 2.0 Query/View/Transformation Specification”, 2016  URL: https://www.omg.org/spec/QVT/About-QVT/
  * (45) Dave Raggett, Arnaud Le Hors and Ian Jacobs  “HTML 4.01 Specification”  In _W3C recommendation_ 24, 1999
  * (46) SAEMobilus  “Architecture Analysis and Design Language (AADL)”, 2004
  * (47) Jesus Sanchez Cuadrado, Loli Burgueno, Manuel Wimmer and Antonio Vallecillo  “Efficient execution of ATL model transformations using static analysis and parallelism”  In _IEEE Transactions on Software Engineering_ , 2020, pp. 1–1  DOI: 10.1109/TSE.2020.3011388
  * (48) Douglas Schmidt  “Guest Editor’s Introduction: Model-Driven Engineering”  In _COMPUTER-IEEE COMPUTER SOCIETY_ , 2006  DOI: 10.1109/MC.2006.58
  * (49) B. Selic  “The pragmatics of model-driven development”  In _IEEE Software_ 20.5, 2003, pp. 19–25  DOI: 10.1109/MS.2003.1231146
  * (50) S. Sendall and W. Kozaczynski  “Model transformation: the heart and soul of model-driven software development”  In _IEEE Software_ , 2003  DOI: 10.1109/MS.2003.1231150
  * (51) J. Sprinkle, M. Mernik, J. Tolvanen and D. Spinellis  “Guest Editors’ Introduction: What Kinds of Nails Need a Domain-Specific Hammer?”  In _IEEE Software_ 26.4, 2009, pp. 15–18  DOI: 10.1109/MS.2009.92
  * (52) Dave Steinberg, Frank Budinsky, Ed Merks and Marcelo Paternostro  “EMF: eclipse modeling framework”  Pearson Education, 2008
  * (53) Daniel Strüber and Anthony Anjorin  “Comparing Reuse Mechanisms for Model Transformation Languages: Design for an Empirical Study.”  In _HuFaMo@ MoDELS_ , 2016, pp. 27–32  Citeseer
  * (54) Arie Van Deursen and Paul Klint  “Domain-specific language design requires feature descriptions”  In _Journal of Computing and Information Technology_ , 2002  DOI: 10.2498/cit.2002.01.01
  * (55) Rolf Weiber and Daniel Mühlhaus  “Strukturgleichungsmodellierung: Eine anwendungsorientierte Einführung in die Kausalanalyse mit Hilfe von AMOS, SmartPLS und SPSS”  Springer-Verlag, 2021  DOI: 10.1007/978-3-658-32660-9

##  Appendix A USM Results for Moderation Effects

See Tables 7, 8, 13, 11, 9, 12, 4, 10, 5 and 6.

Table 4: Overview of moderation effects of meta-model size |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.0009 |  0.1644 |  0.0004 |  0.0341 |  0.0247 |  0.0197 |  0.0218
Incrementality |  0.0174 |  0.0003 |  0.0086 |  0.0335 |  0 |  0.0224 |  0.0054
Mappings |  0.018 |  0.0309 |  0.0058 |  0.0049 |  0 |  0.0012 |  0.0106
Model Management |  0.1389 |  0.0264 |  0.0006 |  0.023 |  0 |  0.0177 |  0.0199
Model Navigation |  0.001 |  0.0438 |  0.0019 |  0.0128 |  0.0032 |  0.0422 |  0.018
Model Traversal |  0.0382 |  0.059 |  0 |  0.0422 |  0.0023 |  0.037 |  0.0106
Pattern Matching |  0.0091 |  0 |  0.0003 |  0.03 |  0.0028 |  0.0418 |  0.0093
Reuse Mechanisms |  0.0495 |  0 |  0.0077 |  0.0542 |  0.0693 |  0.0943 |  0.0021
Traceability |  0.0778 |  0.0464 |  0.00001 |  0.0714 |  0.021 |  0.076 |  0.0223
Table 5: Overview of moderation effects of model size |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.1 |  0.066 |  0.036 |  0.048 |  0.03 |  0.069 |  0.012
Incrementality |  0.065 |  0.14 |  0.019 |  0.036 |  0.031 |  0.052 |  0.009
Mappings |  0.085 |  0.033 |  0.076 |  0.042 |  0.032 |  0.083 |  0.003
Model Management |  0.36 |  0.089 |  0.059 |  0.023 |  0.047 |  0.079 |  0.04
Model Navigation |  0.074 |  0.007 |  0.04 |  0.053 |  0.05 |  0.072 |  0.011
Model Traversal |  0.125 |  0.069 |  0.038 |  0.048 |  0.031 |  0.104 |  0.008
Pattern Matching |  0.133 |  0.052 |  0.038 |  0.056 |  0.024 |  0.036 |  0.014
Reuse Mechanisms |  0.096 |  0.00009 |  0.029 |  0.005 |  0.11 |  0.093 |  0.036
Traceability |  0.078 |  0.173 |  0.02 |  0.048 |  0.096 |  0.108 |  0.016
Table 6: Overview of moderation effects of transformation size |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.1 |  0.15 |  0 |  0.27 |  0.033 |  0.097 |  0.044
Incrementality |  0.086 |  0.0075 |  0.0086 |  0.33 |  0.043 |  0.092 |  0.050
Mappings |  0.13 |  0.0055 |  0 |  0.17 |  0.042 |  0.16 |  0.034
Model Management |  0.32 |  0.094 |  0.021 |  0.23 |  0.012 |  0.13 |  0.056
Model Navigation |  0.15 |  0.058 |  0.0022 |  0.25 |  0.049 |  0.13 |  0.064
Model Traversal |  0.069 |  0.045 |  0.0038 |  0.21 |  0.061 |  0.15 |  0.015
Pattern Matching |  0.16 |  0.055 |  0.040 |  0.28 |  0.075 |  0.12 |  0.055
Reuse Mechanisms |  0.21 |  0.019 |  0.0077 |  0.16 |  0.27 |  0.23 |  0.087
Traceability |  0.11 |  0.11 |  0 |  0.25 |  0.067 |  0.12 |  0.040
Table 7: Overview of moderation effects of the amount of bidirectional use-cases |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.067 |  0.011 |  0.136 |  0.101 |  0.026 |  0.092 |  0.010
Incrementality |  0.036 |  0.005 |  0.120 |  0.115 |  0.003 |  0.099 |  0.019
Mappings |  0.029 |  0.034 |  0.065 |  0.084 |  0.002 |  0.072 |  0.001
Model Management |  0.143 |  0.048 |  0.175 |  0.066 |  0.010 |  0.053 |  0.014
Model Navigation |  0.058 |  0.003 |  0.155 |  0.049 |  0.037 |  0.071 |  0.020
Model Traversal |  0.024 |  0.068 |  0.142 |  0.071 |  0.023 |  0.117 |  0.012
Pattern Matching |  0.045 |  0.006 |  0.110 |  0.092 |  0.002 |  0.058 |  0.002
Reuse Mechanisms |  0.040 |  0.026 |  0.130 |  0.220 |  0.100 |  0.190 |  0.013
Traceability |  0.065 |  0.120 |  0.150 |  0.160 |  0.060 |  0.150 |  0.012
Table 8: Overview of moderation effects of developer experience |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.05 |  8.4E-06 |  0.011 |  0.08 |  0.037 |  0.084 |  0.033
Incrementality |  0.045 |  0.0003 |  0.007 |  0.083 |  0.032 |  0.055 |  0.046
Mappings |  0.00032 |  0 |  0.0013 |  0.072 |  0.052 |  0.1 |  0.039
Model Management |  0.15 |  0.025 |  0.032 |  0.085 |  0.05 |  0.073 |  0.034
Model Navigation |  0.053 |  0.043 |  0.013 |  0.08 |  0.032 |  0.074 |  0.05
Model Traversal |  0.045 |  0.0008 |  0.011 |  0.1 |  0.024 |  0.14 |  0.054
Pattern Matching |  0.013 |  0.024 |  0.0052 |  0.081 |  0.032 |  0.078 |  0.048
Reuse Mechanisms |  0.047 |  0 |  0.0015 |  0.08 |  0.077 |  0.055 |  0.022
Traceability |  0.072 |  0.017 |  0.011 |  0.055 |  0.042 |  0.066 |  0.04
Table 9: Overview of moderation effects of the semantic gap between input and output |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.01 |  0.033 |  0.008 |  0 |  0.089 |  0.056 |  0.041
Incrementality |  0.006 |  0.067 |  0.009 |  0 |  0.119 |  0.046 |  0.045
Mappings |  0.013 |  0.033 |  0.002 |  0 |  0.053 |  0.049 |  0.043
Model Management |  0.1 |  0.023 |  0.021 |  0.009 |  0.163 |  0.052 |  0.054
Model Navigation |  0.027 |  0.01 |  0.001 |  0.007 |  0.112 |  0.1 |  0.043
Model Traversal |  0.023 |  0.032 |  0 |  0.002 |  0.194 |  0.067 |  0.029
Pattern Matching |  0.032 |  0.033 |  0 |  0.029 |  0.239 |  0.096 |  0.032
Reuse Mechanisms |  0.016 |  0.033 |  0.008 |  0.007 |  0.237 |  0.068 |  0.033
Traceability |  0.032 |  0.027 |  0.0003 |  0.012 |  0.184 |  0.058 |  0.038
Table 10: Overview of moderation effects of the sanity of involved meta-models |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0.2 |  0.0001 |  0.00002 |  0.035 |  0.025 |  0.074 |  0.045
Incrementality |  0.06 |  0.06 |  0.001 |  0.03 |  0.013 |  0.097 |  0.047
Mappings |  0.09 |  0.03 |  0.03 |  0.07 |  0.005 |  0.074 |  0.059
Model Management |  0.21 |  0.02 |  0.02 |  0.006 |  0.03 |  0.086 |  0.073
Model Navigation |  0.11 |  0.12 |  0.002 |  0.06 |  0.04 |  0.099 |  0.044
Model Traversal |  0.1 |  0.08 |  0 |  0.08 |  0.02 |  0.13 |  0.069
Pattern Matching |  0.09 |  0 |  0.01 |  0.05 |  0.01 |  0.07 |  0.081
Reuse Mechanisms |  0.11 |  0 |  0.008 |  0.03 |  0.05 |  0.094 |  0.059
Traceability |  0.11 |  0.001 |  0 |  0.05 |  0.05 |  0.14 |  0.079
Table 11: Overview of moderation effects of the amount of incremental use-cases |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0 |  0.06 |  0.039 |  0.0045 |  0.034 |  0 |  0.00031
Incrementality |  0 |  0.00067 |  0.012 |  0.0042 |  0.079 |  0 |  0.006
Mappings |  0 |  0.017 |  0.079 |  0.0018 |  0.061 |  0.0072 |  0.017
Model Management |  0.13 |  0.027 |  0.068 |  0.016 |  0.062 |  0.00038 |  0.039
Model Navigation |  0 |  0.24 |  0.066 |  0.037 |  0.098 |  0.033 |  0.023
Model Traversal |  0 |  0.047 |  0.025 |  0.028 |  0.083 |  0 |  0.0091
Pattern Matching |  0.052 |  0 |  0.024 |  0.00048 |  0.073 |  0.032 |  0.034
Reuse Mechanisms |  0.047 |  0 |  0.024 |  0.0036 |  0.14 |  0.051 |  0.034
Traceability |  0.059 |  0 |  0.088 |  0.005 |  0.13 |  0 |  0.029
Table 12: Overview of moderation effects of the choice of language |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0 |  0 |  0.045 |  0 |  0.022 |  0 |  0
Incrementality |  0 |  0 |  0.058 |  0 |  0.0035 |  0 |  0
Mappings |  0 |  0.0063 |  0.043 |  0 |  0.0047 |  0.0052 |  0
Model Management |  0.080 |  0 |  0.0050 |  0.0084 |  0.0077 |  0.0058 |  0
Model Navigation |  0 |  0.031 |  0.051 |  0.022 |  0.038 |  0.027 |  0.0015
Model Traversal |  0 |  0.00081 |  0.049 |  0.0075 |  0.040 |  0 |  0
Pattern Matching |  0.028 |  0.00081 |  0.018 |  3.4E-05 |  0.019 |  0.0076 |  0.0065
Reuse Mechanisms |  0.037 |  6.2E-09 |  0.042 |  0.0015 |  0.102 |  0.027 |  0.029
Traceability |  0.036 |  0.022 |  0.0042 |  0.026 |  0.024 |  0 |  0.041
Table 13: Overview of moderation effects of the experience in the used languages |  Comprehensibility |  Ease of Writing |  Expressiveness |  Tool Support |  Maintainability |  Productivity |  Reusability
---|---|---|---|---|---|---|---
Bidirectionality |  0 |  0 |  0 |  0 |  0.0006 |  0 |  0
Incrementality |  0 |  0 |  0.0086 |  0 |  0 |  0 |  0
Mappings |  0 |  2.4E-17 |  4E-07 |  0 |  0 |  0.0006 |  0
Model Management |  0.090 |  0 |  0.021 |  0.024 |  0 |  0.0012 |  0
Model Navigation |  0 |  0.008 |  0.0022 |  0.0088 |  0.0014 |  0.043 |  0.015
Model Traversal |  0 |  0.0008 |  0 |  0.0065 |  0.0001 |  0 |  0
Pattern Matching |  0.0005 |  1.5E-19 |  4.4E-05 |  0.00049 |  0.00069 |  0.0011 |  0.0065
Reuse Mechanisms |  0.0056 |  0.0014 |  0.0077 |  0.0071 |  0.018 |  0.025 |  0.0022
Traceability |  0.022 |  2E-17 |  4E-06 |  0.028 |  0.014 |  0 |  0.016

See pages 1 of A_survey-overview See pages 2- of A_survey-overview

##  Appendix C Mail Templates

Dear ${Author Name},

We found your contact information while conducting a structured literature
search on model-to-model transformations. We seek your expertise on that
topic.

We invite you to participate in our study ‘The Impact of Model Transformation
Language Capabilities on the Perception of Language Quality Properties‘ (see
below for the URL). It will only take about 20-25 minutes to complete our
online survey. We believe your answers can provide meaningful insights and
help drive the field further.

Our survey is based on a large-scale interview study that qualitatively
assessed what the community believes to be the main factors that drive the
advantages and disadvantages of Model Transformation Languages for model-to-
model transformations. Our results have been published in the Empirical
Software Engineering journal https://doi.org/10.1007/s10664-022-10194-7

Our survey now quantifies our results to provide a clear picture of which of
our identified factors are most important. The methodology for this survey has
been peer reviewed at the Registered Reports track at ESEM’22 and is available
under https://doi.org/10.48550/arXiv.2209.06570

The survey is available at

https://sp2.informatik.uni-ulm.de/limesurvey/index.php/112652?lang=en

It will be open till January 15, 2023.

All responses are completely anonymous.

Many thanks for supporting our research!

Best regards

Stefan Höppner

##  Appendix D Data Privacy Agreement

To invite people to participate in this survey, we used author information
(first name, last name and email address) from published academic papers in
the domain of model driven software engineering. We will delete your personal
information 2 months after you received the invitation.

The participation in this online survey is anonymous. Your name and email
address are only used to invite you.

For future publications, we will publish and further process the anonymous raw
data collected in this survey. This includes aggregating and statistical
analysis of answers provided by participants. We will perform this in a way
that does not allow inferring the identity of individual participants (e.g.,
by stripping free text answers of information that could identify a
participant). Contact Information

If you have any questions about this survey or the data you provided, please
contact us:

Stefan Höppner
<EMAIL_ADDRESS>
Institute of Software Engineering and Programming Languages,
Ulm University,
James-Franck-Ring, 89069 Ulm, Germany