# Process Extraction from Text
state of the art and challenges for the future

Patrizio Bellan1,2 Mauro Dragoni1 Chiara Ghidini1
1 Fondazione Bruno Kessler (FBK), Trento, Italy
2 Free University of Bozen-Bolzano, Bolzano, Italy
E-mail<EMAIL_ADDRESS>
###### Abstract

Automatic Process Discovery aims at developing algorithmic methodologies for
the extraction and elicitation of process models as described in data. While
Process Discovery from event-log data is a well established area, that has
already moved from research to concrete adoption in a mature manner, Process
Discovery from text is still a research area at an early stage of development,
which rarely scales to real world documents. In this paper we analyze, in a
comparative manner, reference state-of-the-art literature, especially for what
concerns the techniques used, the process elements extracted and the
evaluations performed. As a result of the analysis we discuss important
limitations that hamper the exploitation of recent Natural Language Processing
techniques in this field and we discuss fundamental limitations and challenges
for the future concerning the datasets, the techniques, the experimental
evaluations, and the pipelines currently adopted and to be developed in the
future.

##  1 Introduction

Process Discovery (van der Aalst, 2011) refers to the set of algorithmic
methodologies that aim to automatically extract and a process model as
described in data. While the typical source of data is that of event-logs, and
Process Discovery from event-log data is a well established and mature area
that has already moved from research to concrete adoption in a mature manner,
Process Discovery that uses textual documents as a source of data is a
research area which still lacks the ability to scale to generic real world
texts (Maqbool et al., 2018; van der Aa et al., 2018a)111The early stage of
development of process model extraction from text is even more evident if we
compare this research area to other areas such as the one of ontology learning
from text, where many different approaches exist, and actual techniques have
been also incorporated in concrete modeling tools (Cimiano, 2006; Petrucci et
al., 2018; Ghidini et al., 2012)..

This form of Process Discovery, that we call process extraction from text, can
be regarded as the specific problem of finding an algorithmic function to
transform process description texts into their formal representation (the
process model diagram). The ambiguous nature of natural language, the multiple
possible writing styles, and the great variability of possible domains of
application make this task extremely challenging. Indeed recent papers on this
topic (Riefer et al., 2016; Maqbool et al., 2018; van der Aa et al., 2018a)
highlight that after almost ten years of research, process extraction from
text is a task far from being resolved and further research in this direction
is needed with the purpose of improving the quality of the process model
generation.

In this paper we propose a comparative analysis of well-known recent work,
published specifically in the area of Business Process Management from 2010,
so as to better understand their contributions and limitations and to identify
which are the challenges in this topic the research community should address.
Differently from recent surveys we do not limit to provide a systematic wide
literature review. Instead, we would like to provide complementary directions
along with the ones proposed in the literature reviews papers. The novelty of
our review paper is in a finer comparison of the selected contributions and
the analysis of the experimental evaluation made. Our goal is to focus on a
limited number of 13 reference papers in the field, which target both
declarative (e.g., DECLARE) and procedural (e.g., BPMN) languages, and to
compare and discuss them in depth considering their entire pipeline, the
process elements they extract, and their evaluation procedures so as to
understand their limitations not only in terms of the _techniques_ used, but
also the _data exploited_ , and their _evaluation procedures_. Indeed the
comparative discussion of the 13 papers identifies four groups of problems:
(i) problems with the limited and rather specific data used for training and
testing, which are very far from the variety of process descriptions existing
in the real world; (ii) problems with the techniques adopted, which often do
not make use of the advances of NLP techniques in the last few years; (iii)
the lack of a common evaluation framework where to comparatively judge the
quality of the proposed systems, not only in terms of solid data benchmarks
but also in terms of evaluation metrics able to measure a good extracted
process model (or some of its characteristics); and finally (iv) problems
related to the entire (complex) pipeline that has to be constructed in order
to provide process extraction from text solutions. These limitations can
provide the basis for a discussion on relevant challenges that the community
needs to solve in order to make the task of process extraction from text scale
to real world scenarios.

The paper is structured as follows. We first provide a brief introduction of
process extraction from text, then we ground our paper revising the literature
of review articles before summarize the 13 papers included in the analysis and
we compare them according to the approach and techniques used, the process
elements extracted, and the experimental evaluation conducted. Finally we
provide a discussion of what we believe are crucial critical problems in this
research area concerning the data and techniques used, the evaluations
performed and the setting up of the entire pipeline indicating, by means of
questions, a number of open points that the research community working in this
area should aim to address in the near future.

##  2 Process Extraction from Text

The extraction of a process model from documents is a difficult task due both
to the complex nature of natural language and of its multiple linguistics
levels (syntactical, semantics and pragmatics) and to the multiple (correct)
manners in which a process description can actually be conveyed in a process
model.

Figure 1: An example of text-to-model mapping.

Figure 1, taken from (van der Aa et al., 2018a), provides a concrete example
of process extraction from text. The gray shadow boxes link each process
description sentence on the left-hand side to its corresponding process
element in the process diagram, here represented in BPMN. The figure shows an
example of text-to-model mapping in which only the meaningful activities
described in the process description are mapped in the process model diagram.
van der Aa et al. (2018a) describe several challenges in process extraction
from text. We only mention here, with the help of Figure 1 few examples.
First, not all the sentences correspond to a process model element (that is,
the text can contain uninformative sentences). Second the logical succession
described in the text differs to the written sentences’ order (as for example
happens with sentences 4 and 5). Third, the task labels in the process model
may differ substantially from their description in the text.

Figure 2: Two approaches to perform process extraction from text.

If we look at the way process extraction from text has been tackled in the
literature, we can roughly divide the approaches in two big categories. The
first one aims to directly map a process description into its process model
representation via a single function $f$ as graphically depicted in the top
part of Figure 2222We have chosen BPMN as an illustrative example but the
approach is clearly agnostic to the specific modeling language.. As we will
see in the remaining of the paper, function $f$ is typically implemented via a
complex and ad-hoc tailored pipeline. This approach has the advantage of
defining a tailored transformation that can take into account all the
available contextual information that can help solve the problem. Nonetheless,
this advantage becomes a drawback when we need to devise general solutions or
when the algorithmic function $f$ is applied into different contexts. A
further approach towards the implementation of a direct mapping $f$ is the
exploitation of Artificial Neural Networks. However, the huge quantity of data
required to learn a model, and the small quantity of data available in this
research domain makes this strategy rarely adopted.

The second approach found in the literature performs a two-steps
transformation approach with intermediate representation to extract and create
a process model. As illustrated in the bottom part of Figure 2 the algorithmic
function $f$ is here considered a compound function $f_{a}\circ f_{b}$: first,
function $f_{a}$ extracts process elements from text and populates the
intermediate representation (also called, world model), then function $f_{b}$
builds the process model diagram starting from the structured representation
of the elements contained in the world model.

Note that $f_{a}$ and $f_{b}$ could be further broken down into smaller tasks
that allows to better handle the problem complexity. For example $f_{a}$ could
contain a module that takes care of the resolution of anaphoric references
within text, another that filters out uninformative textual fragments that may
act as noise in the later stages, and then specific modules tailored to the
extraction of instances of specific process elements (e.g., activities or
roles). Similarly, $f_{b}$ may contain modules devoted to the addition of
process elements not explicitly described in the text (but conceptually
required to create a correct model as the activity “ship bicycle to customer”
in Figure 1), modules devoted to the connection of process elements together
following the same logic conveyed in the textual description, or modules
devoted to the generation of the labels of the different elements in the
diagram.

##  3 Related Works

Before presenting a brief summary of the selected key contributions, we revise
important review papers in this topic with the purpose of marking the
differences between existing review papers and our review to ground our
article in the existing literature.

A process model need to be efficient, validated, consistent with the process
it supports, checked for correctness and completeness. Under this perspective,
in the technical report proposed in Mendling et al. (2014, 2019) the authors
identified 25 key challenges related to the task of process model extraction
from textual data. The main point made in this paper regards the challenges
related to the translation of a process model diagram into its textual
description and the other way around. The difference between our paper and
this report is that we presents the problematic aspects with the current
contributions and the experimental evaluation made emerged from the comparison
analysis.

Another important review contribution is proposed in the position paper van
der Aa et al. (2018a). The authors recognize that pipelines proposed in the
literature are still below an optimal performance that could be due to the
heterogeneity of possible domains in which these systems are applied. So, they
recognize a potential use for adopting Natural Language Processing research in
Business Process Management (BPM) through a selection of key challenges. The
analysis groups the challenges into the three levels of BPM: Multi Process
Management, Process Model Management, and Process Instance Management.
Regarding the challenges concerning process extraction, the authors propose to
adopt out-of-the-box NLP tools to analyze the structure of documents to
extract process model elements and their relations. A potential use for NLP
tools in BPM identified by the authors regards the analysis of process models
generated with the aim of improving their quality. Different from this paper,
in which the authors highlight the potential of NLP research in this topic,
our aim is to propose to the community new challenges and open points that
emerge from the comparison analysis.

In Mendling et al. (2017) the authors reviewed prior research in BPM to
emphasize the major contributions that would provide a foundation of smart
business process management. The authors revised the contributions aimed at
solving automatic service composition, and process configuration. The main
topic of the paper is on the adoption of mining techniques to both discover
the use of resources within process models and checking process model quality.
They propose a strategy for the integration of sensory data and social media
data to repository to build a bridge between process repository and processes
instances.

A Systematic Literature Review (SLR) that proposes a comprehensive
investigation of the techniques and tools adopted in the research for the
generation of BPMN models from text is proposed in Maqbool et al. (2018). The
main claim the authors made from their analysis is that existing approaches
are still immature to be adopted in real settings such as industry. The
authors applied a strict selection inclusion criteria (such as that a paper
..must have some defined results and must be sustained by concrete evidences.)
to select the significant contributions published from 2010 to 2018. The
comparison performed by the authors focuses on the specific NLP tools used by
the researchers, the BPMN modeling tools used, and the process elements
extracted from the text. The aim of the authors is not to propose a direct
contrast of the papers but to show the different NLP tools adopted. Our review
differs from this one in several aspects. First of all, we do not propose a
detailed comparison of the tools and frameworks’ settings adopted in the
research papers select. Instead, our goal is to provide a finer comparison of
the techniques adopted, the analysis of input types and their constraints on
the key contributions. Another aspect we differ is about the theoretical level
considered. Indeed, we aim at contrasting the selected contributions in terms
of overall approach taken and techniques adopted. Finally, we consider enlarge
our selection criteria including researches proposed up to the 2020.

Another SRL that compares the methods and techniques adopted in key
contributions published in the field of process extraction from text is
proposed in Riefer et al. (2016). Different from the other reviews, this
review article considers not only the extraction phase, but also the
generative part of the frameworks. In particular, for what concerns the
extraction step, the analysis compare the type of input and the NLP techniques
adopted. Although we also present a comparison of the selected contributions
on the basis of the techniques adopted, we offer a more structured comparison
of the papers in terms of the overall approach adopted.

Finally, a brief comparison of the existing approaches in terms of input,
output, method, and subset of BPMN process elements covered is proposed in
Honkisz et al. (2018). The relevant literature section of this article shows
that current approaches consider a restricted subset of process elements. In
line with this paper, we also analyze the process elements extracted from each
selected contribution, but we differ from it in the more deeper comparison
made on these aspects.

Overall, our main goal is to highlight the specific challenges of process
extraction from text system in order to provide a ground for future research
in this narrow topic. We are interested in discover the challenges and
problematic aspects of text-to-model pipelines. Given the heterogeneous
grounds of the selected contributions and the lack of a common benchmark
procedure, we would highlight gaps, strengths, and weaknesses of each
contribution in order to provide complementary directions along the ones
proposed in the literature reviews papers described above. The novelty of our
review paper is not only in the comparison of the selected contributions at
the macro level, but also in the analysis of the experimental evaluation
performed to evaluate the proposed frameworks.

## A Brief Summary of the Papers

In this Section we briefly summarize the content of the 13 papers considered
in our analysis.333It is worth noting that specific process extraction from
text subtasks may be tackled by papers that address also different tasks. An
example is the comparison (alignment) between a text and a process model,
addressed in van der Aa et al. (2018b). While similar works may provide
additional stimuli to the research in process extraction from text, we have
decided to exclude them from this overview for the sake of maintaining an
homogeneous analysis and comparison. We group the papers according to the two
different approaches described at the end of the previous Section. For each
contribution we describe: (i) the settings of the research, (ii) the procedure
presented, (iii) the evaluation performed to test the ability of the proposed
framework, and (iv) the dataset adopted to perform the evaluation step, if
described.

### The direct approach

Ackermann and Volz (2013) propose a methodology and a prototype 444Not
available at the link cited in the paper at the time of writing. to discover
formal models from natural language texts. The project is grounded on the
philosophy of learning by doing, so it learns how to build up a process model
diagram through user’s feed-backs. The system incrementally learns rules to
cover as many process items as needed to the final user in a specific domain.
Detection rules are coupled with instantiation rules to map text fragments
directly into a process model diagram. This methodology is claimed to be
flexible, extendable, and language-independent. The rules are represented in
terms of Feature Structures (Jurafsky and Martin, 2009), and are used to map
process model diagram elements described in textual fragments into their
corresponding elements in a process model diagram. Whenever a sentence is
provided in input to the tool, the engine checks if a corresponding mapping
rule exist. If there are no rules to describe a process element, the system
exploits the user’s feed-backs to create one. So, the the user has to manually
map a text fragment into its process model element. Then, the system does a
syntactic analysis of the text and creates the corresponding mapping rule. The
set of rules is kept compact, by means of a fuzzy feature unification pruning
mechanism. This strategy is used to avoid the definition of different rules
for the same process model diagram element described in similar textual
fragments, that may differ only for the use of, e.g., a personal pronoun. The
evaluation tests the disambiguation ability of the rules on a sub-set of the
dataset proposed in (Friedrich et al., 2011). The framework evaluation reaches
an overall F1 score of 0.86 on element identification.

Business process constraints discovery from text is the focus of a recent
paper by van der Aa et al. (2019). The discovered constraints are represented
in the Declare language, and are extracted by means of a tailored NLP
pipeline. The work targets several challenges related to process model diagram
elements discovery, such as the discovery of noun-based activities, and the
detection of different _constraint restrictiveness_ due to modal verbs. The
proposed technique identifies activities and their inter-relations in the
input text (a sentence, in this case) after a deep analysis of the text
semantics. Declarative constraints are generated by filling slot-templates
with the elements discovered during the input analysis. The tool is able to
generate five different types of Declare constraints: init, end, response,
precedence, succession. The overall procedure is composed of three main steps:
linguistic processing, activity extraction, and constraint generation. After a
typical text normalization step of the input, linguistic processing begins by
extracting semantic components from the typed dependency relations
representation of the sentence. In this stage, the analysis focuses on the
verb, the subject, and and the object of a sentence. Verbs analysis aims at
identifying if a verb conveys an action, an occurrence, or a states. Process
participants are captured with the analysis of the subject and the object of
the sentence. In particular, the subject of the sentence typically describes
the actor performing an activity whereas the object typically describes the
business object (also called artifact) manipulated by an activity. In order to
enhance the ability of the tool, a particular attention is directed to the
analysis of modal verbs, negations, and prepositional phrases of a sentence.
In particular, the analysis of prepositions allows to detect ordering
relations between activities. Moreover, adverbial clause analysis provides
information on relations that exist between verbs, whereas coordinating
conjunctions analysis detects relations between the entities described. The
activity extraction step aims at detecting verb-based and noun-based
activities (such as “the creation of a claim”) from a sentence. Verb-based
activities are detected using the approach proposed in (Friedrich et al.,
2011). To correctly recognize noun-based activities and avoid false positives,
the analysis of the input concentrates on verbs describing the flow of
activities. Candidate verbs are selected using a list of temporal verbs.
Finally, the information extracted and the relations discovered are used to
fill slot-templates of declarative constraints to generate the specific
constraints.

The evaluation is performed using a collection of 103 pairs consisting of a
constraints description and its corresponding declarative constraints
555available at: https://github.com/hanvanderaa/declareextraction using the
well-known Information Retrieval metrics. The method achieved an overall
precision of 0.77 and a recall of 0.72, yielding an overall F1-score of 0.74.

López et al. (2018) propose a tool, still in a early stage of development, to
extract declarative models from text, and then to maintain the alignment
between the text and the corresponding declarative model represented using
Dynamic Condition Response (DCR) Graphs. The DCR graph represents the model in
which nodes represent activities and edges the constraints relations. The tool
only supports the extraction of three types of constraints between two
activities $A$ and $B$: condition relation, response relation, and exclusion
relation. Model discovery begins with the identification of roles and
activities in the text; then, the system discovers the relations and
constraints between these two types of process elements. However, the paper
does not provide enough details to explain how the declarative constraints are
discovered within the text.

The authors claim that the validation is done with examples from industrial
and academic environments. Nonetheless a detailed description of the
evaluation settings and metrics is lacking, thus making impossible to
carefully assess strengths and limitations of the paper.

Han et al. (2020) propose a deep learning solution to the problem of
automatically discovering a business process and its corresponding process
model diagram (represented in BPMN) without extra human labeled knowledge. The
methodology is based on an Ordered Neurons LSTM (ON-LSTM) model that jointly
learns a language model and its latent semantic hierarchical structure from
Process Definition Documents (PDD). A Recurrent Neural Network based on
Bidirectional-Long Short Term Memory (Bi-LSTM) encodes a process definition
description into a tree-like representation to obtain a contextual
representation of the input. The use of Ordered Neurons enables the
representation of dependencies as pairs for learning a seq2seq model. The
objective function is the maximization of the likelihood of a sentence given
its previous one. The authors found that stacking three BiLSTM layers was a
better solution than using a single BiLSTM layer. Before feeding the neural
model, the input text is enhanced by its semantic representation, built
exploiting the Glove system (Pennington et al., 2014). Then, the embedded
representation of the network is trained to learn the process model.

The proposed system is tested on a dataset built starting from a collection of
210 pairs composed of PDD documents, and diagrams, with Graph Edit Distance as
a measure of the quality of the extraction. The 210 documents are fragmented
into 6000 test samples, with their corresponding fragments of diagrams (in
BPMN), which are then used to train and test the tool. The results show that
the discovered fragments of diagrams have an average similarity of 32% with
the golden standard model. An analysis of the results shows that the proposed
tool is able to correctly generate 57% of the edges and 76% of the nodes in
the process model diagram.

### The two-steps approach

We present here the 9 papers that follow the two steps approach. First we
present 5 papers that cover the entire process extraction from text function
and provide both $f_{a}$ and $f_{b}$, then we describe the 4 papers that focus
only on the extraction (identification) of structured process elements from
the text. These latter papers can be considered works that only realize the
first part of the transformation (that is part of $f_{a}$).

#### Realizing $f_{a}$ and $f_{b}$

de A. R. Gonçalves et al. (2011) present a tool for process elicitation and
modeling of workflows from groups of stories, extending the work proposed by
Santoro et al. (2008). The authors focus on users’ interaction, proposing this
novel methodology, in which business process instances are captured by means
of stories told by the users them-selves. The authors propose a story telling
mining method to ease the work of analysts by automatically eliciting models
from stories. The central idea is to apply text mining, information
extraction, information retrieval, and natural language processing techniques
in association with contextual elements on the collaborative told stories, to
deal with this distributed knowledge. The proposed framework is composed of
three steps. First, the tellers record the stories in a repository. Stories
are classified and clustered together, in order to group those referring to
the same process (as for example those referring to a sending procedure).
Second, the tool extracts process model diagram elements from stories
belonging to the same cluster. At the beginning of this step, uninformative
sentences are excluded from further analysis. The application of a TF/IDF
666Term frequency/Inverse Document Frequency (TF-IDF) “is a numerical
statistic that is intended to reflect how important a word is with respect to
a document in a collection or corpus” cit. Wikipedia filter allows to focus on
the target domain only. Then, syntactical analyses of the informative sentence
of the text, based on two different regular expression grammars G.I and G.II,
aim to find out noun phrases (NP) that could correspond to actors and/or
artifacts, and verb phrases (VP) that could describe activities. In addition,
a list of trigger words are checked to discover gateways. The elements
extracted are memorized into an intermediate representation that adopts the
CREWS (Achour, 1998) model. Third, the process model diagram is built
exploiting templates and the structured data memorized into an intermediate
representation. When several alternative process models are generated, they
are all presented to the process modeler. The analyst has to combine them and
validate the final version manually.

The evaluation measures the ability of the two regular expression grammars G.I
and G.II to retrieve activities present in textual descriptions of two real
world case studies. The coverage of G.I is increased in by coupling it with a
list of trigger words, whereas G.II is used without trigger words. The
evaluation reveals that G.I has an accuracy of 45%, while G.II performs
better, with an accuracy of 78.57%.

The work by Friedrich et al. (2011) is still considered a reference work in
recent surveys and overviews as the ones presented in (Wisniewski et al.,
2019; Riefer et al., 2016). Differently from other approaches, this
investigation does not make any assumptions regarding the input text. In
addition, the work provides an ad-hoc algorithmic solution to anaphora
resolution. The overall procedure first populates a world model with instances
of Actor, Resource, Action, and Flow extracted from the text; then, the
generative module builds the corresponding process model diagram from this
structured intermediate representation. A slightly modified version of the
CREWS World Model is used as intermediate representation. More in detail, the
pipeline proposed in this work is composed of three steps: Sentence Level
Analysis, Text Level Analysis, and Process Model Generation. The Sentence
Level Analysis step starts with common natural language pre-processing tasks
such as text normalization and text cleaning. The pre-processed text is parsed
by the Stanford CoreNLP Library (Manning et al., 2014) to obtain a tree
representation of the sentence. The trees are used to determine if the verb of
the sentence is active or passive and to extract instances of _Actor_ and
_Action_. Moreover, example sentences are filtered out using a keywords-based
list. During the Text Level Analysis, co-references and relative references
are resolved. Conditional markers are checked against a word-list of
conditional indicators to determine the gateway’s type (concurrent, parallel,
inclusive or exclusive). Here, WordNet (Miller et al., 1990) and VerbNet
(Schuler and Palmer, 2005) are used to increase the generalization skills of
the tool. The information contained in the world model is combined with the
information extracted in this step to tackle the problem of actions that span
over multiple sentences. Finally, the Process Model Generation step transforms
the world model into its BPMN-diagram and creates the corresponding model’s
labels for the diagram’s objects.

The paper introduces a publicly available dataset of 47 pairs composed of a
process model description, and a process model diagram for performing the
evaluation. The diagrams extracted with the prototype proposed in the paper
scored an average similarity of 77% compared to the ones of the testing
dataset, measured with the graph edit distance metric. The error analysis
performed by the authors reveals that the Stanford Parser may not be the right
tool to adopt, because it fails during verb classification. Also, since
business process related verbs are not part of VerbNet, this resource is only
partially sufficient to solve the text-to-model task.

The work of Sawant et al. (2014) presents a method to create a process model
diagram and its associated ontology from use case descriptions (descriptions
on how an entity should cooperate with other entities) by mean of
computational linguistics techniques. The authors aim to extract also an
ontology because it can help in further analysis and can provide information
for a knowledge engine. The authors assume that processes are described in an
input text that avoids ambiguous statements. Also, activities of error
handling and alternative flows should be fully specified in the description.
The proposed methodology adopts a set of well-defined rules and templates,
which reduce imprecision and ambiguity. The process extraction method starts
with a linguistic analysis that identifies entity names and generates the
dependency representation of sentences777It is a grammatical structured
representation of a sentence based on word relations.. Then a chunking step is
performed. This step enhances the one presented in (de A. R. Gonçalves et al.,
2011) in two directions: if a noun phrases was not caught by the gazetteer, it
is considered a Business Entity; and if a chunk stands in a subject relation,
it is considered an Actor. Action verbs are classified based on the Levin
classes of VerbNet (Schuler and Palmer, 2005). A Business Entity Co-reference
Resolution system solves the co-reference problem linking together all the
entities that refer to the same process element, extracted in the previous
steps. At this point, all the process relevant information are stored into the
intermediate representation. In the last stage, the tool generates a BPMN
process model diagram from the intermediate representation. In addition,
during this last stage, an ontology is extracted and a set of feature-maps
associates each annotation into Object Properties and Data Properties.

The evaluation is based on a private dataset of 123 real-world use case
descriptions on various business domains. In particular, they test the
extraction of business entity (namely, an artifact), initiating actor,
responding actor, and action (where action is used here to name a sub-set of
BPMN events).

Epure et al. (2015) propose a Text Process Miner that mines diagrams of
process instances from archaeological reports, that is textual descriptions of
process executions in the archaeological domain. While extracting process
instances is a different (and somehow simpler) task than extracting a process
model, we nonetheless decided to include this work as it tackles both the
tasks of process element identification in the text and then the composition
of a diagram. The work is a mixture of natural language processing techniques
to structure the input text and a set of rules to perform the extraction of
the process instance. A special effort is devoted to the handling of active
and passive forms of verb clauses. The pipeline starts with a _text
normalization_ that takes care of removing uninformative sentences and
normalize the remaining textual fragments. Then, a syntactic analysis
discovers the executed activities, and their relations, so as to create the
process instance (execution trace). Rules look for specific syntactic
constructions, such as textual clauses and direct objects, to detect
activities and their type of relations within the process.

The experimental evaluation tests the framework on a single archaeological
study report, and achieves a precision of 0.88. Moreover, a qualitative
analysis performed by experts demonstrates that the BPMN process model diagram
generated from the process instances is representative of the actual process
followed in the archaeological domain.

Honkisz et al. (2018) present a two steps transformation methodology to
automatize the discovery of business process work-flows from natural language
textual documents 888A library of this framework is available at:
https://github.com/KrzyHonk/bpmn-python. The main idea relies on a spread-
sheet-like world model as structured intermediate representation of elements
and their connections, discovered in the text. The system exploits a syntactic
analysis of the input text to extract Subject-Verb-Object constructs (SVO). A
stage of semantic analysis further enhances the extraction by filtering out
unnecessary SVO constructs during the discovery step. The actors are extracted
from text after an analysis of the dependency relations in search for nominal
subject and nominal subject passive, conjunction dependencies, and after a
keyword-based search for pronouns, relative pronouns, and hypernyms that
belong to a specified list of admissible hypernym likes person, organization.
The list is also expanded with common special words and abbreviations. The SVO
triplets are searched for in the parsed tree. The result is a row in the
spread-sheet world model in which subject, activity and the possible object
are recorded. Gateways are extracted exploiting a keywords-based extraction.
Keywords-based search aims to find out signal words that indicate the presence
of a conditional or a parallel gateway as well as the default flow of
conditional gateway. More in detail, this intermediate representation records:
the number of the corresponding phase, the name of the performed action, the
condition of the possible gateway, and the actor that executes the activity.
Finally, the BPMN-diagram is then built up from the intermediate
representation.

A limitation of this paper regards the lack of an experimental evaluation. In
fact, although the authors claim to have tested the described methodology
against a test set gathered from few academic sources, no details are provided
about the dataset, the performance accuracy, and the error analysis.

#### Realizing $f_{a}$ only

Ferreira et al. (2017) present a rule-based methodology, grounded on mapping
rules, to extract business process elements from heterogeneous unstructured
textual sources (such as reports, manuals, and norms). The procedure begins
with a pre-processing step to normalize and a syntactic analysis to determine
the complex structure of information beyond its pure surface realization. The
authors exploit the verb tense to differentiate an activity (present or
future) from an event (past or present perfect). Then, the text is analyzed so
as to extract meaningful features. A set of 33 rules is applied to this
representation to map text into process elements, expressed in BPMN. To
increase the coverage of the extraction step, lists of signal words are
checked to determine the presence (and type) of gateways.

The authors validate the rules they exploit with an empirical evaluation that
involves experts in process modeling. The evaluation is performed by means of
a qualitative and a quantitative evaluation. The first concerns an assessment
of the likelihood of the rules by means of process modeling experts, and
provides positive results. The second evaluates the ability of the tool to
correctly extract a small set of process model diagram elements from a testing
dataset that comprises 56 texts, built by the authors, using manuals and
documentation, and using classical information retrieval metrics. The tool
achieves an overall F1 score of 0.87. Interestingly enough, the extraction of
exclusive and parallel gateways achieved an higher score than the extraction
of activities and events.

The pioneering investigation of Leopold et al. (2018) addresses the unanswered
challenge of classifying activity types in a Robotic Process Automation
domain. This is an important challenge because the identification of the tasks
suitable for automation is vital in such a domain. The work exploits a machine
learning model based on support vector machine (SVM) in order to classify
manual tasks, user tasks, and automated tasks. The pipeline starts with a
linguistic pre-processing that aims to identify relevant linguistic entities
and their relations, adopting the method proposed in (Friedrich et al., 2011).
The result of this pre-processing is an annotated textual process description
representation that is feed to the next step. Then, the system extracts four
different types of features from the annotated representation to create a
machine-like representation, which is the input for the final classification
step. These four types of feature are:  a) a verb feature which extracts the
Levin verb class from the verb of each clause;  b) an object feature that
captures the verb’s object (justified by the fact that certain objects are
more likely to be associated with automated tasks);  c) a resource type set of
feature that leverages all the hypernym relationship within the sentence to
binary discriminate _human_ and _non-human_ words; and  d) a last type of
feature which focuses on the IT domain. This step checks in the sentences the
presence of computer related terms taken from a word-list.  Finally, the
extracted features are feed into a SVM classifier to predict the class of the
task encoded in the features.

In order to perform the experimental evaluation, the authors create a publicly
available dataset starting from the dataset proposed in (Friedrich et al.,
2011), where three independent researchers extracted and labeled 424
descriptions of tasks. The evaluation is performed with a cross validation
leave-out-out strategy on 10-folds. The evaluation reported in the paper shows
an F1 score of 0.81 on the classification task. The results show that verb
features are the most predictive features on this task. Furthermore, the error
analysis reveals that misclassification arise mainly because of insufficient
training data.

The recent work of Qian et al. (2020) proposes a hierarchical neural network
approach, called Multi-Grained Text Classifier (MGTC), to tackle the problem
of classifying sentences in procedural texts without engineering any features.
The overall network architecture is composed of:  (i) an embedding layer to
input the word’s semantics in form of semantic embeddings;  (ii) a recurrent
encoding layer (bi-LSTM) to input an entire sentence to the higher layers;
(iii) a convolutional layer adopted to extract meaningful features from the
input sentence; and finally,  (iv) three classification layers that perform
the three classification tasks.  The objective function is to minimize a
cross-entropy loss for each classification task. The training follows a multi-
task learning procedure. This training mechanism should help to increase the
overall performance of the proposed system because it allows to extract a more
robust and representation of the input semantics. Moreover, the authors claim
that the training follows a coarse-to-fine strategy. Nonetheless, the paper
does not provide clear details on how this step is done. The neural model is
trained to jointly performs: sentence classification, sentence semantics
recognition, and semantic role labeling. The sentence classification task
discriminates sentences describing an action from sentences describing a
statement. The sentences semantics recognition task performs a multi-class
classification of sentences w.r.t different control flow constructs. The last
classification task assigns the proper semantic role to the sentence’s words
in action sentences. The three classifiers allow to extract activity, their
actor and artifact as well as the sequence flows between activities.

An experimental evaluation is conducted on two publicly available datasets,
built by the authors, that contain 2636 and 2172 sentences respectively. They
are built starting from a collection of cooking recipes and maintenance
manuals, and used for for training and testing the proposed tool. The
experimental evaluation tests different configurations of the network and
provides a quantitative evaluation of the tool’s performance. The authors
report a statistically significant high-score of the system in the two
datasets compared to a pattern-matching based method used as baseline.

In another recent contribution, Quishpi et al. (2020) propose a novel
technique to automatically annotate process model elements and their relations
within descriptions that follow a declarative approach. The proposed tool
annotates process model description with Annotated Textual Descriptions of
Processes (ATDP) tags. ATDP annotations can be translated into linear time
temporal logic over finite traces. So, the transformation of text into logic
constituents opens up to the possibility of applying formal reasoning over it.
In particular, the sub-set of ATDP patterns considered in this work are:
response, succession, precedence, weak order, and conflict order (resented in
ATDP as NonCoOccurence). The methodology couples the analysis of natural
language with T-regex rules. T-regex is a regular expression language that
allows to define patterns on dependency trees of a sentence. This approach
makes the tool robust to variations in writing style and word use. The
transformation procedure begins with a common pre-processing and normalization
of the input text. Then, a morphological analysis allows to determine the
grammatical category of each word. At this point, a name entity recognition
(NER) sub-component of the framework detects named entities in the text. In
particular, it detects the named person, location, organization, time and
numerical expressions. A word disambiguation step reduces the problem of
semantic ambiguity of words, using WordNet (Miller et al., 1990) as reference
resource. All the information gathered in the previous steps are fed into a
dependency parser to obtain a semantic tree representation of the input. Now,
it is possible to apply a semantic role labeling tool to obtain the theta
roles (such as the agent, patient, and recipient) described in the text.
Finally, a co-reference resolution sub-component of the tool takes care of
linking referring expression to the mentions of the same entity. After these
steps of analyses, syntactics and semantics are well exposed and it is
possible to apply T-regex rules on them. The tool aims to extract process
entities and their relations. In particular, it extracts the entities of:
activities and events, the role (the actor) who is responsible of an activity;
business object manipulated by an activity; conditional marker that describes
the process flow.

The experimental evaluation provided in the paper evaluates the extraction of
activities and their relations. The evaluation of the activity extraction task
is performed on a collection of 18 pairs of process model description, and
process model diagram manually annotated to obtain a gold standard dataset.
The results shows that the proposed approach achieves an overall F1 score of
0.71, and that the tool performs better in this task than a baseline taken
from (Friedrich et al., 2011). The evaluation of the relation extraction task
is performed by comparing the tool with the one proposed in (van der Aa et
al., 2019). A second dataset, only partially available, was created for this
evaluation. The results shows that the proposed approach achieves an overall
F1 score of 0.61, which is greater than the one of the baseline.

##  4 Comparing the papers

In this section we provide a comparative analysis of the selected
contributions. Given their heterogeneous settings the lack of a common
benchmark procedure, and of a common testing dataset, the comparison is less
than trivial. To overcome this difficulty we have decided to compare them at
the macro level on three main characteristics, described in three different
sub-sections:  (i) the main approach adopted to tackle the problem and the
overall characteristics of the proposed framework;  (ii) the type of business
process process model diagram elements extracted from the input data;  (iii)
the experimental evaluation performed to evaluate the proposed framework.

###  4.1 The approach and techniques used

In this section we compare the selected contributions on the main approach
adopted to tackle the problem and the overall characteristics of the proposed
framework. In particular we characterize them in Table 1 by describing:  (i)
the approach: either direct ($f$) or two step transformation. In the latter
case we report whether the paper presents the entire pipeline ($f_{a}\circ
f_{b}$) or only the first step ($f_{a}$);  (ii) the main strategy used;  (iii)
the level of user involvement in the pipeline;  (iv) the form of natural
language text accepted as input;  (v) the structured intermediate
representation (world model) adopted (if applicable); and  (vi) the form of
output generated.

Table 1: The approach and techniques used.

|  |  |  | Input |  |
---|---|---|---|---|---|---
Paper | Approach | Main Strategy | User inv. | Text | Structure | Description | Intermediate Repr. | Output
Ackermann and Volz (2013) | $f$ | Rules | Semi-automatic | Entire | Unstructured | Model | – | BPMN
van der Aa et al. (2019) | Rules and Templates | Automatic | Sentence | Unstructured | Model | – | DECLARE
López et al. (2018) | not described | Semi-automatic | Entire | Unstructured | Model | – | DCR graph
Han et al. (2020) | Artificial Neural Network | Automatic | Entire | Structured | Model | – | BPMN
de A. R. Gonçalves et al. (2011) | $f_{a}\circ f_{b}$ | Rules and Templates | Semi-automatic | Entire | Unstructured | Model | CREWS | BPMN
Sawant et al. (2014) | Templates | Automatic | Entire | Semi-structured | Model | GATE | BPMN
Friedrich et al. (2011) | Rules | Automatic | Entire | Unstructured | Model | CREWS | BPMN
Epure et al. (2015) | Rules | Automatic | Entire | Semi-structured | Execution | Structured table | BPMN
Honkisz et al. (2018) | Rules | Automatic | Entire | Unstructured | Model | Spreadsheet | BPMN
Ferreira et al. (2017) | $f_{a}$ | Rules | Automatic | Text | Unstructured | Model | – | Tagged entities
Qian et al. (2020) | Artificial Neural Network | Automatic | Entire | Unstructured | Model | – | Tagged entities
Quishpi et al. (2020) | Rules | Automatic | Entire | Unstructured | Model | – | Tagged entities
Leopold et al. (2018) | Support Vector Machine | Automatic | Entire | Unstructured | Model | – | Activities’ list

Let us explore the table more in detail. In the second column the table
summaries in a compact manner the classification into the two main groups
(with sub-groups) already used in summarizing the papers: out of 13 selected
papers, four perform a direct extraction of a process model diagram out of a
process model description; the remaining nine contributions follow a two steps
approach with intermediate representation, but only four implement the
complete pipeline.

Concerning the type of strategy used, we can say that typically the selected
contributions perform a syntactic and semantic analysis of the input text.
They nonetheless differ on the techniques adopted to extract process model
diagram elements from text and to generate the output. Table 1 shows that the
most common techniques used are rule-based techniques (8 papers out of 13). In
particular, in (Epure et al., 2015; Ferreira et al., 2017) rules are applied
to the semantic structures of sentences; in (Friedrich et al., 2011; Honkisz
et al., 2018) rules are enhanced with a list of specific words; the authors of
(Sawant et al., 2014) propose to formulate rules in form of Feature
Unification Rules. This strategy allows to increase the coverage of the rules,
keeping the number of rules relatively small. Another rule-based strategy is
presented in (Quishpi et al., 2020). Here, rules are expressed in form of
Tree-base patterns applied on top of the dependency tree of each sentence. In
(van der Aa et al., 2019) rules are used to extract process model diagram
elements and templates are used to generate declarative constraints. A step
forward to increase coverage and generalization abilities of rules is
presented in (Ackermann and Volz, 2013) whether rules are formulated over
linguistic template for sentence structures. Two contributions of this rule-
based strategy group leveraged a templates-filling strategy to design the
process model diagram elements extracted in their output process model
diagram. A machine learning strategy is adopted in 3 papers. Due to the
complexity of handling multiple linguistic levels and phenomena of a process
description at once, in (Han et al., 2020; Qian et al., 2020) the mapping is
performed via neural transformation. Finally, Leopold et al. (2018) uses a
Support Vector Machine classifier (SVMC) to classify the type of a given
activity as: manual task, user task, or automated task.

Concerning the level of user involvement, Table 1 shows that most of the
papers (10 out of 13) implement a completely automatic pipeline: in these
works the user is not required to make any interaction with the output data.
For papers, instead, assume that the user has to manually choose, correct, and
validate the output data. The output of the framework proposed in (de A. R.
Gonçalves et al., 2011) is a collection of possible alternative process model
diagrams, so, the user is required to choose the correct one and validate it.
Also, during the story creation (that is, the generation of the input data),
the teller (a person who describes a process) has to manually validate actors
and characters of the story. In (Ackermann and Volz, 2013) and (López et al.,
2018) a user interaction with the system is required, since the framework
constantly tries to increase rules coverage and detection accuracy taking
advantage of the user feedback which is created when the user corrects the
process model diagram generated.

The remaining columns summarize the type of input, output and intermediate
representations (when applicable). For the input we describe whether the
pipeline accepts in input an entire portion of text or single sentences;
whether the text is completely unstructured or some sort of structure is
required; and whether it describes the model of a process or its single
executions. The majority of contributions, that is those in (Ackermann and
Volz, 2013; López et al., 2018; Friedrich et al., 2011; Honkisz et al., 2018;
Ferreira et al., 2017; Qian et al., 2020; Leopold et al., 2018) all accept in
input an entire process model description written in a completely unstructured
way. The work in (Quishpi et al., 2020) accepts in input an entire text but
looks for declarative patterns at the sentence level; the work in (van der Aa
et al., 2019) instead analyses an unstructured process description one
sentence at a time; finally the framework described in (de A. R. Gonçalves et
al., 2011) is still unstructured but describes stories that refer to process
executions Two papers focus on semi-structured textual descriptions in the
form of of Use Case Description (UCD) documents (Sawant et al., 2014) and
Process execution traces (Epure et al., 2015). The former are structured
descriptions of the process with an implicit description of a process flow
contained in a list of steps representing the (possibly alternative) flow. The
latter are structured as series of ordered sentences describing the activities
of a process in a sequential manner. Finally, a completely structured
description of a process model, in the form of a Process Definition Document,
is considered in (Han et al., 2020) only.

The five works that realize the entire $f_{a}\circ f_{b}$ pipeline adopt four
different intermediate representations (world models) that can be grouped into
three different sets: the first one is provided by the CREWS (Achour, 1998)
world model, adopted in (de A. R. Gonçalves et al., 2011; Friedrich et al.,
2011), the second one is provided by the GATE annotations (Cunningham et al.,
2013), explored in (Sawant et al., 2014), and the third is provided by a less
structured representation either in forms of structured table or in forms of
spreadsheet, adopted in (Epure et al., 2015) and (Honkisz et al., 2018),
respectively.

Regarding the output we need to make a distinction between the contributions
that propose a complete pipeline and those that only realize $f_{a}$. If we
consider the complete pipeline, Table 1 shows that most papers extract
_imperative_ business process models expressed in the BPMN modeling language.
If we move to the papers that realize $f_{a}$ only, they produce an output
that is either sentences tagged with process entities or directly a list of
process entities. The latter is provided by (Leopold et al., 2018), which
produces in output a classification of activity’s type. The former is provided
in (Ferreira et al., 2017; Quishpi et al., 2020; Qian et al., 2020), and the
specific elements are listed in Table 2. Concerning the modeling paradigm,
(Ferreira et al., 2017),(Leopold et al., 2018), and (Qian et al., 2020) are
inspired by _imperative_ paradigms (and in particular BPMN), while (Quishpi et
al., 2020) analyses the text looking for declarative knowledge.

###  4.2 The process elements extracted

Table 2 reports the list of elements that are extracted by at least one paper.
We group the elements in two subsets: those referring to the control flow
(activity, event999The term “event” is widely used in BPM often to denote
different entity types. The interested reader is referred to (Adamo et al.,
2020) for a discussion. In this paper we use the term event to denote BPMN
events., gateway, sequence flow, control flow, and generic relations between
activities, here extracted only wen referring to Declare relations) and those
referring to elements that participate somehow to the activity (actor, and
artifact) (Adamo et al., 2017). Since the elements extracted depend from the
modeling language, we have grouped the papers according to that101010Since DCR
Graphs focuses on a subset of Declare patterns, for the sake of simplicity we
consider it here a variant of Declare.. In the Table, we use “NO” to indicate
that a process element is present in the target modeling language but it is
not extracted by the pipeline at hand. We instead use “-” to indicate that the
element is not present in the target modeling language, and thus its
extraction does not concern that particular paper. Since approaches exist to
extend Declare with data, we have considered the two participants when
analyzing (van der Aa et al., 2019) and (López et al., 2018).

In addition to a description on the elements extracted from the 13 papers, we
briefly describe and compare the different techniques used to extract the
specific elements.

Table 2: Elements extracted from the input text.

|  | Control Flow | Participant
---|---|---|---
Paper | Output | Activity | Event | Gateway | Sequence flow | Message flow | Declare Relation | Actor | Artifact
Ackermann and Volz (2013) | BPMN | Yes | Yes | Yes | Yes | Yes | – | Yes | Yes
Han et al. (2020) | Yes | Yes | Yes | Yes | Yes | – | No | No
de A. R. Gonçalves et al. (2011) | Yes | Yes | Yes | Yes | Yes | – | Yes | Yes
Sawant et al. (2014) | Yes | Yes | Yes | Yes | Yes | – | Yes | Yes
Friedrich et al. (2011) | Yes | Yes | Yes | Yes | Yes | – | Yes | Yes
Epure et al. (2015) | Yes | No | Yes | Yes | No | – | Yes | No
Honkisz et al. (2018) | Yes | Yes | Yes | Yes | Yes | – | No | Yes
van der Aa et al. (2019) | Declare | Yes | – | – | – | – | Yes | No | No
López et al. (2018) | Yes | – | – | – | – | Yes | No | No
Ferreira et al. (2017) | Tags | Yes | Yes | Yes | No | No | – | No | Yes
Qian et al. (2020) | Yes | No | No | Yes | No | – | Yes | Yes
Quishpi et al. (2020) | Yes | Yes | – | – | – | Yes | Yes | Yes
Leopold et al. (2018) | Yes | No | No | No | No | – | Yes | No

Let us first focus on the elements describing the control flow of a business
process. Since activities are the core element of a process model diagram,
this element is always identified and extracted from a process model
description and is always present in the BPMN and Declarative diagrams.
Activities are extracted using different strategies which can be divided in
two main groups: those which use pre-defined rules and those which either
attempt to learn the rules or avoid using them altogether.

Let us start with the papers that use pre-defined rules. In (de A. R.
Gonçalves et al., 2011) the authors define rules in form of regular expression
grammar rules, which focus on the analysis of the verbal phrases (VP) of a
sentence. This technique in further improved in (Ferreira et al., 2017) with
the help of signal words and is also further developed in (Sawant et al.,
2014; Honkisz et al., 2018; Friedrich et al., 2011; López et al., 2018; van
der Aa et al., 2019; Leopold et al., 2018; Quishpi et al., 2020). In these
latter contributions, the analysis of the verbal phrases of a sentence are
used together with the dependency relations between the subject, the verb.
Moreover, the use of WordNet to extend word-list terms is exploited. This
extension allows to enhance rules coverage and generalizability. In (Epure et
al., 2015; Friedrich et al., 2011; Leopold et al., 2018) rules take into
account verbal and nominal phrases, and are also applied at the clause-level
of the sentence and to subordinate conjunctions. In particular, a transitive
verb and its object together are considered an activity candidate. Then,
WordNet and VerbNet are used to gather information about verb features (such
as verb transitivity) and to decide if a candidate activity is a real activity
that must be represented in a process model diagram or not. Rules defined on
top of the semantic tree representation of a sentence, also called T-regex
patterns, are explored in Quishpi et al. (2020). The definition of pattern
rules over a semantic tree representation of a sentence helps in increasing
the coverage of the rules while keep the number of rules relatively small (if
compared to the set of rules for plain text one may need).

The second group of papers contain two works that aim at learning a set of
rules to perform extraction of activities (Ackermann and Volz, 2013; López et
al., 2018), and other two papers that leverage artificial neural networks to
avoid defining a-priori rules (Han et al., 2020; Qian et al., 2020). In the
formers the system tries to learn as many features as possible when
interacting with the user. User’s feed-backs are helpful in refining rules and
essential to increase their precision and recall.

The elements Event, Gateway, Sequence Flow, and Message Flow are other popular
elements extracted from a process model description especially in the works
that aim at extracting BPMN diagrams. Event is targeted by all the works that
focus on the BPMN language but (Epure et al., 2015), and by two works that aim
at tagging the text. With the exception of (Han et al., 2020), all the works
exploit a rule based strategy similar to the one applied for extracting
activities. Gateway is extracted by all the approaches that focus on the BPMN
language. Moreover it is somehow addressed also by the two works that extract
declarative specifications, as in these works the control flow structures are
not extracted explicitly, but captured by the constraints them-selves. The
majority of papers ((Ackermann and Volz, 2013; Sawant et al., 2014; Epure et
al., 2015; Ferreira et al., 2017; de A. R. Gonçalves et al., 2011; Honkisz et
al., 2018; Friedrich et al., 2011; Leopold et al., 2018)) extract control flow
structures using a rule-based approach. What differentiates the different
works is the list of signal words they adopt to increase coverage of the
rules. A neural approach is instead exploited in (Han et al., 2020; Qian et
al., 2020). Similarly to gateway, Sequence Flow is extracted by all the
approaches that focus on the BPMN language and is always extracted with a
rule-based strategy with the exception of (Han et al., 2020). The same holds
for Message Flow, which is not extracted in (Epure et al., 2015) since this
work does not extract events. Sequence Flow and Message Flow are instead
almost not considered in works that realize $fa$ only.

Control flow elements, in the form of DECLARE relations, are always extracted
in works that address declarative-models (van der Aa et al., 2019; López et
al., 2018) or annotate text with declarative constructs (Quishpi et al.,
2020), adopting the same strategy adopted for activity.

The analysis of actors111111We have classified as actors all mentions of
actors and roles. and artifacts121212We have classified as artifacts all
mentions of artifacts, business entities, and (non agentive) resources., shows
that they are slightly less present than control flow elements in the output
of approaches that target the BPMN languages, completely absent in approaches
that target the Declare language (possibly due to the still preliminary status
of work that extend Declare with data), and rather popular in the four papers
that only implement a partial pipeline, where they are second only to
activity. Regarding the strategy, the situation is similar to the extraction
of the other process model diagram elements. In general, actors and artifacts
are extracted by means of rules applied on top of the analysis of NPs and VPs
of each sentence. Rules in form of patterns or in form of grammar expression
rules are explored in (de A. R. Gonçalves et al., 2011; Ackermann and Volz,
2013; Sawant et al., 2014; Ferreira et al., 2017). In the contributions
proposed in (Sawant et al., 2014; Honkisz et al., 2018), rules are extended by
the use of word-lists and a gazetteer, whereas in the work proposed in
(Honkisz et al., 2018) this step also integrates WordNet. A more elaborated
strategy is proposed in (Friedrich et al., 2011; Leopold et al., 2018). In
these two contributions actors extracted are validated by checking the
presence of terms against a word-list, and the analysis of the dependency
relations between words. T-regex patterns are defined in the work in Quishpi
et al. (2020); whereas the extraction performed via artificial neural network
is explored in (Han et al., 2020)

###  4.3 The experimental evaluation conducted

In this section we collect and compare the studies focusing on the
experimental evaluation they conduct. Since we are interested in process
extraction from text we report here only the evaluation part that refers to
this specific task131313Some papers use the process extraction from text as a
component to address further tasks, which are evaluated in the papers. We do
not report these different evaluations here. We compare the papers collecting
the following information:  (i) what is tested;  (ii) the dataset, and  (iii)
the metric adopted in the experimental evaluation;  (iv) the result of the
evaluation in terms of performance.  The results are reported in Table 3. We
indicate with F1 score∗ those papers that use several accuracy metrics to
evaluate the contribution, which are here omitted for the sake of
presentation. The interested reader can find the other accuracy metrics (such
as precision and recall) in the corresponding papers.

[tbh]

Table 3: The evaluations performed.

|  | Test dataset |  |
---|---|---|---|---
Paper |  What is tested |  dataset | Availability | Metric |  Score
Ackermann and Volz (2013) |  - |  - | - | - |  -
|  Disambiguation ability of rules |  subset of (Friedrich et al., 2011) | Yes | F1 score* |  0.86
van der Aa et al. (2019) |  DECLARE pattern extraction |  103 $\langle$ patterns description, patterns$\rangle$ pairs | Yes | F1 score* |  0.74
López et al. (2018) |  - |  - | - | - |  -
Han et al. (2020) |  Diagram similarity |  6000 $\langle$process description, model diagram$\rangle$ pairs | No | Graph Edit Distance |  Edge:0.57; Node:0.76; Graph:0.32
de A. R. Gonçalves et al. (2011) |  Regular Expression Grammars G1, G2 for process model elements extraction |  2 cases studies (c.s.1 and c.s.2) | No | Accuracy |  G1: c.s.1 33%, c.s.2 45% G2: c.s.1 80%, c.s.2 78%
Sawant et al. (2014) |  Extraction of: (i) Business Entity (artefact); (ii) Initiating Actor; (iii) Responding Actor; (iv) Action (Event) |  123 Use Case Descriptions | No | F1 score* |  (i) 0.79; (ii) 0.98; (iii) 0.879; (iv) 0.977
Friedrich et al. (2011) |  Diagram similarity |  47 $\langle$process description, model diagram$\rangle$ pairs | Yes | Graph Edit Distance |  0.77
Epure et al. (2015) |  Activity extraction |  Archaeological report composed of 34 executions | No | Precision |  0.88
Honkisz et al. (2018) |  - |  - | - | - |  -
Ferreira et al. (2017) |  Extraction of: (i) Activity; (ii) Event; (iii) Exclusive Gateway; (iv) Inclusive Gateway |  56 process model description | No | F1 score* |  (i) 0.807; (ii) 0.868; (iii) 0.816; (iv) 0.927
Qian et al. (2020) |  (i) single sentence classification; (ii) sentence semantics; (iii) semantic role labeling 1 |  A collection of recipes and manuals | Yes | Accuracy |  (i) recipe 93.34% manual 91.74%; (ii) recipe 91.53% manual 86.49%; (iii) recipe 82.39% manual 80.44%
Quishpi et al. (2020) |  Extraction of: (i) Activity; (ii) Relation |  (i) subset of (Friedrich et al., 2011) \+ subset of (Sànchez-Ferreres et al., 2018); (ii)subset of (Friedrich et al., 2011) \+ subset of (Sànchez-Ferreres et al., 2018) \+ (van der Aa et al., 2019) | Partial | F1 score* |  (i) 0.71; (ii) 0.61
Leopold et al. (2018) |  Extraction of different types of Activity |  subset of (Friedrich et al., 2011) | Yes | F1 score* |  0.81

  * 1

Our understanding of the paper is that the single sentence classification task
classifies a sentence (or a textual fragment) as an Activity or not. The
sentence semantic task classifies the semantic conveyed in a sentence (or in a
textual fragment) into gateway, loop, end-event, and sequence flow. The
semantic role labeling task classifies the words of the text into three
categories: role, action and activity data.

The table shows an highly heterogeneous situation both in terms of what is
evaluated, and the datasets and metrics used to perform the evaluation itself.
This makes a direct comparison almost impossible. Nonetheless we provide an
attempt to comparatively analyze the different columns.

Focusing on what is tested, we can divide the contributions in three main
groups:  (i) contributions that evaluate the similarity between the (BPMN, in
our case) process diagram extracted from a text and that provided in the
testing dataset (Diagram Similarity),  (ii) contributions that evaluate the
extraction of process model diagram elements, and  (iii) other contributions.
The works in (Friedrich et al., 2011; Han et al., 2020) belong to the first
group. These contributions evaluated the diagram similarity adopting the Graph
Edit Distance as evaluation metric. However, these two contributions were
tested on different datasets and therefore the evaluations provided in the
papers are not directly comparable. The works in (van der Aa et al., 2019; de
A. R. Gonçalves et al., 2011; Sawant et al., 2014; Epure et al., 2015;
Ferreira et al., 2017; Quishpi et al., 2020; Leopold et al., 2018) belong to
the second group, and evaluate the ability to extract different types of
elements. Finally, the workin (Qian et al., 2020) evaluates the ability to
identify properties of the input sentence and the ability of the proposed rule
to disambiguate sentences.

Table 3 shows that the different works were tested using different datasets or
different sub-set of the same dataset, which are often not publicly available.
This also makes a rigorous comparison extremely hard. The only dataset shared
among different papers is the one proposed by Friedrich et al. (2011). It is a
public dataset consisting of pairs containing a textual description of a
business process and its BPMN representation. Parts of it are adopted by other
three papers, but each paper extracts different parts of it for the
evaluation. As a consequence, while being often considered a reference
dataset, the dataset of (Friedrich et al., 2011) can be hardly considered a
reference dataset in practice. As far as we know, the dataset proposed by van
der Aa et al. (2019) is he only publicly available dataset for the Declare
language, while the dataset of Qian et al. (2020) is the only other publicly
available dataset and contains sentences with words labeled. All the other
datasets are either not available or not publicly available.

Focusing on the metrics, they can be grouped in two groups that reflect the
ones described above when describing what is tested. The works in (Friedrich
et al., 2011; Han et al., 2020) adopt a graph-based metric to quantitatively
evaluate the quality of the diagrams extracted from a textual description of a
process. In particular, the work in (Han et al., 2020) adopts a slightly
modified version of the formula proposed in (Friedrich et al., 2011). Instead,
the works that focus on the extraction of one or more process elements ((van
der Aa et al., 2019; de A. R. Gonçalves et al., 2011; Sawant et al., 2014;
Epure et al., 2015; Ferreira et al., 2017; Quishpi et al., 2020; Leopold et
al., 2018)) all adopt information retrieval metrics.

Analyzing the scores in a unifying manner is a problematic task, since the
different contributions are tested on different test sets, and the
experimental evaluations described in the paper target different aspects of
the extraction pipeline. We can, nonetheless, divide the evaluations in two
main groups. A first group, which comprises (Friedrich et al., 2011) and (Han
et al., 2020) aims at testing the process model diagram generated in output.
Here, the prototype proposed in (Friedrich et al., 2011) achieves a higher
score that the prototype presented in (Han et al., 2020) All the other
contributions aim at testing the extraction of specific elements, but their
comparison is almost impossible. To make an example, activity extraction is
evaluated in (de A. R. Gonçalves et al., 2011; Epure et al., 2015; Ferreira et
al., 2017; Quishpi et al., 2020; Leopold et al., 2018). A direct comparison is
difficult since these works often adopt different metrics. Also, even for
works that exploit the same metric (see e.g., (Ferreira et al., 2017) and
(Quishpi et al., 2020)), it is difficult to say if an higher score is due to a
better tool or on the evaluation performed on input text with different levels
of complexity. Regarding the group of works that tested extraction of
declarative relations, Quishpi et al. (2020) provides a direct comparison with
van der Aa et al. (2019) showing a better performance. If we restrict this
evaluation to the dataset originally introduced in (van der Aa et al., 2019),
we can note that van der Aa et al. (2019) still achieves a better performance
than Quishpi et al. (2020).

##  5 Discussion

The analysis proposed in the previous Sections reveals several problematic
aspects of state of the art work in process extraction from text. In this
section we elaborate on four sets of problematic questions concerning  (i) the
data;  (ii) to the techniques adopted;  (iii) the experimental evaluation
conducted; and  (iv) the pipeline proposed.

###  5.1 The data

A crucial set of questions concern the datasets adopted. Data are problematic
in many ways: First of all, they are often not publicly available. Even worst,
most of the available ones are somehow extracted from the dataset originally
proposed in (Friedrich et al., 2011). The only exception is the dataset
proposed in (Qian et al., 2020). Nonetheless, we are doubtful of the
usefulness of these data in addressing process extraction from text challenges
since it consists of instruction booklets and food recipes, whose complexity
in terms of control flow is often simpler than the one of e.g., an
administrative procedure. Second, the dataset in (Friedrich et al., 2011)
presents some problematic aspects. A major problem is given by the fact that
it is not representative of the variety of $<$process description, process
diagram$>$ pairs one may have in real scenarios. Obviously having such a fully
representative dataset may be too ambitious even for a high quality dataset,
given the variability of language and also the different ways in which the
same intended process can be represented in a diagram. Nonetheless, the
dataset in (Friedrich et al., 2011) was mainly the effort of a limited group
of persons, missing the typical validation steps in the form of multiple
annotations and inter-annotation agreements that are typically used to take
care of the differences that may exist in human judgment. Third, the dataset
in (Friedrich et al., 2011) represents pairs of process description and
process diagram without any intermediate form of annotation. For example, the
specific correspondence between a portion of the text and an activity, or a
gateway, in the diagram is missing. As a consequence the dataset is not ready
to be used to work on and comparatively evaluate sub-tasks of process
extraction from text, such as e.g., activity (gateway, actor, or some specific
relations among them) extraction from text. In our opinion, this lack of
specific annotations has had a negative impact both on the investigation of
specific sub-tasks of process extraction from text, and on the development and
the adoption of reference metrics to comparatively evaluate these specific
sub-tasks. This has contributed to create the heterogeneous situation depicted
in Table 3, where the different contributions can be hardly compared on a
common ground. Fourth, the size of the dataset is not large enough to support
the training and testing required by many sophisticated machine-learning
techniques. We believe that this lack of good quality, large enough, publicly
available annotated datasets, that reflect the real complexity of documents
adopted in, e.g., companies or real organizations, is a core problem that is
likely impeding the development of scalable proposals. In fact, such datasets
could enable the adoption of recent Natural Language Processing techniques and
machine learning solutions to handle writing style variations and word
ambiguities.

We believe that the BPM community, possibly adopting mechanisms similar to the
ones that have made available dozens of event-logs via the BPI challenges
(which, in turn, have had a crucial impact in the development of process
mining from event-log techniques). In creating such datasets the community
will have to answer some crucial questions such as how big a dataset should
be? How varied should it be (both in terms of textual variability and process
elements)? Do we need focus on text annotated with specific elements or on
pairs of textual fragments and their corresponding models? In this latter
case, how do we deal with possibly equivalent models and the biases that may
be introduced by providing only one process model?

###  5.2 The techniques

The lack/availability of data has a direct impact on the techniques that can
be applied in process extraction from text. In particular, the lack of big
quantities of annotated data make the training of neural models unfeasible and
indeed most of the approaches resort to use rule-based strategies. Defining
rules that are precise and general enough at the same time however appears to
be problematic. Often word-lists and external resources such as WordNet,
VerbNet are adopted to increase the coverage and robustness of generic rules
(especially the ones that analyze SVO triples), but doing that makes the rules
extremely tailored to specific scenarios, writing styles, and formatting of
documents with the result of having to refine/re-build the framework for each
domain it is adopted in. Another problem is related to the identification of
activities, actors and data-objects, via the analysis of verbs, and
subject/object relations. An effective contribution of state of the art works
is that the analysis of SVO triples enables the differentiation between those
verbs expressing activities and those verbs expressing events. Also, the
extraction of the subject and of the object of a verb allows the detection of
actors and data-objects. However, all papers ignore the problem of false
positives. This problem could be addressed using classifiers on top of the
extraction mechanism, Nonetheless this would require annotated datasets
containing also non-informative sentences (or more specifically here,
sentences whose SVO triples do not almost always translate into an actor-
activity-data object pattern). A further intriguing aspect of state of the art
work is that most works extract syntactics and semantics information through
the analysis of the plain text and only few papers provide a direct semantic
representation of the text input by mean of semantic word embeddings. Given
the impact that semantic word embeddings has recently had on different tasks
in Information Retrieval and Natural Language Processing, and the importance
of it in enhancing, e.g., the performance of statistical classifiers, this
also opens interesting questions for future research efforts.

All the problems above are somehow related and concern the usage of recent NLP
advancements in process extraction from text. While the usage of these
techniques is related to the availability of data highlighted above, we also
believe that this perspective puts forward interesting specific questions on
the techniques adopted, such as: how can we leverage recent NLP advancements
and increase performance of process extraction from text tasks? Can we adopt
semantics embeddings to represent the input text instead of performing a deep
syntactical and semantical analyses with the available amount of data? And
ultimately, is it possible to build machine-learning based techniques that
overcome the current rule-based ones in terms of performance and generality?

A final observation concerns the usage of external resources and NLP
techniques/tools (such as coreference resolution tools), and in particular the
lack of adoption of new NLP techniques/tools, likely due to the lack of recent
works on process extraction from text. Given the recent advances in NLP, this
observation raises an interesting question for the community: Can we increase
the frameworks’ performance just leveraging recent NLP advancement and up-to-
date resources? Answering this question would help in understanding which
errors generated by State of the art approaches are due to the specific
process extraction from text techniques proposed, and which errors are due to
obsolete NLP resources and instruments adopted.

###  5.3 The experimental evaluations

We have already discussed of the problems related to the data, which obviously
influence also the experimental evaluations, and therefore we we focus here on
problems and challenges related to the metrics adopted to measure the
performance of the proposed techniques.

By looking at Table 3 we can see that most contributions analyzed in this
paper rely either on information retrieval measures, or on graph-based
measures. Nonetheless, a first problematic issue is that no real reference
metrics exist in the community for the different proposed tasks. Another
problem that should deserve some consideration is the fact that these metrics
do not distinguish between possible kinds of errors that can be generated. For
example, would it be better to identify a gateway but assign it to the wrong
type or do not identify it at all? Would it be better to associate an activity
to the wrong actor or just to extract the activity? Answering these questions
is a challenging task as extracting more information may help the manual
refinement of the process, but errors that are not rectified may have
significant negative consequences on the process model. A further problem is
given by the fact that the metrics proposed in the literature mainly consider
the graph structure of the process model and do not make a distinction between
equivalent (very similar) process model fragments. As a very simple example
how should we evaluate when two activities are represented as a compound
activity in the diagram?

Future work on process extraction from text will also require addressing the
challenges related to the metrics and in particular answering questions such
as is it possible to propose reference metrics for the different specific
subtasks of process extraction from text as well as for the overall
performance of the entire pipeline? Shall we distinguish among different types
of errors or not? How do we take into account that different graph structures
may convey the same (or very similar) semantic meanings? The ultimate goal
here would be to address two fundamental high level questions: How can we
measure the performance of a process extraction from text framework? Which are
the aspect of the framework we need to consider during an experimental
evaluation?

###  5.4 The pipeline

Concerning the overall pipeline the analysis of state of the art work shows
efforts in both the direct and the two step approaches. While the direct
approach may appear extremely attractive, due to the popularity of end-to-end
machine learning approaches, the big question here is is there a way to
collect the massive set of data that would require the construction of a
generic direct pipeline?

If we focus on the two step approach, we can identify a big potential
advantage, due to the fact that we can decompose the entire pipeline into
modular subtasks and therefore work in a divide-et-impera manner, but also
several problems and challenges. The first one relates to the composition of
the pipeline, and to the different (sub)tasks in which we may decompose
$f_{a}$ and $f_{b}$. As an example we have noticed that the task of filtering
out uninformative information is rarely considered in state of the art work
but may be extremely important in real world procedural documents. This could
be due to the fact that there are almost no uninformative textual fragments in
the reference dataset of Friedrich et al. (2011) even though the pipeline
should drive the construction of the datasets and not the other way around.
Another challenge concerns the ordering of the different sub-tasks, and the
impact it can have on the overall pipeline. For example, do we need to resolve
coreferences before filtering out uninformative sentences from a text or can
we post-pone this task later? Another problematic aspect regards the labeling
task, and the available datasets for all the specific different su-tasks. As
already mentioned when discussing the data problem, the reference dataset of
Friedrich et al. (2011) lacks and intermediate form of annotation and is
therefore not directly usable for the construction and evaluation of e.g., a
specific module dealing with the subtask of detecting noun-based activities.
Further questions concern the usage of the intermediate representation. First
of all, no clear reasons are given in papers on why a specific world model was
used, and no study exist to assess whether some are better and offer more
robust intermediate representation. Second, sharing the data contained in
different world models built by different process extraction from text
frameworks is currently very hard. This is extremely problematic, as having a
common world model (or at least a way to move data across different ones)
would boost a real modularization of $f_{a}$ and $f_{b}$, allow reuse and the
investigation of single components of the pipeline one at a time.

To sum up, the overall questions concerning the two step approach are: can we
provide a clear definition of the sub-tasks (or modules) that need to be
performed to realize $f_{a}$ and $f_{b}$ and in which order they have to be
composed? Can we support all the different modules with appropriate datasets?
Is there a better world model in terms of robustness and efficient
representation? Which are its characteristics? Can we define a common data-
structure to structure and share the intermediate representation? And
ultimately can the BPM community take advantage of the modular structure of
the two way approach to boost the research in process extraction from text?

##  6 Conclusions

In this paper we have provided a comparative analysis of state of the art work
on process extraction from text, published specifically in the area of
Business Process Management from 2010, so as to better understand their
contributions and limitations and to identify which are important challenges
in this topic the research community should address.

The comparative analysis shows the existence of important problems related to
the data, the proposed techniques, and especially the usage of tailored and
often very specific rule based approaches, the lack of uniformity in the
experimental evaluation, and somehow still a lack of a reference pipeline. In
particular we have emphasized as the lack of representative and realistic
datasets, together with the lack of detailed annotations of the different
elements a process is composed of, is hampering the exploitation of recent
advances in machine learning based NLP techniques. While we believe that the
two-steps transformation approach with intermediate representation is likely
to be the most promising approach in process extraction from text, because of
its modular nature which may enable the adoption of a divide-et-impera
strategy, we also believe that the BPM community should make a coordinated
effort to support and stimulate research in the specific subtasks that compose
a process extraction from text pipeline, for instance by adopting mechanisms
similar to the ones that have made available dozens of event-logs via the BPI
challenges, and have addressed year after year, different specific bits of a
process discovery from event-log pipeline.

## References

  * Achour [1998] C. B. Achour.  Guiding Scenario Authoring.  In _Information Modelling and Knowledge Bases X: 8th European-Japanese Conferences on Information Modelling and Knowledge Bases_ , volume 51 of _Frontiers in Artificial Intelligence and Applications_ , pages 152–171. IOS Press, 1998.
  * Ackermann and Volz [2013] L. Ackermann and B. Volz.  model[NL]generation: natural language model extraction.  In _Proceedings of the 2013 ACM workshop on Domain-specific modeling, DSM@SPLASH 2013_ , pages 45–50. ACM, 2013.  doi: 10.1145/2541928.2541937.
  * Adamo et al. [2017] G. Adamo, S. Borgo, C. Di Francescomarino, C. Ghidini, N. Guarino, and E. M. Sanfilippo.  Business Processes and Their Participants: An Ontological Perspective.  In _AI*IA 2017 Advances in Artificial Intelligence - XVIth International Conference of the Italian Association for Artificial Intelligence, Proceedings_ , volume 10640 of _Lecture Notes in Computer Science_ , pages 215–228. Springer, 2017.  doi: 10.1007/978-3-319-70169-1\\_16.
  * Adamo et al. [2020] G. Adamo, C. Di Francescomarino, and C. Ghidini.  Digging into Business Process Meta-models: A First Ontological Analysis.  In _Advanced Information Systems Engineering - 32nd International Conference, CAiSE 2020, Proceedings_ , volume 12127 of _Lecture_ , pages 384–400. Springer, 2020.  doi: 10.1007/978-3-030-49435-3\\_24.
  * Cimiano [2006] P. Cimiano.  _Ontology learning and population from text - algorithms, evaluation and applications_.  Springer, 2006.  ISBN 978-0-387-30632-2.  doi: 10.1007/978-0-387-39252-3.
  * Cunningham et al. [2013] H. Cunningham, V. Tablan, A. Roberts, and K. Bontcheva.  Getting More Out of Biomedical Documents with GATE’s Full Lifecycle Open Source Text Analytics.  _PLoS Comput. Biol._ , 9(2), 2013.  doi: 10.1371/journal.pcbi.1002854.
  * de A. R. Gonçalves et al. [2011] J. C. de A. R. Gonçalves, F. M. Santoro, and F. A. Baião.  Let Me Tell You a Story - On How to Build Process Models.  _J. UCS_ , 17(2):276–295, 2011.  doi: 10.3217/jucs-017-02-0276.
  * Epure et al. [2015] E. V. Epure, P. Martín-Rodilla, C. Hug, R. Deneckère, and C. Salinesi.  Automatic process model discovery from textual methodologies.  In _9th IEEE International Conference on Research Challenges in Information Science, RCIS 2015_ , pages 19–30. IEEE, 2015.  doi: 10.1109/RCIS.2015.7128860.
  * Ferreira et al. [2017] R. C. B. Ferreira, L. H. Thom, and M. Fantinato.  A Semi-automatic Approach to Identify Business Process Elements in Natural Language Texts.  In _ICEIS 2017 - Proceedings of the 19th International Conference on Enterprise Information Systems, Volume 3_ , pages 250–261. SciTePress, 2017.  doi: 10.5220/0006305902500261.
  * Friedrich et al. [2011] F. Friedrich, J. Mendling, and F. Puhlmann.  Process Model Generation from Natural Language Text.  In _Advanced Information Systems Engineering - 23rd International Conference, CAiSE 2011. Proceedings_ , volume 6741 of _Lecture Notes in Computer Science_ , pages 482–496. Springer, 2011.  doi: 10.1007/978-3-642-21640-4\\_36.
  * Ghidini et al. [2012] C. Ghidini, M. Rospocher, and L. Serafini.  Modeling in a wiki with moki: Reference architecture, implementation, and usages.  _International Journal On Advances in Life Sciences_ , 4(3&4):111–124, 2012.
  * Han et al. [2020] X. Han, L. Hu, L. Mei, Y. Dang, S. Agarwal, X. Zhou, and P. Hu.  A-BPS: automatic business process discovery service using ordered neurons LSTM.  In _2020 IEEE International Conference on Web Services, ICWS 2020_ , pages 428–432. IEEE, 2020.  doi: 10.1109/ICWS49710.2020.00063.
  * Honkisz et al. [2018] K. Honkisz, K. Kluza, and P. Wisniewski.  A Concept for Generating Business Process Models from Natural Language Description.  In _Knowledge Science, Engineering and Management - 11th International Conference, KSEM 2018, Proceedings, Part I_ , volume 11061 of _Lecture Notes in Computer Science_ , pages 91–103. Springer, 2018.  doi: 10.1007/978-3-319-99365-2\\_8.
  * Jurafsky and Martin [2009] D. Jurafsky and J. H. Martin.  _Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition_.  Prentice Hall series in artificial intelligence. Prentice Hall, Pearson Education International, 2009.  ISBN 9780135041963.
  * Leopold et al. [2018] H. Leopold, H. van der Aa, and H. A. Reijers.  Identifying candidate tasks for robotic process automation in textual process descriptions.  In _Enterprise, Business-Process and Information Systems Modeling - 19th International Conference, BPMDS 2018, 23rd International Conference, EMMSAD 2018, Held at CAiSE 2018, Proceedings_ , volume 318 of _Lecture Notes in Business Information Processing_ , pages 67–81. Springer, 2018.  doi: 10.1007/978-3-319-91704-7\\_5.
  * López et al. [2018] H. A. López, S. Debois, T. T. Hildebrandt, and M. Marquard.  The process highlighter: From texts to declarative processes and back.  In _Proceedings of the Dissertation Award, Demonstration, and Industrial Track at BPM 2018 co-located with 16th International Conference on Business Process Management (BPM 2018)_ , volume 2196 of _CEUR Workshop Proceedings_ , pages 66–70. CEUR-WS.org, 2018.
  * Manning et al. [2014] C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel, S. Bethard, and D. McClosky.  The stanford CoreNLP natural language processing toolkit.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, System Demonstrations_ , pages 55–60. The Association for Computer Linguistics, 2014.  doi: 10.3115/v1/p14-5010.
  * Maqbool et al. [2018] B. Maqbool, F. Azam, M. W. Anwar, W. H. Butt, J. Zeb, I. Zafar, A. K. Nazir, and Z. Umair.  A comprehensive investigation of BPMN models generation from textual requirements - techniques, tools and trends.  In _Information Science and Applications - ICISA 2018_ , volume 514 of _Lecture Notes in Electrical Engineering_ , pages 543–557. Springer, 2018.  doi: 10.1007/978-981-13-1056-0\\_54.
  * Mendling et al. [2014] J. Mendling, H. Leopold, and F. Pittke.  25 Challenges of Semantic Process Modeling.  _International Journal of Information Systems and Software Engineering for Big Companies (IJISEBC)_ , 1(1):78–94, 2014\.
  * Mendling et al. [2017] J. Mendling, B. Baesens, A. Bernstein, and M. Fellmann.  Challenges of smart business process management: An introduction to the special issue.  _Decis. Support Syst._ , 100:1–5, 2017.  doi: 10.1016/j.dss.2017.06.009.  URL https://doi.org/10.1016/j.dss.2017.06.009.
  * Mendling et al. [2019] J. Mendling, H. Leopold, L. H. Thom, and H. van der Aa.  Natural language processing with process models (NLP4RE report paper).  In P. Spoletini, P. Mäder, D. M. Berry, F. Dalpiaz, M. Daneva, A. Ferrari, X. Franch, S. Gregory, E. C. Groen, A. Herrmann, A. Hess, F. Houdek, O. Karras, A. Koziolek, K. Lauenroth, C. Palomares, M. Sabetzadeh, N. Seyff, M. Trapp, A. Vogelsang, and T. Weyer, editors, _Joint Proceedings of REFSQ-2019 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 25th International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2019), Essen, Germany, March 18th, 2019_ , volume 2376 of _CEUR Workshop Proceedings_. CEUR-WS.org, 2019.  URL http://ceur-ws.org/Vol-2376/NLP4RE19_paper04.pdf.
  * Miller et al. [1990] G. A. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. Miller.  Wordnet: An on-line lexical database.  _International Journal of Lexicography_ , 3:235–244, 1990\.
  * Pennington et al. [2014] J. Pennington, R. Socher, and C. D. Manning.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014_ , pages 1532–1543. ACL, 2014.  doi: 10.3115/v1/d14-1162.
  * Petrucci et al. [2018] G. Petrucci, M. Rospocher, and C. Ghidini.  Expressive ontology learning as neural machine translation.  _J. Web Semant._ , 52-53:66–82, 2018.  doi: 10.1016/j.websem.2018.10.002.
  * Qian et al. [2020] C. Qian, L. Wen, A. Kumar, L. Lin, L. Lin, Z. Zong, S. Li, and J. Wang.  An approach for process model extraction by multi-grained text classification.  In _Advanced Information Systems Engineering - 32nd Int. Conference, CAiSE 2020, Proceedings_ , volume 12127 of _Lecture Notes in Computer Science_ , pages 268–282. Springer, 2020.  doi: 10.1007/978-3-030-49435-3\\_17.
  * Quishpi et al. [2020] L. Quishpi, J. Carmona, and L. Padró.  Extracting annotations from textual descriptions of processes.  In _Business Process Management - 18th International Conference, BPM 2020, Proceedings_ , volume 12168 of _Lecture Notes in Computer Science_ , pages 184–201. Springer, 2020.  doi: 10.1007/978-3-030-58666-9\\_11.
  * Riefer et al. [2016] M. Riefer, S. F. Ternis, and T. Thaler.  Mining process models from natural language text: A state-of-the-art analysis.  _Multikonferenz Wirtschaftsinformatik (MKWI-16), March_ , pages 9–11, 2016.
  * Sànchez-Ferreres et al. [2018] J. Sànchez-Ferreres, H. van der Aa, J. Carmona, and L. Padró.  Aligning textual and model-based process descriptions.  _Data Knowl. Eng._ , 118:25–40, 2018.  doi: 10.1016/j.datak.2018.09.001.
  * Santoro et al. [2008] F. M. Santoro, M. R. S. Borges, and J. A. Pino.  Tell us your process: A group storytelling approach to cooperative process modeling.  In _Proceedings of the 12th International Conference on CSCW in Design, CSCWD 2008_ , pages 29–34. IEEE, 2008.  doi: 10.1109/CSCWD.2008.4536950.
  * Sawant et al. [2014] K. P. Sawant, S. Roy, S. Sripathi, F. Plesse, and A. S. M. Sajeev.  Deriving requirements model from textual use cases.  In _36th International Conference on Software Engineering, ICSE ’14, Companion Proceedings._ , pages 235–244. ACM, 2014.  doi: 10.1145/2591062.2591193.
  * Schuler and Palmer [2005] K. K. Schuler and M. S. Palmer.  _Verbnet: A Broad-Coverage, Comprehensive Verb Lexicon_.  PhD thesis, University of Pennsylvania, USA, 2005.  AAI3179808.
  * van der Aa et al. [2018a] H. van der Aa, J. Carmona, H. Leopold, J. Mendling, and L. Padró.  Challenges and opportunities of applying natural language processing in business process management.  In _Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018._ , pages 2791–2801. Association for Computational Linguistics, 2018a.
  * van der Aa et al. [2018b] H. van der Aa, H. Leopold, and H. A. Reijers.  Checking process compliance against natural language specifications using behavioral spaces.  _Inf. Syst._ , 78:83–95, 2018b.  doi: 10.1016/j.is.2018.01.007.
  * van der Aa et al. [2019] H. van der Aa, C. Di Ciccio, H. Leopold, and H. A. Reijers.  Extracting declarative process models from natural language.  In _Advanced Information Systems Engineering - 31st International Conference, CAiSE 2019, Proceedings_ , volume 11483 of _Lecture Notes in Computer Science_ , pages 365–382. Springer, 2019.  doi: 10.1007/978-3-030-21290-2\\_23.
  * van der Aalst [2011] W. M. P. van der Aalst.  _Process Mining - Discovery, Conformance and Enhancement of Business Processes_.  Springer, 2011.  ISBN 978-3-642-19344-6.  doi: 10.1007/978-3-642-19345-3.
  * Wisniewski et al. [2019] P. Wisniewski, K. Kluza, K. Jobczyk, B. Stachura-Terlecka, and A. Ligeza.  Overview of generation methods for business process models.  In _Knowledge Science, Engineering and Management - 12th Int. Conference, KSEM 2019, Proceedings, Part II_ , volume 11776 of _Lecture Notes in Computer Science_ , pages 55–60. Springer, 2019.  doi: 10.1007/978-3-030-29563-9\\_6.