missing

#  verBERT: Automating Brazilian Case Law Document Multi-label Categorization
Using BERT

Felipe R. Serras1, Marcelo Finger1

###### Abstract

In this work, we carried out a study about the use of attention-based
algorithms to automate the categorization of Brazilian case law documents. We
used data from the Kollemata Project to produce two distinct datasets with
adequate class systems. Then, we implemented a multi-class and multi-label
version of BERT and fine-tuned different BERT models with the produced
datasets. We evaluated several metrics, adopting the micro-averaged F1-Score
as our main metric for which we obtained a performance value of
$\langle\mathcal{F}_{1}\rangle_{micro}=0.72$ corresponding to gains of 30
percent points over the tested statistical baseline.

##  1 Introduction

In this work, we explore the use of BERT [Devlin et al. 2019] to develop a
prototype to automate the categorization of Brazilian case law documents,
known as “verbetação”. This categorization has been done manually for the past
$25$ years by experts in the field of law, without taking advantage of
knowledge engineering techniques; as a result, the categories are not
organized as any computationally coherent data structure. Its automation
could, therefore, save time and labor, in addition to optimizing its ability
to represent knowledge. The problem was brought to us by the Kollemata Project
team, which organized a library of Brazilian jurisprudence, containing more
than 24,000 case law documents from the property law field, and whose dataset
was used as the basis for this research.

Unlike other problems in NLP, this one was not clearly associated with a
predefined task. We raised several modeling possibilities and chose multi-
label categorization because of the high value of having an unified set of
predetermined thematic categories for this task, manifested both in the
literature of the field and by law professionals we consulted.

Considering the scarce literature on this problem, especially in Brazilian
Portuguese, our goals were (i) to provide an initial proposal for modeling the
problem, compatible with the manifested needs of the consulted professionals,
(ii) to assess the potential performance of the main state-of-the-art methods
(in this case BERT) on this modeling, and (iii) to map the main obstacles to
performance improvement on the adopted modeling. The results of our work are
prototypical and seek, mainly, to start the debate and indicate the future
paths that research on this topic could take.

To achieve these goals, we reformulated the Kollemata dataset with an
appropriate class system, and implemented a set of BERT classifiers, which can
serve as prototypes to automate the categorization of Brazilian case law
documents, deepening our understand of the addressed problem and the used
algorithm and methodology.

##  2 Related Work

Following the proposition of BERT, which included the presentation of the
Multilingual BERT model, the BERTimbau model [Souza et al. 2020] \- the first
BERT model trained entirely in Brazilian Portuguese - was developed and used
in the automation of several tasks in Brazilian Portuguese [Leite et al. 2020,
Souza et al. 2019, Salvatore et al. 2019].

As far as we know, this is the first work to explore the use of BERT to
automate case law documents categorization in Brazilian Portuguese. Other
works explore the same problem in other languages or dialects of Portuguese
[Calambás et al. 2015, Gonçalves and Quaresma 2003, Gonçalves and Quaresma
2005, Feinerer and Hornik 2008, Mencía and Fürnkranz 2010], and one work
explores a single-label version of the same problem with less data in
Brazilian Portuguese [de Colla Furquim and De Lima 2012]. All of these
articles studied algorithms created before BERT.

Regarding our efforts to reorganize the data class system, we used clustering
techniques to produce an organized class hierarchy, which is not a new ideia
for general [Aggarwal and Zhai 2012] or legal applications [Moens 2001].
However, here we adopted a methodology designed to meet the specific needs of
this task.

Only few recent works relate to ours, both in method and type of task, as the
articles of [Villata et al. 2020] and [Chalkidis et al. 2019]. In the first
one, the authors explore different applications of machine learning to Italian
law documents, including the use of the Multilingual BERT model to perform the
categorization of case law documents under a proprietary taxonomy. In the
second one, several algorithms, including BERT, were tested to generate a
multi-label classifier for European legislative documents in English, based on
a large system of topics. These two works are those whose results are most
comparable to ours.

##  3 Problem

In Brazil, case law documents are called “acórdãos” and are used to register
the decision of a court and to serve as reference to similar cases in the
future (jurisprudence). Searching for these documents is a recurrent task for
law professionals and to facilitate it, all case law documents contain a
component of jurisprudence (“ementa jurisprudencial”), formed by (i) a brief
summary of the documents content and (ii) a header, containing a set of
descriptive terms that represent the main legal topics covered in the
decision. In Portuguese, these terms are called “verbetes” and the header is
called “verbetação”. Figure 1 contains a gold standard example of a Brazilian
component of jurisprudence [Guimarães 2004].

Figure 1: A gold standard example of a component of jurisprudence from a
Brazilian case law document, extracted from [Guimarães 2004].

The purpose of this component of jurisprudence is to help readers in assessing
whether a document is relevant or not to them. However, [Guimarães 2004]
identifies problems in the ways these headers are produced, including
redundancy, usage of generic terms, lack of a consistent vocabulary, and lack
of well-defined syntactic structures, both within the terms and between them.
These problems affect the appropriate usage of the component of jurisprudence
and get in the way of having a well formed and nationally validated set of
case law descriptor terms, seen as an important asset within the Brazilian
theory of law.

As stated in Section 1, given the importance of having this set of
predetermined descriptor terms, we decided to model it as a task of multi-
class and multi-label text categorization i.e. the task of assigning to each
document $d_{j}$ in a set of natural language documents $\mathcal{D}$, a
variable number of categories $c_{i}$, taken from a set of multiple possible
predefined categories $\mathcal{C}$ [Sebastiani 2002]. Also, fitting the data
to this model could serve as basis for the resolution of several problems
identified by [Guimarães 2004].

##  4 Materials and Methodology

###  4.1 Kollemata and verBERT

We used data from the Kollemata Project111https://www.kollemata.com.br/. Its
database contains more than 24,000 entries, each one corresponding to one case
law document from the property law field and containing its summary and
header222The Kollemata Project dataset was given to our research by the
Kollemata Project team. The data set will not be publicly available for now,
as Brazilian authorities are reviewing the correct way to make these documents
available on a large scale without exposing sensitive data..

Our goal was to use the pairs of summary and header content as the examples to
BERT fine-tuning, aiming to automatically generate the terms of the header
from the summary content. The procedures performed to prepare this data for
this usage are explained in Section 5.

We implemented a Python3 program called verBERT, that uses the Hugging Face
Transformers Package [Wolf et al. 2019] to fine-tune, evaluate and test BERT
multi-label classifiers using pre-treined BERT models. In doing so, we
refactored, adapted and incremented the examples of Hugging Face for
generating BERT Classifiers over GLUE [Wang et al. 2019], with copyright from
Hugging Face, Google and Nvidia, but available under the Apache 2.0
License333The verBERT source code, as well as other resources, are available
in our repository: https://github.com/frserras/verbert-categorization.

Our multi-label categorization architecture was inspired by [Trivedi 2019] and
uses a simple feed-forward network coupled with sigmoid functions $\sigma$ as
the categorization layer (Equations 1 and 2, where $\mathbf{a}$ is the BERT
output representation, $W^{C}:W^{C}\in\mathbb{R}^{d\times|\mathcal{C}|}$ a
linear transformation matrix, $\mathbf{b}$ a bias vector, and $d$ is the
characteristic dimension of the model) [Han and Moraga 1995], and the mean of
the binary cross-entropy [Goodfellow et al. 2016, p. 65-66] over the labels as
the loss function to be minimized using Adam optimizer [Kalchbrenner et al.
2014, Goodfellow et al. 2016, p. 305-306] with weight decay regularization
[Loshchilov and Hutter 2017].

$\mathbf{c}=\varsigma(\mathbf{a}W^{C}+\mathbf{b})$ (1)
$\varsigma(\mathbf{x}=\\{x_{1},...,x_{n}\\})=\\{\sigma(x_{1}),...,\sigma(x_{n})\\}$
(2)

We evaluated and compared three pre-trained BERT models as bases for our
classifier generation: Multilingual BERT, a base model trained over $104$
languages on Wikipedia data [Devlin et al. 2019], and the two variants (base
and large) of the BERTimbau Model, pre-trained entirely in Portuguese [Souza
et al. 2020, Souza et al. 2019].

###  4.2 Metrics and Baseline

To compare the models produced by verBERT, we adopted three performance
metrics commonly used in problems of categorization and information retrieval:
precision $\mathcal{P}$, recall $\mathcal{R}$, and F1-Score $\mathcal{F}_{1}$
[Godbole and Sarawagi 2004]. For each metric $\mathcal{E}$ we computed its
three averaged multi-label versions: the micro-average
$\langle\mathcal{E}\rangle_{micro}$, the macro-average
$\langle\mathcal{E}\rangle_{macro}$ and the average by instance
$\langle\mathcal{E}\rangle_{inst}$ [Koyejo et al. 2015]

We also used the accuracy metric [Godbole and Sarawagi 2004], for which we
selected two versions for the multi-label case: the Hamming accuracy
$\mathcal{A}$ i.e. the percentage of labels correctly predicted over all
examples, and the subset accuracy $\dot{\mathcal{A}}$, the percentage of
examples for which all labels were correctly predicted.

We selected the $\langle\mathcal{F}_{1}\rangle_{micro}$ as our main metric,
since it tends to be more appropriate for information retrieval cases with
strong unbalanced categories, like ours. As secondary metrics, we adopted (i)
the $\langle\mathcal{F}_{1}\rangle_{macro}$, which shows us the impact of
category imbalance on performance, if compared with its micro-averaged
counterpart, and (ii) the subset accuracy $\dot{\mathcal{A}}$, because is a
more robust metric for problems of categorization, that serves as a lower
bound to our performance.

Besides the models produced by verBERT, we implemented a simple statistical
baseline method, where the $n=5$ most common categories of the dataset were
used to categorize all of its entries444We tested different values for $n$ and
choose the one that maximized the baseline performance for our main metrics..
Its results are presented in Table 2. This type of baseline, despite its
simplicity, is a robust option for unbalanced category systems, where betting
on the most likely categories can be a good strategy for blindly maximizing
performance.

##  5 Data preparation and Ontological Adjustment

We used the pairs [summary, header], from the Kollemata Project dataset and
pre-processed them by (i) filtering corrupted HTML notation from the
summaries, (ii) unifying the terms for law entities across the dataset, and
(iii) mapping and unifying different segmentation symbols between descriptor
terms. The data was structured in a list of entries, each one corresponding to
a case law document and containing the summary and an ordered list of its
header descriptor terms.

We then performed an exploratory analysis on the dataset, to identify its main
properties and eventual obstacles for automation. We evaluated the
distributions, across the data, of summary size, number of categories per
header, presence of the descriptor terms in their respective summary, and
category occurrence.

From the summary size distribution, we observed that the expected number of
sub-words by summary is less than the internal BERT dimensionality $d$ for our
three models, which confirms that this configuration is in the range where
BERT is expected to be more efficient than recurrent models [Devlin et al.
2019].

The categories per header distribution revealed an average of $5$ descriptor
terms per document, while the distribution of presence of the descriptor terms
in their respective summary revealed that most summaries contain a fraction of
their respective descriptor terms, but not all. Both of these results
characterize a scenario compatible with modeling the problem as a multi-label
classification task.

On the other hand, the category occurrence distribution revealed a total
number of categories (20,780) close to the number of documents in the dataset,
most of them with low occurrence. This is due to the high specificity of the
descriptor terms, that contain many concepts and sub-concepts within
themselves. The descriptor terms “ineficácia da adjudiação”, “ineficácia da
alienação” and “ineficácia da caução” are a good example. All of them
represent, at the same time, the concept of “ineficácia” and another sub-
concept, unique to each of these terms. Ideally, the concept and sub-concept
would be separated somehow.

This corroborates the problems pointed out by [Guimarães 2004], highlighted in
Section 3. To enable the treatment of the problem via multi-label
categorization, we performed an ontological adjustment step, in an attempt to
extract the concepts contained in each descriptor term and organize them in a
class system with reasonable size.

Before that, however, we also examined the correlation between the orders of
appearance of any two terms in the headers, expecting to detect any type of
underlying hierarchy between them that could be used in our ontological
adjustment. We observed a strong pairwise correlation, but not to the point of
building longer and more complex hierarchical lineages between the categories.
From this we concluded that segmenting the categories in their components and
then build a hierarchical structure from them, taking advantage of these co-
occurrences was the best way to adequate the class system.

During this ontological adjustment, category descriptor terms were tokenized
in its component words, stop words were removed using the
NLTK555https://www.nltk.org/ Portuguese stop words list, and then stemmed with
a manually enriched version of the RSLP Stemmer [Huyck and Orengo 2001]. We
removed the terms with less then 5 occurrences to eliminate spurious terms and
then assigned paternity relations between terms with high co-occurrence.

The terms in the top of the obtained hierarchy with low occurrence were
grouped under the “Others” category, while the ones with high occurrence were
automatically clustered in $25$ super-categories, each connected to the
component terms of the original categories by the hierarchical structure.

We clustered the categories using a K-means clusterizer applied to arrays with
identifiers of the documents in which each category appeared666This clustering
was performed as a pre-processing step. The classifier performs the
categorization on already clustered data and therefore has no access to the
arrays with identifiers of the documents in which each category appeared,
which could introduce a bias towards the actual labels. [Manning and Schütze
1999, p. 515-518]. These arrays had their dimension previously reduced to
$d=50$ using the scikit-learn Trucated SVD Algorithm777https://scikit-
learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html. We
tested different values of reduced dimension $d$ and different clustering
methods. Each combination resulted in different amounts of clusters, and the
chosen combination was the one whose clustering groups were the most cohesive.

Despite being justified by the properties of the data, the ontological
adjustment carried out here is extensive. For this reason, the performance
values obtained should be understood as the joint performance between the
categorization method and the adopted data reorganization methodology.

##  6 Experiments and Results

To evaluate and compare the performance of the models produced by verBERT and
our hierarchy of categories, we executed a series of experiments. We used
three pre-trained models, Multilingual BERT, BERTimbau base, and BERTimbau
large; and two different versions of the produced hierarchy and dataset:
dataset 1, produced with a smaller grouping rate and maintaining the “Others”
category, and dataset 2, a slightly more homogeneous option, produced with
higher grouping rate and excluding the “Others” category.

Each combination of model and dataset prompted a group of experiments, in
which we varied the maximum learning rate $\lceil\eta\rceil$, reached after
$50$ linear warming-up steps ($2\cdot 10^{-5}$, $4\cdot 10^{-5}$, $5\cdot
10^{-5}$, $1\cdot 10^{-4}$, $1\cdot 10^{-3}$, $5\cdot 10^{-4}$, $5\cdot
10^{-3}$, and $1\cdot 10^{-2}$)888We used learning rate values based on the
ones used in [Devlin et al. 2019] and [Souza et al. 2019], and included larger
values to get a more general picture of the performance evolution., the
maximum input sentence size $|\mathcal{S}|$ ($52$, $68$, $131$, and $200$
tokens) and the categorization threshold probability $P_{ct}$, above which a
category is assigned to a document by the model ($0.25$, $0.50$ and $0.75$).

We divided each dataset in a training set ($72\%$), a validation set ($8\%$)
and a test set ($20\%$). In each experiment the model was fine-tuned with the
training set for $10$ epochs and evaluated at regular intervals against the
validation set. The intermediate model with the best performance on the
validation set was then evaluated on the test set.

For that, we used two machines: (i) one with $48$ cores, $378$ GB of primary
memory and a NVIDIA Tesla K40c GPU ($11.4$ GB), and the other (ii) with $40$
cores, $62.8$ GB of primary memory, and a NVIDIA GeForce GTX1080Ti GPU ($11.7$
GB). Since both machines had secondary memory limitations, verBERT was planned
to use little storage as possible.

Comparing the results of the experiments, we observe an improvement in
performance with higher values of maximum learning rate $\lceil\eta\rceil$ and
input sentence size $|\mathcal{S}|$.The increase in performance with input
sentence size is observed for all values of $|\mathcal{S}|$, but the increase
rate decreases progressively, indicating saturation of this trend. On the
other hand, the growth in performance with maximum learning rate
$\lceil\eta\rceil$ is observed only up to $\lceil\eta\rceil=5\cdot 10^{-4}$,
from which point on, performance rates fall sharply. Our investigations
suggest that this is due to a local minimum, where the optimizer got stuck.

We also observed that the categorization threshold probability dictates the
optimization strategy adopted by the algorithm: with higher values of
$P_{ct}$, it is easier to maximize precision first and then slightly decrease
it to increase recall and F1-score. In contrast, with smaller threshold
probability values, it is easier to maximize recall first and then decrease it
optimally, to enhance precision and F1-score.

To provide a global view, Table 1 presents the main performance metrics in the
best case for each group of experiments, and the parameters for which these
results were obtained.

Table 1: Best results for the main metrics and respective parameters from each group of experiments on the test set. $\downarrow$Dataset/Model$\rightarrow$ | BERTimbau LARGE
---|---
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.71$ | $\dot{\mathcal{A}}=0.24$
Dataset 1 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.66$ | $(P_{limiar}=0.25;\lceil\eta\rceil=1\cdot 10^{-4};|\mathcal{S}|=200)$
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.72$ | $\dot{\mathcal{A}}=0.38$
Dataset 2 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.71$ | $(P_{limiar}=0.50;\lceil\eta\rceil=1\cdot 10^{-4};|\mathcal{S}|=131)$
$\downarrow$Dataset/Model$\rightarrow$ | Multilingual BASE
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.69$ | $\dot{\mathcal{A}}=0.24$
Dataset 1 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.59$ | $(P_{limiar}=0.50;\lceil\eta\rceil=1\cdot 10^{-4};|\mathcal{S}|=200)$
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.71$ | $\dot{\mathcal{A}}=0.37$
Dataset 2 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.70$ | $(P_{limiar}=0.50;\lceil\eta\rceil=5\cdot 10^{-5};|\mathcal{S}|=200)$
$\downarrow$Dataset/Model$\rightarrow$ | BERTimbau BASE
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.70$ | $\dot{\mathcal{A}}=0.24$
Dataset 1 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.59$ | $(P_{limiar}=0.50;\lceil\eta\rceil=1\cdot 10^{-4};|\mathcal{S}|=200)$
| $\langle\mathcal{F}_{1}\rangle_{micro}=0.72$ | $\dot{\mathcal{A}}=0.38$
Dataset 2 | $\langle\mathcal{F}_{1}\rangle_{macro}=0.70$ | $(P_{limiar}=0.25;\lceil\eta\rceil=4\cdot 10^{-5};|\mathcal{S}|=200)$

We observed better results on average for the large model, than for the two
base models, for both datasets. However, this improvement is low: only two
percent point increase in $\langle\mathcal{F}_{1}\rangle_{micro}$ for the best
case. This seems to indicate the superiority of the larger model and the
robustness of the smaller ones at the same time. This improvement is larger
for macro-F1 over the dataset 1. Since the dataset 1 is less homogeneous and
the macro-F1 is a metric more sensitive to the lack of homogeneity, this
suggests that the large model is better at dealing with less homogeneous
category systems.

Comparing the results for the BERTimbau models with the results for the
Multilingual model, we see that using Portuguese based models improves average
performance, but again, this improvement is low, what may indicate the
robustness of the Multilingual model.

Looking at the impact of data homogeneity over performance, we see that the
use of dataset 2 instead of dataset 1 brings an improvement to all models,
specially in the subset accuracy $\dot{\mathcal{A}}$, which indicates a
significant increase in our performance lower bound. Furthermore, we also see
an increase in the macro-averaged F1, which reveals that a small improvement
in homogeneity was enough to cause a great impact in the gap between the
micro-averaged and the macro-averaged F1 scores. These results should
nonetheless be interpreted with caution, given the difference between the test
sets of dataset 1 and dataset 2 . Also, with dataset 2 we obtained our best
model, the $verBERT\star$, obtained by fine-tuning BERTimbau large with
$P_{ct}=0.5$, $\lceil\eta\rceil=1\cdot 10^{-4}$ and $|\mathcal{S}|=131$. In
Table 2 we compare all the performance metrics obtained by $verBERT\star$ and
by our baseline, described in Section 4

Table 2: A comparison of the performance results obtained by the $verBERT\star$ model and by our statistical baseline, for all performance metrics. | $\langle\mathcal{P}\rangle_{micro}$ | $\langle\mathcal{P}\rangle_{macro}$ | $\langle\mathcal{P}\rangle_{inst}$ | $\langle\mathcal{R}\rangle_{micro}$ | $\langle\mathcal{R}\rangle_{macro}$ | $\langle\mathcal{R}\rangle_{inst}$ |
---|---|---|---|---|---|---|---
Baseline | 0.33 | 0.06 | 0.33 | 0.47 | 0.19 | 0.49 |
$verBERT\star$ | 0.77 | 0.77 | 0.79 | 0.67 | 0.66 | 0.73 |
| $\langle\mathcal{F}_{1}\rangle_{micro}$ | $\langle\mathcal{F}_{1}\rangle_{macro}$ | $\langle\mathcal{F}_{1}\rangle_{inst}$ | $\mathcal{A}$ | $\dot{\mathcal{A}}$ |  |
Baseline | 0.39 | 0.09 | 0.37 | 0.80 | 0.00 |  |
$verBERT\star$ | 0.72 | 0.71 | 0.73 | 0.95 | 0.38 |  |

##  7 Conclusions

The $verBERT\star$ model achieves results significantly superior to the
baseline for all metrics. Beyond that, the value obtained by $verBERT\star$
for our main metric, the micro-averaged F1 score, is close to the $0.732$
value obtained by [Chalkidis et al. 2019], and is higher then the $0.505$
value, obtained by [Villata et al. 2020] 999As mentioned in Section 2, these
two works are those whose methods, applications, and therefore results, are
most comparable to ours..

These results are quite positive given the difficulties in modeling the
problem, and indicate that our approach is promising. However, (i) a direct
comparison would be inappropriate, given the methodological differences
between our work and those cited, (ii) the observed performance is still far
from the optimal values obtained for other applications in NLP, and (iii) to
obtain such a level of performance it was necessary to carry out a heavy pre-
processing in which we observed several of the taxonomic problems of the
generation of headers, pointed out in the literature. This indicates that the
lack of standardization in headers is the main obstacle to achieving better
performance results.

Despite the usefulness of our models as categorization prototypes that could
already be used to assist human classifiers, we believe that, given these
results, better performance values could only be obtained (i) by exploring
models other than the categorization, which would compromise the taxonomic
value of the descriptor terms pointed by field specialists, or (ii) by
revisiting the production system of the headers in its core, in order to
manually generate a system of categories that is appropriate for the main
activity and that could be used in the automation process without so much pre-
processing . In addition, (iii) repetitions of our experiments could bring
greater statistical value to these results, mitigating the uncertainty brought
by the differences between the class systems of our two datasets, and (iv) the
exploration of more robust baselines based on other natural language
processing techniques could help us better understand the quality of the
performance obtained by our method. These are the main issues we would like to
address in future works.

## Acknowledgements

This work was carried out as a master’s research project [Serras 2021] at the
Center for Artificial Intelligence (C4AI-USP), with support by the São Paulo
Research Foundation (FAPESP grant #2019/07665-4) and by the IBM Corporation.
This study was financed in part by the Coordenação de Aperfeiçoamento de
Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001. We would like
to thank Dr. Sérgio Jacomino, his team from the Kollemata Project, Professors
Adriana Unger, Juliano Maranhão, Renata Vieira, Denis Mauá and the anonymous
reviwers for their help.

## References

  * [Aggarwal and Zhai 2012] Aggarwal, C. C. and Zhai, C. (2012).  A survey of text clustering algorithms.  In Mining text data, pages 77–128. Springer.
  * [Calambás et al. 2015] Calambás, M. A., Ordóñez, A., Chacón, A., and Ordoñez, H. (2015).  Judicial precedents search supported by natural language processing and clustering.  In 2015 10th Computing Colombian Conference (10CCC), pages 372–377. IEEE.
  * [Chalkidis et al. 2019] Chalkidis, I., Fergadiotis, M., Malakasiotis, P., and Androutsopoulos, I. (2019).  Large-scale multi-label text classification on eu legislation.  arXiv preprint arXiv:1906.02192.
  * [de Colla Furquim and De Lima 2012] de Colla Furquim, L. O. and De Lima, V. L. S. (2012).  Clustering and categorization of brazilian portuguese legal documents.  In International Conference on Computational Processing of the Portuguese Language, pages 272–283. Springer.
  * [Devlin et al. 2019] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019).  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * [Feinerer and Hornik 2008] Feinerer, I. and Hornik, K. (2008).  Text mining of supreme administrative court jurisdictions.  In Data Analysis, Machine Learning and Applications, pages 569–576. Springer.
  * [Godbole and Sarawagi 2004] Godbole, S. and Sarawagi, S. (2004).  Discriminative methods for multi-labeled classification.  In Pacific-Asia conference on knowledge discovery and data mining, pages 22–30. Springer.
  * [Gonçalves and Quaresma 2003] Gonçalves, T. and Quaresma, P. (2003).  A preliminary approach to the multilabel classification problem of portuguese juridical documents.  In Portuguese Conference on Artificial Intelligence, pages 435–444. Springer.
  * [Gonçalves and Quaresma 2005] Gonçalves, T. and Quaresma, P. (2005).  Is linguistic information relevant for the classification of legal texts?  In Proceedings of the 10th international conference on Artificial intelligence and law, pages 168–176.
  * [Goodfellow et al. 2016] Goodfellow, I., Bengio, Y., and Courville, A. (2016).  Deep Learning.  MIT Press.  http://www.deeplearningbook.org.
  * [Guimarães 2004] Guimarães, J. A. C. (2004).  ELABORAÇÃO DE EMENTAS JURISPRUDENCIAIS: elementos teórico-metodológicos, volume 9 of Série Monografias do CEJ.
  * [Han and Moraga 1995] Han, J. and Moraga, C. (1995).  The influence of the sigmoid function parameters on the speed of backpropagation learning.  In International workshop on artificial neural networks, pages 195–201. Springer.
  * [Huyck and Orengo 2001] Huyck, V. and Orengo, V. (2001).  A stemming algorithm for the portuguese language.  In SPIRE, volume 1, pages 186–193.
  * [Kalchbrenner et al. 2014] Kalchbrenner, N., Grefenstette, E., and Blunsom, P. (2014).  A convolutional neural network for modelling sentences.  In 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference.
  * [Koyejo et al. 2015] Koyejo, O., Natarajan, N., Ravikumar, P., and Dhillon, I. S. (2015).  Consistent multilabel classification.  In Advances in Neural Information Processing Systems.
  * [Leite et al. 2020] Leite, J. A., Silva, D. F., Bontcheva, K., and Scarton, C. (2020).  Toxic language detection in social media for brazilian portuguese: New dataset and multilingual analysis.  arXiv preprint arXiv:2010.04543.
  * [Loshchilov and Hutter 2017] Loshchilov, I. and Hutter, F. (2017).  Decoupled weight decay regularization.  arXiv preprint arXiv:1711.05101.
  * [Manning and Schütze 1999] Manning, C. and Schütze, H. (1999).  Foundations of Statistical Natural Language Processing.  Mit Press. MIT Press.
  * [Mencía and Fürnkranz 2010] Mencía, E. L. and Fürnkranz, J. (2010).  Efficient multilabel classification algorithms for large-scale problems in the legal domain.  In Semantic Processing of Legal Texts, pages 192–215. Springer.
  * [Moens 2001] Moens, M.-F. (2001).  Innovative techniques for legal text retrieval.  Artificial Intelligence and Law, 9(1):29–57.
  * [Salvatore et al. 2019] Salvatore, F., Finger, M., and Hirata Jr, R. (2019).  A logical-based corpus for cross-lingual evaluation.  arXiv preprint arXiv:1905.05704.
  * [Sebastiani 2002] Sebastiani, F. (2002).  Machine Learning in Automated Text Categorization.  ACM Computing Surveys, 34(1):1–47.
  * [Serras 2021] Serras, F. R. (2021).  Algoritmos baseados em atenção neural para a automação da classificação multirrótulo de acórdãos jurídicos.  Master’s thesis, Instituto de Matemática e Estatística (IME).
  * [Souza et al. 2019] Souza, F., Nogueira, R., and Lotufo, R. (2019).  Portuguese named entity recognition using bert-crf.  arXiv preprint arXiv:1909.10649.
  * [Souza et al. 2020] Souza, F., Nogueira, R., and Lotufo, R. (2020).  BERTimbau: pretrained BERT models for Brazilian Portuguese.  In 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23 (to appear).
  * [Trivedi 2019] Trivedi, K. (2019).  Multi-label text classification using bert – the mighty transformer.  https://medium.com/huggingface/multi-label-text-classification-using-bert-the-mighty
-transformer-69714fa3fb3d. Acessado em 02/03/2020.
  * [Villata et al. 2020] Villata, S. et al. (2020).  Natural language processing applications in case-law text publishing.  In Legal Knowledge and Information Systems: JURIX 2020: The Thirty-third Annual Conference, Brno, Czech Republic, December 9-11, 2020, volume 334, page 154. IOS Press.
  * [Wang et al. 2019] Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. (2019).  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In the Proceedings of ICLR.
  * [Wolf et al. 2019] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019).  Huggingface’s transformers: State-of-the-art natural language processing.  arXiv preprint arXiv:1910.03771.


$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-endpoint
on the boundary of $\mathcal{P}_{t}$. However, there needs to be an arc
$\gamma$ on the boundary of $\mathcal{P}_{t}$ that follows $\sigma$ in the
counter-clockwise order around the
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-endpoint
implying an existence of a map $R\to P_{(\gamma,c)}$ for some grading $c$, a
contradiction as $P_{(\gamma,c)}\in\tilde{\mathcal{L}}$. If $\sigma$ is a
closed arc, it is, up to homotopy, a loop that winds around $t$ a given number
of times as $\mathcal{P}_{t}$ is homotopy equivalent to a circle whose
fundamental group is $\mathbb{Z}$. Such a closed arc cannot be equipped with a
grading to yield an object of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ as
each time a loop winds around $t$ the grading decreases or increases by more
than $1$ (Definition 2.10 and Remark 2.11 in [18]):

$-1$$-1$$-?$$-?$$-1$$-1$ Figure 7. Changes in grading for graded curves in
$\mathcal{Q}_{t}$

This can be observed by taking the admissible dissection $\Delta$ of the
underlying marked surface that encodes the structure of $\Lambda$. The
$\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bullet$-point
puncture $t$ is enclosed in a connected component $\mathcal{Q}_{t}$ of the
complement of $\Delta$ in $S\backslash P$ by
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$
$\delta_{1},\dots,\delta_{n}\in\Delta$ that follow each other in the counter-
clockwise order around $t$. In the dual admissible $\Delta^{*}$ (cf.
Proposition 1.34), there is a
$\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bullet$-arc
$\delta^{*}_{i}$ for each $\delta_{i}$ such that crossing two following dual
$\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bullet$-point
arcs in the counter-clockwise order decreases the grading by 1 and vice versa
by Definition 1.39. A loop that winds around $t$ in $\mathcal{P}_{t}$ can be
assumed to lie in $\mathcal{Q}_{t}$ as well. So every time, this loop winds
around its grading would increase or decrease by $n\geq 1$; such a loop cannot
represent a one-dimensional band object as it is not possible to grade it
properly.

Lastly, $\sigma$ may be a
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-infinite
arc. By our discussion above, $\sigma$ needs to be contained in
$\mathcal{P}_{t}$, otherwise crossing a boundary
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arc
of $\mathcal{P}_{t}$ yielding a morphism from $\tilde{\mathcal{R}}$ to
$\tilde{\mathcal{L}}$. In this instance, however, it has have a
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-endpoint
on the boundary of $\mathcal{P}_{t}$ and a boundary
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arc
of $\mathcal{P}_{t}$ following it in the counter-clockwise order around the
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-enpoint,
which is also not permitted by our discussion above.

The other case, when the boundary of $\mathcal{P}_{t}$ is formed by
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arcs
in $\Gamma$ that give rise to objects in $\mathcal{R}$, is discussed dually. ∎

######  Theorem 3.17.

There is a one-to-one correspondence between semiorthogonal decompositions of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ and
good cuts of the marked surface associated to $\Lambda$.

###### Proof.

This a simple combination of Theorem 3.16 and Theorem 3.15. ∎

###  3.4. Semiorthogonal decompositions of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ with
multiple terms

We conclude this section by a discussion on how the results on
characterization of two-term semiorthogonal decompositons in Theorem 3.17
extends to semiorthogonal decomposition of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ with
more than two terms.

######  Definition 3.18.

Let $(S,M,P)$ be marked surface. A sequence of good cuts
$(\Omega_{1},\dots,\Omega_{n})$, is called a proper if the good cuts are pair-
wise different up to homotopy and if for every $1\leq i\leq n-1$:

  1. (i)

each $\omega\in\Omega_{i+1}$ that lies on a connected component of the
complement of $\Omega_{i}$ in $S\backslash P$ with right added marked points
is homotopic to a component of the boundary thereof;

  2. (ii)

each $\omega\in\Omega_{i}$ that lies on a connected component of the
complement of $\Omega_{i_{+}1}$ in $S\backslash P$ with left added marked
points is homotopic to a component of the boundary thereof.

######  Proposition 3.19.

Let $(S,M,P)$ be marked surface, and let $(\Omega_{1},\Omega_{2})$ be a proper
sequence of good cuts. Denote $\langle\mathcal{L}_{i},\mathcal{R}_{i}\rangle$
the semiorthogonal decomposition of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$
corresponding to $\Omega_{i}$ by Theorem 3.17, for $i=1,2$. Then,
$\mathcal{L}_{1}\subsetneq\mathcal{L}_{2}$.

###### Proof.

Consider a
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arc
$\gamma$ on $(S,M,P)$ whose corresponding objects in
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ lie in
$\mathcal{L}_{1}$ of the semiorthogonal decomposition
$\langle\mathcal{L}_{1},\mathcal{R}_{1}\rangle$ thereof. The
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arc
$\gamma$ is contained within a connected component $\mathcal{P}$ of the
complement of $\Omega_{1}$ in $S\backslash P$. By definition, the component
$\mathcal{P}$ would be with right added marked points. Pick
$\omega\in\Omega_{1}$ that lies on the boundary of $\mathcal{P}$ and a
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-arc
$\gamma^{\prime}$ such that: it begins in $g_{\omega}$, the
$\color[rgb]{0.0546875,0.58984375,0.0078125}\definecolor[named]{pgfstrokecolor}{rgb}{0.0546875,0.58984375,0.0078125}{\circ}$-endpoint
of $\omega$, and it is contained in $\mathcal{P}$. Therefore,
$\gamma^{\prime}$ has to precede $\omega$ in the counter-clockwise order
around $g_{\omega}$.

Because $(\Omega_{1},\Omega_{2})$ is proper, the connected component
$\mathcal{P}$ does not contain any $\omega\in\Omega_{2}$ not homotopic to a
segment of its boundary. For this reason, neither $\gamma$ nor
$\gamma^{\prime}$ cross an arc in $\Omega_{2}$, and the corresponding objects
of $\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$
need to both lie either in $\mathcal{L}_{\Omega_{2}}$ or in
$\mathcal{R}_{\Omega_{2}}$ by Lemma 3.7 combined with Proposition 3.13.

Furthermore, as $(\Omega_{1},\Omega_{2})$ is proper, we may assume that
without loss of generality $\omega$ lies within a connected component
$\mathcal{Q}$ of the complement of $\Omega_{2}$ in $S\backslash P$ with right
added marked points or on a boundary thereof. If $\omega$ lies on $Q$, then so
does $\gamma^{\prime}$, which does not cross any arc in $\Omega_{2}$ and
$\gamma^{\prime}$ precedes $\omega$ in the counter-clockwise order around
$g_{\omega}$. Hence, the objects of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$
corresponding to $\gamma^{\prime}$ lie in $\mathcal{L}_{\Omega_{2}}$; it is
also the case for the objects of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$
corresponding to $\gamma$, so $\mathcal{L}_{1}\subseteq\mathcal{L}_{2}$.

The fact that $\mathcal{L}_{1}\subsetneq\mathcal{L}_{2}$ follows from the
assumption that $\Omega_{1}$ and $\Omega_{2}$ are not equal up to homotopy.
Theorem 3.17 then assures that the corresponding semiorthogonal decompositions
differ as well. ∎

######  Theorem 3.20.

There is a one-to-one correspondence between semiorthogonal decompositions of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ with
arbitrarily many terms and proper sequences of good cuts of the marked surface
associated to $\Lambda$.

###### Proof.

At first, we note that it is a straight-forward consequence of Proposition 1.3
that a semiorthogonal decomposition
$\langle\mathcal{C}_{1},\dots,\mathcal{C}_{n}\rangle$ of $\mathcal{T}$
uniquely corresponds to a sequence of two-term semirothogonal decompositions
$\langle\mathcal{L}_{i},\mathcal{R}_{i}\rangle$ of $\mathcal{T}$, for $1\leq
i\leq n-1$, such that $\mathcal{L}_{i}$ is the full triangulated subcategory
of $\mathcal{T}$ generated by $\mathcal{C}_{1},\dots,\mathcal{C}_{i}$ and
$\mathcal{R}_{i}$ is the full triangulated subcategory of $\mathcal{T}$
generated by $\mathcal{C}_{i+1},\dots,\mathcal{C}_{n}$, and vice versa.

Combining Theorem 3.17 with Proposition 3.19, we obtain that there is a one-
to-one correspondence with proper sequence of good cuts of the marked surface
$(S,M,P)$ associated to $\Lambda$ and a sequence
$\langle\mathcal{L}_{i},\mathcal{R}_{i}\rangle$ of semiorthogonal
decompositions of
$\operatorname{\mathsf{D}^{\mathsf{b}}}(\Lambda\textrm{-}\mathrm{mod})$ with
$\mathcal{L}_{i}\subsetneq\mathcal{L}_{i+1}$ for each $i$ smaller than the
length of the sequence. This correspondence is carried forward to a one-to-one
correspondence between proper sequences of good cuts by our observation above.
∎

## References

  * [1] Claire Amiot, Pierre-Guy Plamondon, and Sibylle Schroll.  A complete derived invariant for gentle algebras via winding numbers and arf invariants.  arXiv preprint arXiv:1904.02555, 2019.
  * [2] Kristin Krogh Arnesen, Rosanna Laking, and David Pauksztello.  Morphisms between indecomposable complexes in the bounded derived category of a gentle algebra.  Journal of Algebra, 467:1–46, 2016.
  * [3] Ibrahim Assem and Dieter Happel.  Generalized tilted algebras of type an.  Communications in Algebra, 9(20):2101–2125, 1981.
  * [4] Ibrahim Assem and Andrzej Skowronski.  Iterated tilted algebras of type an.  Math. Z, 195(2):269–290, 1987.
  * [5] Karin Baur and Raquel Coelho Simões.  A geometric model for the module category of a gentle algebra.  Int. Math. Res. Not. IMRN, (15):11357–11392, 2021.
  * [6] A. A. Beilinson, J. Bernstein, and P. Deligne.  Faisceaux pervers, Analyse et Topologie sur les Espaces Singulier (i).  Asterisque, 100, 1982.
  * [7] Viktor Bekkert and Héctor A. Merklen.  Indecomposables in derived categories of gentle algebras.  Algebras and Representation Theory, 6(3):285–302, 2003.
  * [8] Alexei Igorevich Bondal.  Representation of associative algebras and coherent sheaves.  Mathematics of the USSR-Izvestiya, 34(1):23, 1990.
  * [9] İlke Çanakçı, David Pauksztello, and Sibylle Schroll.  Mapping cones in the bounded derived category of a gentle algebra.  Journal of Algebra, 530:163–194, 2019.
  * [10] İlke Çanakçı, David Pauksztello, and Sibylle Schroll.  Corrigendum to “Mapping cones for morphisms involving a band complex in the bounded derived category of a gentle algebra” [J. Algebra 530 (2019) 163–194].  Journal of Algebra, 569:856–874, 2021.
  * [11] Wen Chang, Haibo Jin, and Sibylle Schroll.  Recollements of derived categories of graded gentle algebras and surface cuts.  arXiv preprint arXiv:2206.11196, 2022.
  * [12] Wen Chang and Sibylle Schroll.  A geometric realization of silting theory for gentle algebras.  arXiv preprint arXiv:2012.12663, 2020.
  * [13] Fabian Haiden, Ludmil Katzarkov, and Maxim Kontsevich.  Flat surfaces and stability structures.  Publ. Math. Inst. Hautes Études Sci., 126:247–318, 2017.
  * [14] Thorsten Holm, Peter Jørgensen, and Raphaël Rouquier.  Triangulated categories, volume (London Mathematical Society Lecture Note Series) 375.  Cambridge University Press, 2010.
  * [15] Mark Hovey, John H. Palmieri, and Neil P. Strickland.  Axiomatic stable homotopy theory.  Mem. Amer. Math. Soc., 128(610):x+114, 1997.
  * [16] Alexander Kuznetsov.  Semiorthogonal decompositions in algebraic geometry.  In Proceedings of the International Congress of Mathematicians—Seoul 2014. Vol. II, pages 635–660. Kyung Moon Sa, Seoul, 2014.
  * [17] Sebastian Opper.  On auto-equivalences and complete derived invariants of gentle algebras.  arXiv preprint arXiv:1904.04859, 2019.
  * [18] Sebastian Opper, Pierre-Guy Plamondon, and Sibylle Schroll.  A geometric model for the derived category of gentle algebras.  arXiv preprint arXiv:1801.09659, 2018.
  * [19] Michel Van den Bergh.  Abstract blowing down.  Proceedings of the American Mathematical Society, 128(2):375–381, 2000.