\begin{table}[h!]
\centering
\begin{tabular}{lll}
\hline
                             & Validation Acc. \\ \hline
\textbf{Greek BERT with MLM}    &\textbf{83.8\%}  \\
Greek BERT without MLM & 83.4\%  \\
\textbf{M-BERT with MLM}       & \textbf{83.8\%}  \\
M-BERT without MLM     & 81.6\%  \\
Naive Bayes            & 75.8\%   \\
Random Author Assignment  & 11.2\%   \\\hline
\end{tabular}
\caption{Accuracy of the authorship classifier on the test set for the document-balanced splitting.}
\label{appd:allmodels}
\end{table}
\begin{itemize}
    \item we use a transfer learning approach to train BERT for Ancient Greek;
    \item we demonstrate that this language model is useful for authorship attribution of Ancient Greek texts;
    \item we obtain results that may be used as evidence in the process of authorship attribution of the Pseudo-Plutarchean texts;
    \item we obtain new insights into the paths along which the reception of ancient philosophy was developed.
\end{itemize}

\subsection{Tokenizers}
The tokenization of words into sub-word tokens is a crucial preprocessing step that can affect the performance of the model. Up to this point, we were using "words" as a linguistic term, whereas we understand tokens as the output of a tokenization algorithm. Thus, there would be one or more tokens that represent every word. Counting the number of tokens used on average to represent a word or, conversely, an average number of words per token gives an estimate of how fit  the tokenization is for the data set. In particular, the average number of words per token varies from 0 to 1, and the closer it is to 1, the more words are represented with one token. Various researchers have shown that corpus-specific tokenization could be beneficial for an NLP task. For example, \citet{sennrich-etal-2016-neural} show that optimal vocabulary is dependent on the frequencies of the words in the target corpus. \citet{lakew2019controlling} and \citet{aji2020neural} partially discuss the tokenization in the setting of cross-language transfer. Though \citet{aji2020neural} demonstrate that there is no clear evidence that one parent is better than another for cross-lingual transfer learning, they also show that token matching and joint vocabulary \cite{nguyen2017transfer} are the best ways to handle the embedding layer during transfer learning. 

Since each model has its own specific tokenizer before pretraining, one wants to measure how well each of them works with Ancient Greek. To do that, one could take two sample data sets: a sample of Modern Greek
Wikipedia (referred to in Table \ref{tab:tok} as “modern”) and a comparable sample of Ancient Greek (“ancient”  in Table \ref{tab:tok}). Each data sample is tokenized with both the Modern Greek BERT tokenizer\footnote{https://huggingface.co/nlpaueb/bert-base-greek-uncased-v} and the Multilingual BERT tokenizer\footnote{https://huggingface.co/bert-base-multilingual-cased}. One can speculate that the model uses shorter tokens to adopt grammatical information and deal with longer, rarely observed words. In contrast, the representations with longer tokens could be useful for semantically intensive problems. These longer, semantically charged tokens may vary significantly on various downstream tasks. Thus, the average length of a token and the average number of words per token, shown in Table \ref{tab:tok}, could be coarsely used as an estimate of the resulting tokenization. One could claim that the higher these values, the more apt the tokenizer is for the task. Indeed, higher average length of a token and number of words per token mean that longer, more semantically charged tokens could be matched for transfer; see \cite{singh2019bert,aji2020neural,samenko2021fine} for a detailed discussion of various tokenization properties.

We use this data set to train BERT Classifiers similarly to \cite{fabien2020bertaa}. Table \ref{tab:au} shows the validation accuracy for Modern Greek and Multilingual BERTs after MLM on Ancient Greek and 10 epochs of classifier training\footnote{AdamW, LR = 2e-5, eps = 1e-8,  linear\_schedule}. Table \ref{tab:au}  also shows a standard NLTK Naive Bayes Classifier trained on the 2 000 most frequent unigrams as a reference point for authorship attribution accuracy.

Since Ancient Greek was used in various regions and territories, one might expect that the texts also show regional peculiarities. Such peculiarities
then should also be detectable by a dedicated regional BERT Classifier. We constructed three coarse regions for the origins of the authors in the data set: a region surrounding Delphi, where Plutarch was working, a region in the proximity of Alexandria, and the region of ancient Ionia, namely the ancient region on the central part of the western coast of modern Anatolia, see Figure \ref{fig:map}. After balancing texts written by authors in these three regions with the fourth label that includes random sentences from authors outside of these regions, we train another BERT-based classifier to achieve a $0.79$ validation accuracy for the region of the author. Table \ref{tab:region} shows the results of the obtained classifier on the validation set.

Let us now split these three Pseudo-Plutarchean texts into the separate sentences and apply the author classifier described above to these texts.
Table \ref{tab:p} shows the authors that are most frequently attributed within a particular document, along with the share of sentences attributed to them. We have double-checked these results using an alternative scoring method. Instead of classifying every sentence in a document and then averaging the classifier's results across all sentences, one could obtain probability scores for every author that the model estimates for every sentence. Averaging those probabilities throughout the document, one could obtain the three most-probable author candidates. These three most-probable authors turn out to be exactly the same for all three documents as the ones in Table \ref{tab:p}. Moreover, the resulting probabilities of the authorship are also the same for all three most probable authors across all three documents under examination. 

Since we are interested in a particular historical time-span around Plutarch, we did not provide details on other potential applications for the resulting model. However, one could list several further applications that might interest historians and, to our knowledge, are not developed to this day. Stylistic attributes of text include author-specific attributes (see \cite{xu} or \cite{Jhamtani} on 'shakespearization'), politeness \cite{Sennrich}, gender or political slant \cite{Prabhumoye}, formality of speech \cite{Rao} but most importantly for the scope of this work the \textit{'style of the time'} \cite{Hughes}. Using the provided approach for the historical dating of the documents is a feasible option that might bring new historical insights. It is only the question of data and their quality. For example, one could try to use a corpus spanning several hundreds of years, fine-tune the developed model for the task of date attribution and see if it could work reasonably well. This is a possible further line of work to pursue, yet it is outside of the scope of this contribution.

We have studied two different ways of data splitting to address possible questions on the validity of the proposed machine learning pipeline. The first preprocessing described above splits the texts into sentences and then randomly splits them into training, validation, and testing. This splitting might create some dependencies in the evaluation sets since sentences from the same text could be in training, validation, and testing sets. This potentially could lead to higher evaluation scores, so here we briefly discuss another way of splitting data that could not be prone to such data' leak'. Let us refine our data set leaving only the authors that have produced several documents. Let's do the test-train split so that whole documents end up in the train or the test set. This split limits the list of authors even further, yet it is not a problem in this context. Table \ref{appd:allmodels} summarizes the result on the test set.
{"value": "83.8", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "Greek BERT with MLM", "model settings": {"xx": "yy"}} {"value": "83.4", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "Greek BERT without MLM", "model settings": {"xx": "yy"}} {"value": "83.8", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "M-BERT with MLM", "model settings": {"xx": "yy"}} {"value": "81.6", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "M-BERT without MLM", "model settings": {"xx": "yy"}} {"value": "75.8", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "Naive Bayes", "model settings": {"xx": "yy"}} {"value": "11.2", "type": "Result", "training data/set": ["xx", "authorship attribution data set"], "test data/set": ["xx", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc."], "experimental settings": {"xx": "yy"}, "model": "Random Author Assignment", "model settings": {"xx": "yy"}}