\begin{table}[h]
\centering
\begin{tabular}{lr}
\hline
& Validation\\
& accuracy  \\ 
\hline
Greek BERT  & \textbf{80\%}                                              \\
Greek BERT no MLM-transfer  & 78\%                                              \\
Multilingual  BERT                 & 78\%                                                  \\ 
Naive Bayes Classifier &   43\%                                                 \\ 
Random authorship attribution &   6\%                                                 \\ 
\hline
\end{tabular}
\caption{After MLM training and ten epoch of fine-tuning for authorship attribution, the validation accuracy of  Modern Greek BERT  is slightly higher than that of the Multilingual BERT after similar fine-tuning procedures. Modern Greek BERT fine-tuned for authorship attribution without MLM transfer learning phase shows lower validation accuracy. All BERT-based classifiers significantly outperform the Naive Bayes Classifier that uses the two thousand most frequent unigrams. Another baseline attributes one of seventeen labels to the text at random.}
\label{tab:au}
\end{table}
Transformer artificial neural networks have shown spectacular success in machine translation and answering search queries by internally extracting, after extensive pretraining, abstract patterns, and long-range dependencies in human language samples through encoder hierarchies \cite{vaswani2017attention}. Therefore, it seems natural to apply such schemes to other tasks that traditionally depended on human language understanding. Thus, this paper uses versions of BERT to investigate questions of authorship in Ancient Greek literature. Such models have already been used for authorship attribution, but the
actual number of cases is still limited. \citet{fabien2020bertaa} demonstrate that fine-tuning of a pretrained BERT  language model with an additional dense layer and a softmax activation allows performing authorship classification. \citet{polignano2020contextualized} use BERT for author profiling in social media and conclude that despite encouraging results in terms of reliability, the computational power required for running such a model is too demanding for the task. These results show that transformers could be successfully used for authorship attribution, yet the application of these models to historical texts is still limited. Specifically, \citet{bamman2020latin} develop BERT for Latin and \cite{assael2019restoring} train an LSTM language model of Ancient Greek. There is also a char-BERT implementation\footnote{https://github.com/brennannicholson/ancient-greek-char-bert}, but we do not know of any public full BERT model trained to work with Ancient Greek. We are also unaware of any examples when BERT was used successfully for authorship attribution of historical documents. 

Since each model has its own specific tokenizer before pretraining, one wants to measure how well each of them works with Ancient Greek. To do that, one could take two sample data sets: a sample of Modern Greek
Wikipedia (referred to in Table \ref{tab:tok} as “modern”) and a comparable sample of Ancient Greek (“ancient”  in Table \ref{tab:tok}). Each data sample is tokenized with both the Modern Greek BERT tokenizer\footnote{https://huggingface.co/nlpaueb/bert-base-greek-uncased-v} and the Multilingual BERT tokenizer\footnote{https://huggingface.co/bert-base-multilingual-cased}. One can speculate that the model uses shorter tokens to adopt grammatical information and deal with longer, rarely observed words. In contrast, the representations with longer tokens could be useful for semantically intensive problems. These longer, semantically charged tokens may vary significantly on various downstream tasks. Thus, the average length of a token and the average number of words per token, shown in Table \ref{tab:tok}, could be coarsely used as an estimate of the resulting tokenization. One could claim that the higher these values, the more apt the tokenizer is for the task. Indeed, higher average length of a token and number of words per token mean that longer, more semantically charged tokens could be matched for transfer; see \cite{singh2019bert,aji2020neural,samenko2021fine} for a detailed discussion of various tokenization properties.

We use this data set to train BERT Classifiers similarly to \cite{fabien2020bertaa}. Table \ref{tab:au} shows the validation accuracy for Modern Greek and Multilingual BERTs after MLM on Ancient Greek and 10 epochs of classifier training\footnote{AdamW, LR = 2e-5, eps = 1e-8,  linear\_schedule}. Table \ref{tab:au}  also shows a standard NLTK Naive Bayes Classifier trained on the 2 000 most frequent unigrams as a reference point for authorship attribution accuracy.

Since Ancient Greek was used in various regions and territories, one might expect that the texts also show regional peculiarities. Such peculiarities
then should also be detectable by a dedicated regional BERT Classifier. We constructed three coarse regions for the origins of the authors in the data set: a region surrounding Delphi, where Plutarch was working, a region in the proximity of Alexandria, and the region of ancient Ionia, namely the ancient region on the central part of the western coast of modern Anatolia, see Figure \ref{fig:map}. After balancing texts written by authors in these three regions with the fourth label that includes random sentences from authors outside of these regions, we train another BERT-based classifier to achieve a $0.79$ validation accuracy for the region of the author. Table \ref{tab:region} shows the results of the obtained classifier on the validation set.

Table \ref{tab:p}  demonstrates that the resulting authorship profiles differ for all three documents. However, the sample size for De Fluviis and De Musica is smaller than the sample size for Placita Philosophorum. Plutarch ends up being in the top three of the most frequently predicted authors only in De Musica. For De Fluviis and De Musica Athenaeus ends up being the most frequent guess of the BERT authorship classifier, while for Placita Philosophorum "other" is the most frequent attribution. Claudius Ptolemaeus and Sextus Empiricus are, respectively, the second and the third most frequent guesses. 

Figure \ref{fig:reg} shows the regional profile obtained with the BERT-based regional classifier. Once again, all three works show different structural properties. While the model associates every second sentence from De Fluviis and De Musica with the Delphi region, it only attributes $15\%$ of the sentences from Placita Philosophorum to Delphi. The model is far more uncertain about the third document. Every fourth sentence in Placita Philosophorum is associated with the Alexandrian region, while almost half is labeled as "other". 

All in all, the result shows that the Placita Philosophorum is closely related to a philosophical-scientific tradition from the 1st century CE to ca. 220/250 CE. Possibly one could even see an embedding into a specifically Alexandrian context, which despite very different contents of the works (Strabo as a geographer, Ptolemy as mathematician and geographer, Athenaeus as an anthologist, and Sextus as a Skeptic), is related to the Pseudo-Plutarchian Placita. The parallels between Pseudo-Plutarch and Sextus Empiricus have already been pointed out several times in the literature \cite{mansfeld2020}; connections to Claudius Ptolemaeus have not been considered so far. It is fascinating that in this group of three (Pseudo-Plutarch, Claudius Ptolemaeus, and Sextus
Empiricus), a mathematical-cosmologically oriented context for very different topics of philosophy, natural science, and medicine, is the basis. We can thus state a new hypothesis that the Placita Philosophorum presumably originates from an Alexandrian scientific context of the 2nd century CE. We also should notice that De Fluviis and De Musica, and Placita Philosophorum might have three different authors. There are some similarities between De Fluviis and De Musica (both have approximately 50\% of the text attributed to the Delphi region and approximately 20\% attributed to Athenaeus). At the same time, one should be cautious in the conclusions since Placita Philosophorum is longer and have a different topical structure than the other two works. This might explain the lower percentage of Delphi-attributed sentences in the document. Athenaeus was allegedly born on the territory of modern Egypt, as well as Claudius Ptolemaeus. 

Sextus Empiricus lived in different places, however, like the authors of De Fluviis and
De Musica, he seems to have been more influenced by the Alexandrian tradition than has been seen so far. Furthermore, the results  shown in Table  \ref{tab:au}, and in figure \ref{fig:reg}  indicate that the three works analyzed here (De Fluviis, De Musica, Placita Philosophorum) cannot be securely linked to one identifiable author. Since all three works have a strongly compilatory character due to the many  quotations, references and summaries, this result is quite plausible and confirms the current scholarly opinion against an attribution to Plutarch.
{"value": "80", "type": "Result", "training data/set": ["acient greek", "authorship attribution data set"], "test data/set": ["acient greek", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc"], "experimental settings": {"xx": "yy"}, "model": "Greek BERT", "model settings": {"xx": "yy"}} {"value": "78", "type": "Result", "training data/set": ["acient greek", "authorship attribution data set"], "test data/set": ["acient greek", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc"], "experimental settings": {"xx": "yy"}, "model": "Greek BERT no MLM-transfer", "model settings": {"xx": "yy"}} {"value": "78", "type": "Result", "training data/set": ["acient greek", "authorship attribution data set"], "test data/set": ["acient greek", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc"], "experimental settings": {"xx": "yy"}, "model": "Multilingual BERT", "model settings": {"xx": "yy"}} {"value": "43", "type": "Result", "training data/set": ["acient greek", "authorship attribution data set"], "test data/set": ["acient greek", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc"], "experimental settings": {"xx": "yy"}, "model": "Naive Bayes Classifier", "model settings": {"xx": "yy"}} {"value": "6", "type": "Result", "training data/set": ["acient greek", "authorship attribution data set"], "test data/set": ["acient greek", "authorship attribution data set"], "task": "authorship attribution", "metric": ["Accuracy", "Acc"], "experimental settings": {"xx": "yy"}, "model": "Random authorship attribution", "model settings": {"xx": "yy"}}