We present a quantitative study of the Annals of the Joseon Dynasty, the daily written records of the five hundred years of a monarchy in Korea. We first introduce the corpus, which is a series of books describing the historical facts during the Joseon dynasty. We then define three categories of the monarchial ruling styles based on the written records and compare the twentyfive kings in the monarchy. Finally, we investigate how kings show different ruling styles for various topics within the corpus. Through this study, we introduce a very unique corpus of monarchial records that span an entire monarchy of five hundred years and illustrate how text mining can be applied to answer important historical questions.
Introduction Historical documents are usually studied qualitatively by researchers focusing on a close reading of a small number of documents. However, for a large corpus of historical texts, qualitative methods have limitations, thus quantitative approaches have been introduced recently (Moretti, 2005; Jockers, 2013) . There is also research in applying text mining and natural language processing methods to identify patterns in a corpus of large and longitudinal documents (Mimno, 2012) . In this paper, we introduce a unique corpus of historical documents from the written records that span almost five hundred years from the fourteenth century up to the late nineteenth century within the Korean peninsula. We apply text mining to this corpus to show the power of a computational approach in answering historical questions. We first introduce The Annals of the Joseon Dynasty (Chunchugwan, 1863) . Joseon is the last monarchial nation in the Korean Peninsula from its founding in 1392 up to 1910. The Annals of the Joseon Dynasty are a series of books of historical facts, recorded almost daily during the Joseon dynasty. Whenever a king abdicated the throne, the Chunchugwan (office for annals compilation) updated the Annals for that king from all related official and unofficial documents. The Annals contain political, economic, social and cultural topics during the corresponding time periods. To illustrate the application of a text mining approach, we analyze each king's ruling style from the Annals of the Joseon dynasty. Being a monarchial system, almost all decisions within the government are confirmed by the king, where the king can make the decision on his own, or after discussing it with the government officials. We identify the patterns of each king's decision making and compare the patterns among the kings. The results show interesting patterns of the kings' ruling styles, including the tendency to make arbitrary decisions of the kings who were later dethroned because of tyranny. Additionally, we apply a topic model to the corpus and analyze the kings' ruling style for each topic. The Annals of the Joseon Dynasty In this section, we describe the details of The Annals of the Joseon Dynasty (from here referred to as the AJD) (Chunchugwan, 1863) and our process for building a corpus of the AJD. In its entirety, the AJD consists of records from twentyseven kings over 519 years. However, the last two kings' (Gojong, Sunjong) books are usually excluded from research by historians because many facts are distorted. We follow that convention and use the books of the first twenty-five kings. These records, in their original Chinese text and in the Korean translations, are available publicly through   Joseon was a monarchy, but a king could not make all decisions by himself. Instead, Joseon adopted a government system that most of the public issues are discussed with the government officials (Park, 1983; Kim, 2008) before the king made the decisions, which are all recorded in the AJD. Hence, by analyzing the decision making process in the AJD, we can understand each king's ruling style. Categorizing ruling style In Joseon dynasty, the king was the final decision maker. Even when the government officials discussed the public issues, a king's approval was needed. We can categorize each king's decision making process into three types. First, a king can order directly without discussion, which we call Arbitrary Decision (AD). Second, a king can discuss an issue with the officials and then direct his   order, which we call Discussion and Order (DO). Third, a king can discuss an issue with the officials and then decide to follow the officials' suggestion, which we call Discussion and Follow (DF). The difference between DO and DF is that in DO, the king acts aggressively with his own opinion. From these observations, we ask two research questions: 1) Can we identify and categorize kings with different ruling styles? 2) Do kings' ruling styles differ depending on the topic? Method To understand each king's ruling style, we first identify relevant articles that contain the king's decision making because many of the articles describe non-governmental affairs, such as the weather, or simple status reports. These relevant articles contain direct quotations of the words of the king or the government official. The original texts do not contain any quotation marks, but translators added them to distinguish explicit quotations, which we can use to identify these relevant articles. Its size is 126K, 36% over all articles. Each article contains who said what for an issue, and king's final actions are written mostly in the last part of the article. For example, the underlined last sentence in Figure 1b says that the king followed the official's suggestion. Hence, to identify king's action for each issue, we focus on the last sentence in each article. First, we identify that the setence subject is the king, because some issues are dealt by others. For example, Sunjo, Heonjong and Cheoljong's mother or grandmother ruled as regent, so her decisions are recorded in the AJD. To identify the part of speech in Korean, we used HanNanum (Choi et al., 2012) . And, we investigate the verbs that indicating decisions including order, follow, approval and reject. We use sixty verbs that describe king's decision specifically, and table 2 shows example words. Finally, we classify these decisions into three types: 1) the king orders without discussions with the officials, and we label them as AD, 2) the king orders, approves, or rejects verbs in which their original Chinese characters show active decision making by the king, and we label them as DO, and 3) the king follows or discusses verbs which show passive submission by the king, and we label them as DF. To identify topics, we use a Bayesian topic model, LDA (Blei et al., 2003) . We implement it using Gibbs sampling (Griffiths and Steyvers, 2004) , set 300 topics, and optimized hyperparameters after 100 iterations (Asuncion et al., 2009) . We remove stopwords and words with document frequency of 30 or smaller. Results and Discussions We investigate the difference of ruling style between kings. We run multinomial test (Read and Cressie, 1988 ) between king's ruling style distributions. Result shows that almost all kings are different significant from others (p < 0.001). It means that each king has his own ruling style. Figure 2 shows the distribution of each category of ruling style. Overall, many kings do not act arbitrary. They discuss about many of the national affairs with officials. But, Taejo who is the   founder of the Joseon dynasty shows high value of AD. And Yeonsangun and Gwanghaegun who are evaluated as a tyrant also show high value of it. So we can imagine that tyrants tend to act arbitrarily. We also identified those kings whose ruling style differed most from other kings. We use JS divergence which is the symmetric measure of the difference between two probability distributions. We compute JS divergence with each king pair's ruling style distributions. Result shows that Heonjong (0.1220) and Yeonsangun (0.0998) have highest distance value. It means their ruling style are quite different from other kings. Because Heonjong's grandmother governed the Joseon each year, so his actions are quite few. But, unlike Yeonsangun, Gwanghaegun (0.0454) who is known as a tyrant has similar value mean distance from other kings (0.0434). It means his ruling style is quite similar to other kings, and this result supports previous results in Korean historical study (Kye, 2008 ) that re-evaluate his reputation. We investigate the difference of king's ruling style based on the topic. We run multinomial test (Read and Cressie, 1988 ) between king's overall ruling style distribution and specific distribution given a topic. Results show that some ruling styles given a topic are different significant from overall (p < 0.01). It means that the king's ruling style when the topic is given is different from his usual style. Table 3 shows examples of topic. Figure 3 shows four kings' overall ruling style and specific one given a topic. Comparing with the leftmost bars which is overall ruling style of the king, each ruling style given a topic is different from it. And, we can see that kings show similar/different ruling style for a topic. For example, kings tend to discuss and order (DO) to officials for retirement and remission topics. And, Sejong the Great and Gwanghaegun discuss and follow (DF) officials' words for agricultural topic. But, for grants topic, Yeonsangun and Gwanghaegun act more arbitrarily (AD) than overall ruling style, and Sejong the Great follows more official's opinions (DF). Conclusion and Future Work We introduced long and large historical documents, The Annals of the Joseon Dynasty. It contains lots of topics such as political, economic, social and cultural over 500 years. We looked at the ruling style of kings in Joseon dynasty and its difference by topics by computational methods. This is ongoing work, and we are looking to find more hidden structures in the AJD. Currently, historians evaluate the king's reputations (Park, 2004; Lee, 2010) . This evaluation is done by many aspects, but one of the important feature is king's ruling style (Kim, 2008) . So we are looking to improve methods for analyzing ruling style more specifically. For example, we will look at the relationship with officials, especially who can make the king follows his opinion. This approach can be used to measure king's leadership.
1 2 3.1 3.2 3.3 4
Introduction The Annals of the Joseon Dynasty Categorizing ruling style Method Results and Discussions Conclusion and Future Work
Results Members
Investigate the effects of the participants in a discussion
Compute the mutual information among ruling styles
Agency officials who remonstrate to the king
Investigate the effects of the participants in a discussion
Compute the mutual information among ruling styles
Agency officials who remonstrate to the king