SentText: A Tool for Lexicon-Based Sentiment Analysis in Digital Humanities
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Schmidt, Thomas
Dangel, Johanna
Wolff, Christian
We present SentText, a web-based tool to perform and explore lexicon-based sentiment analysis on texts, specifically developed for the Digital Humanities (DH) community. The tool was developed integrating ideas of the user centered design process and we gathered requirements via semi-structured interviews. The tool offers the functionality to perform sentiment analysis with predefined sentiment lexicons or self-adjusted lexicons. Users can explore results of sentiment analysis via various visualizations like bar or pie charts and word clouds. It is also possible to analyze and compare collections of documents. Furthermore, we have added a close reading function enabling researchers to examine the applicability of sentiment lexicons for specific text sorts. We report upon the first usability tests with positive results. We argue that the tool is beneficial to explore lexicon-based sentiment analysis in the DH but can also be integrated in DH-teaching.
 I hope everyone sees my slide. I will gently switch to English as you hear. Do you see my slides? Okay. So, my name is Thomas Schmidt from the Media Informatics Group of the University of Regensburg and I will present Cent text at Tugel for Lexicon based sentiment analysis in digital humanities. As already was said, Johanna Dangell was the primary developer of this tool, Christian Wojty, the supervisor of diseases that's connected to this tool development. And yes, this is a theme, this is originally a demo slash short paper contribution. So I will talk a little bit about background and how we approach this tool development, but I will mainly focus on showing the tool and by using it. So I will start with a little bit of background about sentiment analysis. Here you see just a standard definition of sentiment analysis. Also called opinion mining, the field of study that analyzes people's opinions, sentiments, appraisals, attitudes and emotions, mostly in written text. Overall the idea of sentiment analysis is to try to predict the sentiment, express sentiment in a text unit, if it's rather positive or rather negative, if it's rather neutral or something like mixed. And that's the basic premise of this method. And it has become a rather popular method in recent years in the text mining community and information science in general and is applied in various research areas like social media analysis, analysis of product reviews, sentiment analysis on Twitter into a similar and research branches. If you would approach sentiment analysis nowadays, the state of the art idea would probably be to use some sort of large pre-annotated corpus about sentiment annotations. For example, you want to do Twitter sentiment analysis. There are a lot of large Twitter corpora with annotated sentiment per tweet. You would use these as training corpus and then perform some machine learning algorithm to train a model, probably nowadays with something like a large word embedding like bird or similar stuff. Of course, not every topic and not every research area has large chunks of annotated corpora. And of course, especially nowadays, the modern machine learning approaches are not that transparent and as other methods. So there's also another very popular, still very popular method, which is a little bit more simpler, which are lexicon based methods. And this is also what is still kind of popular in the DH community due to some sort of lack of annotated corpora, although there are developments that are going more into the direction of machine learning. Shortly about lexicon based approaches and sentiment analysis, because this is the main idea of our tool, it's a rather simple concept. Instead of having a pre-annotated corpus, you have a pre-annotated list of words, so-called sentiment bearing words, which are annotated concerning the sentiment polarity. So if a word is rather positive or negative in the general usage of this word, and this can be a number from like one or minus one, or it can be some metric scale, and you use these lists, which are created in various forms, sometimes expert based, sometimes by some sort of semi-automatic process. And then you perform rather trivial calculations for a text unit. Here's just a very simple example. You simply count the words you detect as positive, you count the words you detect as negative, and then you perform some basic mathematical calculations to get an overall value of the polarity, the expressed valence of the text. Of course, there are some approaches to perform this more sophisticated. You can try to look at negation words. You can try to look at valence intensifiers. You can perform limitization to improve this. But the overall premise is a basic calculation of the words detected in the text. As I already mentioned, due to the lack of corpora, and because it's very transparent also and very easy to perform, for a long time in DH context, lexicon based approaches, where the approach is to use when performing sentiment analysis. But some example what people are doing, they are, for example, looking at the most negative or the negative sentiment words used in plays by Shakespeare, for example, here in Humblett. They look at relationships between characters. This is a graph that visualizes the accumulated sentiment and speeches expressed by a specific character in a Shakespeare play, or tello towards the Stemona. You can see in the visualization that the accumulated sentiment becomes more and more negative, which is in line with the real content of this play. And similar stuff was also explored in various other areas that are close to digital humanities, like literary studies, but also for historical language and other research areas. The ideas are the same. We also contributed to this branch of research. We explored lexicon based sentiment analysis for another tool we developed for quantitative drama analysis, which I will shamelessly plug in here. In this specific case, we performed lexicon based sentiment analysis on place of lessing, and we evaluated approaches, and we explored visualizations. We developed a web tool to explore sentiment analysis in lessing's place. A tool that performs analysis like this and produces visualizations like this. This is, for example, a bar graph visualizing how the sentiment becomes more and more negative in a play from lessing. This is something you can identify in all of this plays. And we developed a lot of visualizations for this method. But as we did this and as more users used the tool and gave us feedback on what is interesting and so on, the feedback we received was more and more towards the idea that people wanted to perform their own lexicon based sentiment analysis. And people also wanted to have more transparency about how these results actually came to existence. So we saw a lack of tool existence in this context, and we decided to actually develop such a tool that people could use to perform their own lexicon based sentiment analysis. And the idea was to focus primarily on the DH community. We wanted to make it as accessible as possible. My impression is that tools that need a lot of installation, need a lot of dependencies are not used that frequently. And then tools that are more accessible like web tools. And we also wanted to give some transparency in the context of how calculations came to existence. We also applied some methods of the user-centered design process. We performed a requirements analysis. We integrated methods of usability engineering. For example, we performed interviews with a couple of people from the DH context, but also from literary scholars. We performed some usability tests to gain feedback and to develop a tool that is as adapted to the specific community we are designing it for as possible. I will not go into detail of all of these methods since I really want to show the tool. Just some of the requirements we acquired via this process are shown here. As I already said, people wanted to primarily use their own material. They wanted to adjust lexicons. As you can imagine, text sorts in DH are not really contemporary language. It's most of the time historical language or other specific domains, specific language. And people wanted to adjust this. They wanted transparent results. Since the web is not that... And the web is seen as a very accessible platform to use, which of course causes also a lot of problems. But this is the idea we followed here. So I summarized some of the overall functionality. We do perform some advanced stuff with our tool like limitations and negations. Right now everything is focused on German, but we plan to extend this also on other languages. But instead of talking about the functionality, I will actually show the tool with a live demo. And it will actually be also the first time that I explored this specific text. So as you already know, in IT, there can go nothing wrong with a live demo. So let's look at the tool. So I hope you all see my browser now. Can you give me short feedback? Okay. So this is the start page of 10 text. We integrated a lot of documentation and explanation as we already made the experience that people are very interested in all of this. And where we explain the different pre-processing steps, what types of data you can upload, what you can download, and so on. If you go to the sentiment analysis branch of this tool, you then can indeed upload your files and perform sentiment analysis. You offer some basic German sentiment lexicon, but there's also a possibility to upload your own lexicon if you follow a specific data standard, a rather simple standard, and that you can read more about in our documentation of the tool. From the advanced options, as I already said, we can perform limitization with a German limitizer, and the process is very time consuming. Again, since we are in the web, everything performance is a thing, so to speak, but you can also, you also have some other adjustment possibilities like stop words list and to use negations as well and shifters and so on. So I looked for an interesting example where I hope that I find some interesting results for state of the art research, so to speak, and what we will do, we will compare German rep lyrics with German Schlager lyrics. I looked a lot for the English word of Schlager, but I didn't find anything. It seems to be German Schlager. And I will prepare sentiment analysis. This might take a little time, so I of course prepared this beforehand just to give you an insight how this looks. This corpus is basically a list of lyrics from various artists of the specific rep or Schlager genre. I don't know much about rep, but maybe I have some artists, yes, some well-known German artists of Schlager. Not super representative, but of course I just want to show how the tool is applied. So please don't, this is not about German lyrics or something like this. Nevertheless, if you perform the analysis, what you get is a screen like this. On the left screen, in this specific case, the entire corpus of rep lyrics and the entire corpus Schlager lyrics are seen as one document, so to speak. So we just compare one document to each other. You can of course import more documents and then create so-called folders to compare document collections to each other. We will just focus on this one type. We currently look at the Schlager results. You get a normalized score. It's always very small, since it is normalized by the numbers of tokens, which helps to compare documents of different sizes. But nevertheless, the overall impression is indeed that the Schlager texts do look actually less negative than the rep texts, although the difference is quite small. You produce some visualizations that you can look for. For example, a pie chart of the negative detected words and the positive detected words. You can look at the strongest sentiment bearing words. This is just for Schlager in this example. So these are the very positive words that are most frequently used for Schlager. These are the most negative words that are used for Schlager. The word Hölle, I think this is very specific song that is the reason for this result here. And you can explore other analysis. We also try to connect a little bit of closed reading, which is actually something that we found out that users really like to explore how the sentiment analysis actually works. In the right text, you can explore your document and look at the specific words that are detected and what sentiment value they actually have. So this is actually the part where people oftentimes go into and look at, okay, a wrong word was detected. This word shouldn't be positive and so on. There are some words that sounds and sound intuitive. I was looking a bit for an example where negation comes to play, but I'm not sure if I can find one off the top of my head. So if a negation word is close to a sentiment bearing word, the valence gets thrown into the other polarity. Yes, but this is something that people actually quite like. You can also compare documents with each other. In this case, we would compare Rep to Schlager, for example. For example, if I want to compare the most negative words in Schlager and the most negative words in Rep, you can look at something like this and then you can get an overall impression of what is happening and I would argue that some of the results here are already telling of the specific genre. Of course, you can also look at this from a more quantitative standpoint of the specific word distribution. You would see here, indeed, there are more negative words, 46 percent than in Schlager, which is apparently a bit more positive according to the specific methods and all the limitations that are connected to this specific method. Yes, so I have all the links in my presentation. I will later on also share the presentation. Of course, you also find all the links to the tool and so on on the specific paper to explore the tool. That's always the most fun with tools to explore them better yourself. From my standpoint, we also performed a usability test. I will try to go back to the slides here. I hope you all see now my slides again. It was a rather small usability test, but the feedback was rather positive with very good results. Of course, we had some sort of iterative development sort of tool was constantly tried to improve. Overall, there are still a lot of missing features. Other than that, I'm rather glad the tool exists. It seems to be used rather often. I have a lot of my access numbers are surprisingly high. I don't know how these people always find the tool and every now and then I receive some mails and what people still would like the most is of course more lexicals and more languages. Could you integrate something for Spanish and so on? This is something to do. Of course, some sort of user management. Right now, everything is more or less, it's more or less, we basically saved nothing from a user. We don't even save the text, so you can save your overall dashboard. You can save the PNGs and you can download some tables with the results. You can download an XML with your results, but you can't save your entire process, so to speak. Overall, since I'm also doing sentiment analysis with other methods, I would say that if you really want to do a very scientific research project performing lexicon-based sentiment analysis, you usually need more control than this tool can offer, but I still think that this tool has its value. Nevertheless, you can explore a text sort, get first results, get first insights, get a first understanding of your specific text that might be problematic. I also think this tool is very nice to use for education purposes, just to show the tool to students to explore sentiment analysis and its method of law to clean. If you want to have more information, the good friends of Fortex created a very large tutorial for the tool that I can recommend if you want to get to know more. And other than this, I thank you for all your attention. I thank for Johanna for the great tool. Her contact data is also here. I hope you had some fun with the talk. Thank you very much.
 I hope everyone sees my slide.  I will gently switch to English as you hear.  Do you see my slides?  Okay.  So, my name is Thomas Schmidt from the Media Informatics Group of the University of Regensburg  and I will present Cent text at Tugel for Lexicon based sentiment analysis in digital  humanities.  As already was said, Johanna Dangell was the primary developer of this tool, Christian  Wojty, the supervisor of diseases that's connected to this tool development.  And yes, this is a theme, this is originally a demo slash short paper contribution.  So I will talk a little bit about background and how we approach this tool development,  but I will mainly focus on showing the tool and by using it.  So I will start with a little bit of background about sentiment analysis.  Here you see just a standard definition of sentiment analysis.  Also called opinion mining, the field of study that analyzes people's opinions, sentiments,  appraisals, attitudes and emotions, mostly in written text.  Overall the idea of sentiment analysis is to try to predict the sentiment, express sentiment  in a text unit, if it's rather positive or rather negative, if it's rather neutral or  something like mixed.  And that's the basic premise of this method.  And it has become a rather popular method in recent years in the text mining community  and information science in general and is applied in various research areas like social  media analysis, analysis of product reviews, sentiment analysis on Twitter into a similar  and research branches.  If you would approach sentiment analysis nowadays, the state of the art idea would probably be  to use some sort of large pre-annotated corpus about sentiment annotations.  For example, you want to do Twitter sentiment analysis.  There are a lot of large Twitter corpora with annotated sentiment per tweet.  You would use these as training corpus and then perform some machine learning algorithm  to train a model, probably nowadays with something like a large word embedding like bird or similar  stuff.  Of course, not every topic and not every research area has large chunks of annotated corpora.  And of course, especially nowadays, the modern machine learning approaches are not that transparent  and as other methods.  So there's also another very popular, still very popular method, which is a little bit  more simpler, which are lexicon based methods.  And this is also what is still kind of popular in the DH community due to some sort of lack  of annotated corpora, although there are developments that are going more into the direction of machine  learning.  Shortly about lexicon based approaches and sentiment analysis, because this is the main  idea of our tool, it's a rather simple concept.  Instead of having a pre-annotated corpus, you have a pre-annotated list of words, so-called  sentiment bearing words, which are annotated concerning the sentiment polarity.  So if a word is rather positive or negative in the general usage of this word, and this  can be a number from like one or minus one, or it can be some metric scale, and you use  these lists, which are created in various forms, sometimes expert based, sometimes by  some sort of semi-automatic process.  And then you perform rather trivial calculations for a text unit.  Here's just a very simple example.  You simply count the words you detect as positive, you count the words you detect as negative,  and then you perform some basic mathematical calculations to get an overall value of the  polarity, the expressed valence of the text.  Of course, there are some approaches to perform this more sophisticated.  You can try to look at negation words.  You can try to look at valence intensifiers.  You can perform limitization to improve this.  But the overall premise is a basic calculation of the words detected in the text.  As I already mentioned, due to the lack of corpora, and because it's very transparent  also and very easy to perform, for a long time in DH context, lexicon based approaches,  where the approach is to use when performing sentiment analysis.  But some example what people are doing, they are, for example, looking at the most negative  or the negative sentiment words used in plays by Shakespeare, for example, here in Humblett.  They look at relationships between characters.  This is a graph that visualizes the accumulated sentiment and speeches expressed by a specific  character in a Shakespeare play, or tello towards the Stemona.  You can see in the visualization that the accumulated sentiment becomes more and more  negative, which is in line with the real content of this play.  And similar stuff was also explored in various other areas that are close to digital humanities,  like literary studies, but also for historical language and other research areas.  The ideas are the same.  We also contributed to this branch of research.  We explored lexicon based sentiment analysis for another tool we developed for quantitative  drama analysis, which I will shamelessly plug in here.  In this specific case, we performed lexicon based sentiment analysis on place of lessing,  and we evaluated approaches, and we explored visualizations.  We developed a web tool to explore sentiment analysis in lessing's place.  A tool that performs analysis like this and produces visualizations like this.  This is, for example, a bar graph visualizing how the sentiment becomes more and more negative  in a play from lessing.  This is something you can identify in all of this plays.  And we developed a lot of visualizations for this method.  But as we did this and as more users used the tool and gave us feedback on what is interesting  and so on, the feedback we received was more and more towards the idea that people wanted  to perform their own lexicon based sentiment analysis.  And people also wanted to have more transparency about how these results actually came to existence.  So we saw a lack of tool existence in this context, and we decided to actually develop  such a tool that people could use to perform their own lexicon based sentiment analysis.  And the idea was to focus primarily on the DH community.  We wanted to make it as accessible as possible.  My impression is that tools that need a lot of installation, need a lot of dependencies  are not used that frequently.  And then tools that are more accessible like web tools.  And we also wanted to give some transparency in the context of how calculations came to  existence.  We also applied some methods of the user-centered design process.  We performed a requirements analysis.  We integrated methods of usability engineering.  For example, we performed interviews with a couple of people from the DH context, but  also from literary scholars.  We performed some usability tests to gain feedback and to develop a tool that is as  adapted to the specific community we are designing it for as possible.  I will not go into detail of all of these methods since I really want to show the tool.  Just some of the requirements we acquired via this process are shown here.  As I already said, people wanted to primarily use their own material.  They wanted to adjust lexicons.  As you can imagine, text sorts in DH are not really contemporary language.  It's most of the time historical language or other specific domains, specific language.  And people wanted to adjust this.  They wanted transparent results.  Since the web is not that...  And the web is seen as a very accessible platform to use, which of course causes also a lot  of problems.  But this is the idea we followed here.  So I summarized some of the overall functionality.  We do perform some advanced stuff with our tool like limitations and negations.  Right now everything is focused on German, but we plan to extend this also on other languages.  But instead of talking about the functionality, I will actually show the tool with a live  demo.  And it will actually be also the first time that I explored this specific text.  So as you already know, in IT, there can go nothing wrong with a live demo.  So let's look at the tool.  So I hope you all see my browser now.  Can you give me short feedback?  Okay.  So this is the start page of 10 text.  We integrated a lot of documentation and explanation as we already made the experience  that people are very interested in all of this.  And where we explain the different pre-processing steps, what types of data you can upload,  what you can download, and so on.  If you go to the sentiment analysis branch of this tool, you then can indeed upload your  files and perform sentiment analysis.  You offer some basic German sentiment lexicon, but there's also a possibility to upload your  own lexicon if you follow a specific data standard, a rather simple standard, and that  you can read more about in our documentation of the tool.  From the advanced options, as I already said, we can perform limitization with a German  limitizer, and the process is very time consuming.  Again, since we are in the web, everything performance is a thing, so to speak, but you  can also, you also have some other adjustment possibilities like stop words list and to  use negations as well and shifters and so on.  So I looked for an interesting example where I hope that I find some interesting results  for state of the art research, so to speak, and what we will do, we will compare German  rep lyrics with German Schlager lyrics.  I looked a lot for the English word of Schlager, but I didn't find anything.  It seems to be German Schlager.  And I will prepare sentiment analysis.  This might take a little time, so I of course prepared this beforehand just to give you an  insight how this looks.  This corpus is basically a list of lyrics from various artists of the specific rep or  Schlager genre.  I don't know much about rep, but maybe I have some artists, yes, some well-known German  artists of Schlager.  Not super representative, but of course I just want to show how the tool is applied.  So please don't, this is not about German lyrics or something like this.  Nevertheless, if you perform the analysis, what you get is a screen like this.  On the left screen, in this specific case, the entire corpus of rep lyrics and the entire  corpus Schlager lyrics are seen as one document, so to speak.  So we just compare one document to each other.  You can of course import more documents and then create so-called folders to compare document  collections to each other.  We will just focus on this one type.  We currently look at the Schlager results.  You get a normalized score.  It's always very small, since it is normalized by the numbers of tokens, which helps to compare  documents of different sizes.  But nevertheless, the overall impression is indeed that the Schlager texts do look actually  less negative than the rep texts, although the difference is quite small.  You produce some visualizations that you can look for.  For example, a pie chart of the negative detected words and the positive detected words.  You can look at the strongest sentiment bearing words.  This is just for Schlager in this example.  So these are the very positive words that are most frequently used for Schlager.  These are the most negative words that are used for Schlager.  The word Hölle, I think this is very specific song that is the reason for this result here.  And you can explore other analysis.  We also try to connect a little bit of closed reading, which is actually something that  we found out that users really like to explore how the sentiment analysis actually works.  In the right text, you can explore your document and look at the specific words that are detected  and what sentiment value they actually have.  So this is actually the part where people oftentimes go into and look at, okay, a wrong  word was detected.  This word shouldn't be positive and so on.  There are some words that sounds and sound intuitive.  I was looking a bit for an example where negation comes to play, but I'm not sure if I can find  one off the top of my head.  So if a negation word is close to a sentiment bearing word, the valence gets thrown into  the other polarity.  Yes, but this is something that people actually quite like.  You can also compare documents with each other.  In this case, we would compare Rep to Schlager, for example.  For example, if I want to compare the most negative words in Schlager and the most negative  words in Rep, you can look at something like this and then you can get an overall impression  of what is happening and I would argue that some of the results here are already telling  of the specific genre.  Of course, you can also look at this from a more quantitative standpoint of the specific  word distribution.  You would see here, indeed, there are more negative words, 46 percent than in Schlager,  which is apparently a bit more positive according to the specific methods and all the limitations  that are connected to this specific method.  Yes, so I have all the links in my presentation.  I will later on also share the presentation.  Of course, you also find all the links to the tool and so on on the specific paper to  explore the tool.  That's always the most fun with tools to explore them better yourself.  From my standpoint, we also performed a usability test.  I will try to go back to the slides here.  I hope you all see now my slides again.  It was a rather small usability test, but the feedback was rather positive with very  good results.  Of course, we had some sort of iterative development sort of tool was constantly tried to improve.  Overall, there are still a lot of missing features.  Other than that, I'm rather glad the tool exists.  It seems to be used rather often.  I have a lot of my access numbers are surprisingly high.  I don't know how these people always find the tool and every now and then I receive  some mails and what people still would like the most is of course more lexicals and more  languages.  Could you integrate something for Spanish and so on?  This is something to do.  Of course, some sort of user management.  Right now, everything is more or less, it's more or less, we basically saved nothing  from a user.  We don't even save the text, so you can save your overall dashboard.  You can save the PNGs and you can download some tables with the results.  You can download an XML with your results, but you can't save your entire process, so  to speak.  Overall, since I'm also doing sentiment analysis with other methods, I would say that if you  really want to do a very scientific research project performing lexicon-based sentiment  analysis, you usually need more control than this tool can offer, but I still think that  this tool has its value.  Nevertheless, you can explore a text sort, get first results, get first insights, get  a first understanding of your specific text that might be problematic.  I also think this tool is very nice to use for education purposes, just to show the tool  to students to explore sentiment analysis and its method of law to clean.  If you want to have more information, the good friends of Fortex created a very large  tutorial for the tool that I can recommend if you want to get to know more.  And other than this, I thank you for all your attention.  I thank for Johanna for the great tool.  Her contact data is also here.  I hope you had some fun with the talk.  Thank you very much.
ISI-2021-TIB-22-Schmidt_0_1415.png ISI-2021-TIB-22-Schmidt_1416_2951.png ISI-2021-TIB-22-Schmidt_2952_2999.png ISI-2021-TIB-22-Schmidt_3000_4835.png ISI-2021-TIB-22-Schmidt_4836_5015.png ISI-2021-TIB-22-Schmidt_5016_7547.png ISI-2021-TIB-22-Schmidt_7548_9071.png ISI-2021-TIB-22-Schmidt_9072_9815.png ISI-2021-TIB-22-Schmidt_9816_11375.png ISI-2021-TIB-22-Schmidt_11376_14171.png ISI-2021-TIB-22-Schmidt_14172_15287.png ISI-2021-TIB-22-Schmidt_15288_15347.png ISI-2021-TIB-22-Schmidt_15348_15395.png ISI-2021-TIB-22-Schmidt_15396_15959.png ISI-2021-TIB-22-Schmidt_15960_16043.png ISI-2021-TIB-22-Schmidt_16044_16703.png ISI-2021-TIB-22-Schmidt_16704_17435.png ISI-2021-TIB-22-Schmidt_17436_18395.png ISI-2021-TIB-22-Schmidt_18396_18503.png ISI-2021-TIB-22-Schmidt_18504_18575.png ISI-2021-TIB-22-Schmidt_18576_18791.png ISI-2021-TIB-22-Schmidt_18792_18935.png ISI-2021-TIB-22-Schmidt_18936_18995.png ISI-2021-TIB-22-Schmidt_18996_19151.png ISI-2021-TIB-22-Schmidt_19152_25871.png ISI-2021-TIB-22-Schmidt_25872_26003.png ISI-2021-TIB-22-Schmidt_26004_26123.png ISI-2021-TIB-22-Schmidt_26124_29411.png ISI-2021-TIB-22-Schmidt_29412_29687.png ISI-2021-TIB-22-Schmidt_29688_30133.png