PADI-web: a health monitoring system to analyze the emergence and spread of animal diseases
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Rabatel, Julien
Julien Rabatel is a freelance developer with a PhD in Computer Science. Nowadays, he uses his development and research experience to participate as a freelancer in various projects, frequently involving public institutions such as CIRAD or universities. PADI-web is an online tool that collects and processes documents from the Web. Its goal is to allow daily surveillance in epidemiology, by exploiting unofficial sources such as local news. This talk will describe how PADI-web is working, and what it can provide to the end-user. Also, we will take a look at the future of PADI-web and how we can address its current limitations.
 Hello, as a developer of PaddyWeb, I'm going to give you an overview of this platform, which is a platform for automated extraction of animal disease information and from the web. So what is PaddyWeb? It initially comes from the need of the VSI, the French Epidemic Intelligence Organization in France, which aims at identifying, monitoring different sources in order to identify different signals of sanitary dangers in animal health. And in order to do so, they have to monitor different types of sources, such as official sources, OIE and PESI, but also a number of experts. And they also have to monitor the web in order to find potential relevant news containing such signals of dangers. And this is where PaddyWeb is coming because this task is huge and demanding. And PaddyWeb is here to help them automate this process. So as a result, the first definition for PaddyWeb is that it is a tool for monitoring online news for the detection of emerging animal infectious diseases and the extraction of outbreak information. As compared to other similar tools, PaddyWeb focuses mainly on animal health and has a fully automated pipeline based on multiple machine learning methods. And as for the challenges encountered in the development and design of PaddyWeb, it's important to notice that first, it's a very generic and highly customizable tool. Even if I said before that it was focusing on animal health, this is true, but still in the development of PaddyWeb, we always had in mind that we wanted it to be generic so that it could be applied to other domains of application than animal epidemiology. Also it uses a lot of existing tools and libraries as much as possible in its processing pipeline. And it's a task also to combine them and keep a pipeline that is complex but still is a robust processing pipeline. And finally, another challenge in the development of PaddyWeb is that we have different types of end users. We can have domain experts, for instance epidemiologists, but we can also have data scientists as users or even developers. And of course, they expect different things from PaddyWeb so we have to provide them with these different outputs. So here is the PaddyWeb processing pipeline. So the first step of PaddyWeb is the data collection step because we cannot monitor the whole web of course. We made the decision to focus on Google News mainly, which means that PaddyWeb every few hours query Google News with different combinations of keywords like disease names, like symptoms like host names, in order to retrieve only potentially relevant articles for animal epidemiology. Google News provides us with some results that are in the form of RSS feeds, which basically is a structured list of articles with different information. And then for each article that is written, we have the step of the web page processing. So here is an example of such a collection process step. We have an example of an article from a website. And first of all, we have to retrieve some metadata for this article directly from the RSS feed. So we have the publication date, for instance, the title of the article, and of course a link to the actual web page of the article. Then we have to clean this web page. We have to visit it and extract the content that will be used inside PaddyWeb. So we have to clean everything, which means getting rid of everything that is not needed, like ads, navigation menus, images, videos, comments, and so on. Then we can obtain a clean text, which is here in French. As indeed we have different types of sources in PaddyWeb, different articles coming from different countries in various languages. And we made the choice of processing texts only in English within PaddyWeb in order to simplify the processing pipeline. So we have to translate the text into English. For that we have first a language section step. And then we apply machine learning, machine translation, sorry, using the Microsoft translator here in order to obtain a text that is clean, that is in English, and that is ready to be processed within PaddyWeb. Now that we are able to collect some articles and clean them and store them into our database, we have a data classification step, which mainly initially is here to decide whether an article is relevant or not. Indeed in PaddyWeb we collect a lot of articles daily. Every day we have a lot of articles collected, but not all of them are of epidemiological interest, despite the fact that we created, we made some quite precise queries to Google News. So we had to build an automatic classification system based on supervised machine learning methods. And so we did that and actually it's quite good at removing non-redeven articles in PaddyWeb. But another thing that is important to notice here is that the classification module here again is very customizable, is very generic, and it can be used for other tasks, other classification tasks. And actually this is what we do. So for instance we use the specification module for designing what is the topic of an article, is it about an outbreak declaration here, is it about the consequences of an outbreak and so on. We can also use it for sentence classification, which is we do not classify the whole article, but also we can classify each sentence in an article, for instance to tell the users what type of information is contained in a sentence. And in PaddyWeb we can also define other classification tasks. It's quite easy to do it within PaddyWeb. Another thing to notice here is that the users within PaddyWeb can rectify wrong classifications, which means that if a user sees a classification that is not correct within the PaddyWeb user interface he can rectify it and then every day the models, the classification models are retrained based on this new example so that they can get better and better over time. Now here is an example of an article that has been classified. So in the PaddyWeb user interface, so here you can see that this article, for instance, which is about an outbreak of Avian influenza has been classified as relevant first, but also has been classified as being about an outbreak declaration. And these type of things allow the users, for instance, to study only articles that are about outbreak declarations, for instance. Now another module that I would like to speak about today because it is very important in PaddyWeb is the information extraction module. So the goal of the information extraction in PaddyWeb is to automatically detect some pieces of information within the text that we collected, pieces of information that are of interest for animal epidemiology here so they can be location names, hosts, or species, disease names, case numbers, dates and so on. Here you have an example of a sentence where everything in blue is a type of information that we would like to be able to extract automatically. So in order to do so, we use a named entity recognition tool which is well known in the machine learning and text mining community, which is called SPACY. And actually SPACY already includes a model to recognize some generic entities in English, entities like names of locations, dates, names of organizations, names of people, things like that. But of course it is not designed to extract epidemiology information. So what we had to do here was first to build a dataset of around 500 articles which has been user labeled, each entity in these articles has been labeled. And then we have trained a specific model for SPACY for animal epidemiology in order to recognize entities that are specific to PaddyWeb topics, names of disease and so on. And this is what allows us in PaddyWeb user interface to have things like this where in the text of an article, the African swine fever has been automatically detected as being the name of disease. And we have also types of information that are automatically detected like this. And this is very important then for the outputs that the end users are going to use in the end. Okay, finally, for this presentation, I would like to speak about the last step of the processing pipeline of PaddyWeb which is about the outputs of PaddyWeb for the end users. As I said before, PaddyWeb has different types of users, very different types of users. So of course we have to provide different types of outputs for them. So we have different types of outputs that are here. We have, for instance, first of all, the PaddyWeb user interface which is a website. And by the way, it is publicly available. Then we have notification emails for helping the continuous monitoring of use collected by PaddyWeb. This is typically very useful for the French epidemic intelligence surveillance members who have to daily see what is happening on PaddyWeb. And then we have values exporting capabilities in PaddyWeb in order to extract different types of data generated by PaddyWeb. This is useful for data modules for computer scientists, for instance. And finally, we have also a simple JSON API for developers which can, for instance, if developers want to use some of the capabilities of PaddyWeb without actually using the PaddyWeb user interface, the website, if they want to do something else with the capabilities of PaddyWeb. So these are the different types of outputs. Here is a very quick example of email notifications to be able to vote. This is the kind of email that users can receive daily or weekly and they show the last article that have been collecting during the last 24 hours, for instance, by PaddyWeb with some basic information. Actually, they are organized by disease and by location. And another thing here is that the user can select the disease she or she wants to follow. So for instance, if I am only interested in ADN influenza, I can specify it with my subscription on the PaddyWeb user interface. So this is one type of output. And another type is about exporting capabilities of PaddyWeb. So here is an example of probably one of the most useful exporting formats for PaddyWeb, which is about exporting location-based extracted information from PaddyWeb, which is a simple file, CSV file, for instance, with every word being one location found by PaddyWeb in an article. And then we provide different types of information. So a lot of information about the location and they are coming from your name's API, actually, this type of information. So you have, for instance, the spatial coordinates of the location, which is very useful if you want to put them on the map. Then we have different, all the other types of information that have been extracted by PaddyWeb around the location mentioned. And also different types of information about the article it has been found in and things like that. So this is one type of export in PaddyWeb, but I like to say that yes, PaddyWeb is also able to export other types of data. So almost every type of data that is handled by PaddyWeb or that is generated by PaddyWeb can be exported between articles, extracted information, keywords, sentences, RSS feeds and so on. Now to conclude the talk, I would like to speak a bit about the current and future work because indeed PaddyWeb is becoming a very large platform and there is room for improvement in almost every step of the pipeline, of course. But we have identified a few ones that are more important, such as, for instance, improving the geolocation, which is a work about extracting spatial information from the text. We have to refine the geolocation used in notification emails, so sometimes the countries that we associate with the article is wrong and we have to fix that. And we need also some disambiguation of location entities, which is basically the problem of when I see the name of a city, for instance, maybe there are different places in the world with the same name and which one is it that is mentioned in the text. And we have in progress some work on visual analytics of detected events with some of other wood partners. For instance, there is work on space time visualization in progress of detected outbreaks in collaboration with partners in France. There is some work on risk mapping using extracted outbreak data in collaboration with partners in Belgium. And of course, also, there are some work on improving some specific modules in Paddyweb or creating new modules in Paddyweb. And there is a lot of research work done by, for instance, by PhD students or postdoc researchers here. And of course, one part of my job as a developer is to know how to integrate them to Paddyweb while maintaining the main ideas of the platform. So thank you very much for your attention. Here you have some of the members of the Paddyweb team. And I'd like to say that if you want to use Paddyweb, if you want to try it, or if you have some ideas to collaborate with the Paddyweb team, you can contact us at the given email address here. Thank you. Let me kick off first question. How many interests you have at the moment? Okay, actually, I won't have the exact numbers, current numbers, because it evolves quite quickly. But I think we have a bit less than 400,000 articles in the database. Because it started collecting articles a few years ago, in 2016, I think maybe. And not all of them are relevant. But okay, it gives an idea of the size of the database actually. And for each article, we have hundreds of extracted pieces of information and so on. So the database is quite big, actually. And geographically, have you tried, because I see that you geocode, using geonames, geocode geography, have you tried making a map to see the density of this? Well, actually, not directly us. In Paddyweb, there is no map visualization, but there is some discussion actually to integrate some map visualization within Paddyweb user interface currently. But still, there are another partner, a good partner, at Lyon in Montpellier in France. And they have been doing somewhere to actually show a map, but not only a map, but also a time-related map. A time slider to see. Yes, and a map in order to visualize the outbreaks and so on. So it can be done with a type of exporting ability that I showed before from Paddyweb. So the moment you don't have it, right? You don't have that. Not within Paddyweb. It's not a part of Paddyweb. It's something that has been done outside Paddyweb using the data generated by Paddyweb. And do you have you tried putting, testing the Paddyweb, how it reacts to noise? And I know that today there's a lot of work done on detecting inflated information or spam or something. So how do you work? Or is it then done on the Google side because you use the Google news, right? Actually, what we noticed quite quickly when we made the first prototype for Paddyweb was that even if we have very precise queries to Google news with some very specific keywords, disease names and so on, even in that case, there is a majority of articles that are collected that are not really even at all. And the way we dealt with that problem was to introduce a classification module. And the fact is that there is this classification module, which is able, because we provided it with some labeled data, some examples of redevent and non-relevant articles. It's able now to make this work of removing everything that is not relevant. And actually, it's quite good at doing that because if I remember where we have something like 95% accuracy score for descending whether an article is relevant or not. So this is something that works pretty good for designing whether an article is relevant. But then we have also the noise of everything that comes behind, which is mainly the information extraction module. Of course, when we sometimes we extract information that is not correct. Sometimes the location names are not the good ones, for instance, things like this. So of course, there is the amount of things that we extract, so it's a bit of a part of the problem because we can see that if a lot of things say that something happens here, then probably there is something happening here. And if it is completely an outlier, maybe it's wrong. But we don't have for now a way to really eliminate wrongly identified expected information. Okay. We have a lot of questions in the chat. So maybe I can read them out to you. Let's start from the first one from Elena. Can PadiWeb use other languages besides English? Like for example, Serbian, since we have also Serbian partners. And how could it be used from the colleagues in other countries? Okay. Well, actually, yes, PadiWeb has really been designed with generosity in mind, which means also being able to work with different languages, other than English, for instance. As you saw, we have a translation step in our pipeline, which aims at translating everything that is not in English into English so that we can work with it. But it's also completely possible in PadiWeb to just work with another language and decide that we are working in something different than English, or we're just not translated everything. But in that case, the main difficulty actually is not on the PadiWeb site in terms of code or development. It's more in the terms of how to feed the machine learning models that we use. It means that if you want now to use Serbian, then you have to build classification models for Serbian for deciding whether it's relevant or not and so on. So this is a lot of work to prepare the machine learning models in order to be able to use a language. And this is mainly why we decided to just use one language, because otherwise it's too much work. Hi. Can you give a concrete example of use of PadiWeb developers? Yes. Well, actually a very simple example, quite trivial one actually was a few years ago, I think. Someone wanted to use the abilities of PadiWeb, the ability to collect data and to index them and do a classification on data, but they just not wanted to use PadiWeb user interface. They wanted to have their own module doing that within their own user interface for another type of project, for instance. And in that case, they were using the PadiWeb JSON API in order to do that. So they could use some of the features of PadiWeb, but completely outside the, and some of the database of PadiWeb as well, completely outside the user interface. That was one example of usage. Okay, thank you. And do you know how many people subscribe to this notification system and if you have any idea who they are? Well, actually, I don't know how many there are today. I know that each time there is a presentation, there are some people asking for a user account and the kind of course, but I know that, okay, typically the notification emails was designed for people from the French Epidemic Intelligence Surveillance Team, because every day there were almost every day their work is to completely go to the computer and search for what has been happening in the last two days, for instance, what are the news, et cetera, that are relevant to our problem. And for them, it's very useful to have this type of notification emails directly in their emails every day so that they don't have to search by themselves. They don't have to go to PadiWeb website and do a search on it. They receive the information directly. And one last question from the chat. How do you deal with fake news? Okay, this is not a problem that we specifically addressed. So well, I don't have in mind an example of some fake news that we saw appearing in PadiWeb that could add some noise, for instance, to the data. I know there are people in the audience from the PadiWeb team have an ID, but for now we're just not addressing it. We haven't seen the need for it for now. Okay, thanks. And then I see one hand raised from Frank. Frank, would you like to ask a question? Yeah. Yeah. Thanks, Julian. And thanks, everyone. It's really a very nice platform. I just have one or two questions. So firstly, on the location, sorry, my name is Frank from ISIDM. So on the location part, I mean, the geocoding of the data collected at what spatial resolution is that done? I mean, at what geographic scale? Is it at country level, at state level, or at sub-state level? And then for places whose coordinates or who don't have good geocodes, right? How do you do to manage that? I'm based in Nigeria for now, and a couple of places in Nigeria can't be geocoded. So how do you manage such? Then on the API, is it tied to the subscriptions? So does it mean developers have access to the API endpoints for the parts they have subscribed to on your platform? Or are the APIs, does the API expose any needed endpoint from a developer? And finally, how often do you bring together the team of experts, perhaps your modulators, to review the classification results to be sure of what have been classified correctly after the initial 500 classified articles? So those are these few questions. Thanks. Okay. I tried to remember everything as much as possible. So I think the first question was about geocoding and using geonames, and sometimes it was about the granularity of the data from geonames, maybe. And about that, okay, we don't have any pre-specified granularity that we want to use. We just find names of locations in text, and then we query geonames to know whether there is something that has this name and this geonames entity. So we don't have any constraint on the granularity, and this is geonames that give us, we find a name, and geonames give us information, and such as what type of entity it is, what type of land of special entity it is. And this is the kind of information that you have here on the exported data that is completely provided by geonames. We don't filter or process this information here. And if something is not present in geonames, then unfortunately, we just know that potentially it is a location name because we extract it as a location name, but we don't have any information about it because geonames won't recognize it. So that can happen a lot, and that can happen. And sometimes it shows that the information extraction made a mistake, said that something was a location name, but it is not. That can happen. And sometimes maybe it's just that geonames don't know it. Okay, so the next question I think was about the JSON API. The JSON API for now is very simple. And in order to actually it was designed very specifically for a given project. And the question of whether it's publicly available for other developers and so on. Actually, it has to be really discussed with a party web team specifically for each case because for now we have something that is that protects the API so that it cannot be used by other people if they are not being authorized to do so. So you would have to send an email to the party web contact email actually. And the last question I think was about the classification, how we make sure that the classifications are correct or not, I think. And the fact is that in, okay, I try to come back here. Okay, one thing is that if you're a signed in user on party web, each time you see an article, you have the list of the classification labels that have been assigned to each article. So here, for instance, in my example, it is a relevant article and it's an outbreak declaration according to the automatic classification module. Now if you see that there is a mistake here, a user can just click on the button and then it will take precedence and says, okay, this one has been wrongly classified. And now we consider it as the user said. And then we have when we train a model automatically every day, actually, we retrain the model, the classification models. We have a cross validation process that is here to validate the fact that the classification model according to the examples we have is doing good work or not. And we have one more, a couple of questions still from the chat if from Isabelle, if party web could be used for videos or spoken news. So not only text. Well, this is a very interesting question actually because there is a current discussion currently on working with YouTube videos because indeed they can be a different type of source. Currently, there is no way that we're going to do something that is able to recognize from the voice of something speaking to convert it into something that we can work with. But on YouTube, there is something very interesting, which is that there are transcriptions, transcripts for the videos that can be user written or they can be automatically generated. And we plan maybe to use these kind of transcripts in order to work with video interaction. Can I also one more question Julia about the data size? What would be a dream size where you think you could do like also some like very reliable modeling? You said you have 400,000 articles. What would be a dream size where you think that like, I mean, I don't know how much Google News produces per year articles or would you like to have like a, you know, I don't know, 40 million articles that you can analyze, including video. So what's what would be a dream size? Yes. Well, actually, it really depends on each model has its own requirements in terms of size of data in order to become reliable. For instance, for if I take the example of the classification models, okay, the classification models for redevance, is it redevanced or not? Okay, there are only two possibilities. Texts are usually very different from their redevanced or not redevanced. Okay, we've been able to build a model that is quite effective with quite with not a lot of data, but for other types of specifications, for instance, it's much more difficult, but it's also much more difficult to build data sets, user labeled data sets that will be really useful. So well, I'd say I would like to have as much data as possible, of course, but it means that actually the actual work, then the actual effort is on the experts that have to manually label the data. And for instance, I have an example of the model that we built for the information extraction module. It was about 500 articles, which is not that huge in terms of machine learning data set. But Elena Afebska actually did the work of labeling each entity within each text, and it was a huge, huge task. So yes, we have to find the right balance between visibility for users, for experts, and usefulness for the models. So while there is not one right answer to that, I'm afraid. And the historic news articles, like most of your articles, like articles that Google News harvest is just digital news articles, but they're also historic articles. And I know Google digitized the books, right? And the European libraries digitized the books. What about the historic news articles going like 20, 30 years back? Would this be something interesting for Padiweb also? Well, I guess for the epidemiologist, it could be something of interest to see how things evolved over time. For the developer point of view, from my point of view, yes, it could be feasible if we can have an access at some time in terms of raw text that we can process. And this is the main, the most important thing for us is to be able to convert everything into some text that we can process. Initially Padiweb comes from collaboration between some epidemiologists from the French Epidemic Intelligence team and some computer scientists. And while the epidemiologists have to find potential threats to animal health by monitoring some unofficial news articles on the web and some computer scientists, then so that maybe it was possible to automate this part of the process by collecting articles, by processing them automatically. And the idea was to build an automated pipeline, exploiting some works in machine learning in order to provide something that we felt would really help the epidemiologists in their daily tasks. So that was the main idea behind the platform, Padiweb, and that was how the first prototype for Padiweb was developed a few years ago. And since then, a lot of features have been added or improved in Padiweb, such as the article classification module, for instance. And actually also by discussing with other research groups by other people, we realized that this type of platform like Padiweb would be really useful to a lot of domains of applications, of applications. So monitoring news or documents and the automaticity, having an idea of the information they contain, this is something that had to be done for multiple applications, actually not only for animal epidemiology. Well, yes, there are very different channels of distribution for the information that has been extracted by Padiweb. And it is because we have different types of users and they expect different things from Padiweb. We have different types of those channels of distribution. So the first one is about domain experts, so epidemiologists, for instance, from the French Epidemic Surveillance Team. And they are interested in following almost daily what is happening so that they can have a feel of the current disease outbreaks and also they need to be able to react quickly if something of PachiBrain interest happens. And another thing is that they don't want to go to Padiweb every day to research the new article. So that's why we have the notifications, notification emails in Padiweb that are here to provide them directly with some of the information that has been collected lately during the last 24 hours, for instance. And then if something of interest appears in the notification email, they can still go to Padiweb and then do some research about the thing that happened. And then we have a completely different set of users that are data scientists, that are computer scientists, that are developers. And for them, actually, they are more interested in the data collected by Padiweb and extracted by Padiweb, but they want to be able to analyze the data, process them differently than what we do in Padiweb. So for instance, one example is about building map visualizations given the break data that Padiweb has extracted. And in this case, we provide to users, these type of users, with exporting files in different formats that can be processed by them. So these are the main ways to distribute information that is collected by Padiweb. Well, yes, of course, there are lots of plans to improve Padiweb because it's becoming a very large, very big platform. And well, for almost every step that is presented in Padiweb, there is a way to improve it. So there are maybe different levels of improvement that we can work on. One is to improve each module individually. So maybe improve the information extraction module, maybe improve the collection process and so on. One example of this type of improvements is the geolocation geocoding part of Padiweb, which is about extracting spatial information from the articles. And here we identified some weaknesses in our process. So currently there is some, for instance, some PhD students who are currently working on this problem trying to find different ways to do it. And later on, maybe we will integrate this type of work into Padiweb in order to improve it. And then I think there are another level of improvement, which is very important here. It's about improving the data sets, the data that we use in order to feed the different machine learning models that we have in Padiweb. So for instance, the classification models, they really rely on the type of data that we gave them as examples. And because it is difficult to ask experts to do that a lot, it's a lot of efforts from the experts to manually build data sets to do something that is still very important. And I think this is one of the main things that we really need to do in the future to keep Padiweb pretty good.
 Hello, as a developer of PaddyWeb, I'm going to give you an overview of this platform,  which is a platform for automated extraction of animal disease information and from the  web.  So what is PaddyWeb?  It initially comes from the need of the VSI, the French Epidemic Intelligence Organization  in France, which aims at identifying, monitoring different sources in order to identify different  signals of sanitary dangers in animal health.  And in order to do so, they have to monitor different types of sources, such as official  sources, OIE and PESI, but also a number of experts.  And they also have to monitor the web in order to find potential relevant news containing  such signals of dangers.  And this is where PaddyWeb is coming because this task is huge and demanding.  And PaddyWeb is here to help them automate this process.  So as a result, the first definition for PaddyWeb is that it is a tool for monitoring  online news for the detection of emerging animal infectious diseases and the extraction  of outbreak information.  As compared to other similar tools, PaddyWeb focuses mainly on animal health and has a fully  automated pipeline based on multiple machine learning methods.  And as for the challenges encountered in the development and design of PaddyWeb, it's important  to notice that first, it's a very generic and highly customizable tool.  Even if I said before that it was focusing on animal health, this is true, but still in  the development of PaddyWeb, we always had in mind that we wanted it to be generic so  that it could be applied to other domains of application than animal epidemiology.  Also it uses a lot of existing tools and libraries as much as possible in its processing pipeline.  And it's a task also to combine them and keep a pipeline that is complex but still is a  robust processing pipeline.  And finally, another challenge in the development of PaddyWeb is that we have different types  of end users.  We can have domain experts, for instance epidemiologists, but we can also have data scientists as users  or even developers.  And of course, they expect different things from PaddyWeb so we have to provide them with  these different outputs.  So here is the PaddyWeb processing pipeline.  So the first step of PaddyWeb is the data collection step because we cannot monitor the whole web  of course.  We made the decision to focus on Google News mainly, which means that PaddyWeb every few  hours query Google News with different combinations of keywords like disease names, like symptoms  like host names, in order to retrieve only potentially relevant articles for animal epidemiology.  Google News provides us with some results that are in the form of RSS feeds, which basically  is a structured list of articles with different information.  And then for each article that is written, we have the step of the web page processing.  So here is an example of such a collection process step.  We have an example of an article from a website.  And first of all, we have to retrieve some metadata for this article directly from the  RSS feed.  So we have the publication date, for instance, the title of the article, and of course a link  to the actual web page of the article.  Then we have to clean this web page.  We have to visit it and extract the content that will be used inside PaddyWeb.  So we have to clean everything, which means getting rid of everything that is not needed,  like ads, navigation menus, images, videos, comments, and so on.  Then we can obtain a clean text, which is here in French.  As indeed we have different types of sources in PaddyWeb, different articles coming from  different countries in various languages.  And we made the choice of processing texts only in English within PaddyWeb in order to  simplify the processing pipeline.  So we have to translate the text into English.  For that we have first a language section step.  And then we apply machine learning, machine translation, sorry, using the Microsoft translator  here in order to obtain a text that is clean, that is in English, and that is ready to be  processed within PaddyWeb.  Now that we are able to collect some articles and clean them and store them into our database,  we have a data classification step, which mainly initially is here to decide whether  an article is relevant or not.  Indeed in PaddyWeb we collect a lot of articles daily.  Every day we have a lot of articles collected, but not all of them are of epidemiological  interest, despite the fact that we created, we made some quite precise queries to Google  News.  So we had to build an automatic classification system based on supervised machine learning  methods.  And so we did that and actually it's quite good at removing non-redeven articles in PaddyWeb.  But another thing that is important to notice here is that the classification module here  again is very customizable, is very generic, and it can be used for other tasks, other  classification tasks.  And actually this is what we do.  So for instance we use the specification module for designing what is the topic of an article,  is it about an outbreak declaration here, is it about the consequences of an outbreak  and so on.  We can also use it for sentence classification, which is we do not classify the whole article,  but also we can classify each sentence in an article, for instance to tell the users  what type of information is contained in a sentence.  And in PaddyWeb we can also define other classification tasks.  It's quite easy to do it within PaddyWeb.  Another thing to notice here is that the users within PaddyWeb can rectify wrong classifications,  which means that if a user sees a classification that is not correct within the PaddyWeb user  interface he can rectify it and then every day the models, the classification models  are retrained based on this new example so that they can get better and better over time.  Now here is an example of an article that has been classified.  So in the PaddyWeb user interface, so here you can see that this article, for instance,  which is about an outbreak of Avian influenza has been classified as relevant first, but  also has been classified as being about an outbreak declaration.  And these type of things allow the users, for instance, to study only articles that  are about outbreak declarations, for instance.  Now another module that I would like to speak about today because it is very important in  PaddyWeb is the information extraction module.  So the goal of the information extraction in PaddyWeb is to automatically detect some  pieces of information within the text that we collected, pieces of information that are  of interest for animal epidemiology here so they can be location names, hosts, or species,  disease names, case numbers, dates and so on.  Here you have an example of a sentence where everything in blue is a type of information  that we would like to be able to extract automatically.  So in order to do so, we use a named entity recognition tool which is well known in the  machine learning and text mining community, which is called SPACY.  And actually SPACY already includes a model to recognize some generic entities in English,  entities like names of locations, dates, names of organizations, names of people, things  like that.  But of course it is not designed to extract epidemiology information.  So what we had to do here was first to build a dataset of around 500 articles which has  been user labeled, each entity in these articles has been labeled.  And then we have trained a specific model for SPACY for animal epidemiology in order  to recognize entities that are specific to PaddyWeb topics, names of disease and so on.  And this is what allows us in PaddyWeb user interface to have things like this where in  the text of an article, the African swine fever has been automatically detected as being  the name of disease.  And we have also types of information that are automatically detected like this.  And this is very important then for the outputs that the end users are going to use in the  end.  Okay, finally, for this presentation, I would like to speak about the last step of the processing  pipeline of PaddyWeb which is about the outputs of PaddyWeb for the end users.  As I said before, PaddyWeb has different types of users, very different types of users.  So of course we have to provide different types of outputs for them.  So we have different types of outputs that are here.  We have, for instance, first of all, the PaddyWeb user interface which is a website.  And by the way, it is publicly available.  Then we have notification emails for helping the continuous monitoring of use collected  by PaddyWeb.  This is typically very useful for the French epidemic intelligence surveillance members  who have to daily see what is happening on PaddyWeb.  And then we have values exporting capabilities in PaddyWeb in order to extract different types  of data generated by PaddyWeb.  This is useful for data modules for computer scientists, for instance.  And finally, we have also a simple JSON API for developers which can, for instance, if  developers want to use some of the capabilities of PaddyWeb without actually using the PaddyWeb  user interface, the website, if they want to do something else with the capabilities  of PaddyWeb.  So these are the different types of outputs.  Here is a very quick example of email notifications to be able to vote.  This is the kind of email that users can receive daily or weekly and they show the last article  that have been collecting during the last 24 hours, for instance, by PaddyWeb with some  basic information.  Actually, they are organized by disease and by location.  And another thing here is that the user can select the disease she or she wants to follow.  So for instance, if I am only interested in ADN influenza, I can specify it with my subscription  on the PaddyWeb user interface.  So this is one type of output.  And another type is about exporting capabilities of PaddyWeb.  So here is an example of probably one of the most useful exporting formats for PaddyWeb,  which is about exporting location-based extracted information from PaddyWeb, which is a simple  file, CSV file, for instance, with every word being one location found by PaddyWeb in an  article.  And then we provide different types of information.  So a lot of information about the location and they are coming from your name's API, actually,  this type of information.  So you have, for instance, the spatial coordinates of the location, which is very useful if you  want to put them on the map.  Then we have different, all the other types of information that have been extracted by  PaddyWeb around the location mentioned.  And also different types of information about the article it has been found in and things  like that.  So this is one type of export in PaddyWeb, but I like to say that yes, PaddyWeb is also  able to export other types of data.  So almost every type of data that is handled by PaddyWeb or that is generated by PaddyWeb  can be exported between articles, extracted information, keywords, sentences, RSS feeds  and so on.  Now to conclude the talk, I would like to speak a bit about the current and future work  because indeed PaddyWeb is becoming a very large platform and there is room for improvement  in almost every step of the pipeline, of course.  But we have identified a few ones that are more important, such as, for instance, improving  the geolocation, which is a work about extracting spatial information from the text.  We have to refine the geolocation used in notification emails, so sometimes the countries  that we associate with the article is wrong and we have to fix that.  And we need also some disambiguation of location entities, which is basically the problem of  when I see the name of a city, for instance, maybe there are different places in the world  with the same name and which one is it that is mentioned in the text.  And we have in progress some work on visual analytics of detected events with some of  other wood partners.  For instance, there is work on space time visualization in progress of detected outbreaks  in collaboration with partners in France.  There is some work on risk mapping using extracted outbreak data in collaboration with partners  in Belgium.  And of course, also, there are some work on improving some specific modules in Paddyweb  or creating new modules in Paddyweb.  And there is a lot of research work done by, for instance, by PhD students or postdoc  researchers here.  And of course, one part of my job as a developer is to know how to integrate them to Paddyweb  while maintaining the main ideas of the platform.  So thank you very much for your attention.  Here you have some of the members of the Paddyweb team.  And I'd like to say that if you want to use Paddyweb, if you want to try it, or if you  have some ideas to collaborate with the Paddyweb team, you can contact us at the given email  address here.  Thank you.  Let me kick off first question.  How many interests you have at the moment?  Okay, actually, I won't have the exact numbers, current numbers, because it evolves quite  quickly.  But I think we have a bit less than 400,000 articles in the database.  Because it started collecting articles a few years ago, in 2016, I think maybe.  And not all of them are relevant.  But okay, it gives an idea of the size of the database actually.  And for each article, we have hundreds of extracted pieces of information and so on.  So the database is quite big, actually.  And geographically, have you tried, because I see that you geocode, using geonames, geocode  geography, have you tried making a map to see the density of this?  Well, actually, not directly us.  In Paddyweb, there is no map visualization, but there is some discussion actually to integrate  some map visualization within Paddyweb user interface currently.  But still, there are another partner, a good partner, at Lyon in Montpellier in France.  And they have been doing somewhere to actually show a map, but not only a map, but also a  time-related map.  A time slider to see.  Yes, and a map in order to visualize the outbreaks and so on.  So it can be done with a type of exporting ability that I showed before from Paddyweb.  So the moment you don't have it, right?  You don't have that.  Not within Paddyweb.  It's not a part of Paddyweb.  It's something that has been done outside Paddyweb using the data generated by Paddyweb.  And do you have you tried putting, testing the Paddyweb, how it reacts to noise?  And I know that today there's a lot of work done on detecting inflated information or  spam or something.  So how do you work?  Or is it then done on the Google side because you use the Google news, right?  Actually, what we noticed quite quickly when we made the first prototype for Paddyweb was  that even if we have very precise queries to Google news with some very specific keywords,  disease names and so on, even in that case, there is a majority of articles that are collected  that are not really even at all.  And the way we dealt with that problem was to introduce a classification module.  And the fact is that there is this classification module, which is able, because we provided  it with some labeled data, some examples of redevent and non-relevant articles.  It's able now to make this work of removing everything that is not relevant.  And actually, it's quite good at doing that because if I remember where we have something  like 95% accuracy score for descending whether an article is relevant or not.  So this is something that works pretty good for designing whether an article is relevant.  But then we have also the noise of everything that comes behind, which is mainly the information  extraction module.  Of course, when we sometimes we extract information that is not correct.  Sometimes the location names are not the good ones, for instance, things like this.  So of course, there is the amount of things that we extract, so it's a bit of a part of  the problem because we can see that if a lot of things say that something happens here,  then probably there is something happening here.  And if it is completely an outlier, maybe it's wrong.  But we don't have for now a way to really eliminate wrongly identified expected information.  Okay.  We have a lot of questions in the chat.  So maybe I can read them out to you.  Let's start from the first one from Elena.  Can PadiWeb use other languages besides English?  Like for example, Serbian, since we have also Serbian partners.  And how could it be used from the colleagues in other countries?  Okay.  Well, actually, yes, PadiWeb has really been designed with generosity in mind, which means  also being able to work with different languages, other than English, for instance.  As you saw, we have a translation step in our pipeline, which aims at translating everything  that is not in English into English so that we can work with it.  But it's also completely possible in PadiWeb to just work with another language and decide  that we are working in something different than English, or we're just not translated  everything.  But in that case, the main difficulty actually is not on the PadiWeb site in terms of code  or development.  It's more in the terms of how to feed the machine learning models that we use.  It means that if you want now to use Serbian, then you have to build classification models  for Serbian for deciding whether it's relevant or not and so on.  So this is a lot of work to prepare the machine learning models in order to be able to use  a language.  And this is mainly why we decided to just use one language, because otherwise it's too  much work.  Hi.  Can you give a concrete example of use of PadiWeb developers?  Yes.  Well, actually a very simple example, quite trivial one actually was a few years ago,  I think.  Someone wanted to use the abilities of PadiWeb, the ability to collect data and to index them  and do a classification on data, but they just not wanted to use PadiWeb user interface.  They wanted to have their own module doing that within their own user interface for another  type of project, for instance.  And in that case, they were using the PadiWeb JSON API in order to do that.  So they could use some of the features of PadiWeb, but completely outside the, and some  of the database of PadiWeb as well, completely outside the user interface.  That was one example of usage.  Okay, thank you.  And do you know how many people subscribe to this notification system and if you have  any idea who they are?  Well, actually, I don't know how many there are today.  I know that each time there is a presentation, there are some people asking for a user account  and the kind of course, but I know that, okay, typically the notification emails was designed  for people from the French Epidemic Intelligence Surveillance Team, because every day there  were almost every day their work is to completely go to the computer and search for what has  been happening in the last two days, for instance, what are the news, et cetera, that are relevant  to our problem.  And for them, it's very useful to have this type of notification emails directly in their  emails every day so that they don't have to search by themselves.  They don't have to go to PadiWeb website and do a search on it.  They receive the information directly.  And one last question from the chat.  How do you deal with fake news?  Okay, this is not a problem that we specifically addressed.  So well, I don't have in mind an example of some fake news that we saw appearing in PadiWeb  that could add some noise, for instance, to the data.  I know there are people in the audience from the PadiWeb team have an ID, but for now we're  just not addressing it.  We haven't seen the need for it for now.  Okay, thanks.  And then I see one hand raised from Frank.  Frank, would you like to ask a question?  Yeah.  Yeah.  Thanks, Julian.  And thanks, everyone.  It's really a very nice platform.  I just have one or two questions.  So firstly, on the location, sorry, my name is Frank from ISIDM.  So on the location part, I mean, the geocoding of the data collected at what spatial resolution  is that done?  I mean, at what geographic scale?  Is it at country level, at state level, or at sub-state level?  And then for places whose coordinates or who don't have good geocodes, right?  How do you do to manage that?  I'm based in Nigeria for now, and a couple of places in Nigeria can't be geocoded.  So how do you manage such?  Then on the API, is it tied to the subscriptions?  So does it mean developers have access to the API endpoints for the parts they have subscribed  to on your platform?  Or are the APIs, does the API expose any needed endpoint from a developer?  And finally, how often do you bring together the team of experts, perhaps your modulators,  to review the classification results to be sure of what have been classified correctly  after the initial 500 classified articles?  So those are these few questions.  Thanks.  Okay.  I tried to remember everything as much as possible.  So I think the first question was about geocoding and using geonames, and sometimes it was about  the granularity of the data from geonames, maybe.  And about that, okay, we don't have any pre-specified granularity that we want to use.  We just find names of locations in text, and then we query geonames to know whether there  is something that has this name and this geonames entity.  So we don't have any constraint on the granularity, and this is geonames that give us, we find  a name, and geonames give us information, and such as what type of entity it is, what  type of land of special entity it is.  And this is the kind of information that you have here on the exported data that is completely  provided by geonames.  We don't filter or process this information here.  And if something is not present in geonames, then unfortunately, we just know that potentially  it is a location name because we extract it as a location name, but we don't have any  information about it because geonames won't recognize it.  So that can happen a lot, and that can happen.  And sometimes it shows that the information extraction made a mistake, said that something  was a location name, but it is not.  That can happen.  And sometimes maybe it's just that geonames don't know it.  Okay, so the next question I think was about the JSON API.  The JSON API for now is very simple.  And in order to actually it was designed very specifically for a given project.  And the question of whether it's publicly available for other developers and so on.  Actually, it has to be really discussed with a party web team specifically for each case  because for now we have something that is that protects the API so that it cannot be  used by other people if they are not being authorized to do so.  So you would have to send an email to the party web contact email actually.  And the last question I think was about the classification, how we make sure that the  classifications are correct or not, I think.  And the fact is that in, okay, I try to come back here.  Okay, one thing is that if you're a signed in user on party web, each time you see an  article, you have the list of the classification labels that have been assigned to each article.  So here, for instance, in my example, it is a relevant article and it's an outbreak declaration  according to the automatic classification module.  Now if you see that there is a mistake here, a user can just click on the button and then  it will take precedence and says, okay, this one has been wrongly classified.  And now we consider it as the user said.  And then we have when we train a model automatically every day, actually, we retrain the model,  the classification models.  We have a cross validation process that is here to validate the fact that the classification  model according to the examples we have is doing good work or not.  And we have one more, a couple of questions still from the chat if from Isabelle, if party  web could be used for videos or spoken news.  So not only text.  Well, this is a very interesting question actually because there is a current discussion  currently on working with YouTube videos because indeed they can be a different type  of source.  Currently, there is no way that we're going to do something that is able to recognize  from the voice of something speaking to convert it into something that we can work with.  But on YouTube, there is something very interesting, which is that there are transcriptions, transcripts  for the videos that can be user written or they can be automatically generated.  And we plan maybe to use these kind of transcripts in order to work with video interaction.  Can I also one more question Julia about the data size?  What would be a dream size where you think you could do like also some like very reliable  modeling?  You said you have 400,000 articles.  What would be a dream size where you think that like, I mean, I don't know how much Google  News produces per year articles or would you like to have like a, you know, I don't know,  40 million articles that you can analyze, including video.  So what's what would be a dream size?  Yes.  Well, actually, it really depends on each model has its own requirements in terms of  size of data in order to become reliable.  For instance, for if I take the example of the classification models, okay, the classification  models for redevance, is it redevanced or not?  Okay, there are only two possibilities.  Texts are usually very different from their redevanced or not redevanced.  Okay, we've been able to build a model that is quite effective with quite with not a lot  of data, but for other types of specifications, for instance, it's much more difficult, but  it's also much more difficult to build data sets, user labeled data sets that will be  really useful.  So well, I'd say I would like to have as much data as possible, of course, but it means  that actually the actual work, then the actual effort is on the experts that have to manually  label the data.  And for instance, I have an example of the model that we built for the information extraction  module.  It was about 500 articles, which is not that huge in terms of machine learning data set.  But Elena Afebska actually did the work of labeling each entity within each text, and  it was a huge, huge task.  So yes, we have to find the right balance between visibility for users, for experts,  and usefulness for the models.  So while there is not one right answer to that, I'm afraid.  And the historic news articles, like most of your articles, like articles that Google  News harvest is just digital news articles, but they're also historic articles.  And I know Google digitized the books, right?  And the European libraries digitized the books.  What about the historic news articles going like 20, 30 years back?  Would this be something interesting for Padiweb also?  Well, I guess for the epidemiologist, it could be something of interest to see how things  evolved over time.  For the developer point of view, from my point of view, yes, it could be feasible if we can  have an access at some time in terms of raw text that we can process.  And this is the main, the most important thing for us is to be able to convert everything  into some text that we can process.  Initially Padiweb comes from collaboration between some epidemiologists from the French  Epidemic Intelligence team and some computer scientists.  And while the epidemiologists have to find potential threats to animal health by monitoring  some unofficial news articles on the web and some computer scientists, then so that maybe  it was possible to automate this part of the process by collecting articles, by processing  them automatically.  And the idea was to build an automated pipeline, exploiting some works in machine learning  in order to provide something that we felt would really help the epidemiologists in their  daily tasks.  So that was the main idea behind the platform, Padiweb, and that was how the first prototype  for Padiweb was developed a few years ago.  And since then, a lot of features have been added or improved in Padiweb, such as the  article classification module, for instance.  And actually also by discussing with other research groups by other people, we realized  that this type of platform like Padiweb would be really useful to a lot of domains of applications,  of applications.  So monitoring news or documents and the automaticity, having an idea of the information they  contain, this is something that had to be done for multiple applications, actually not  only for animal epidemiology.  Well, yes, there are very different channels of distribution for the information that has  been extracted by Padiweb.  And it is because we have different types of users and they expect different things from  Padiweb.  We have different types of those channels of distribution.  So the first one is about domain experts, so epidemiologists, for instance, from the  French Epidemic Surveillance Team.  And they are interested in following almost daily what is happening so that they can have  a feel of the current disease outbreaks and also they need to be able to react quickly  if something of PachiBrain interest happens.  And another thing is that they don't want to go to Padiweb every day to research the  new article.  So that's why we have the notifications, notification emails in Padiweb that are here to provide  them directly with some of the information that has been collected lately during the  last 24 hours, for instance.  And then if something of interest appears in the notification email, they can still go  to Padiweb and then do some research about the thing that happened.  And then we have a completely different set of users that are data scientists, that are  computer scientists, that are developers.  And for them, actually, they are more interested in the data collected by Padiweb and extracted  by Padiweb, but they want to be able to analyze the data, process them differently than what  we do in Padiweb.  So for instance, one example is about building map visualizations given the break data that  Padiweb has extracted.  And in this case, we provide to users, these type of users, with exporting files in different  formats that can be processed by them.  So these are the main ways to distribute information that is collected by Padiweb.  Well, yes, of course, there are lots of plans to improve Padiweb because it's becoming a  very large, very big platform.  And well, for almost every step that is presented in Padiweb, there is a way to improve it.  So there are maybe different levels of improvement that we can work on.  One is to improve each module individually.  So maybe improve the information extraction module, maybe improve the collection process  and so on.  One example of this type of improvements is the geolocation geocoding part of Padiweb,  which is about extracting spatial information from the articles.  And here we identified some weaknesses in our process.  So currently there is some, for instance, some PhD students who are currently working  on this problem trying to find different ways to do it.  And later on, maybe we will integrate this type of work into Padiweb in order to improve  it.  And then I think there are another level of improvement, which is very important here.  It's about improving the data sets, the data that we use in order to feed the different  machine learning models that we have in Padiweb.  So for instance, the classification models, they really rely on the type of data that  we gave them as examples.  And because it is difficult to ask experts to do that a lot, it's a lot of efforts from  the experts to manually build data sets to do something that is still very important.  And I think this is one of the main things that we really need to do in the future to  keep Padiweb pretty good.
MOOD_science_webinar_Julien_Rabatel_22March2022__2__0_131.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__132_251.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__252_695.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__696_1967.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__1968_4439.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__4440_5891.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__5892_6167.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__6168_6419.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__6420_8531.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__8532_12119.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__12120_14783.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__14784_17927.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__17928_19115.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__19116_21347.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__21348_22031.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__22032_23003.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__23004_23375.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__23376_24107.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__24108_38963.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__38964_44651.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__44652_44699.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__44700_52991.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__52992_53351.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__53352_53783.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__53784_53915.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__53916_54707.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__54708_55283.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__55284_56771.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__56772_56879.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__56880_58691.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__58692_59435.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__59436_60263.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__60264_61055.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__61056_61103.png MOOD_science_webinar_Julien_Rabatel_22March2022__2__61104_61676.png