Finnish National Bibliography Fennica as Linked Data
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Suominen, Osma
The National Library of Finland is making our national bibliography Fennica available as Linked Open Data. We are converting the data from 1 million MARC bibliographic records first to BIBFRAME 2.0 and then further to the Schema.org data model. In the process, we are clustering works extracted from the bibliographic records, reconciling entities against internal and external authorities, cleaning up many aspects of the data and linking it to further resources. The Linked Data set is CC0 licensed and served using HDT technology. The publishing of Linked Data supports other aspects of metadata development at the National Library. For some aspects of the Linked Data, we are relying on the RDA conversion of MARC records that was completed in early 2016. The work clustering methods, and their limitations, inform the discussions about potentially establishing a work authority, which is a prerequisite for real RDA cataloguing. This presentation will discuss lessons learned during the publishing process, including the selection and design of the data model, the construction of the conversion pipeline using pre-existing tools, the methods used for work clustering, reconciliation and linking as well as the infrastructure for publishing the data and keeping it up to date.
 Yes, I'm Osmo Soaminin. Hello again. Yeah, I'm going to talk about the Finnish National Bibliography Fennika as link data. So basically, this is the mission that I'm trying to accomplish here, that I'm trying to make our National Bibliography a bit more web-ish, but not too web-ish, so just to write the amount. And I'm still not sure how long it will take, but there's been some progress since last year, so I'm going to present that. So why are we doing this? Well, first of all, we want to make our data, our metadata, more visible also internationally, because right now it's locked down in silos, basically. And in order for libraries to stay relevant in the web-based world, I think it's important for them to share what they have. And of course also to, this is a great exercise for finding out problems with the quality of data and an opportunity to make it more interoperable with the rest of the world. And we're also sort of as an institution trying to prepare for the future, because it, well, yeah, I was pointed out yesterday that it's 15 years since Mark was sort of, yeah, Mark must die again, but it still hasn't died, but eventually there will be something else. And I think this is sort of exercising, trying to find out what the issues are with the current data and how to move it to some other, more modern model. And of course, why not do this? Because, yeah, we're here. So this is basically what we started with. This is our snowflake. I don't know if you can see it there in the background. But we had, in the national bibliography, which is mostly about books, serials, e-books, maps, we have one million bibliographic records. So this is our main bibliographic database, but we have several others. For example, we have a separate database for music. So that's not included here. And we have some authority records. So we have person names and corporate names, and then we have subject records. And these are all Mark records. But we want to turn this into a graph, something like this, to find out the main entities there, the works, the basis, organizations, subjects, persons. But in order to do this, it's not a straightforward mapping, as you probably know. So you basically need to blow up all the Mark records and then reassemble all the bits and pieces into something that looks a bit like this marshmallow and toothpick structure. So you build a graph out of the pieces that you get from the Mark records. But how to do this, what's the best approach to find out how to do some research? And that's sort of culminated in my last year's talk at SWIP. Some of you might remember this diagram and picture with the silos. So I gave a talk called, from Mark silos to link data silos, and then did a follow-up webinar with DCMI, and then finally a journal article. But basically, the point was just to review all the different approaches that had been used in different libraries to make there a bibliographic metadata, but make it into link data. And it turned out that everybody is basically doing it differently. And yeah. But we tried to learn from that, and sort of, since we're not exactly the first ones to do this, so it was possible to learn from the others. And basically, we found out that for our purposes, schema.org would be a good model because it's sort of allows us to describe our resources from a common sense for web users perspective. So it means that, but we still wanted to do it in sort of a bit more advanced than just converting one record at a time. So we wanted to do separate out the works from the instances in the sense of bib frame. So, and it's possible to do that with the bibliographic extensions of schema. And, but it means that we're not converting every detail. It's, in a sense, we get a metadata haircut for free because we're just, at least at the moment, we're trying to find out what are the main entities, but some of their attributes or fields are still not converted. And we got some help from Richard Wallace in planning this. But how to get from the mark to the link data? So we need something like a black box where you can put in mark and then you get out link data. And I looked at the alternatives for, there are various tools for doing this, but none of them seemed just right for our purposes. There were some, some, but there were some interesting approaches and good tools that sort of got you part of the way, but not all the way. So we decided to build this pipeline for conversion that sort of stitches together existing tools as much as possible and then sort of glues them together and then adds the missing parts. And we started from a dump of our database. It's an ALEF system. So the easiest way was to start with an ALEF sequence. It's basically a textual format for mark records. So one million records in the form of an ALEF sequence. And then convert them using a tool called Katmandu, which is like a Swiss Army knife for metadata, into mark XML and also do some fixes there on at the same time. And then from mark XML, we could use the Library of Congress mark to be a friend to converter. We started with the earlier one, the mark to be a friend, but then switched to this one as soon as it was released. So we get in basically two steps we already get to BIB frame RDF. But it's pretty verbose and it's not really linked anywhere even within the dataset. So there's a lot of internal duplication in there and yeah, it doesn't have the URIs for the things that are referenced. We would like there. Anyway, from there, we could start using RDF technologies. So I made this monster sparkle query to turn the BIB frame into schema, basic schema without any external links yet. And so we have it in a basic schema format. And then another sparkle to connect that with external vocabularies and like resources. So some of them were our own like the subjects and the corporate names, but we also linked to the RDA media content and carrier vocabularies because we're already using RDA in our mark records. So it was just getting the right URIs into the RDF version. But the problem is that within there, there's still a lot of internal duplications. So for example, one record becomes at least one work. And that doesn't seem right because often you have many additions of the same work or you have a print and an ebook or something. And to bring these together, I had to create some work piece that tried to connect the work entities that actually represent the same intellectual work and then to merge these into the same entity, again, using sparkle mostly. But then we still have a problem with the persons because, well, many of the sort of the important authors are listed in the person names authority and we could use their identifiers. But some are not and there's a big long tail of people mentioned in the mark records which we don't have identifiers for. And it's when you merge the works, you also get sort of many duplicates of the same person. And to merge these, we also created keys for them and tried to deduplicate some of them. But it's difficult because you can't be sure that if you just have a name and the same name is mentioned in many mark records, without any authority control, you can't be sure whether it's the same person or not. So we have to be a bit careful there and sort of err on the safe side which means that we have a duplicate. We have some duplicates there still. Anyway, then the result we can put in an RDF store. And from there on, we can then publish it. So we have this, a lot of systems working together to make this available for humans and machines. So first of all, we have this triple store based on Fuseki which provides its workland point. And then a small application that makes this available also as web pages and the link data and an open search API. And then also we provide the same data as downloads. And one of the download formats is Htt which is very convenient for publishing this sort of a static data set in a compressed RDF file. And we can also serve that using a link data fragment server. So it's an alternative to using the Spark Lendpoint when you want to do complex queries that might otherwise bring down the Spark Lendpoint. You can move some of the processing over to the client side using link data fragments. Okay. So this is the publishing. I'm going to show a little demo of this. Bear with me. So this is... Make it a little bigger. So this is how it looks on the web. Of course, the main point is the data, not the user interface. This is just for taking a look at, I mean, getting familiar with the data. This is IonLithit history. This is a finished translation of Stephen Hawking's work, A Brief History of Time. And up here we have the work attributes. So what it's about, who is the author and so on. And then down here we have several instances. So the first edition in Finnish was published in 1988. But then we have several other editions published later. And the last one is an e-book in 2012. And these have all been brought together into sort of that they are instances of the same work. Which wasn't, of course, the case in Mark because each one of them was a separate Mark record. But we can also, I mean, this is not very user-friendly and it doesn't have any holdings information. So for somebody using this who wants to know more, we can also go to FINN now, which is the main discovery interface for all kinds of cultural heritage. And which has more information based. This is just, this is not linked data, but it provides access to the holdings information, for example. So it's sort of linked to the more user-friendly but less linked UI. Okay. Or then we can look at, but this is browsable. We can, all of these are links because this is linked data. So we can go and look at Stephen Hawking. And we can see what he has authored, what he has contributed to, and works about Stephen Hawking. So everything here is sort of linked together and browsable. It's very different from FINN because FINN now, the main interface is just a search box. So I don't know if this qualifies as a spelunking UI. Maybe not because it, yeah, it doesn't have all those nice visualizations. But anyway, the point is not to search, even though there is a search box to get started. And also, this work is a translation, so it's linked to the original work, a brief history of time. And if there were other translations, they would also be shown here. We can also look at the subjects here. So this one is about, for example, black holes. So black holes are cool. Okay. There is some information about the subject itself. This is the SCOS information. But when we scroll down, we find all the works about black holes. So this way, you can also browse by subject. This is also different. I mean, in FINN, you would have to search for the subject. Maybe not so different. But anyway, it's, yeah. And we can take any of these, like, these one aspects of quantum fields and strings. This is maybe a thesis or something. And again, we get the instances. But this is all data, so we got links to other representations. So for example, we can look at this in JSON-LD. And I mean, you probably can't see it, but the URL ends with that JSON. So this is, I mean, at the same time, in the same way, we can look at RDF XML here. So it's all provided. And the data is embedded. I can't see it, but it's somewhere here. It's embedded within the page, so that as a script tag. So a search engine could pick it up from there as well. And we can, yeah, there is a basic search box, which is also provided as open search. So we can search for Mark Twain and find out, well, we have a few duplicates of the person Mark Twain, and then we have some works about him. But we can look at what we have about him in this database. Okay. Getting back to my presentation. This is also available as downloadable dumps. So these are our hairballs. If you want to do something more interesting, just go and pick it up from there, if you like. And there are also, the Mark records are also provided there. So if you want to look at that, instead, it's possible. And yeah, I already mentioned we have a linked data fragment server, just very experimental, but it was easy to set up. So it's possible to use that. This is the full data model currently. So you can see there's not that many attributes in here. The main point is to get the entities right. And the main classes here are work and instance. And then there are some relationships between them. And this is basically just the same division as in the frame. And then we have person, organization, publication event, place. And then we have the concepts that are the subjects. Yeah. And then we have some series. And there's a full documentation there, if you're interested in the form of tables. Some of the challenges here in this, doing this, first of all, the work extraction is really hard to do properly. I mean, in principle, it's like this, that you extract the works using some kind of work keys from the Mark records. And then this will never be right, because the metadata is always a little messy and there are problems with older records, for example, or missing information about original works. So you have to, probably you would have to eventually make a work authority to actually have stable identities for those works. And then start using it and maintaining it for the purposes of cataloging. This is what I would hope to do eventually, but currently we're at step one. And it's not, I mean, it's hard to find the resources and the motivation to do this. Although, I mean, some libraries like the Swedish National Library and also in Norway and Germany, I know there are projects around this, doing this. So yeah, I hope eventually you would get some kind of use of this, but it seems like a big investment to get started. So it's not so easy to do in practice. Then the linking, this is, I mean, the blue blob there in the middle is basically this bibliographic data set. And the yellow boxes are the things that we currently link to. So the subjects are linked to the Library of Congress subject headings. Places are linked to the finished place in registry and Wiki data currently. We're still working on the Wiki data mapping, but it's pretty far along. So this could be thought in terms of the stars, link data or the data stars, maybe we're at four stars now because there's no really links for the works and the instances which are the core entities. But we could, I mean, potentially link, for example, the persons could be linked to Istni and VF and the works could be linked to WorldCat works or, for example, the Libris Excel system, which is coming in Sweden's pursuit. They use the bit frame approach for works so we could link to their works or we could link the instances to WorldCat and or to other national libraries, for example, using ISBNs and as keys for the linking. Yeah, so these are some just potential ideas for linking. Then there's the problem with persistence of identifiers because the data is still being maintained as marked records. And the conversion is done in a way that we can do it all over again tomorrow. But and it's stable as long as the data is unchanged, but there are lots of things that have to be given identifiers, not just the records themselves, but entities extracted from the records. So, and if the records change, for example, two records get merged together or somebody adds a new contributor information or something, then it's often the identifiers in the result also change. So it's like trying to build a castle on sand. So it keeps moving and it's hard to maintain the persistence. So this is something that I don't really have a good solution for. Of course, one way would be is to push back those identifiers into the marked records themselves so that they would be maintained in the same place as the data itself. This would help, but you can't do that in all cases. I think there are some facilities for doing this, like the zero, subfield zero and subfield one. But it's not doesn't work for all cases. This is open data. We could look at it from a fair perspective. Is it findable? Yes. Well, we use your eyes. We have rich metadata. Is it accessible? We provide several ways of accessing it. Is it interoperable? Well, we use RDF, which is standard when we use schema, which is a standard, a little bit of RDAU. It's easy zero licensed and we use this entities that are also used in other databases. So we hope that it's somewhat reusable at least. What to do next? Well, first of all, want to continue enriching and cleaning the RDF data. For example, maps are now just creative works, but they could be schema maps. And to add more links to other link datasets, like I already said, and then to expand this the same idea to other datasets. For example, the music discography viola and the article database. It happens to be the 100 year anniversary of Finland today. So this is also my birthday present. We have this this logo everywhere right now. Like you see it on when you buy milk or something. It's all it's everywhere. Okay, but this was my present. Thank you. Thank you, Osma. Any questions? Thank you. Thank you for this talk. This was very nice to see. And for me, although it's not very important for you, it's the user interface. I found it quite nice. Especially you could see all these links and for people like us, it's clear that this is linked data. But I think many users, they don't know what link data are. And I think basically you have something from it that this is linked data. But do you tell the users that this is linked data or do you ask them anything? Well, I guess first of all, we have to know who is the user here. And it's a new service. So right now we don't have any. I mean, you could be. But it says link data service right at the top. So yeah, in that way, it's telling that this is linked data. But I don't know what else could we do to make the point clear. If you have ideas, then I'm very happy. Yeah. One question which came to my mind is you have some, you have to do a lot of teamwork with the works and the deduplication. Do you consider opening an opportunity for users to take part in this work? Kind of a crowdsourcing approach. So let's check if these three might wins are the same person or not. Is anything that as a consideration to open this kind of work up? That's a good question. And we haven't considered that approach. I think the challenge is sort of relating the data here, which is the sort of the end result of a pretty long chain of operations and trying to relate that back to the original data. So, well, in the case of Mark Twain, it would be actually fairly simple. What it would mean is that the original mark records, they should be enhanced with the person identifier, which was missing. And that's why those duplicates were created. So, but this is also something that we are sort of doing internally. So having those identifiers there in the mark records is actually a very new thing. They haven't been there for like two months, maybe. So before opening up that into crowdsourcing, I think we first have to see what we can do internally. Like, we still haven't picked all the low-hanging fruits in that area. We're just getting started. So, but yeah, that's a good idea. And we will think about it. If you were to, Osmo, if you were to run the conversion again tomorrow, what kind of hurdle would that be? So how are these different pieces connected to each other? Or is there one script that you can use or how does it work? I can easily run it anytime. The full conversion takes about five hours and then the loading to the triple store and generating all the downloads and stuff. That takes one more hour, so maybe about six hours. And the result should be the same if the records have not changed. So all the links will be generated the same way and all the URIs should be the same. Did that answer the question? Or, yeah, okay. So just a webby question. So with the frame, there's the activity pattern and your schema use shortcut properties. Was that a deliberate decision when you could have used actions, schema actions? Or could you say a little bit more about that decision? Yeah, I would say we just took the easiest approach. And in this case, this was the direct property. And also, it's because in the data there is, for example, those roles, who is a translator and who's an illustrator, those are only available in some of the records and not all of them. I mean, it doesn't prevent you from using the more advanced pattern, but it just makes it less useful, I think. I think we could do some of that with sub-properties that we could say, for example, that this person is not only a contributor but also illustrator or translator and assert those both properties. Okay, hello Esmo. Thanks for the presentation. When you're linking, when you would be linking the mark records or blown up mark records to other databases, let's say to other national libraries to find the similar items, would you inject the links back to the original mark records? Yeah, I think that would make sense. I haven't really thought that far, but right now there is basically all the data is coming from somewhere. This system itself, well, it has a few triples about this dataset, but other than that, everything comes from a data source. And I think for those links, we would have to decide where to put them and it would probably make sense to put them in the mark records rather than somewhere else. Thank you Rosmar. It's very nice to follow your work. I've got a question about if you're planning to integrate more kinds of relations between your works, like aggregations or works that are based on other works or that kind of links in your mark record. Yes, I would hope to do that, but I haven't really looked very deeply at what would be possible based on the records we have, because of course it has to be, I mean, we have to get the information from somewhere. So I mainly concentrated, in terms of work-to-work relations, I mainly concentrated on the translation relationship, because it seemed to be by far the most common that was in the records, and it's also sort of interesting from a user perspective and from, I mean, being a national library of a small country where a lot of the literature is a translation of something else. Thank you so much, Osmar, for these insights. I think it's very inspiring how you use the bit frame core classes to group the things and then use schema to comply to the search engines. I think this should really increase the interoperability of the data, considering that data models are harder to overcome than individual properties that can be mapped. Was it something you did deliberately to be able to align to other datasets likely to come up in bit frame, or is it just because it fitted the structure you wanted to achieve? I wouldn't say those two options are contradictory, but I think it's what's more about that we saw the problems of duplication when you have a large number of marked records, which are basically about the same work. And especially when it comes to, when you start to have lots of electronic versions or digitized versions of the same work, so we have sort of more and more of these marked records that are disconnected even though they are about the same thing. So it seemed to be useful to do something in this area, and rather than chose just a flat model that maps one marked record to one entity in the RDF to do something a little bit more advanced with works. And so the, also the available tools of course affected so this decision. So having the bit frame converter available from the Library of Congress and seeing how it seems to give you at least the starting point for doing this grouping or clustering by work, it sort of seemed like a good idea to try to apply here. Okay, one last question before we go into the coffee break. Thank you for your talk. And I'm wondering, it's a very nice dataset I think. Do you have any idea who will be your users or are you planning to do any kind of marketing for special user groups to take advantage of that data? Well, yes, this is actually, this is part of a larger opening of our data. So the data, in addition to this linked dataset, we are, we have created a data catalog. Because we already provide lots of many open APIs into, in more traditional APIs usually. But they were all sort of not very well documented and sort of hidden in specific systems. And so we wanted to, we were also encouraged by the ministries and so on that want to see our data put out in the open. So we created this data catalog that tries to document all the datasets and all the APIs we have. And this is one of them or actually several of them because there are many APIs available for the same dataset. So and we are trying to, but we haven't yet really, I mean, this was the official launch in a way for the linked dataset and we will announce the open data service sometime next week and try to organize also maybe some event around it to make it more known to developers. So yes, we are trying to get some publicity for this. Okay. Thanks again, Osmar.
 Yes, I'm Osmo Soaminin. Hello again. Yeah, I'm going to talk about the Finnish National  Bibliography Fennika as link data. So basically, this is the mission that I'm trying to accomplish  here, that I'm trying to make our National Bibliography a bit more web-ish, but not too  web-ish, so just to write the amount. And I'm still not sure how long it will take, but  there's been some progress since last year, so I'm going to present that. So why are we  doing this? Well, first of all, we want to make our data, our metadata, more visible  also internationally, because right now it's locked down in silos, basically. And in order  for libraries to stay relevant in the web-based world, I think it's important for them to  share what they have. And of course also to, this is a great exercise for finding out problems  with the quality of data and an opportunity to make it more interoperable with the rest  of the world. And we're also sort of as an institution trying to prepare for the future,  because it, well, yeah, I was pointed out yesterday that it's 15 years since Mark was  sort of, yeah, Mark must die again, but it still hasn't died, but eventually there will  be something else. And I think this is sort of exercising, trying to find out what the  issues are with the current data and how to move it to some other, more modern model.  And of course, why not do this? Because, yeah, we're here. So this is basically what we started  with. This is our snowflake. I don't know if you can see it there in the background.  But we had, in the national bibliography, which is mostly about books, serials, e-books,  maps, we have one million bibliographic records. So this is our main bibliographic database,  but we have several others. For example, we have a separate database for music. So that's  not included here. And we have some authority records. So we have person names and corporate  names, and then we have subject records. And these are all Mark records. But we want to  turn this into a graph, something like this, to find out the main entities there, the works,  the basis, organizations, subjects, persons. But in order to do this, it's not a straightforward  mapping, as you probably know. So you basically need to blow up all the Mark records and then  reassemble all the bits and pieces into something that looks a bit like this marshmallow and  toothpick structure. So you build a graph out of the pieces that you get from the Mark records.  But how to do this, what's the best approach to find out how to do some research? And that's  sort of culminated in my last year's talk at SWIP. Some of you might remember this diagram  and picture with the silos. So I gave a talk called, from Mark silos to link data silos,  and then did a follow-up webinar with DCMI, and then finally a journal article. But basically,  the point was just to review all the different approaches that had been used in different  libraries to make there a bibliographic metadata, but make it into link data. And it turned out  that everybody is basically doing it differently. And yeah. But we tried to learn from that,  and sort of, since we're not exactly the first ones to do this, so it was possible to learn  from the others. And basically, we found out that for our purposes, schema.org would be  a good model because it's sort of allows us to describe our resources from a common sense  for web users perspective. So it means that, but we still wanted to do it in sort of a  bit more advanced than just converting one record at a time. So we wanted to do separate  out the works from the instances in the sense of bib frame. So, and it's possible to do  that with the bibliographic extensions of schema. And, but it means that we're not converting  every detail. It's, in a sense, we get a metadata haircut for free because we're just, at least  at the moment, we're trying to find out what are the main entities, but some of their attributes  or fields are still not converted. And we got some help from Richard Wallace in planning  this. But how to get from the mark to the link data? So we need something like a black  box where you can put in mark and then you get out link data. And I looked at the alternatives  for, there are various tools for doing this, but none of them seemed just right for our  purposes. There were some, some, but there were some interesting approaches and good  tools that sort of got you part of the way, but not all the way. So we decided to build  this pipeline for conversion that sort of stitches together existing tools as much  as possible and then sort of glues them together and then adds the missing parts. And we started  from a dump of our database. It's an ALEF system. So the easiest way was to start with  an ALEF sequence. It's basically a textual format for mark records. So one million records  in the form of an ALEF sequence. And then convert them using a tool called Katmandu,  which is like a Swiss Army knife for metadata, into mark XML and also do some fixes there  on at the same time. And then from mark XML, we could use the Library of Congress mark  to be a friend to converter. We started with the earlier one, the mark to be a friend,  but then switched to this one as soon as it was released. So we get in basically two  steps we already get to BIB frame RDF. But it's pretty verbose and it's not really linked  anywhere even within the dataset. So there's a lot of internal duplication in there and  yeah, it doesn't have the URIs for the things that are referenced. We would like there.  Anyway, from there, we could start using RDF technologies. So I made this monster sparkle  query to turn the BIB frame into schema, basic schema without any external links yet. And  so we have it in a basic schema format. And then another sparkle to connect that with  external vocabularies and like resources. So some of them were our own like the subjects  and the corporate names, but we also linked to the RDA media content and carrier vocabularies  because we're already using RDA in our mark records. So it was just getting the right  URIs into the RDF version. But the problem is that within there, there's still a lot  of internal duplications. So for example, one record becomes at least one work. And that  doesn't seem right because often you have many additions of the same work or you have  a print and an ebook or something. And to bring these together, I had to create some  work piece that tried to connect the work entities that actually represent the same intellectual  work and then to merge these into the same entity, again, using sparkle mostly. But then  we still have a problem with the persons because, well, many of the sort of the important  authors are listed in the person names authority and we could use their identifiers. But some  are not and there's a big long tail of people mentioned in the mark records which we don't  have identifiers for. And it's when you merge the works, you also get sort of many duplicates  of the same person. And to merge these, we also created keys for them and tried to deduplicate  some of them. But it's difficult because you can't be sure that if you just have a name  and the same name is mentioned in many mark records, without any authority control, you  can't be sure whether it's the same person or not. So we have to be a bit careful there  and sort of err on the safe side which means that we have a duplicate. We have some duplicates  there still. Anyway, then the result we can put in an RDF store. And from there on, we  can then publish it. So we have this, a lot of systems working together to make this available  for humans and machines. So first of all, we have this triple store based on Fuseki  which provides its workland point. And then a small application that makes this available  also as web pages and the link data and an open search API. And then also we provide the  same data as downloads. And one of the download formats is Htt which is very convenient for  publishing this sort of a static data set in a compressed RDF file. And we can also  serve that using a link data fragment server. So it's an alternative to using the Spark  Lendpoint when you want to do complex queries that might otherwise bring down the Spark  Lendpoint. You can move some of the processing over to the client side using link data fragments.  Okay. So this is the publishing. I'm going to show a little demo of this. Bear with me.  So this is... Make it a little bigger. So this is how it looks on the web. Of course,  the main point is the data, not the user interface. This is just for taking a look at, I mean,  getting familiar with the data. This is IonLithit history. This is a finished translation of  Stephen Hawking's work, A Brief History of Time. And up here we have the work attributes.  So what it's about, who is the author and so on. And then down here we have several  instances. So the first edition in Finnish was published in 1988. But then we have several  other editions published later. And the last one is an e-book in 2012. And these have all  been brought together into sort of that they are instances of the same work. Which wasn't,  of course, the case in Mark because each one of them was a separate Mark record. But we  can also, I mean, this is not very user-friendly and it doesn't have any holdings information.  So for somebody using this who wants to know more, we can also go to FINN now, which is  the main discovery interface for all kinds of cultural heritage. And which has more information  based. This is just, this is not linked data, but it provides access to the holdings information,  for example. So it's sort of linked to the more user-friendly but less linked UI. Okay.  Or then we can look at, but this is browsable. We can, all of these are links because this  is linked data. So we can go and look at Stephen Hawking. And we can see what he has authored,  what he has contributed to, and works about Stephen Hawking. So everything here is sort  of linked together and browsable. It's very different from FINN because FINN now, the  main interface is just a search box. So I don't know if this qualifies as a spelunking  UI. Maybe not because it, yeah, it doesn't have all those nice visualizations. But anyway,  the point is not to search, even though there is a search box to get started. And also,  this work is a translation, so it's linked to the original work, a brief history of time.  And if there were other translations, they would also be shown here. We can also look  at the subjects here. So this one is about, for example, black holes. So black holes are  cool. Okay. There is some information about the subject itself. This is the SCOS information.  But when we scroll down, we find all the works about black holes. So this way, you can also  browse by subject. This is also different. I mean, in FINN, you would have to search for  the subject. Maybe not so different. But anyway, it's, yeah. And we can take any of these,  like, these one aspects of quantum fields and strings. This is maybe a thesis or something.  And again, we get the instances. But this is all data, so we got links to other representations.  So for example, we can look at this in JSON-LD. And I mean, you probably can't see it, but  the URL ends with that JSON. So this is, I mean, at the same time, in the same way,  we can look at RDF XML here. So it's all provided. And the data is embedded. I can't see it,  but it's somewhere here. It's embedded within the page, so that as a script tag. So a search  engine could pick it up from there as well. And we can, yeah, there is a basic search box,  which is also provided as open search. So we can search for Mark Twain and find out, well,  we have a few duplicates of the person Mark Twain, and then we have some works about him.  But we can look at what we have about him in this database. Okay. Getting back to my  presentation. This is also available as downloadable dumps. So these are our hairballs. If you  want to do something more interesting, just go and pick it up from there, if you like.  And there are also, the Mark records are also provided there. So if you want to look at that,  instead, it's possible. And yeah, I already mentioned we have a linked data fragment server,  just very experimental, but it was easy to set up. So it's possible to use that.  This is the full data model currently. So you can see there's not that many attributes in  here. The main point is to get the entities right. And the main classes here are work and  instance. And then there are some relationships between them. And this is basically just the  same division as in the frame. And then we have person, organization, publication event, place.  And then we have the concepts that are the subjects. Yeah. And then we have some series.  And there's a full documentation there, if you're interested in the form of tables.  Some of the challenges here in this, doing this, first of all, the work extraction is really hard  to do properly. I mean, in principle, it's like this, that you extract the works using some kind  of work keys from the Mark records. And then this will never be right, because the metadata is  always a little messy and there are problems with older records, for example, or missing information  about original works. So you have to, probably you would have to eventually make a work authority  to actually have stable identities for those works. And then start using it and maintaining it  for the purposes of cataloging. This is what I would hope to do eventually, but currently we're  at step one. And it's not, I mean, it's hard to find the resources and the motivation to do this.  Although, I mean, some libraries like the Swedish National Library and also in Norway and Germany,  I know there are projects around this, doing this. So yeah, I hope eventually you would get some kind  of use of this, but it seems like a big investment to get started. So it's not so easy to do in  practice. Then the linking, this is, I mean, the blue blob there in the middle is basically this  bibliographic data set. And the yellow boxes are the things that we currently link to. So the subjects  are linked to the Library of Congress subject headings. Places are linked to the finished  place in registry and Wiki data currently. We're still working on the Wiki data mapping,  but it's pretty far along. So this could be thought in terms of the stars, link data or the data  stars, maybe we're at four stars now because there's no really links for the works and the  instances which are the core entities. But we could, I mean, potentially link, for example,  the persons could be linked to Istni and VF and the works could be linked to WorldCat works or,  for example, the Libris Excel system, which is coming in Sweden's pursuit. They use the  bit frame approach for works so we could link to their works or we could link the instances to  WorldCat and or to other national libraries, for example, using ISBNs and as keys for the linking.  Yeah, so these are some just potential ideas for linking. Then there's the problem with  persistence of identifiers because the data is still being maintained as marked records. And  the conversion is done in a way that we can do it all over again tomorrow. But and it's stable as  long as the data is unchanged, but there are lots of things that have to be given identifiers,  not just the records themselves, but entities extracted from the records. So, and if the  records change, for example, two records get merged together or somebody adds a new  contributor information or something, then it's often the identifiers in the result also change.  So it's like trying to build a castle on sand. So it keeps moving and it's hard to maintain the  persistence. So this is something that I don't really have a good solution for. Of course,  one way would be is to push back those identifiers into the marked records themselves so that they  would be maintained in the same place as the data itself. This would help, but you can't do that in  all cases. I think there are some facilities for doing this, like the zero, subfield zero and  subfield one. But it's not doesn't work for all cases. This is open data. We could look at it  from a fair perspective. Is it findable? Yes. Well, we use your eyes. We have rich metadata. Is it  accessible? We provide several ways of accessing it. Is it interoperable? Well, we use RDF, which is  standard when we use schema, which is a standard, a little bit of RDAU. It's easy zero licensed and  we use this entities that are also used in other databases. So we hope that it's somewhat  reusable at least. What to do next? Well, first of all, want to continue enriching and cleaning  the RDF data. For example, maps are now just creative works, but they could be schema maps.  And to add more links to other link datasets, like I already said, and then to expand this the  same idea to other datasets. For example, the music discography viola and the article database.  It happens to be the 100 year anniversary of Finland today. So this is also my birthday present.  We have this this logo everywhere right now. Like you see it on when you buy milk or something.  It's all it's everywhere. Okay, but this was my present. Thank you. Thank you, Osma. Any questions?  Thank you. Thank you for this talk. This was very nice to see. And for me, although it's not very  important for you, it's the user interface. I found it quite nice. Especially you could see all  these links and for people like us, it's clear that this is linked data. But I think many users,  they don't know what link data are. And I think basically you have something from it that this  is linked data. But do you tell the users that this is linked data or do you ask them anything?  Well, I guess first of all, we have to know who is the user here. And it's a new service. So  right now we don't have any. I mean, you could be. But it says link data service right at the top.  So yeah, in that way, it's telling that this is linked data. But I don't know what else could  we do to make the point clear. If you have ideas, then I'm very happy. Yeah. One question  which came to my mind is you have some, you have to do a lot of teamwork with the works and  the deduplication. Do you consider opening an opportunity for users to take part in this work?  Kind of a crowdsourcing approach. So let's check if these three might wins are the same person or  not. Is anything that as a consideration to open this kind of work up? That's a good question.  And we haven't considered that approach. I think the challenge is sort of relating the data here,  which is the sort of the end result of a pretty long chain of operations and trying to relate  that back to the original data. So, well, in the case of Mark Twain, it would be actually fairly  simple. What it would mean is that the original mark records, they should be enhanced with the  person identifier, which was missing. And that's why those duplicates were created. So, but this  is also something that we are sort of doing internally. So having those identifiers there in  the mark records is actually a very new thing. They haven't been there for like two months,  maybe. So before opening up that into crowdsourcing, I think we first have to see what we can do  internally. Like, we still haven't picked all the low-hanging fruits in that area. We're just  getting started. So, but yeah, that's a good idea. And we will think about it.  If you were to, Osmo, if you were to run the conversion again tomorrow, what kind of hurdle  would that be? So how are these different pieces connected to each other? Or is there  one script that you can use or how does it work? I can easily run it anytime. The full  conversion takes about five hours and then the loading to the triple store and generating  all the downloads and stuff. That takes one more hour, so maybe about six hours. And the  result should be the same if the records have not changed. So all the links will be generated  the same way and all the URIs should be the same. Did that answer the question?  Or, yeah, okay. So just a webby question. So with the frame, there's the activity pattern  and your schema use shortcut properties. Was that a deliberate decision when you could  have used actions, schema actions? Or could you say a little bit more about that decision?  Yeah, I would say we just took the easiest approach. And in this case, this was the direct  property. And also, it's because in the data there is, for example, those roles, who is  a translator and who's an illustrator, those are only available in some of the records  and not all of them. I mean, it doesn't prevent you from using the more advanced pattern,  but it just makes it less useful, I think. I think we could do some of that with sub-properties  that we could say, for example, that this person is not only a contributor but also  illustrator or translator and assert those both properties.  Okay, hello Esmo. Thanks for the presentation. When you're linking, when you would be linking  the mark records or blown up mark records to other databases, let's say to other national  libraries to find the similar items, would you inject the links back to the original  mark records?  Yeah, I think that would make sense. I haven't really thought that far, but right now there  is basically all the data is coming from somewhere. This system itself, well, it has a few triples  about this dataset, but other than that, everything comes from a data source. And I think for  those links, we would have to decide where to put them and it would probably make sense  to put them in the mark records rather than somewhere else.  Thank you Rosmar. It's very nice to follow your work. I've got a question about if you're  planning to integrate more kinds of relations between your works, like aggregations or works  that are based on other works or that kind of links in your mark record.  Yes, I would hope to do that, but I haven't really looked very deeply at what would be  possible based on the records we have, because of course it has to be, I mean, we have to  get the information from somewhere. So I mainly concentrated, in terms of work-to-work relations,  I mainly concentrated on the translation relationship, because it seemed to be by far the most common  that was in the records, and it's also sort of interesting from a user perspective and  from, I mean, being a national library of a small country where a lot of the literature  is a translation of something else.  Thank you so much, Osmar, for these insights. I think it's very inspiring how you use the  bit frame core classes to group the things and then use schema to comply to the search  engines. I think this should really increase the interoperability of the data, considering  that data models are harder to overcome than individual properties that can be mapped.  Was it something you did deliberately to be able to align to other datasets likely to  come up in bit frame, or is it just because it fitted the structure you wanted to achieve?  I wouldn't say those two options are contradictory, but I think it's what's more about that we  saw the problems of duplication when you have a large number of marked records, which are  basically about the same work. And especially when it comes to, when you start to have lots  of electronic versions or digitized versions of the same work, so we have sort of more  and more of these marked records that are disconnected even though they are about the  same thing. So it seemed to be useful to do something in this area, and rather than chose  just a flat model that maps one marked record to one entity in the RDF to do something a  little bit more advanced with works. And so the, also the available tools of course affected  so this decision. So having the bit frame converter available from the Library of Congress  and seeing how it seems to give you at least the starting point for doing this grouping  or clustering by work, it sort of seemed like a good idea to try to apply here.  Okay, one last question before we go into the coffee break.  Thank you for your talk. And I'm wondering, it's a very nice dataset I think. Do you have  any idea who will be your users or are you planning to do any kind of marketing for special  user groups to take advantage of that data?  Well, yes, this is actually, this is part of a larger opening of our data. So the data,  in addition to this linked dataset, we are, we have created a data catalog. Because we  already provide lots of many open APIs into, in more traditional APIs usually. But they  were all sort of not very well documented and sort of hidden in specific systems. And  so we wanted to, we were also encouraged by the ministries and so on that want to see  our data put out in the open. So we created this data catalog that tries to document all  the datasets and all the APIs we have. And this is one of them or actually several of  them because there are many APIs available for the same dataset.  So and we are trying to, but we haven't yet really, I mean, this was the official launch  in a way for the linked dataset and we will announce the open data service sometime next  week and try to organize also maybe some event around it to make it more known to developers.  So yes, we are trying to get some publicity for this.  Okay. Thanks again, Osmar.
Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_0_923.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_924_2663.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_2664_8723.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_8724_10799.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_10800_14483.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_14484_16655.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_16656_17699.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_17700_17795.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_17796_18959.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_18960_19451.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_19452_19799.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_19800_20231.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_20232_20327.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_20328_20771.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_20772_20843.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_20844_21683.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_21684_21731.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_21732_21875.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_21876_21923.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_21924_22727.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_22728_22775.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_22776_22907.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_22908_22979.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_22980_23759.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_23760_33815.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_33816_33911.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_33912_33971.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_33972_34463.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_34464_34559.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_34560_37031.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_37032_39107.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_39108_39155.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_39156_39203.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_39204_41987.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_41988_42275.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_42276_42647.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_42648_43439.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_43440_44159.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_44160_45563.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_45564_48683.png Finnish_National_Bibliography_Fennica_as_Linked_Da_sLMxALIQKmQ_48684_51821.png