Indexing of Special Collections for Increased Accessibility
CC Attribution 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Russell, Judith C.
Hlava, Marjorie M. K.
Challenge of Discovery: Recent large-scale initiatives focused the attention of the Smathers Libraries at the University of Florida on the need for significantly expanded and enhanced metadata for our digital collections, both retrospective and prospective. This requires new tools and changing roles and responsibilities for cataloging/metadata staff, including the application of automated processes, Improved and consistent metadata practices, and the development of new taxonomies. Projects that are described include the new genealogical initiatives with Internet Archive and Family Search, Portal of Florida History, the Digital Library of the Caribbean (dLOC) and the Cuban Heritage Initiatives. We have concluded the pilot project on automated indexing and metadata generation for the Portal of Florida History using 25,000 full text records and the JSTOR thesaurus instead of the Library of Congress subject headings to automatically index the collection and create a search portal. This paper gives a report on the results of the pilot and early application of the process to the entire UF Digital Collections to extract information from our complete digital collections (over 600,000 items/12 million pages) for the Portal. Results of the pilot show significantly increased retrievability, greater depth of accessibility via detailed subject metadata and then explores application to the entire digital collection. The idea of a more automated processes going forward to allow traditional cataloging to focus on the things that need individual attention and use automated tools to develop and improve metadata for other materials is explored since the large number of items (over 12 million and growing at 100,000 pages per month) makes it impractical to use traditional means. We are working with tools that have been developed for information products and services, but can be applied effectively to library collections. This paper covers three of the main themes of the conference: 1) Exposing Grey Literature to Wider Audiences using an open search portal 2) Confronting Obstacles in Accessing Grey Literature through limited metadata tagging and keeping the collection access obscure as sub sets of the Library OPAC, and 3) the Digital Preservation project covering everything from old newspapers, personal papers and county historical collections is the Lifeline for Grey Resources but if not widely available through the internet and deeply tagged using a controlled vocabulary the simple scanning of papers only creates the new microfilm dilemma of locking data away in in accessible places.
 are going to do this as a team because we want to share with you this project that we're doing with the assistance of Access Innovations, the libraries at the University of Florida are transforming access to library collections by transforming the way we catalog our collections and acknowledging the decreasing reliance on the online public access catalog for discovery. The vision statement of the library is calls for providing service at the point of need and digital collections are essential to that. Charles mentioned that Georgetown is a research one university. The University of Florida is also where a very large university, about 53,000 students, over a third of them are in graduate school or professional degree programs. We have six medical and health sciences colleges over $770,000 million in research. So it's a very large complex university. We're also a land grant university, so we have agricultural extension offices in every county in Florida and we also have numerous research centers and other facilities outside of our home campus in Gainesville. Most on-campus students take one or more online courses per semester and remote online degree programs are growing. The first two users who are not physically in the libraries is very important even though student use of the libraries is very high and growing. Historically, when we digitized materials from our print collections, the mark record was the initial and often the only descriptive metadata and library congress subject headings from the only controlled vocabulary assigned to the digital item. I'm not a fan of library of congress subject headings. Google has taught people to search with natural language and using LCSH is often like asking our users to learn a foreign language. I'm sorry, I need to get back to three. I'm sorry. Apologize. I'm sorry. I can't set my paper. So our UF digital collections are very large and growing very quickly so we are looking for automated processes to expand and enhance our metadata. Improved and consistent metadata practices must be defined and then rigorously and prospectively and metadata for existing content must be brought up to these new standards. This requires new tools and changing roles and responsibilities for our cataloging and metadata staff. We have a very large repository, the UF digital collections, which more than 300 digital collections, over 545,000 items and over 13 million pages submitted by UF libraries and partner institutions. We have several major initiatives that are driving us to make these changes in our metadata. For example, Florida history is one of our preeminent collections at UF, both in print and digital form, but the content is drawn from many different collections, rare books, manuscripts, political papers, newspapers, maps, government documents, university archives, etc. The challenge is to identify all of that digital content aggregated and presented in a coherent body of information which we call the portal of Florida history. As a result of this project, we turned to access innovations for assistance with this effort and we jointly decided to begin with the digital and digitized theses and dissertations produced by UF graduates. So what you see here is the portal for Florida history. In 2016, we began a pilot project and we worked specifically with the these stations. As you probably well know, if you are also involved in digitization, digital could be just a new microfilm, which is straightforward to create, but very difficult to search and retrieve. The Florida thesis project, as we called it, was taking all the digital and digitized theses and dissertations, a collection of about 25,000 items, which covers a broad discipline and see if we couldn't enhance the metadata for those to make a more retrievable. For about the last 10 years, University of Florida has been receiving digital deposit of electronic theses and the dissertations and performing retrospective scanning on the older dissertations. So it's a good size collection and they have also begun the retrospective scanning of all the previous theses. What we did was take software from the data harmony collection, the access innovation sys, and build a project to hold the data, to build a repository. The data was extracted from the Internet of Digital Collection, including the existing metadata, which is in Mark Metz XML. We tested against three different controlled vocabulary, three thesari, which were probably, which were very broadly, the coverage of them was very broad. News indexer, the National Information Center for Education Media, and JSTOR, which covers a large repository. After we did all the testing and assessment, we determined that we really needed two thesari. One would be the topical one. In this case, we chose JSTOR and the second one would need to be Florida specific authority terms. So we have a combination of the JSTOR thesari modified for Florida and a geographic and great Floridians file as well. We also tagged the data for the Florida thesis project. This shows you some before and after sample records. I don't know if this will work to set. I don't see them else. On your left, you see the original catalog record, which is very thin. There aren't very many keywords. There's not much in the line of genre. On the right, you see the enhanced record, which has considerably more listed those that come from JSTOR as well as the additional ones from the Florida thes. You can see that there's a great deal more information for you to hang on to. Students can add their keywords as they submit their thesis or dissertations, but they're not metadata. Libraries often add the LCSH headings, but as you know, particularly for special or great literature collections, those are very thin. They use up with words like biography, theses, and electronic theses or dissertations. Out of the four genre headings, three of them are not particularly useful for searching, particularly for conceptual searching. As you see from these examples, the first two and the last four lines remain the same in both records, but the automated process added 12 additional controlled vocabulary terms, which are thesis and topical specific to the geography of Florida, and therefore made the records automatically inclusive for the portal of Florida history. What you see here on the left is an example of some of the University of Florida collections. And then we move to the arrow on the top, which shows XML records exported from the University of Florida digital collections into the CIS repository. From that repository, we can export mark records to OCLC. We can export staff internal use and review. We can put them into a local repository with the enhanced metadata and return them to the University of Florida digital collections. So as a result of all this work, University of Florida now has a taxonomy of Florida specific terms that it can maintain and expand to use and manage both print and digital collections. But also because of the use of the taxonomy as the default search in the University of Florida digital collection, Lucene software, instead of using the normal default in Lucene, which is full text. We get 80% or better accuracy in retrieval. Retrieval in just full text would be 55% to 60%, which gives an awful lot of false drops and frustration to the user groups. If we just add the terms to the full text, we only get a 6% to 7% increase in accuracy. So searching on the control vocabulary terms is very, very important. So we have done the theses. We have proved the workflow and began work on some additional collections as well. Crosswalk between the UDC collections and the SOPEX catalog at the University of Florida. And now that we've done the assessment of the search results and enhanced the metadata, the pilot itself is concili- Do I want to dance here? I will say that we were very pleased with the results of the pilot. The one record is just a tiny sample, but many of the records ended up with that equivalent of quality of search terms. So we're very eager now to apply these tools and processes to the rest of our digital collections. So SOPEX, which Margie mentioned, is the open-source software platform that we use. And it supports 11 subject metadata fields, which have not been used consistently over time. This is certainly not a unique occurrence, but it's one that needs to be addressed to improve access to our digital collections. We need to standardize the use of the fields. That's essential to development and application of the enhanced metadata and to support advanced search focused on one or more fields. All topical subject terms will be assigned to a single field. Geographic terms, place names, corporate names will each be assigned a single field. They'll have authority files. The existing terms will be mapped to the appropriate fields and replaced with control vocabulary terms. This will, as Margie said, be much greater precision. So for example, Jose Martí could be an author, he could be a subject, he could be part of a place or corporate name. We're doing a major project with the Bibliotech Nacional de Cuba Jose Martí, and it's going to show us a corporate name in that context. As with the Portal Florida History Project, we need to apply automated tools and techniques to existing collections in the digital library of the Caribbean, and also to new collections as it's submitted to D-LOC, including our new Cuban Heritage collections. This will be particularly important for improving retrieval of cross-collection that have come from different institutions with different metadata, schema, vocabulary, and languages. We hope that this process will allow us to continue to have metadata in the native language of the submitted material, but also in English so that there will be an easier common search across disciplines. One of the new collections that will go into the digital library of the Caribbean is resulting from our agreement with the Bibliotech National de Cuba to create deep broad open access to the Cuban Heritage collections and put them into D-LOC. The British of the Bibliotech Nacional estimated that 58 percent of its Cuban Heritage materials are uniquely held in Cuba, and it is committed to digitizing those materials, but it's asking UF and its partners outside of Cuba to identify sources and digitize the other 42 percent of them, making bibliographic control essential to avoid duplication of effort and make the collection as comprehensive as possible. They have shared their bibliographic records with UF, and we have agreed to establish an OCLC symbol for them, which we will manage and make sure that all of their Cuban Heritage records are available in WorldCat. We will also use the project management database. However, 16,000 of the records that they provided to us while digital are not in marked format, they merely scan images of catalog cards, such as you can see on the slide. So we once again turned to access innovations for assistance with conversion of these catalog cards. Since we've already had the three minute thing, I'll just point out to you very quickly that what you see here is a number of those of you that are of an older generation might actually recognize what those are. And on the left you see some of the conversion that the NACI and NACI did, but for those where there was not a conversion mark, what we got was a card that was undifferentiated in any way, so it was just text. And so we tried to separate that text, and we're fairly successful in separating the text into a full bibliographic record. And for search purposes, we needed to separate out not a call number and the name and the title and so on, but we also want to place a publication and data publication. And the publisher, because for searching in great literature in general, those are important fields, and so we didn't want just a blob of an undifferentiated material. What you find through this thing that we fairly quickly described to you is that we are actually creating the records in the SIS database, no longer creating them in the cataloging application or the ILS, and that means that with the records created in SIS, we are exporting to OCLC and the OPEC as appropriate to the University of Florida Digital Collections as well. So this is a more up-to-date version of the image that we showed you on the SIS input panel, then you see the repository itself, and from the repository, we're exporting the records to all of the other places. So we are not cataloging originally in Mark, but rather inverting the cataloging process to create a metadata record in SIS and export it. This gives us a very streamlined workflow. It goes title by title, and catalogers find it a very fast way to work. Once it's in the repository, we can output in any number of formats, including HTML, and all the records can be both print and electronic. Okay, so to conclude, I want to go back, assuming it will let me change page down. Oops. There we go. I wanted to go back to the original title, which Margie actually created for us, the Death of the Library of Catalog, with a big question mark. And I want to come back to that title because I think when we look at these new metadata tools and techniques, you can ask the question, does this signal the death of the library catalog? And clearly most libraries continue to invest significant time and resources in cataloging and in the integrated library systems that host our catalogs, but our students who have grown up with Google are much less likely to turn to the OPAC for discovery. So the answer to the question is not yet, but as research libraries like ours continue to place more and more emphasis on digital collections, there will be reduced emphasis on traditional title by title cataloging and greater emphasis on automated metadata generation. As Margie said, we're inverting the traditional cataloging process. Medi-data will become the source of mark records rather than mark records continuing to be the source of metadata.
 are going to do this as a team because we want to share with you this project that we're  doing with the assistance of Access Innovations, the libraries at the University of Florida  are transforming access to library collections by transforming the way we catalog our collections  and acknowledging the decreasing reliance on the online public access catalog for discovery.  The vision statement of the library is calls for providing service at the point of need  and digital collections are essential to that.  Charles mentioned that Georgetown is a research one university.  The University of Florida is also where a very large university, about 53,000 students,  over a third of them are in graduate school or professional degree programs.  We have six medical and health sciences colleges over $770,000 million in research.  So it's a very large complex university.  We're also a land grant university, so we have agricultural extension offices in every  county in Florida and we also have numerous research centers and other facilities outside  of our home campus in Gainesville.  Most on-campus students take one or more online courses per semester and remote online degree  programs are growing.  The first two users who are not physically in the libraries is very important even though  student use of the libraries is very high and growing.  Historically, when we digitized materials from our print collections, the mark record  was the initial and often the only descriptive metadata and library congress subject headings  from the only controlled vocabulary assigned to the digital item.  I'm not a fan of library of congress subject headings.  Google has taught people to search with natural language and using LCSH is often like asking  our users to learn a foreign language.  I'm sorry, I need to get back to three.  I'm sorry.  Apologize.  I'm sorry.  I can't set my paper.  So our UF digital collections are very large and growing very quickly so we are looking  for automated processes to expand and enhance our metadata.  Improved and consistent metadata practices must be defined and then rigorously and prospectively  and metadata for existing content must be brought up to these new standards.  This requires new tools and changing roles and responsibilities for our cataloging and  metadata staff.  We have a very large repository, the UF digital collections, which more than 300 digital collections,  over 545,000 items and over 13 million pages submitted by UF libraries and partner institutions.  We have several major initiatives that are driving us to make these changes in our metadata.  For example, Florida history is one of our preeminent collections at UF, both in print  and digital form, but the content is drawn from many different collections, rare books,  manuscripts, political papers, newspapers, maps, government documents, university archives,  etc.  The challenge is to identify all of that digital content aggregated and presented in a coherent  body of information which we call the portal of Florida history.  As a result of this project, we turned to access innovations for assistance with this  effort and we jointly decided to begin with the digital and digitized theses and dissertations  produced by UF graduates.  So what you see here is the portal for Florida history.  In 2016, we began a pilot project and we worked specifically with the these stations.  As you probably well know, if you are also involved in digitization, digital could be  just a new microfilm, which is straightforward to create, but very difficult to search and  retrieve.  The Florida thesis project, as we called it, was taking all the digital and digitized theses  and dissertations, a collection of about 25,000 items, which covers a broad discipline and  see if we couldn't enhance the metadata for those to make a more retrievable.  For about the last 10 years, University of Florida has been receiving digital deposit  of electronic theses and the dissertations and performing retrospective scanning on the  older dissertations.  So it's a good size collection and they have also begun the retrospective scanning of all  the previous theses.  What we did was take software from the data harmony collection, the access innovation  sys, and build a project to hold the data, to build a repository.  The data was extracted from the Internet of Digital Collection, including the existing  metadata, which is in Mark Metz XML.  We tested against three different controlled vocabulary, three thesari, which were probably,  which were very broadly, the coverage of them was very broad.  News indexer, the National Information Center for Education Media, and JSTOR, which covers  a large repository.  After we did all the testing and assessment, we determined that we really needed two thesari.  One would be the topical one.  In this case, we chose JSTOR and the second one would need to be Florida specific authority  terms.  So we have a combination of the JSTOR thesari modified for Florida and a geographic and  great Floridians file as well.  We also tagged the data for the Florida thesis project.  This shows you some before and after sample records.  I don't know if this will work to set.  I don't see them else.  On your left, you see the original catalog record, which is very thin.  There aren't very many keywords.  There's not much in the line of genre.  On the right, you see the enhanced record, which has considerably more listed those that  come from JSTOR as well as the additional ones from the Florida thes.  You can see that there's a great deal more information for you to hang on to.  Students can add their keywords as they submit their thesis or dissertations, but they're  not metadata.  Libraries often add the LCSH headings, but as you know, particularly for special or great  literature collections, those are very thin.  They use up with words like biography, theses, and electronic theses or dissertations.  Out of the four genre headings, three of them are not particularly useful for searching,  particularly for conceptual searching.  As you see from these examples, the first two and the last four lines remain the same  in both records, but the automated process added 12 additional controlled vocabulary  terms, which are thesis and topical specific to the geography of Florida, and therefore  made the records automatically inclusive for the portal of Florida history.  What you see here on the left is an example of some of the University of Florida collections.  And then we move to the arrow on the top, which shows XML records exported from the  University of Florida digital collections into the CIS repository.  From that repository, we can export mark records to OCLC.  We can export staff internal use and review.  We can put them into a local repository with the enhanced metadata and return them to the  University of Florida digital collections.  So as a result of all this work, University of Florida now has a taxonomy of Florida specific  terms that it can maintain and expand to use and manage both print and digital collections.  But also because of the use of the taxonomy as the default search in the University of  Florida digital collection, Lucene software, instead of using the normal default in Lucene,  which is full text.  We get 80% or better accuracy in retrieval.  Retrieval in just full text would be 55% to 60%, which gives an awful lot of false drops  and frustration to the user groups.  If we just add the terms to the full text, we only get a 6% to 7% increase in accuracy.  So searching on the control vocabulary terms is very, very important.  So we have done the theses.  We have proved the workflow and began work on some additional collections as well.  Crosswalk between the UDC collections and the SOPEX catalog at the University of Florida.  And now that we've done the assessment of the search results and enhanced the metadata,  the pilot itself is concili- Do I want to dance here?  I will say that we were very pleased with the results of the pilot.  The one record is just a tiny sample, but many of the records ended up with that equivalent  of quality of search terms.  So we're very eager now to apply these tools and processes to the rest of our digital collections.  So SOPEX, which Margie mentioned, is the open-source software platform that we use.  And it supports 11 subject metadata fields, which have not been used consistently over  time.  This is certainly not a unique occurrence, but it's one that needs to be addressed to  improve access to our digital collections.  We need to standardize the use of the fields.  That's essential to development and application of the enhanced metadata and to support advanced  search focused on one or more fields.  All topical subject terms will be assigned to a single field.  Geographic terms, place names, corporate names will each be assigned a single field.  They'll have authority files.  The existing terms will be mapped to the appropriate fields and replaced with control vocabulary  terms.  This will, as Margie said, be much greater precision.  So for example, Jose Martí could be an author, he could be a subject, he could be part of  a place or corporate name.  We're doing a major project with the Bibliotech Nacional de Cuba Jose Martí, and it's going  to show us a corporate name in that context.  As with the Portal Florida History Project, we need to apply automated tools and techniques  to existing collections in the digital library of the Caribbean, and also to new collections  as it's submitted to D-LOC, including our new Cuban Heritage collections.  This will be particularly important for improving retrieval of cross-collection that have come  from different institutions with different metadata, schema, vocabulary, and languages.  We hope that this process will allow us to continue to have metadata in the native language  of the submitted material, but also in English so that there will be an easier common search  across disciplines.  One of the new collections that will go into the digital library of the Caribbean is resulting  from our agreement with the Bibliotech National de Cuba to create deep broad open access to  the Cuban Heritage collections and put them into D-LOC.  The British of the Bibliotech Nacional estimated that 58 percent of its Cuban Heritage materials  are uniquely held in Cuba, and it is committed to digitizing those materials, but it's asking  UF and its partners outside of Cuba to identify sources and digitize the other 42 percent  of them, making bibliographic control essential to avoid duplication of effort and make the  collection as comprehensive as possible.  They have shared their bibliographic records with UF, and we have agreed to establish an  OCLC symbol for them, which we will manage and make sure that all of their Cuban Heritage  records are available in WorldCat.  We will also use the project management database.  However, 16,000 of the records that they provided to us while digital are not in marked format,  they merely scan images of catalog cards, such as you can see on the slide.  So we once again turned to access innovations for assistance with conversion of these catalog  cards.  Since we've already had the three minute thing, I'll just point out to you very quickly that  what you see here is a number of those of you that are of an older generation might  actually recognize what those are.  And on the left you see some of the conversion that the NACI and NACI did, but for those  where there was not a conversion mark, what we got was a card that was undifferentiated  in any way, so it was just text.  And so we tried to separate that text, and we're fairly successful in separating the  text into a full bibliographic record.  And for search purposes, we needed to separate out not a call number and the name and the  title and so on, but we also want to place a publication and data publication.  And the publisher, because for searching in great literature in general, those are important  fields, and so we didn't want just a blob of an undifferentiated material.  What you find through this thing that we fairly quickly described to you is that we are actually  creating the records in the SIS database, no longer creating them in the cataloging application  or the ILS, and that means that with the records created in SIS, we are exporting to OCLC and  the OPEC as appropriate to the University of Florida Digital Collections as well.  So this is a more up-to-date version of the image that we showed you on the SIS input  panel, then you see the repository itself, and from the repository, we're exporting  the records to all of the other places.  So we are not cataloging originally in Mark, but rather inverting the cataloging process  to create a metadata record in SIS and export it.  This gives us a very streamlined workflow.  It goes title by title, and catalogers find it a very fast way to work.  Once it's in the repository, we can output in any number of formats, including HTML,  and all the records can be both print and electronic.  Okay, so to conclude, I want to go back, assuming it will let me change page down.  Oops.  There we go.  I wanted to go back to the original title, which Margie actually created for us, the  Death of the Library of Catalog, with a big question mark.  And I want to come back to that title because I think when we look at these new metadata  tools and techniques, you can ask the question, does this signal the death of the library  catalog?  And clearly most libraries continue to invest significant time and resources in cataloging  and in the integrated library systems that host our catalogs, but our students who have  grown up with Google are much less likely to turn to the OPAC for discovery.  So the answer to the question is not yet, but as research libraries like ours continue  to place more and more emphasis on digital collections, there will be reduced emphasis  on traditional title by title cataloging and greater emphasis on automated metadata generation.  As Margie said, we're inverting the traditional cataloging process.  Medi-data will become the source of mark records rather than mark records continuing to be  the source of metadata.
Vortrag_Russell_Hlava_0_923.png Vortrag_Russell_Hlava_924_3227.png Vortrag_Russell_Hlava_3228_3311.png Vortrag_Russell_Hlava_3312_3359.png Vortrag_Russell_Hlava_3360_3671.png Vortrag_Russell_Hlava_3672_3899.png Vortrag_Russell_Hlava_3900_5411.png Vortrag_Russell_Hlava_5412_6239.png Vortrag_Russell_Hlava_6240_7247.png Vortrag_Russell_Hlava_7248_7451.png Vortrag_Russell_Hlava_7452_7547.png Vortrag_Russell_Hlava_7548_16103.png Vortrag_Russell_Hlava_16104_17123.png Vortrag_Russell_Hlava_17124_21791.png Vortrag_Russell_Hlava_21792_22007.png Vortrag_Russell_Hlava_22008_22607.png Vortrag_Russell_Hlava_22608_23519.png Vortrag_Russell_Hlava_23520_24083.png Vortrag_Russell_Hlava_24084_27461.png