Image courtesy of Flickr user marttj

Image courtesy of Flickr user law_kevin

Question: Will the way volumes are digitized today change the way that work is done in the future: less close reading more broad reading/more data mining and deduction across corpora. Are the traditional monograph-driven disciplines a strong enough force against the tides of Google-ization (i.e. the “close enough” answer is the good enough answer)?
These questions are being explored by author David Weinberger in an upcoming book he’s working on called Too Big To Know and he gives some insight into his writing process and the arguments he is pulling together for his book on his Joho the Blog. It’s interesting both from the perspective of getting an inside look at an author’s process and for the sake of the arguments themselves. Check it out.

Here’s where we need your help (and I was able to work in reference to the term “crowdsourcing”).

We need a name that makes it clear that we aren’t NC ECHO, the Digital Heritage Center, or NC State (all of which either are in some way related to the State Library or State Archives or have the words “North Carolina” and “State”  in their name). The name should (a) make reference to the Archives & Library or (b) be completely neutral (and I think that we prefer a “neutral”  name as it allows our partnerships to grow, potentially).  We’ll be providing access to the historic and contemporary records and publications of state government (as we do now), and the name needs to reflect that (probably).  These materials can be as diverse as the Library’s 2003 document about prosecuting computer crimes from the Department of Crime Control and Public Safety to the Archives’s Black Mountain College publications and promotional photographs from the Department of Tourism. And, we’re both divisions within the Department of Cultural Resources, so there is some question as to whether we need to work a reference to that in?

Now, I realize that a name containing all of this information would be ridiculous, but I’m providing this information to give you some background.

OK. Let the crowdsourcing begin…

The project puts online more than 7,000 images taken for the local newspaper, mostly between the late 40s and late 60s. It’s actually kind of a unique collection, since lots of material from this date is all locked up in copyright restrictions.

Although we are kind of trying to get to the point where adding a new collection isn’t all that big of a deal (for example, we also launched three other “collections” in the last month), this one was a little different. First, it makes up roughly 2/3 of the entire digital collection. Second, it’s the result of the first year of a two-year grant project (partially supported with federal LSTA funds made possible through a grant from the Institute of Museum and Library Services, administered by the State Library of North Carolina, a division of the Department of Cultural Resources), and finally it’s got it’s own look and feel and lots of contextual features.

The greatest thing for us, is that it is actually just part of Digital Collections, so all the content is findable in our main portal and when you are surfing through the Seeds of Change site, you can easily bounce back to the main site. No more silos! (at least between these two collections…baby steps, you know). This is another example of our overall philosophy of trying to build one great infrastructure so that collections can be as customized (or not) as we want them to be.

Anyway, enough babbling, please check it out and let us know what you think!

Blimp advertising The Outlaw (film) from UNLV's digital collection "Howard Hughes: Welcome Home, Howard"

UNLV just announced a couple of new collections this past week. Both of them have pretty great interfaces, especially considering that they are built on CONTENTdm. We all know about CONTENTdm’s weaknesses and strengths, but those of us who use it everyday are intimately familiar with the challenges that its interface poses. Not built in XML and not built with CSS (or, just barely), making the interface “your own” involves following cryptically named links back to multiple “style sheet pages.” It is sort of like trying to figure out which one of the cords in the back of your desk is attached to the printer. . . in the dark.

Anyway, UNLV seems to have solved this problem with something they are calling “dmbridge.” I won’t go into the details here, because they’ve already done that at this link (http://digital.library.unlv.edu/dmbridge). But, I will say that they have incorporated lots of Web 2.0 features that are reflected in their two new collections, “Welcome Home, Howard” and “Southern Nevada: The Boomtown Years.” And, apparently, they will be sharing the code.

The first dealt with legal and ethical issues surrounding digitization. While I was prepared for a lot of information about copyright, I was surprised by a lively and enlightening discussion of privacy…something that I have not seen addressed in many other forums.

The second symposium addressed “The sustainability of Large-Scale Digitization of Manuscript Collections.” I arrived expecting to discuss the difficulties many of us are facing in creating long-lasting digitization programs, but instead found that a large portion of the day addressed the mass digitization of books with presenters John Wilkin of Michigan, Oya Rieger of Cornell, and Liz Bishoff of BCR. While they noted that many of the materials being scanned this way are in fact from “Special Collections” in libraries, they are doing a fundamentally different thing (in my opinion!) than the mass digitization of manuscript material.

All of this is not to criticize the coordinators of the symposium — as I’ve found in just about every other venue, when Google Books is in the room, it attains a Paris Hilton-like level of notoriety. Rather, I bring this up to raise the question of why it is fundamentally different?

There are of course the obvious reasons: formats, lack of MARC metadata. Then there are the what I’ll call context-related reasons of archival organization which make fitting into the item-level mold difficult. But I think what is most interesting are the infrastructure reasons: namely, what constitutes “mass” digitization of special collections at all and how our online systems support the presentation and sharing of this information.

The final panel of the syposium featured the excellent Collections Online project at the Smithsonian Institution Archives of American Art, who have come closest to defining what mass digitization of special collections can really mean. By forsaking item-level metadata, they’ve recreated the experience of browsing through their physical collections in the online environment. The amount of material they have created, while not rivaling that of of Google, is impressive. But more remarkable is the viewpoint they have taken: that digitization of this material isn’t “special” isn’t a “project”, it’s day to day work. Therefore, since they would not describe items in their analog collection at the item level, they do not do so in the online version (providing navigation through the finding aid instead). While I’m not sure that this is the answer at my own institution, thinking of the whole, rather than the parts is what is impressive. In my opinion, it is part of what gives it “mass.”

The issue of sharing was also much discussed at the forum, both in the sense of sharing digitization services and in sharing information in a more open way. John Wilkin made the point more than once that libraries in general need to think outside the box of the ILS. A point of view shared by many of our colleagues, and indeed, one which even has a catchy name attached: the Next Generation Catalog. But where is the next generation catalog for special collections and digital collections?

One reason that Google Books and Internet Archive have been successful, (and, I believe, one reason their mass digitization works) is that they have created a “union catalog” that is universal and reusable to some extent. Google has even done some opening up of their data and offering of APIs to reuse it. So how would a “union catalog” for special collections even work? I know this is a pipe dream anyway, but how would we even began to share on a larger level? Do we have standards in place for this to work? What kinds of APIs would we want for digital collections?

This work is relevant for several reasons. Someday, these contemporary documents will be historical documents. That is, if they exist at all. If they are not saved, our history goes with them. That’s just a simple fact. But the value of this work goes far deeper. I mean, we preserve the physical (read: paper) copy of the Declaration of Independence to ensure that we do not forget our ideals of democratic government; we preserve Abraham Lincoln’s letters not only as a testament to his strength, influence, and unconquerable spirit, but also for the historical importance of their content in documenting the activities of a sitting president. It is no less important to preserve the intellectual output of our public servants today because they create it with computers and not quill pens. In 2008, we elected the first female governor in North Carolina’s history. Shouldn’t her tenure be documented and saved?

On another level, this work is important for those who cannot (or can no longer) afford to travel to pursue their research. Because of budget cuts, many of us cannot travel right now, no matter how important it is to the work we do. This will continue to worsen before it gets better. For researchers, limiting funds for travel can be devastating. Content offered through digital repositories can play a huge part in aiding these patrons. And, as more researchers opt for online over in-person visits, many of us have begun tracking their needs and digitizing in response to them.

The problem is that many libraries around the country are facing significant budget and staffing cuts. Our own Library is, as well. The funny thing is that many indicators* point to library usage increasing during economic downturns. In this era of online everything, will our web stats increase as much, or more, than our turnstile counts? And if so, are we ready to respond to that usage? How do we offer the services that patrons need with a smaller budget and fewer staff? I’m betting on our online services to help us weather this storm. And, we’ll continue to do our part in preserving democracy by providing free access to government information.

* See, for instance, http://seattletimes.nwsource.com/html/localnews/2008630228_libraries15.html, http://www.wickedlocal.com/braintree/archive/x511410985/Library-use-booms-during-economic-downturn, http://www.stevenspointjournal.com/article/20081215/SPJ0101/812150372/1657, and http://blog.njla.org/archives/2009/01/

I now realize that nearly everything you do like this is going to take a long time, if: a) you want to do it right and b) your institution won’t quit asking you to do other projects!

All joking aside, we are enormously proud of our repository. The guiding principles were:

• one common database and infrastructure that could be utilized for multiple “collections”
• Collections are nice and we do like them, but we will always have “uncollected” items in there and we need to design it so that you can still discover these
• This repository will gather everything digitized for a patron request, for an exhibit, for a book, for preservation, or because it would be cool to digitize it

Based on the above principles we used a back-end XML database (TextML) and a METS/MODS record for every object. In addition, we included Dublin Core records in the METS (scripted from the MODS) for the purposes of OAI harvesting, appropriate admin metatadata (MIX in most cases since we have a lot of images), and even the full text of TEI records when those were available. To date we only have one audio object, but should be ingesting another 20 or so by March.

We had the details of the backend database worked out in the first third of 2008, and then set ourselves to the task of figuring out how to present this stuff to people who would have no idea what the site was, and in many cases, who ECU and Joyner Library were. We took the approach that the site had to invite people into the content — we had to have lots of ways that they could come to page, click on one thing and get to content.

So we created “subject clouds” and created collections post-hoc (i.e. we picked some relevant themes, then looked at what we had and organized them together). Both of these things were created on the premise that the organization of physical items in the library isn’t necessarily the best organization for things on the web. The web can do both more and less than that.

We also provided more tools for navigating through materials once you found some by including hyperlinks in the records, and a faceted drill-down of the subject headings. These, as well as the subject cloud and the organization of the post-hoc collections, were all based on LCSH subject headings. A simple look at size of the facet list or the subject cloud gives you a good indication of both the benefits and drawbacks of this approach. But we figured we would use what we had and I think in some sense it is successful.

What is most interesting to me, is that many of these design elements were things that I’ve been thinking about since my days at UM. Being able to realize them here with a fresh new project was really exciting. Certainly, it isn’t perfect, but I think it’s all too rare in our jobs that we get to come up with an idea, implement it, and then evaluate how it worked out. We are often saddled with maintaining past projects and living within the bounds of past decisions. Of course, admitting that means that I must acknowledge that one day the decisions we’ve made on this project will limit us somewhere else. But I think we’ve created enough leeway and enough to think about, that we should be busy for quite some time.