With scanning technology being easier and cheaper than ever before, why aren’t more genealogy materials online? The answer isn’t, “Just scan it.” Here’s a look at everything involved in making genealogy records available online.
Recently, FamilySearch announced that they have completed digitizing their collection of 2.4 million rolls of microfilm. But… they aren’t all online yet. And when you think about the archives, libraries, government agencies, and other organizations that have records—why aren’t more of them online?
Let’s walk through a hypothetical (yet realistic) scenario. Let’s pretend you’re an archivist and there’s this really cool collection of thousands of letters that you just know would be useful to researchers. Wouldn’t it be great to digitize it and get it online? Let’s see what it would take.
Getting Permission
For an archive or library to make their materials available online, they first need to make sure they have permission to do so. Sometimes that’s straightforward, like when it’s something in the public domain. But if it isn’t in the public domain and it’s something that was donated to them—such as original materials or manuscripts—they need to make sure they have the permission of the donor. Donor agreements sometimes restrict what that library or archive can do with the material. Also, if it isn’t spelled out that the repository does have rights to digitize and distribute, they might need to re-negotiate an agreement allowing them to do so.
If it’s a government agency, sometimes the records are restricted by law. The records might be from a time period that is still in “embargo” (such as states that restrict death records for 50 years). The record type itself might be restricted; this is often seen with state hospital and state asylum records.
In your (hypothetical) archive, you first need to find the donor agreement. When you find that it doesn’t have the necessary permission, you have to try reaching out to the donor, which isn’t always an easy or fast thing to do. But for this example, we’ll say that it only took a month to reach the donor and get his permission.
Getting Funding
As you’ll see, there’s a fair amount of money involved in digitizing. Most archives have razor-thin budgets, and extra projects simply don’t make the cut.
You opt to go for grant funding for the necessary equipment and additional staffing. Fast-forward several months, and you’re notified you got the grant. (You aren’t always so fortunate. It isn’t unusual to go through several rounds of applications to various foundations before securing funding.)
Setting Some Ground Rules
Are you going to scan the front and back of all pages, or just the ones with writing? Are you going to scan the envelope? (I hope the answer to that one is, “Yes.”) How are you going to handle oversize pages that either won’t fit on the scanner or in the field of the camera without being really tiny?
What format will they be scanned into? What resolution? How will you keep together the files of the letters that are multiple pages? File naming conventions?
Arranging and Preparing the Material
Before you set up your scanner or digital camera, there is work to do. Those letters need to be opened and the papers unfolded, unstapled, and un-paperclipped. This takes time. (So. Much. Time.) And there needs to be a way to keep things in order so papers don’t get mixed up in the process.
Getting the Necessary Equipment
There needs to be a scanner or digital camera with the necessary accessories, such as batteries, lighting, camera stand, etc.
And people. You can’t digitize without the people do it.
Digitizing
Honestly, this is usually the easiest part of the whole process, but it still takes time. Scanning a book can be fairly fast, but if you’re working with unbound material (like your hypothetical letter collection), you’re going to go a lot slower. Even if you can do a new image every 5 seconds — which would be lightning speed for some unbound materials, it adds up. Let’s say it’s 5 second per page. Multiply that by a letter that’s 6 pages long plus an envelope… and there are 2000 letters to do. That’s 1167 minutes or almost 20 hours. That’s presuming nothing slows you down. (Head’s up: There’s always something that will slow you down.)
Describing/Cataloging
Those images don’t do anyone good if nobody knows what they are. That requires someone to set up metadata—information about something that makes it more usable. At a minimum, you need some kind of title for this group of images, but there usually needs to be a more robust description.
There’s also something called “structural metadata,” which shows how the images relate to one another. This includes things like the sequence of the images, so that page 5 comes after page 4 but before page 6. It can also be assigning “waypoints.” Essentially, this allows users to see where sections of the work are, much like a book’s table of contents does. This also takes time and someone to do it. (See a pattern here?)
Hosting
If these images are going to be online, they need to be hosted somewhere. (This isn’t the same thing as the archive’s hard drive where they are stored. By the way, that’s more equipment that’s needed.) There also needs to be some sort of website. There are frameworks called “content management systems” that help libraries and archives manage this, but they’re often too expensive for small organizations to use. Even the organizations that do have a CMS still need to pay for the service and have people to work on the technical aspect. (I hope you figured that into your grant proposal.)
Storage and Backups
If you’ve ever had a hard drive fail, you’ll understand the need for a good system of backups. With computers, it isn’t a matter of if they will fail; it’s a matter of when. You’re going to need a good backup system (which includes a clear way to recover data), as well as a plan to migrate data to new media and/or formats when necessary.
Staffing
You might be able to get some of the work done with the staff you already have at your (hypothetical) archive, but this is a big project. You were smart to include in your grant proposal some funding for a part-time employee or intern to work with the project. Even if you had opted to use all volunteers for this, volunteer labor is not free. It still requires time to train, supervise, and manage.
These Issues Affect Everyone
Whether it’s your (hypothetical) small archive or one of the major players in the genealogy space, these issues affect everyone who is trying to get things digitized and bring them online. True, the big players don’t need to apply for grant funding, but they still have issues of time constraints, staffing, and technology. Even for them, there are limits of people, time, and money. It’s tougher for smaller organizations, because they don’t have the economy of scale that the larger ones do.
So when you get frustrated (like I do sometimes) when that record you need isn’t online, remember that there’s more to bringing records online than just scanning.
Hi Amy:
The Kentucky Genealogical Society has a digitization grant program to provide funding for Kentucky based collections. We have already helped many records find a digital home.
Learn more here: https://kygs.org/what-we-do/kentucky-grant-program/grant-program-recipients.html
FamilySearch.org continues to provide free digitizing of records with genealogical value. They have more than 300 cameras and crews travelling the globe in this effort.
If you know of records that could be digitized, click on the link below to read more and then click on the button “Find a Representative.” Fill out the form to have someone from FamilySearch contact you.
https://www.familysearch.org/records/archives/web/services/#preservation
great write-up. thanks so much.
Great post! As one who was involved in a digitization project during my library school practicum, I thank you for pointing out just how much goes into bringing records into the digital world.
Interesting article. I’ve been collecting records, scanning my family’s and borrowed photos for 30 years, and in the past tried to find time to scan the family records and correspondence as I came across them. It’s a lot of work. Now we’ve gotten fancy and are calling it digitizing. Have reached a point where I’m spending 6 months with an active subscription to Ancestry and Newspapers and 6 months on hiatus, so I don’t feel like I’m throwing away my research dollars.
This allows me to select notebooks to go through and see if the material is online or not. Funeral cards, photographs of 2 pages of births and deaths in a great-grandmother’s bible my mother obtained for me 2 years before she passed away and 3 years before the aunt who had it passed away, not online. Obituaries and other articles from old newspapers that aren’t on the Library of Congress Chronicling America or Newspapers.com, not online. Correspondence, materials received from my early mentors, photos they took of old headstones, churches and buildings in the local communities decades ago when they were in much better shape, not online.
Once scanned or digitized, post in my Ancestry galleries, and possibly on the appropriate personal and county genealogy Facebook pages. Whenever I get discouraged about the amount of work, only have to look at different family members profiles and see how many people have shared my uploaded work to their Ancestry and more recently, MyHeritage and FamilySearch, accounts.
A few months ago on a Facebook genealogy page a man was complaining about not finding his ancestors photos on Ancestry and it was a scam. Pointed out to him that Ancestry and the other genealogy websites don’t own the photos, families do. He needed to find out if anyone in his family had older identified photos, whether albums full or just one, scan, and post on the profiles of whichever relatives were in the photo. That would make him a contributor, and possibly a distant cousin would be grateful to see the photo and his tree, share to his tree, and get in touch.
This is my contribution to changing the title from “Why More Genealogy Records Aren’t Online” to “How More Genealogy Records Are Being Placed Online” one tiny step at a time. Have noticed in the past few years that a new generation of researchers are asking questions and moving on to posting information on genealogy Facebook pages.
Great article!
Our little County Archives thankfully has the equipment and staffing to digitize records, but we cannot afford the Hosting part. The annual grants that our State Archives provides don’t include Hosting costs, or only include them for the duration of the grant cycle. The Hosting issue is what stands in the way of making our records more accessible.
Have you tried checking with FamilySearch.org
https://www.familysearch.org/records/archives/web/services/#online-access
to see if some of your collections might qualify for free hosting?
One thing to keep in mind is that all those films came from efforts that began in the 1930s-40s by LDS to film records from all over the U.S. That was the labor intensive part of this collection and was long completed before digitization started.
As far as hosting images, if anyone has *original* documents at least 100 years old that are already digitized and have anything to do with North Carolina, even personal ones like letters, please contact NC Historical Records Online and we can put them online for you on our free website. See http://nchistoricalrecords.org
Yes, the microfilming that FamilySearch and its predecessor organizations have done over the years represents a tremendous amount of work. But even with that microfilming, they still have to get permission from the original record holders to put the images online. (That’s why some are restricted to Family History Centers and affiliate libraries.) Having worked with record holders when I worked for another company, I know how long that process can take.
A small point of clarification — FamilySearch was still capturing new collections on microfilm when they started digitizing the film in the vault. (One reason why they originally thought the project wouldn’t be completed in our lifetime — they were creating microfilm almost as fast as they were digitizing what they already had.) It wasn’t until digital cameras became less expensive, more portable, and gave the image quality that they needed that original collections were captured digitally, skipping the microfilm stage altogether.
Thank you for the very informative article and for this clarification!
Patience isn’t one of my virtues but it certainly is needed in accessing desired genealogy information.
I’ll add one more, from the perspective of a small historical group, but primarily from my personal viewpoint. I spent thousand of hours a year, for multiple years, creating a master database/index to help those researching families in our area. They could easily find out where their family names appeared in our records/books within minutes. It was my time and my money to put it online. The intent was to sell CDs and hard copies of the collated data, to put small markers on unmarked graves in our area…a dream which shattered within days as only a week or two after putting the information online, we received a mailing which was selling the data I gathered. They had no ties to our group or our area, nor had they talked to me or any other member of our group. It was a bitter pill to swallow, that someone would take years worth of research, download it, and then selfishly sell it. I still do indexing, still gather and merge the personal stories supporting the documents I continue to find…but I’m not sure I’ll ever make it totally public again just so others will profit from it.
Interesting topic. Do you have any examples of donated records that you’ve heard about getting permission to access them?
Thank you, Amy, for this realistic picture of what it takes to digitize something and make it available to the public. I work for a public library and, in conjunction with our local historical society, we have a pretty good online collection, but it has taken years and years to get here, and we are by no means finished. I hope your article enlightens people and gives them patience with the process! I plan to share your post with all my genelaogy patrons! (And, by the way, are you a librarian or archivist? You have a very thorough knowledge of this process). Keep up the good work!
I have a Masters in Library Science (though I don’t currently work in a library) and I’ve spent a lot of time in archives and special collections.
Excellent eye-opener, thanks for this!
Thank you for an excellent description of the process of digitizing records. I have the privilege of being one of those who works on entering the data once the records have been filmed.
Such an informative article of what all goes into getting records online!