Saturday, January 23, 2016

The Trouble with Indexes

I am a huge proponent of indexing, especially as a volunteer. I believe that the volume of records left to be made available to the public is so enormous, that volunteer indexing is the only way many records will ever see the light of day. There simply aren't enough resources--in money or time--for any one organization to pay for what records exist to be indexed.

So it falls to every genealogist, the users of these records, to seek volunteer opportunities to index records wherever possible. Everyone who cares about these records, and the people listed on them, must become a well-trained custodian of the past. How we index records is a part of that stewardship.

So when we consider what makes up a good index of a record set, I think we can agree on some core elements.

    Accuracy

    The index that is created must be an accurate representation of what is written on the records. Some errors will always present themselves due to transcription, and these are especially understandable with handwritten records. But an index greatly loses value when all names and details that appear on the records are not included, or the index misrepresents any data points.

    Searchable

    The index should be searchable by all relevant data points. And I'm consistently in the camp that says that if it appears on the record, it's relevant information. While it may not be practical to make every record searchable by every data point, an index's search functionality should be as inclusive as possible to every name that appears on the record.

    One significant example of a commonly missing search parameter is race. At the time of this writing, the only website of the Big Four (Ancestry.com, My Heritage, FindMyPast, and FamilySearch) that allows you to search universally by race is Ancestry.com. This creates a significant impediment to doing research for anyone of African descent.

    The problem becomes compounded increasingly when the black individuals for which I'm searching are not African American, and have no connection to the United States. But the example I'll be addressing below applies specifically to the challenges of indexing African American records.

    Availability of Original Images

    Not every index can provide images for original records. To do so is often cost prohibitive, especially for smaller organizations that are new to indexing.

    However, every index should communicate the origins for the information being indexed. Date ranges, specific locations, repository, and all other information necessary to obtain copies of the original images should be present in the index description and item descriptions.

    Not only does this aid in the crafting of quality source citations, but makes it possible for interested parties to request original copies.

    I think we can all agree that the organization that makes it easiest for genealogists to index records on a large scale is FamilySearch. Their interface has provided the standard for any organization wanting to engage their communities in indexing efforts. What they deliver, especially since it relies almost entirely on volunteers, is impressive.

    But sometimes even they can miss the mark.

    Pittsylvania County, Virignia Death Index on FamilySearch: A Bad Example

    What happens when an index is created, but many of the above points are ignored? How does it affect a person's ability to find desired records?

    After the Ben Affleck/Finding Your Roots controversy hit the fan, it really got me thinking about what the best approach is to document slavery. And while I think everyone has a right of privacy to share or withhold whatever they want about their family to an international audience, I think slavery demands more of genealogists and family historians. Because I have slave holders and slaves throughout various lines of my family, I've decided to document both groups with total openness and objectivity.

    Pittsylvania County, Virginia has a treasure trove of records in comparison to other communities in the South, including for African Americans. I decided to start there, which led me to Virginia Deaths and Burials, 1853-1912 on FamilySearch.

    When I first sat down to do this project in September of last year, there was no way to search this collection by race. As of this writing, that has since changed. Because I was specifically interested in slaves held by the Keatts family in Pittsylvania County, my search parameters included the years 1853-1865.

    The following are examples of records I found.




    One issue I noticed, having seen both the originals and another index of these records, is the placement of the slave holder's name. As you can see here, it has been placed in the father's name position. While I cannot comment on the paternity of any of the slaves owned by my family, I can vouch for the fact that the original records made no such attempt. On the original records as provided by the other index, Richard Keatts is labelled in a column specifically designated for the owner of deceased slaves. He is also listed as the Consort, or informant.





    The practice of putting the slave owner's name in the father's name field is consistent throughout the collection, regardless of the gender or relationship of the owner. Aletha "Letty" Keatts is female, the sister of Richard Keatts, and her name also appears in the father's field.

    Upon closer inspection of the FamilySearch collection, giving the owner's name, followed by "(Owner)," and providing that information as the father's name appears to be the convention for all records related to slaves in Pittsylvania County. In order to have that degree of compliance with this many records, this has to be what the indexers of the collection were instructed to do. However, without the original images, I cannot say whether every slave holder in the FamilySearch collection has such a designation. Additionally, that convention is not disclosed in any description of the collection, or in the Known Issues page of the collection that I could find.

    Additionally, the emancipation status of every African American was stated plainly on the original records. Whether the deceased was white was answered with a Yes or No. In a second column, labeled "Colored," their emancipation status was listed as either "Slave" or "Free." However, that information was not indexed in the FamilySearch collection.

    While it may be possible to isolate all of the enslaved African Americans by using the race search box and searching for "Owner" in the father's name field, there are some issues with this approach. The first is that I cannot determine if every slave holder has such a designation. The second issue is that every search result with that designation appears twice--once as a result for the deceased individual, and once for the so-called father/owner.

    As a result, anyone trying to find enslaved ancestors in Pittsylvania County for this time period has to comb through a results list full of duplicates. Anyone trying to find emancipated ancestors for the same time period is unable to isolate these results from everything else. Given that this information is so clearly stated on the records, the real issue here is the way the records were indexed. The indexing program simply didn't have fields to index the names of slave holders.

    But if real efforts are going to be made to index records pertaining to African Americans and their ancestors, these improvements to the indexing program need to be made. And the fastest, simplest way to correct all of the issues related to these records is to re-index the collection.

    While it is possible to rely on user submitted corrections to individual records, which FamilySearch stated to me as their proposed solution, the lack of thorough correction to all affected records shows a real lack of accountability for the situation they've created. Some might also say that an inaccurate or "quirky" index is better than no index at all. But for errors that so disproportionately affect the African American community, such a glib response is unconscionable. It leaves us to wonder how many other collections related to slavery have similarly botched indexes, and what FamilySearch is doing to identify and correct these issues.

    A Lesson Learned

    The most important lesson to take from this example is that indexing efforts should be well-planned if we expect them to be well-executed. Taking shortcuts, or trying to avoid proper adaptation of current resources to created a true derivative, ultimately creates more work than it alleviates.

    Because of the way computer databases are constructed on the back end, the only chance there is to address multiple records at once is when the collection is indexed. After that, it is a one-by-one, tedious effort to do any corrections.

    Indexes can be incredibly useful. They serve a necessary, low-cost function in providing free access to records. But unfortunately, there is a dark and messy underbelly to them of which every genealogist should be aware before using them.



    The issues that come with them sometimes make them about as useful as another brick in the wall.

    Sunday, January 3, 2016

    Keatts Family Wills of Pittsylvania County, Virginia: A Wish Granted

    On my research trip to the Pittsylvania County Courthouse, I was saddened that I wasn't beginning to scratch the surface of their vast collection. While my time was well focused on the research questions I had at the time, I knew full well that there were hundreds of other records I would never have time to see. Included in these passed over records were many of the wills for my earliest connections to Pittsylvania County in the Keatts family. It pained me to pass over their surnames in the record books I knew were pertinent to me, in exchange for the records I'd planned to collect that day.

    So the weekend that the Ancestry.com probate collection launched, I was full of hope that the records would include some of these. And when I got my first Keatts hints to the collection, I celebrated...




    However, it was premature. 


     
     Is there anything worse than an interrupted happy dance?
    (The answer is no. No, there isn't.)

    You see, the hint lead to an error page. And I sat there for a good five minutes, refreshing in disbelief. I felt lied to. Betrayed. Filled with first world rage. And as I tried to navigate to the collection directly from the Card Catalog, only to be greeted by "No results found" for every combination of search criteria, I finally gave up. No matter how hard I tried, I couldn't get any results for anyone in Pittsylvania County to appear. Something was wrong, broken, or both.

    As the hints continued to accumulate for this ghost collection, and each of them led to the same error page, my frustration mounted. I contacted Ancestry directly to point out the issue, and they said they'd investigate. 

    Meanwhile, I ignored every single hint and told the matching algorithm to stick them in its eye...



    ...among other things I won't repeat in polite company.

    Here's the thing y'all. Just because we subscribe to record collections which are held by large corporations, it doesn't mean we shouldn't communicate with them. Providing user feedback is all the more important when the companies we deal with are large. I think sometimes we have the impression that this does no good because we don't feel like we're talking to a person. Instead, it feels like we're talking to a mechanical collective of voices with no thoughts of its own.

    But corporations like Ancestry.com are made up of people, including other genealogists. And they really do want to see us happy. Like a genie of sorts, they want to give us everything we wish for. And like magic, their money and influence give them the power to deliver many things we can't do for ourselves. And just like a genie, even their power has its limits.

    But perhaps the ways in which Ancestry.com (and other companies too) are most like genies, is that the only way to never get what you wish for is to never tell them what you want.

    So I reached out. I told them what was wrong. I wanted Pittsylvania County, Virginia probates and wills. I wanted the error message to go away and let me get on with my happy dancing. Because there is absolutely nothing worse than taking back a happy dance. It should say right on the receipt  NO REFUNDS in big, ugly letters.

    And I don't know whether they finally got the records, or the database working, or what. I don't care which of my wishes they granted. The point is, the records have arrived!

    The Pittsylvania County, Virginia probate and will database is up, running, and open for business!





    If you, like me, are looking for the Keatts family, allow me to get you started. And let me know in comments, because I want to know who you are!

    Index


    Virginia, Wills and Probate Records, 1652-1983. Pittsylvania County, "Wills, 1800-1870" Ancestry.com (http://search.ancestry.com/search/db.aspx?dbid=9085 : accessed 2 January 2016), index entries for Keatts family members, Image #33, Pg. 27.


    Curtis Keatts: My 6x Great Grandfather


    Virginia, Wills and Probate Records, 1652-1983. Pittsylvania County, "Deeds and Will Book, Vol. 11, 1780-1820," Ancestry.com (http://search.ancestry.com/search/db.aspx?dbid=9085 : accessed 2 January 2016), entry for Curtis Keatts, Image #145, Pg. 268.

    Charles Keatts: My 5x Great Grandfather



    Virginia, Wills and Probate Records, 1652-1983. Pittsylvania County, "General Index to Wills, Vol. 1, 1767-1948," Ancestry.com (http://search.ancestry.com/search/db.aspx?dbid=9085 : accessed 2 January 2016), entry for Charles Keatts, Image #377-378, Pg. 185-186.

    Randolph Keatts: My 4x Great Grandfather


    Virginia, Wills and Probate Records, 1652-1983. Pittsylvania County, "General Index to Wills, Vol. 1, 1767-1948," Ancestry.com (http://search.ancestry.com/search/db.aspx?dbid=9085 : accessed 2 January 2016), entry for Randolph Keatts, Image #813, Pg. 480.

    Popular Posts