On the Edge: A New ERA for NARA

Technology and records experts agree that many of the electronic files being created today will not be accessible 50, 40, or even 30 years from now. Ever-changing formats and computer systems will leave government, business, and personal e-records incapable of being viewed because there is no current technology that can preserve digital files in a manner that will ensure they can be accessed in the future.

Nikki Swartz

Many years ago, the National Archives and Records Administration (NARA) recognized this problem and launched a strategic initiative to preserve and provide long-term access to e-records created by the U.S. government. The Electronic Records Archives (ERA) system also is designed to manage the ever-increasing volumes of e-mail and other e-records created and received by more than 500 U.S. federal agencies.

But that’s not all: NARA officials say ERA will not only ensure the long-term viability of government records, but it also will change how NARA works and how it interacts with other agencies, other institutions, and the public.

A Critical System

According to NARA, ERA will be a comprehensive, systematic, and dynamic means for preserving virtually any kind of e-record, regardless of the specific hardware or software used to create it. When fully operational, ERA will support the National Archives’ mission by allowing government officials and the public to easily access government records and enabling NARA to deliver digital records in usable formats.

ERA is a mission-critical system for NARA, whose vision is to preserve any type of e-record created on any type of computing platform anywhere in the U.S. government and provide discovery and delivery for such records to anyone who has a legal right to access them.

In doing so, it must help the more than 500 federal agencies manage their records; provide access to all types of records in the National Archives, presidential libraries, and federal records centers; and find a way to preserve and provide long-term access to an ever larger and increasingly complex collection of e-records in a way that ensures records’ authenticity.

The system’s importance to NARA cannot be overstated. “Frankly there can’t be National Archives for the 21st century without ERA,” Ken Thibodeau, director of the ERA program, recently told FCW.com.

According to archives.gov, the ERA system will:

• Provide a way for NARA to take in, preserve, and provide continued access to e-records created by the three branches of the U.S. government
• Make it easier for federal government agencies and NARA to conduct business and to pool critical information quickly to make important decisions
• Provide NARA researchers with e-records they can access on both today’s computer systems and future systems
• Improve the public’s ability to view and retrieve records without having to physically visit a NARA facility

To fulfill NARA’s mission to protect and preserve government records, the ERA system has to enable digital preservation of records as well as support the lifecycle management of all types of records. In response to these challenges, NARA and contractor Lockheed Martin Corp. are creating a systemwithin a system. The external system, according to “The Electronic Records Archives Program at the National Archives and Records Administration,” authored by Thibodeau, will support lifecycle management processes for all government records, physical and digital. An internal system will allow NARA to ingest, preserve, and provide access to e-records.

According to NARA, when all its planned capabilities are added, ERA will provide the following major benefits:

• Improved, easier, and faster access to NARA’s records and to the services it offers the government and the public
• A single portal, providing “one-stop shopping” for access to records and services
• Automated aids for searching NARA records, for managing the lifecycle of government records, and for maximizing access to those records, while respecting confidentiality
• A comprehensive, consistent approach to managing records throughout their entire lifecycle
• Preservation and sustained access to enormous quantities and unlimited varieties of e-records
• Online communication and collaboration tools to allow NARA to better interact with the various stakeholders it serves

To accomplish all that, aswell as preserve and provide access to e-records indefinitely into the future, NARA said the ERA systemis designed to be:

• Evolvable: To enable preservation and access over generations of information technology, any and all hardware or software must be replaceable, while the system as a whole continues to provide all required functionality and preserve the authenticity of the records stored in it.
• Scalable in both directions: ERAwill need to handle one billion images from the Department of Defense, for example, and manage relatively small, specialized collections that must be kept separate.
• Extensible: NARA must be able to modify ERA to manage, preserve, and provide access to new e-records formats that are created as technology progresses over the next five, 10, or 100 years.

System Status

In September 2005, NARA awarded Lockheed Martin a $300 million contract for the development of the ERA system. Since then, NARA and Lockheed Martin have worked together to refine the systemrequirements, articulate the system architecture, design for the functions NARA needs in the first deployment of the system, and conduct related activities to ensure the delivery of a quality system.

ERA development will be incremental, with Lockheed Martin delivering capabilities in five deployments through 2012, according to Rita Cacas, ERA communications officer. ERA’s initial operating capability (IOC) was initially scheduled for September 2007, but it was pushed back to June 2008 to ensure enough time for development and testing. On August 16, 2007, Lockheed Martin delivered an “IOC Forward Plan” to NARA, proposing the delivery of a pilot version of the ERA system in three parts, or “drops,” through June 2008. Each of the three software drops will increase the pilot system’s functions covering specific NARA records management processes, to provide the following capabilities:

• Create and process records schedules
• Request transfer of e-records
• Transfer e-records
• Automatically inspect the transferred e-records

The three drops under the Forward Plan were delivered, on time, September 28 and December 21, 2007, and March 7, 2008. NARA’s tests of the first two drops showed that Lockheed Martin satisfied the requirements for those stages. As this issue of The Information Management Journal went to press, NARA was evaluating the third drop. The next step is for Lockheed Martin to deliver a revised system that corrects problems identified in testing the pilot drops. NARAwill thoroughly test the system for end-to-end functionality, performance, and sustainability.

After Lockheed Martin addresses any concerns identified during NARA’s testing, it plans to release the IOC system for a final round of government testing in May. Cacas said NARA will complete product and operational testing and put the system into operation by the end of June.

Then, NARA will implement the system for use by its staff and four federal agencies: the U.S. Patent and Trademark Office, the Department of Energy’s National Nuclear Security Administration/Kansas City Plant, the Department of Labor’s Bureau of Labor Statistics, and the Naval Oceanographic Office. According to Cacas, NARA and Lockheed Martin expect that lessons learned from the initial use by the agencies and NARA staff will help improve the system and will factor into NARA’s collaboration with other agencies as it gradually opens the system for use by the entire federal government.

Beyond the IOC milestone, which marks the end of Increment 1, there are four more increments to be deployed before ERA reaches its full operating capability (FOC) sometime around 2012. Cacas said each increment will add new capabilities, such as increased storage, records appraisal, public access, more robust preservation features, and support for additional file formats.

Past and Present Challenges

To meet the current and future challenges of preserving e-records and ensuring their long-term accessibility, the ERA system has to include a full range of capabilities. The five planned increments, which will lead ERA to FOC, can only be completed as Congress approves funding for ERA – something that will, no doubt, remain a big challenge for NARA in today’s economic climate.

So far, lawmakers have recognized the importance of ERA and have given it budget increases each of the past three years. According to Thibodeau, the enacted appropriations for ERA for fiscal year 2007 were $45.5 million, and it received $58 million for 2008. The total potential value of the Lockheed Martin contract, if all options are exercised through 2012, remains at approximately $310 million, he said.

In its current, Increment-1 stage, ERA enables NARA to ingest or take in records, create records schedules and online records forms, and then verify records received from agencies. As ERA moves toward the four subsequent increments, Cacas said preservation, storage, and workflow capabilities are planned. She said she is hopeful that these capabilities and more will be part of ERA at FOC.

Another big challenge for ERA concerns the sheer volume of records and the plethora of different records formats that exist. On its website, NARA provides guidance for transferring just six types of e-records files it can accept, including PDFs and e-mail attachments. But by Cacas’ estimation, more than 4,500 types of records formats exist today.

NARA must be able to take in records created by more than 500 federal agencies and their bureaus and divisions. Typically, Cacas said, each creates its own records, uses different formats and systems, and employs its own records management processes. Some agencies create terabytes (1 trillion bytes), even petabytes (1,024 terabytes), of data on a daily basis.

“We’re dealing with four agencies now – imagine when we open the floodgates to those 500 agencies,” she said.

The complex e-records challenges faced by NARA and other organizations around the world – technological obsolescence, increasing digital information formats, increasing complexity of digital files, and enormous growth in volumes – are so universal that collaborating to find technological solutions seems a no-brainer. NARA, in fact, is working with many other groups in and outside of the federal government in its efforts to meet these challenges, at least for government e-records, via ERA.

One of the main components of the system is research, so NARA is working with broad research communities – including computer scientists, engineers, archivists, and information scientists in government, academia, and the private sector, across the United States and internationally.

The research aims at both improving NARA’s understanding of the requirements for preserving authentic e-records and investigating technologies that will be available within the next five to 10 years to address those requirements. Because the ERA system will be flexible, it should be able to take advantage of new technologies. The ERA research team has developed collaborative partnerships with the National Science Foundation, the San Diego Super Computer Center, the Library of Congress, and the Digital Library Federation, among others, to support such developments.

Will It Change the World?

According to Cacas, NARA plans to gradually roll out the ERA system to federal agencies. There are both technical and organizational drivers for gradual expansion in the number of agencies using the system. Technically, to accommodate additional agencies the system has to expand, which means NARA must balance expansion costs with the costs of developing additional capacities needed to realize the vision for ERA. Organizationally, Cacas said ERA plans to change the way NARA interacts with other agencies by both automating and systematizing processes that are currently manual and paper-based. Before ERA’s potential can be fully realized, however, she said NARA must provide training for agency staffs as well as ensure that NARA liaisons can support agencies using ERA.

One thing seems certain: ERA will change NARA and the federal government’s records management procedures and enable them to communicate and collaborate much more easily and effectively than they have in the past. For instance, in the past, agencies and NARA traded paper forms in an effort to schedule records. ERA will enable business transactions to be negotiated and forms to be proposed online. With a workflow capability planned to be added the system in the near future,managers will also be able to approve such forms online.

The larger potential of ERA is that it probably will influence the e-records management procedures of state governments and possibly even those of private and public organizations, both in the United States and around the world.

Records managers everywhere will be watching closely.

Nikki Swartz can be contacted at nikkiswartz@hotmail.com.

From May - June 2008