Federal electronic records that document our nation's history, preserve our rights and maintain critical archival materials are at great risk.
There is growing concern, inside and outside of Congress, about federal agencies' management of vital electronic records. Many agencies do not have updated or workable plans to preserve them, and in the digital age, that is totally unacceptable.
During a recent self-assessment administered by the National Archives, 79 percent of agencies were found to be at moderate or high risk of improper destruction of their electronic records.
Records management has received — and continues to receive — low priority within agencies, especially at the highest levels. According to the National Archives, agencies are "falling short in carrying out their records management responsibilities, particularly regarding the exponential use and growth of electronic records."
I have begun to examine solutions — including legislation — that would ensure these records are not lost. H.R. 1387, the Electronic Message Preservation Act, of which I am an original co-sponsor, is a good start.
The bill passed the House in March, and I hope the Senate will act on it as soon as possible. H.R. 1387 would establish uniform standards for the capture, management, preservation and retrieval of federal agency and presidential electronic records.
We also need to update the Federal Records Act to reflect the unique challenges of all electronic records and provide additional oversight and enforcement authority for the archivist of the United States. Recently, some people have even called for expanding private rights of action under the records act.
All federal agencies create electronic records, such as word processing documents, spreadsheets, databases, Web 2.0 systems and e-mail. But not every record that an agency creates is considered to be permanently valuable.
In fact, only 3 percent are eventually transferred to the National Archives for preservation. The other 97 percent are temporary and must be managed by the agency, for various lengths of time, according to statutes and records schedules.
Agencies are required to preserve records of the conduct of government business that is "complete and accurate to the extent required to document the organization, functions, policies, decisions, procedures, and essential transactions of the agency," according to federal rules. Agencies must also preserve records that "furnish the information necessary to protect the legal and financial rights of the Government and of persons directly affected by the agency's activities."
It is the responsibility of each agency, with the support of the National Archives, to economically and effectively create and manage these records.
The National Archives has an additional responsibility to make sure that records of permanent archival value are preserved and made available for future generations.
The threat to the preservation of federal electronic records is not new. As early as the 1960s, the National Archives began examining methods for long-term preservation of "machine-readable" records. However, progress has been slow for both the archives and agencies.
As recently as the late 1980s, the preferred solution to long-term preservation of these records was to put them on microfilm. And as recently as a few years ago, the National Archives was suggesting that agencies print and file their electronic records.
However, archivists, historians and open-government advocates now understand that not only the content of electronic records must be preserved, but their essential characteristics as well. We need to protect not only the data, but also the metadata — the data about the data.
Electronic record preservation must include information on who created the record, in what application it originated and many more data points that assist in not only the retrieval but also the understanding of a record. Manually searching millions of e-mails that have been printed and filed into boxes is certainly time-consuming. However, even when a printed e-mail is located, valuable information that would have been retained in a records management application is irrevocably lost.
The National Archives has endorsed for all federal agencies the Department of Defense Standard 5015.2-STD version 2, which defines requirements that must be met by records management application products.
In 2003, the Government Accountability Office testified at a subcommittee hearing that "most electronic records ... remain unscheduled ... and as a result, they were at risk of loss."
Last month, the GAO again testified that "if records are poorly managed, individuals might lose access to benefits for which they are entitled, the government could be exposed to legal liabilities, and records of historical interest could be lost forever."
Initiatives adopted by the Obama administration over the past year, as well as renewed Congressional oversight, have produced signs of hope.
As chairman of the Subcommittee on Information Policy, Census and National Archives, I am committed to exploring innovative ways to significantly improve federal electronic records management policies and practices.
The goal is threefold: to ensure that agencies appropriately preserve electronic records, that the National Archives properly maintain and make available those records, and that the ultimate users — citizens, historians, courts, agencies or Congress — be able to utilize the records to the fullest extent possible.
Rep. William Lacy Clay (D-Mo.) is chairman of the Oversight and Government Reform Subcommittee on Information Policy, Census and National Archives.