Legal Watch: Shedding Light on Backup Tape E-Discovery
The rise of electronic discovery has shed light into areas of IT back office operations previously known only to system and network administrators. The practice of making backup copies of computer data is a well-established practice that protects against the loss of data should computer hard drives fail – an inevitable event.
Traditionally, these backups have been made to tape media, as that represents the best balance between cost and efficiency to achieve the objective of copying large quantities of data to the backup media in a reasonable amount of time.
W. Lawrence Wescott II, Esq.
Backups usually occur after hours, because files in use by users are generally locked and will not be copied to the backup media. A backup represents a snapshot of the computer at the time the backup was made.
Original Intent of Backups – Disaster Recovery
Backup tapes are best suited for disaster recovery. Under that scenario, an organization will keep the most recent two or three backups of its computers because if the hard drive crashes, the goal is to restore the most recent available data. Assuming a complete backup in a single backup cycle, if the first backup is bad – due to a tape failure, for example – the next good backup is utilized. After the third or fourth backup, the tapes are recycled or otherwise reused or replaced.
If an organization uses backup tapes in this manner, these tapes will never be subject to electronic discovery. By the time a lawsuit is filed (or the duty to preserve information for the suit arises, which can be before the suit is filed), the data is long gone; the tapes having been recycled many times over.
Common Use of Backups – Data History
However, the far more common practice in the IT backup environment is to keep certain tapes for longer periods of time. Assuming a five-day work week, nightly backups are made Monday through Thursday, and Friday is designated as a weekly backup.
While the Monday-Thursday tapes are reused every Monday through Thursday, the Friday weekly tapes may be saved for the entire month and then reused for the next month. Or, they may be saved for several months before being recycled.
A monthly tape is made at the end of the month, and the monthly tapes may be saved for the entire year before being recycled. Or, they may be kept for several years.
Annual tapes can be made at year-end, and they may be saved forever. This is called the “grandfather, father, son” tape rotation protocol, and it is one of the most frequently used protocols.
One vendor has stated that “by incorporating a mix of daily, weekly, monthly, and quarterly backups, you can be sure that you have a complete history of your data from various points in time.” However, capturing data history is an entirely different purpose than preparing for disaster recovery.
Backup Tapes Subject to E-Discovery
Under the broad definition of “electronically stored information,” or ESI, as provided in the 2006 amendments to the U.S. Federal Rules of Civil Procedure, these backup tapes have become the subject of discovery. The original purpose of backup tapes as insurance against a hard drive crash has been transformed into something else altogether – the nightmare of backup tape electronic discovery.
Backup tapes have affected electronic discovery inways entirely unanticipated from when they were used for their original purpose. Organizations now have to consider the ramification of backup tapes relative to the following e-discovery issues:
1. The duty to preserve information
A long-standing principle in the court system is that a litigant must keep information relevant to a lawsuit. In recent years, this has been expanded to require potential litigants to keep information if they have reason to believe that a claim will turn into a lawsuit.
The scope of the legal hold has extended to backup tapes – in a big way. Anecdotal evidence exists of companies being forced to save tens of thousands of backup tapes pursuant to this obligation. In The Sedona Principles: Best Practices Recommendations & Principles for Addressing Electronic Document Production, The Sedona Conference®, a think tank concerned with issues in the area of antitrust law, complex litigation, and intellectual property rights, noted that “the Working Group has firsthand experience of unreasonable and unfair burdens in producing electronic documents in litigation. These unfair burdens have included … preserving at great cost thousands of backup tapes that were subsequently not even sought by the opposing party later in discovery.”
This requirement has its own set of costs:
• Tape costs
Because the tapes must be preserved, they cannot be recycled, and new tapes must be bought to capture the data under hold. When the claim involves a continuing violation, tapes must be purchased to continuously capture the data.
• Storage costs
As the tapes pile up, they must be put somewhere. “Somewhere” is usually offsite in a secure location to protect the tapes against disaster.
2. The resources needed to collect data
In many instances, organizations may no longer have the equipment to restore the tapes, and they must hire an outside vendor for this purpose. Even if they have the equipment, scarce IT resources (personnel and machines) must be used for the restoration process, which can vary greatly. One court observed that “each backup tape may take anywhere from several minutes to five days to restore.” Multiply this by the number of tapes for one machine, and then again for several machines, and costs can quickly mount.
3. The need to cull data
Once data is retrieved froma tape, processing is required to get rid of the extraneous data. Tapes will contain
system and application data, as well as word processing documents and spreadsheets. Restoration of e-mail can pose a special challenge, as many e-mail systems store e-mail in a large, undifferentiated database, which can span several tapes and may require a computer at the server level to access.
Duplication of data is a huge problemas well. Since backup tapes merely represent a snapshot in time, many documents will appear on multiple tapes. E-mail duplication is its own special category, as copies of documents may multiply exponentially as e-mails are forwarded to users or become part of e-mail chains. Ironically, many relevant documents may not appear on backup tapes at all. If a document is created and deleted between backups, it will never be captured on a tape.
4. The costs of review
“Culling” is the result of a first pass at the data. Sometimes, a reviewer might get lucky and discover all the relevant material neatly categorized and stored in a folder. The more likely outcome is that documents are scattered over the machine – and someone has to make the call as to whether a particular document is relevant to the claim.
The volume of documents that remains after the culling process, particularly on a network server backup tape, makes loading the data into some type of repository almost mandatory. Although the costs of such repositories can be substantial, they are dwarfed by the costs of review. The Sedona Conference®, assuming review costs of $200 per hour, estimates the costs of review of one gigabyte of data at $32,000.
5. The neutering of document retention policies
One of the objectives of companies in implementing a document retention policy is to reduce the volume of information in the organization. Only corporate records having value to the organization are retained. Backup tapes subvert this process. Documents that are removed from the system at the end of their lifecycles are never removed from the backup tapes. If they happen to exist on those yearly backups that are kept forever, those documents also live forever.
Backup tapes become a tempting target for counsel whose object is to increase the cost of litigation to the point where the organization is forced to settle rather than engage in ruinous discovery – they can argue the tapes contain unique content no longer present on the organization’s servers.
6. Spoliation of evidence
“Spoliation” is a legal doctrine involving destruction of evidence. A spoliation dispute can change the focus of a lawsuit away from the merits of the action to discovery problems. Backup tapes have played key roles in spoliation matters.
A typical scenario can play out as follows: A party in panic mode who has been sued may try to delete incriminating documents, or the deletion may simply be by mistake. When the evidence is originally collected from the active system, the documents are missing.
The plaintiffs might have copies of the documents, perhaps forwarded by a sympathetic insider. The plaintiffs point to their own documents, which are missing from the defendants’ production. The plaintiffs claim justification to search the defendants’ backup tapes and find the missing documents are on the tapes.
But, recall that backup tapes are only snapshots of the system at a particular point in time. The plaintiffs argue this point, painting a picture of an even more sweeping destruction of documents. An extreme example of this occurred in Coleman v. Morgan Stanley, No. 502003CA005045XXOCAI. (Fla. Cir. Ct. Mar. 1, 2005). Despite certifying their production was complete, Morgan Stanley kept finding backup tapes. The direct result of backup tape production issues was a judgment in compensatory and punitive damages against Morgan Stanley in excess of $1 billion.
Accessible – or Not?
In the legal arena, it appeared early on that backup tapes might not have a significant role in e-discovery. In Zubulake v. UBS Warburg, 217 F.R.D. 309 (S.D.N.Y. 2003), the court identified backup tapes as falling within the category of inaccessible data or data that was unduly burdensome or expensive to produce.
In a later opinion, the Zubulake judge made exceptions for backup tapes “actively used for information retrieval,” or tapes that could be identified as containing documents of the key players in the litigation. Such tapes would be considered accessible. See Zubulake v. UBS Warburg, 229 F.R.D. 422 (2004).
Normally, parties that produce data must bear the cost of production. However, data considered inaccessible is eligible for cost-shifting, or shifting of the cost of production, partially or entirely to the requesting party. The court observed that cost-shifting has the potential to end discovery, because of the potentially huge costs.
It is perhaps for this reason that backup tapes are often the subject of discovery disputes. If counsel can force the other side to search and produce data from backup tapes, and to pay for it, they can acquire increased leverage for settling the case.
However, some courts have found that searching backup tapes is not worth the effort. In Petcou v. C.H. Robinson Worldwide, Inc., 2008 U.S. Dist. LEXIS 13723 (N.D. Ga. Feb. 25 2008), an employment discrimination action, plaintiffs sought e-mails “of a sexual or gender derogatory nature sent from 1998 through 2006” from the defendant’s backup tapes.
The court denied the request, noting the high cost ($79,300 to retrieve two years worth of e-mails from just one employee) and the extremely broad nature of the request. In addition, in order to determine whether an e-mail was “of a sexual or gender derogatory nature,” each e-mail would have to be examined and a judgment made as to whether it was or was not. Finally, the e-mails would not provide any new information not already provided by the plaintiffs.
High costs in themselves are not necessarily indicative of an “undue burden.” In Young v. Pleasant Valley School District, 2008 U.S.Dist. LEXIS 55585 (M.D. Pa. Jul. 21, 2008), the court refused to order the school district to rebuild an e-mail server no longer used, even when the cost would have been $5,000. While not high in itself, this costwas “a significant burden to a public school system.” More important, the information was available from alternative, less costly sources – the plaintiffs could ask the parents of the students in the defendant teacher’s classes whether they had filed any complaints against the teacher.
Warning for the Unwary
An example of how backup tapes can be a trap for the unwary can be found in In re Fannie Mae Securities Litigation, 552 F.3d 814 (D.C. Cir. 2009). After a series of disputes, the lower court ruled that the Office of Federal Housing Enterprise Oversight (OFHEO) had to search what the appellate court referred to as its “disaster recovery backup tapes,” despite OFHEO’s protests it had never agreed with the plaintiffs that those tapes be searched.
The agreement stated OFHEO would work with the defendants to develop “appropriate search terms.” When the defendants came back with 400 search terms, OFHEO protested the term “appropriate” placed a limitation on the terms, but to no avail. The court held that the agreement “gave the…defendants sole discretion to specify search terms and imposed no limits on permissible terms.”
The 400 terms produced 660,000 documents, ultimately costing more than $6 million to review and produce, an amount that represented 9% of the agency’s entire budget. In addition, the agency got behind schedule on the review as it repeatedly underestimated the level of effort required to undertake the review. The court ended up imposing sanctions on OFHEO for its discovery failures.
A side issue in the Fannie Mae case, unaddressed by the courts, was the reference to “disaster recovery backup tapes.” Had the tapes actually been used for disaster recovery, the requested data would have been long gone. There was no Zubulake-type analysis as to whether the tapes were used “solely for disaster recovery” or “actively used for information retrieval.” However, the fact the requested information existed at all strongly points to the latter scenario.
Strategies to Minimize Risks
Before the advent of electronic discovery, a grandfather, father, son backup tape rotation policy was probably a good idea. Tapes were an inexpensive means of insurance against the inadvertent deletion of data. Electronic discovery has driven the cost of that insurance into the stratosphere. What can organizations do to address the problem?
1. Implement a records retention schedule.
A retention schedule forces organizations to focus on business information that is truly important. It also gives them permission to destroy information no longer needed (as long as it is not relevant to pending lawsuits, audits, or investigations).
2. Consider an archiving solution.
Archiving solutions, which allow information to be moved from primary production and storage servers to less expensive resources, are designed to preserve data by archiving older data fromprimary systems only as new data is created. They can also be used to set and enforce retention policies. When the costs of backup tape e-discovery are compared to the benefits of improving information retrieval, an archiving solution may actually be cost-effective, particularly when a records retention schedule is also implemented.
3. Focus on appropriate use of backup tapes.
There may be situations in which backup tapes make sense; for example, a year-end backup of the accounting system to ensure the information is available for tax purposes. However, the value of a year-end backup must be carefully weighed against the potential negative consequences discussed earlier.
The days of keeping backup tapes “just in case” are over. Companies that are (or may become) involved in electronic discovery should refocus their backup strategy on disaster recovery – before their backup tapes become the source of a new disaster.
W. Lawrence Wescott II, Esq., can be contacted at firstname.lastname@example.org.
From September - October 2009