Databus Issue: 2004 2 04/01/2004
Shadow Data
Andy Prestage Management AnalystWhy The Data You Don’t Want Found is Probably Still There
PDF
You may be surprised to learn how difficult it is to permanently render data inaccessible. Deleting a file and then removing it from the Recycle Bin only erases the first character of the filename and removes the pointer to the file within the file allocation table (FAT), master file table (MFT), or other file storage scheme used to locate the file on the disk.
Even though the pointer is gone, the file is still present on the drive and can easily be retrieved with a little determination and the proper utility software. Formatting the drive and even repartitioning it using FDISK, Partition Magic, or a similar utility software is equally ineffective since the data still exists on the hard drive, even after all of these activities.
Now, take it one step further and destroy the file by overwriting it. Even after overwriting, due to the physical properties of data storage and the electromechanical process of how data is written, fragments of the original file, known as “shadow data” are still accessible and can be retrieved using special forensic tools. Furthermore, shadow data can yield a wealth of information, including incriminating evidence of wrongdoing or criminal activity.
So what does all of this mean? Well, if you have data you want to get rid of, it may not be as easy as you think, unless you have a blowtorch, a ball peen hammer, a very strong magnet, and a little time. On the bright side, it also means that if you have a user who is panicked over data that was inadvertently lost, there’s a high likelihood of getting it back.
Data Rescue
There are several programs on the market capable of pulling data back from the brink of obliteration. The easiest (and most expensive) way is to use a commercial data recovery service. Another way is to use a software package designed to recover data in “unallocated space.” Program options range from Executive Software’s Undelete to Ontrack’s EasyRecovery DataRecovery to full-strength forensics products marketed only to law enforcement and other authorized purchasers. Products in this category include New Technology Inc.’s software that allows you to recover, not only data in unallocated space, but also “ambient data,” “shadow data” and data stored in NTFS alternate data streams.
What is Shadow Data?
As a K-12 technologist, you are likely familiar with the need to overwrite data multiple times, at least seven times in fact, to totally destroy it. But, have you ever wondered why seven times? Have you ever wondered what data might still be available on the hard drives contained within the computers you disposed of as surplus last summer? Your concerns are well justified due to the security hazards and confidentiality issues raised by the existence of shadow data.
During a write operation, the read-write armature stores patterns of binary data (zeroes and ones) on the surface of the hard drive. Although pinpoint accurate to a microscopic degree, this process is, in fact, limited in its precision by the physical reality that the armature is a mechanical device capable of accuracy to within a tolerable margin of error. Horizontal and vertical head alignment is just a bit different every time data is written to or read from the hard drive. An analogy may best explain this phenomenon.
Imagine you are in a kayak traveling down a familiar stretch of river. Each successive time you pass through the same stretch of river, your kayak may take a slightly different course as you negotiate your path. Imagine now the armature of a hard drive as it passes over sectors and tracks during a write operation. Each successive pass lays down new data over the top of data that may have been present before. However, due to “play” within the horizontal alignment of the write head, fringes of the prior data that lay just outside the track’s most recent write path will not be destroyed during the new write operation, leaving behind remnants and traces of legacy data, otherwise known as shadow data.
Similarly, variances in signal strength and vertical spacing between the write head and the disk surface plays a role in the dispersion of magnetic encoding during a write operation. The greater the distance between the write head and the disk surface, the wider the dispersion path of magnetic encoding.
Yet a third, and even stranger reason for shadow data exists. During a write operation, data can “seep” into cracks and imperfections of the platter surface. Successive write operations may not seep into the surface in the same manner, resulting in the layering of new data over the top of legacy data. This layering occurs due to physical imperfections in the storage media and variances in the ability of the platters coating to hold a magnetic charge. It may now be obvious why data-scrubbing software performs multiple write operations to ensure that legacy data is destroyed.
Conclusion
Shadow data can provide useful insights into the data present on a hard drive. Simply knowing that shadow data exists may help you assess the risks of data leakage, leading to more prudent decisions regarding equipment disposal. Ultimately, your informed decisions regarding the use of data-scrubbing software and/or degaussing hardware in managing data security risks pays an intangible benefit – peace of mind, knowing that you have reduced the likelihood of a security breach through data leakage in the form of shadow data.
Although it renders a disk totally useless, properly performed high-intensity degaussing (demagnetizing the disk) is probably the most effective process available to ensure the elimination of threats associated with shadow data when storage devices are being retired from service. For floppy diskettes, zip disks, and tapes, incineration is an effective method to reduce security threats associated with shadow data.
Although it is theoretically impossible to guarantee the secure destruction of data from a storage media, multiple overwrites can increase the effectiveness of a data scrub. Peter Gutman of the University of Auckland speculated, in his document “Secure Deletion of Data from Magnetic and Solid-State Memory”, that overwriting a drive 35 times may force the write head to vary magnetic effect to such an extent as to remove all traces of shadow data. Still, there is no guarantee that this extreme solution effectively wipes out all information because the process still relies on an electromechanical device—the drive controller.
With proper training and the availability of forensics-grade tools, evidence grade results can be produced from a computer forensics standpoint. However, these tools are subject to the same mechanical limitations discussed above, increasing the difficulty of producing reliable results for evidentiary use. Thankfully, these limitations are less imposing on U.S. government agencies, where the tools and techniques used in the identification and extraction of shadow data are far more accurate. Of course, these tools are not available to foreign or non-governmental agencies.
School district technology policies concerning the disposal of obsolete equipment and associated computer storage media should take into account the potentials of shadow data as a security risk. Although you don’t have to resort to the measures taken to dispose of Jimmy Hoffa’s body, with proper disposal techniques and controls in place, the possibility that shadow data can become a security problem is greatly reduced.
References:
Curt Bryson and Michael R. Anderson, Shadow Data: The Fifth Dimension of Data Security Risk [WWW document]. URL http://www.forensics-intl.com/art15.html
