Every so often, a new research report (often conducted by an ITAD service provider) will surface that surveys IT asset management professionals regarding their top concerns in IT asset disposition. Invariably, data security is the #1 priority. The need to protect sensitive hard drive when IT equipment is being decommissioned is also the top reason why businesses will seek out a properly certified and experience ITAD service provider.
And rightfully so, considering that each data breach costs companies $4 million, as I pointed out in my last blog To Erase or Not to Erase: Never the Question in Data Security Best Practices. Further justifying data security as the top ITAD concern, since 2015 more than 7,600 data breaches have occurred in the United States, affecting nearly one billion records, this according to the Privacy Rights Clearinghouse.
Of course, these statistics only include the data breaches that have been made public based on data security breach notification laws, which require businesses to take prescribed steps to notify customers about a breach and remediate injuries caused by the breach.
Additionally, the Privacy Rights Clearinghouse totals do not include the exponential number of data breach remediation events that are undertaken when data-bearing IT assets are potentially lost or stolen. A remediation event often ensues when a company cannot account for a data-bearing device but isn’t 100% sure whether the asset has actually been lost or stolen. This commonly comes to light when the detailed equipment inventory report provided by the ITAD vendor does not identically match the client’s asset management system records. Thus, triggering a remediation event—some days or weeks after the equipment had been recovered and processed by the vendor—to determine if a data security breach actually occurred, and whether the incident legally must be reported. While these remediation events are not as costly as a reported data breach, they can require hundreds of investigative hours trying to locate the missing device(s), if they can even be found, and determining the cause.
All this underscores two critical, yet interrelated points. First, ITAD security needs to start with serialized accountability for every IT asset, and for all storage devices within each IT asset. Second, it should end with a certificate of proof that every fragment of data has been verifiably destroyed. Plus, if you’re truly interested in eliminating data security breach risk rather than just mitigating it, as I discussed in my previous blog, then both IT security functions must be conducted onsite—before equipment ever leaves the facility.
But what makes asset and data security particularly challenging in the data center, and why is it so critically important to utilize the services of an ITAD specialist for your during data center decommissioning needs? We’ll discuss the IT asset security challenges in this blog and address the data security challenges in my next blog.
IT Asset Security Challenges of Data Center ITAD
While certifiably destroying all data should be the ultimate objective of IT security during data center decommissioning, you cannot erase hard drives you cannot account for or locate. You cannot shred, degauss, or otherwise destroy those hard drives either. And if your data security strategy calls for secure shipment, then you certainly do not want to transport dirty drives you don’t even know you have or may be missing. As such, regardless of your data security strategy, the data security process must begin with asset accountability. The following are some of the more considerable challenges in accounting for IT assets onsite in the data center:
- The volume of data center IT assets complicates security and accountability. As discussed in this blog, at ITRenew our typical data center decommissioning jobs entail processing about 30-50 racks, yet we’ve managed projects to decommission more than 500 racks at a time. Each rack can contain upwards of 60 servers, each of which can contain anywhere from several hard drives to a few dozen data-storing components (more on this below). So, you’re not just talking about 500 racks, but upwards of 30,000 servers and hundreds of thousands of individual components that can be storing sensitive data that must be protected.
In the typical ITAD process, the client will provide an inventory list, often in an Excel spreadsheet format. Given this extreme volume, however, conventional means such as spreadsheets are highly prone to error and are not going to provide a through-enough level of serialized IT asset tracking. Therefore, there needs to be an asset management database. The challenge therein is whether we can export the correct data that we need to reconcile IT assets with 100% accountability while onsite?
- The high propensity for inventory inaccuracies in data center ITAD. Data center servers are quite often replaced, upgraded, consolidated, or retired. Hard drives routinely fail under the enormous stress of hyperscale computing. For many organizations, it’s very hard to keep IT asset management repositories current with the huge volume of IT asset refresh and break-fix maintenance in the data center. As a result, the decommission inventory lists provided by the client can be inaccurate—sometimes quite substantially.
Conversely, the process of capturing IT asset inventory data onsite is suspect to both manual and systematic error. Not all vendors adhere to a straightforward format of model number, serial number, and part number. An Intel SSD, for example, has many different numbers on the label called ISN, SA, PBA, WO, MODEL, and some unknown IDs. The ISN would represent the serial number in this case. Some of the older LSI flash cards also have a confusing serial numbering, as there are multiple numbers and finding the correct physical label that matches the serial number discovered by the software requires experience.
Also, certain storage manufacturers will report a 20-byte serial number, which includes the serial number as well as other unique identifying codes or numbers. However, when scanning or taking a physical read of the asset label, you will get the 8-byte serial number. While these are actually the same IT assets, because of the byte differential it’s easy to think they are different assets if you don’t have the right people and/or systems that can detect otherwise.
- The need for systematic inventory discovery and reconciliation of data center IT equipment onsite including the host-to-disk relationships. The typical ITAD process only tracks data center IT assets at the host level. But as we just discussed, servers and hard drives are commonly replaced over a data center rack’s lifecycle. So, the precise configuration of the data center server rack at decommission can be quite different than when it was originally deployed, and every variance not reconciled onsite can be a huge asset and data security vulnerability.
Let’s say, for example, a server is being decommissioned that originally contained 12 hard drives, however:
- Now the server contains only 10 hard drives. So, theoretically there are two hard drives missing.
- Yet, of the 12 drives that appear on the client’s inventory list, seven serial numbers don’t even match what has been discovered onsite. So now, only three of the original 12 match.
- What happened with the nine original hard drives that are no longer in this server? Were they decommissioned, returned on RMA, or other? Was the data sanitized?
- Why haven’t the asset management records been updated to reflect the seven replacement hard drives?
And, considering that each rack can have as many as 60 servers, the challenge is not only exponential, but there are different rack/server/hard drive configurations that must be accounted for in asset management systems as well. But if the typical ITAD process only calls for tracking data center IT assets at the host level, how then can these variances be discovered and accounted for onsite—before there is exposure to data security risk?
That’s why it’s critical to document parent-child relationships between the racks, the servers, and the internal hard drives, and to perform 100% serialized reconciliation for every IT asset before anything leaves the data center. Otherwise, you may have no idea if several hard drives went missing until it’s too late.
As an aside, some ITRenew clients will use our Teraware software in pre-production to not only ensure erasure compatibility before data center IT equipment goes into their live environment, but to also discover the full equipment details and establish the asset repository. This way, they can have complete accountability and be able to easily and effectively reconcile inventory at decommissioning. We consider this a best practice for both asset and data security purposes.
- Risk of missing hard drives and other data-bearing components in data center IT equipment. The following are just a few of the more common examples of hard drives or other data-bearing components that could get missed onsite and thus not be erased or be at risk in-transit:
- Any non-volatile storage component that is used as a caching tier will be vulnerable for data breach because it may not be systematically recognized as a storage drive. For example, some will have SIM cards containing data that are hidden behind the RAID controller and go undetected.
- Local storage devices such as CompactFlash (CF) or Secure Digital (SD) cards that are used in high-end hardware for data logging, capturing boot up processes, or imaging. It may not be obvious to a simple technician that motherboard components can also contain sensitive data that must be protected, let alone which components, where they are located, and what to do.
- For high-density chassis that utilize internal and external drive slots for storage, the internal drives are usually missed. The technician may be expecting 12 hard drives on the front of the chassis, but is completely unaware of the existence of internal boot drives. While some of these components may not have traditional data such as consumer identity information, employee records, or proprietary/confidential company information, it is still data that companies do not want to be made public, such as the host name of the server.
- In systems that have data drive slots in the front and boot drive slots in the back, the rear drives are usually missed. Again, technicians will pull the front drives that are easily detectable but miss the small 2.5” drives in the back.
- Drives that are pulled out and replaced with spares during the service life. In some instances, we have found failed drives lying loose inside the rack or even on top of the cabinet.
- Proper maintenance and discipline of the asset management system is difficult over the service life of the asset. Most of the assets being decommissioned today are already 3-5 years old. When they were initially deployed, there was a process and procedure. Over the course of the service life, did those change? If a drive fails or motherboard is replaced in the asset, has the asset repository been updated to reflect changed parts or sanitized/destroyed disks?
Technicians typically do not have access to the client’s asset management system to update the databased records. So, it’s not uncommon for a “bogus” serial number to get logged for the break-fix device. Usually, it’s a bunch a lowercase letters that the technician quickly keyed in lieu of being able to update the database with the actual serial number. In these instances, for every IT asset missing from the inventory list may be a corresponding “non-conformance” IT asset—meaning a serialized device that has been discovered and is not included in the inventory list.
- Properly trained technicians and systems capabilities. As they say, there is no substitute for experience. This includes:
- Making sure that anyone and everyone who touches the IT equipment—from client onsite staff to ITAD service provider resources to third-party resources—is familiar with what equipment is being decommissioned.
- Knowing which devices and components contain sensitive data and how the information should be logged and/or updated in the asset management databases.
- And ensuring that the systems and processes utilized onsite have been developed specifically for these data center nuances, and to mitigate the risks of human error onsite.
It’s also important to call out that, with employee turnover, mergers and acquisition, and general outsourcing, the client resources that we directly engage with during onsite projects may know very little about the data center IT equipment being decommissioned. This makes it even more important to utilize an ITAD specialist that employs technicians experienced in onsite data center decommissioning. A data center ITAD provider should also have specialized system capabilities to spot and rectify asset and data security vulnerabilities—before they manifest into data breach security risk.
In the end, every data center decommissioning project must first have 100% asset accountability and reconciliation if the ultimate goal is to eliminate data security risks. Once this is accomplished, then you can move onto the next challenge, which is properly sanitizing data onsite. We’ll dive into the data security challenges of data center ITAD next. Stay tuned.