Because many organizations - including credit unions - rely heavily on network data to conduct business operations, interact with members, process payments and provide remote access for off-site employees, they believe they are protected from data loss or network failure because they have a backup and recovery system in place. 

Unfortunately, this false sense of security can leave an organization vulnerable to a lengthy and potentially crippling business interruption. So much so that many businesses each year experience significant loss of revenue, diminished customer confidence, and yes, even go out of business completely — despite having some type of backup system in place.

Vulnerabilities of network backup and recovery

The “I have a backup system, so I’m covered” mentality ignores several critical factors. 

First, recovering data from backups can take many hours and even days if associated with hardware failures. Backups can also become corrupted and may not be reliable when needed.

Second, in addition to the critical nature of the data, if there is a failure of the application, database or e-mail server it can leave the business unable to access that data.

For this reason, any solution that does not restore the network to full functionality within minutes — no matter the cause or point of failure — is simply unacceptable.

“In healthcare, the ability to access data is extremely important,” said Justin Huckaby, IT manager at CMA Healthcare, an independent, multi-specialty medical practice in South Carolina. “Physicians now rely on electronic medical records to make good healthcare decisions for their patients. In extreme cases, it can actually be a life or death situation because physicians make decisions based on the information they can access."

Immediate access to data is just as important for the National Service Center, a depot repair shop and spare parts distributor for a variety of industrial barcode and label printers, digital signage, wireless networking and RFID systems. 

“We have terabytes and terabytes of data on our servers. The entire company, as well as field employees and customers are logged into the system throughout the entire day,” IT Manager John Campbell said. “If our accounting systems were to die, our business would close.”

Business continuity in layman’s terms

The odds of a significant network event are much higher than most businesses realize. In fact, the IT industry now shies away from the term disaster recovery because it implies rare events such as a fire or other natural disasters. However, data can be corrupted or lost for a variety of reasons including hardware failures of servers, hard drives, user error, malware, file sharing and poor connection issues during remote access. 

As a result, data backup and recovery has evolved into what the IT industry now calls business continuity. The term intentionally emphasizes the fact that many businesses which rely heavily on data cannot afford to be interrupted or have limited access to records for more than the shortest period of time. 

At its core, there are two fundamental aspects to business continuity. Backups involve copying network data to tapes, network attached storage, local hardware appliances or the cloud. There are many inexpensive solutions on the market and most companies have some form of a backup system in place. 

The other aspect is recovering that information quickly in the event of data loss, corruption or server failure. For many businesses, it is this aspect that is the blind spot in the system and leaves them vulnerable to periods of extended downtime.

The reality is many difficulties can arise when it is time to retrieve stored information. Retrieving terabytes of data from tape or the cloud, for example, can take hours or even days depending on download speeds. If the cause of the data loss is hardware-related, the restoration cannot even begin until the system is repaired or replaced.

Depending on the severity of the issue, this can extend the amount of downtime to several days. Many businesses cannot afford to be offline for even a few minutes.

And, according to IT experts, just because a backup exists does not mean the data is pristine. In fact, backup data can be corrupted as easily as network data.

Comprehensive business continuity solutions

So what does a complete business continuity solution look like for those with only a working knowledge of IT?

Following are five factors that should be found in a comprehensive business continuity solution:

1. Provide image-based backup of key systems.

Some basic backup systems copy select files to tape or other network storage devices. This allows for limited retrieval of data, with some of it potentially excluded. The process of both backing up and recovering the data (when needed) is typically very slow.

When CMA Healthcare’s Huckaby was first hired, the business relied on tape backups that were time-consuming, difficult to manage and easily corrupted.   

Although he lobbied for a better solution, it was only after a major hardware failure to the server which stored electronic medical records that he was able to convince ownership to make a change. The event took the system offline for three or four days.

Since then, Huckaby said CMA Healthcare has entrusted its data backup and recovery to Unified Network Group, Inc., which offers a business continuity solution designed to restore key systems and data within minutes regardless of the cause or point of failure.

UNG offers more advanced image-based backups that create a copy of the operating system and all the data associated with it, including the system state and application configurations. The backup is saved as a single file called an image.

The advantages of this approach are that select files or the entire server can be restored within minutes.

2. Backups occur frequently to a local hardware appliance.

Ideally, backups would be completed hourly and possibly even more frequently for critical real-time data. More frequent backups are limited by the type of system. Tape backups are often set to perform this work during the night and, depending on the amount of data, may not even be finished by the morning. 

Image-based backup, on the other hand, can be completed much faster.

“The system that UNG uses can be configured to take ‘snapshots’ of our servers as frequently as every five minutes,” Huckaby explained. “So we identified the servers that handle all of our healthcare information and we back them up multiple times per hour. Less critical servers are backed up once an hour, so we know that if there is ever an issue we have a backup that is no more than an hour old that we can restore quickly.” 

3. For redundancy, locally stored data would also be backed up in the cloud.

When anticipating every possible scenario where the system could break down, it makes sense to backup any local hardware appliances to the cloud. 

Even less technical business owners are now at least somewhat familiar with the concept of the cloud. This essentially means the entire network is also backed up to remote servers located elsewhere in the United States. Cloud servers are extremely secure with their own redundant protections and backups.

If the local hardware appliance fails or is destroyed in a fire, flood, earthquake or other natural disaster, the entire network can be accessed directly from the cloud. While rare, these events happen more frequently than many realize.

4. Backups are tested daily to ensure the data is not corrupted.

Although there are some rudimentary tools for ensuring a backup was completed successfully and that the data was not corrupted, they are limited and often infrequent. For smaller companies, this monitoring is often not assigned to IT, but to office managers or other staff.

To protect against potential data corruption of the backups, some vendors are now going a step further and conducting daily testing and verification of image-based backups. The daily reports are sent to the client once a week to show that the backup was tested and is in good working order.

5. Backups, whether local or in the cloud, can act as virtual servers in a pinch.

Once the hardware is repaired or replaced, tested and back online, all of the data (including everything changed or added during the downtime) is copied to it and the switch is made back to the actual server.

“Now if the server dies, the data backup system we have can boot a virtual server and be up in just a couple minutes,” Campbell said. “This makes the system we have infinitely easier to use. It is effortless to recover a file and … the entire system if needed as well.”

Cost of network recovery in minutes.

As with any expense, businesses are rightfully concerned with the cost of a more comprehensive backup and recovery system.

Some providers are moving toward a fixed monthly fee model for business continuity solutions based on the amount of data involved. The flat rate includes all software, hardware appliances, cloud storage, monitoring, and even support and assistance when an event occurs.

This stands in contrast to IT support billed by the hour along with additional fees to purchase or lease hardware appliances for cloud storage. These fees add up and can even spike in the event of significant data loss or a network crash.

According to Huckaby, businesses should think of a comprehensive business continuity solution like insurance.

“Nobody likes paying for insurance, but they are sure glad they pay for it when they need to use it,” he said.