Download as PDF

Open File Backups

1.1 Open File Support in BackupAssist

In order to maintain a reliable and recoverable backup strategy it is essential that your backup solution provides support for backing up open files. Without having open file protection mission critical data files may not be backed up. Consequently, your backup sets will be incomplete and unreliable, simply because certain application files were in use during the backup process.

BackupAssist provides two different technologies for backing up open files in Windows environments:

• Snapshot Technology - using the Microsoft® Volume Shadow Copy Service in Windows XP and 2003, BackupAssist is able to perform a backup of all open files by taking a live snapshot of your data before it is backed up. The Microsoft® VSS snapshot will only work, however, if the application data being backed up is compliant with the Microsoft® Shadow Copy Service framework. As an alternative, BackupAssist offers an optional add-on to BackupAssist, the Advanced Open Files Add-on. This Add-on provides complete open file support for Windows NT, 2000, XP, 2003 and SBS and performs point-in-time snapshots of open file data and includes the added advantage of being able to backup open files for applications that are non-VSS compliant.
• Conventional File Copy - by enhancing the functionality of NTBackup, the standard license of BackupAssist can backup all open files that are not exclusively locked on Windows NT, 2000, XP, 2003 and SBS. This method can also be used to backup open files on network shares, where snapshot technology cannot be applied.

Both Snapshot technology and Conventional File Copy can be used together with BackupAssist to ensure complete open file data protection. The two technologies act to cover the range of open file backup requirements. BackupAssist's dual technologies for handling open file backups help businesses choose a cost-effective solution tailored to their backup requirements and budgets. The following chapters explain how each technology works, where they apply, and the advantages and disadvantages of each method.

2.1 Introduction to Snapshot technology

2.1.1 Issues with applications and open files

Active applications will often keep their files continuously open. This can present a real problem for backup applications as it prevents them from accessing and copying these files to the designated backup media (tape, hard-disk, DVD/CD, etc). While the Conventional File Copy method (see section 3) can work for some open files, it cannot, however, be used to backup exclusively locked files.

Additionally, even when applications do not have open files, the data backed up from these applications may not be consistent with state of the original data at point in time of the backup; this is due to the time needed to open, backup and close a file.

Even more important, some servers run critical applications such as databases, which often process and record massive flows of transactions and data messages, swapping this data constantly between memory and disk. This makes it very difficult to distinguish which data needs to be backed to maintain consistency and reliability.

While it is possible to simply disable these applications during the backup process to ensure that the data is in a stable state, this results in unacceptable amounts of downtime, particularly for applications that need to be running 24 hours a day. Additionally, if you are continuously starting and stopping applications, you run the risk of the application not re-starting on request every time, and this will greatly hinder the productivity of your business operations.

For these open file problems, Snapshot technology can be used to take a snapshot or 'picture' of the data at a particular instant in time. A backup application can then backup from this snapshot or point-in-time copy, rather than the original data files that may be open and inaccessible to backup requests.

2.1.2 The BackupAssist Snapshot Options

BackupAssist currently gives you the ability to use either Microsoft® VSS (Windows XP / 2003 only) or its own Advanced Open Files Add-on to perform point-in-time snapshots.

You can choose your preferred Snapshot Manager within the 'Open files' tab of a BackupAssist job, as shown below:

2.2 Using Microsoft® VSS with BackupAssist

On Windows XP and 2003, files on a local drive that are currently in use can be backed up completely as if a snapshot image was taken at the time of the backup. This is done through a feature known as Volume Shadow Copy or VSS which ensures data integrity and reliable backup sets.

VSS is a storage management framework specifically designed for Windows to create point-in-time copies or snapshots of a hard disk volume. VSS uses a combination of requestors (backup applications), writers (business applications) and providers (snapshot technology) to perform the snap-shotting process, and works as follows

1. The backup application or requestor (NTBackup) launches to perform a backup and sends a request to VSS to perform the snapshot.

2. VSS communicates with running applications or writers to complete existing transactions and halt new ones.

3. VSS then takes the snapshot using the snapshot technology or provider.

4. Once the snapshot is complete, VSS communicates with the running applications so they can resume normal operations.

5. The backup application then performs a backup of the snapshot created, where no open files are present, which means that no files are skipped.

6. Once the backup operation on the snapshot completes, VSS removes the snapshot from the system.

To enable VSS within BackupAssist, simply choose Microsoft® Volume Shadow Copy Service as your open file snapshot manager in the 'Open files' tab of the job settings section:

If you are not using Windows XP, 2003 or SBS 2003, then the option for Microsoft Shadow Copy will be grayed out and not available:

2.3 The Advanced Open Files Add-on - advantages over VSS and its usage

2.3.1 VSS Limitations

While the Microsoft® VSS framework does provide an excellent solution for storage management and point-in-time snapshots to cater for backing up applications with open files, it does have certain limitations:

• VSS is currently only available through Windows XP and Windows 2003

This means that businesses running Windows 2000 or later servers cannot use VSS to backup their open files.

• VSS is a framework that only supports VSS compliant applications

Applications must be VSS compliant in order to interface with the VSS framework. Currently, many business applications are not VSS compliant, which includes a large number of legacy applications. This means that a large number of applications will not be protected by VSS.

2.3.2 When applications are not VSS compliant

For applications that are not VSS compliant, the VSS framework only provides crash consistency on files that are open or in use during the backup procedure. Crash consistency only guarantees file system integrity; it does not guarantee transaction integrity.

This means that if an application transaction is in progress at the time of a snapshot, the actual snapshot may only contain a partial record of that transaction; consequently the data backed up is incomplete.

For an example of this inconsistency we can relate to a typical banking transaction, involving the following two actions:

The transaction above involves two simple actions: a withdrawal of a sum of money from one bank account and a deposit of the same amount into another account. These two actions must be recorded within the one transaction or the accounts will be out of balance.

Now, we can take a look at a scenario using a VSS snapshot to backup a non-VSS compliant application performing this operation:

In this scenario, because the application being backed up by VSS is not VSS compliant, the VSS framework has no indication of any transactions in progress. In this particular case the snapshot has been taken in the middle of a transaction and as a consequence the backup of this snapshot will only be of the withdraw action and not the deposit; the accounts will be out of balance.

This can obviously cause major issues when it comes time to perform a restoration because your backup may be missing vital actions within various transactions.

2.3.3 The Advanced Open Files Add-on advantage

To overcome these limitations, users of BackupAssist can opt for the Advanced Open Files Add-on.

The add-on is an advanced open file management solution which ensures that you have complete protection for all your mission critical data and actually supports legacy and other custom applications that are not compliant with the VSS framework.

To overcome the VSS issue of performing a snapshot during an open or unfinished transaction, the add-on will monitor write operations occurring across all of your active applications. Once BackupAssist has determined a point in time where significant time has elapsed since the last write operation, it will perform the snapshot. The time that BackupAssist waits until performing the snapshot is known as the Write Inactivity Period or WIP; for mass storage systems a safe WIP is usually from 2 to 5 seconds. Basically when the WIP elapses, it is assumed that the application data is consistent (no write operations) and that it is safe to perform the snapshot.

2.3.4 When to use the Open Files Add-on

You should use the Open Files Add-on if you are:

• Running Windows 2000 Server or earlier where Microsoft® VSS is not available

· Running any applications that are not compliant with the VSS framework

· Using the NTBackup Engine in BackupAssist to back up your data (Windows XP and 2003 only; newer Operating Systems, such as Windows Server 2008 and Windows Vista, are not supported.)

2.3.5 How to install the Advanced Open Files Add-on

To install the add-on ensure that you select the Open Files Add-on when installing BackupAssist:

If you did not initially install the Open Files Add-on you can re-run the installer, which can be downloaded from the BackupAssist website, and choose 'Modify' to install it:

Once installed, access the 'Open files' tab within the job settings window (see section 2.1.2) to enable the Advanced Open Files Add-on. Once a backup runs and completes, the backup report will indicate the success of the Open File Snapshot that was taken before the backup commenced.

The following screenshot shows the monitor screen in BackupAssist while a job is running. You will now notice an extra step in the backup process, 'Activate open file manager' which involves taking a live snapshot of local drives that you are backing up data from using the add-on.

2.4 Useful References

Microsoft® VSS - Microsoft Technet: How Volume Shadow Copy Works

Snapshot Technology - Search Storage Articles

Chapter 3 below covers how you can use the Conventional File Copy Method to further ensure that you have a secure backup of open files, including backups of files residing on a network share that do not have any exclusive locks.

3.1 How Conventional File Copy Works

When using the Windows Backup Utility, NTBackup, (which BackupAssist uses to perform the backup operation of common data files) files that are in use or open during the backup process are skipped and as a consequence, not backed up. BackupAssist uses a multi-phase backup process to work around the issue of skipped files. This multi-phase process is unique to BackupAssist, and extends the capabilities of NTBackup.

Phase One

Your selected drives and directories are backed up. Open files are skipped by NTBackup. BackupAssist will detect these skipped files and attempt to copy them to a temporary directory.

Phase T w o

Open files copied to this temporary are backed up. Open files that are locked with a shared lock or no lock at all are successfully backed up using this method. Examples of such files include Outlook .pst files. Files that have exclusive locks, however, cannot be copied, and as a consequence cannot be backed up using this method. These open files are typically from database applications.

For specific databases such as Microsoft Exchange or SQL Server that have exclusively locked files, you can use the respective BackupAssist modules (built-in Exchange module and the SQL Add-on) to backup these resources.

3.1.1 Resolving issues with exclusively locked files

For all other application data with exclusively locked files, you need to use either VSS or the Advanced Open Files Add-on, as described in the previous section of this document.

VSS (Windows XP/2003 only) and the Advanced Open Files Add-on both overcome the issue backing up exclusively locked files by first taking a live snapshot/point-in-time copy of the files before the backup commences and then backing up from this snapshot rather than the original open files.

3.2 When Does Conventional File Copy Apply?

Conventional File Copy for open file backups can apply in two separate situations:

1. When no snapshot technology is available - either VSS or the Advanced Open Files Add-on.

2. When backing up data from a network share - you will require the Conventional File Copy method if you are attempting to backup open files residing on a network share because current snapshot technology does not support snapshots of network volumes.

3.3 Utilizing Conventional File Copy

To utilize Conventional File Copy check Enable conventional file copy in the 'Open files' tab:

You will also need to specify a directory where the open files are to be copied to. This is the temporary location that any open files detected in phase one of the backup process are copied to before being backed up during phase two.

3.4 A practical example of how the Conventional File Copy Method works

The following simple example illustrates how BackupAssist handles open file backups using the Conventional File Copy Method.

3.4.1 Example details

The following system was used for this example:

• Operating System: Windows Server 2003
• Directory to backup : C:\TestDir containing three files:

3.4.2 Backup procedure

When BackupAssist first runs NTBackup to backup your files in phase one of the backup process, all open files are skipped. There were two open files in the test directory, so we expect both files to be skipped and not backed up. BackupAssist will then attempt to copy these skipped files to the Open Files Directory which has been set to: C:\Program Files\BackupAssist\OpenFiles in BackupAssist.

Only the shared lock file can be copied. The exclusively locked file cannot be copied because the Windows Operating System prevents programs from accessing it.

BackupAssist will then run NTBackup again for phase two of the backup process, and backup files copied to the Open Files Directory. From this directory, BackupAssist recreates the full directory structure of the open files. The table below provides instructive examples of how this works:

This means that you can determine the original location of an open file. The backup report will indicate if any files were skipped during the backup process:

The NTBackup log file is also included in the backup report and will highlight what files were skipped:

The backup report additionally includes an 'Open File Copy Report' which summarizes all of the open files that could and could not be copied using Conventional File Copy:

3.5 Restoring Open Files from Conventional File Copy Backup

To perform a restoration of open files, run NTBackup. Navigate to the Restore tab and locate your backup file or tape:

• Set 1 corresponds to the first backup phase where files and directories were backed up. Any open files would have been skipped during this phase and as a result do not appear in this set.
• Set 2 corresponds to the second backup phase where open files that were copied to the Open Files Directory are backed up, along with additional data such as Exchange Server.

Phase One Backup

During the Phase One backup of c:\TestDir, only one file was closed - Normal file.txt.

When viewing the contents of the backup file, you can see that file is available for restoration. To restore it, select the file and click 'Start Restore':

Phase Two Backup

During the Phase Two backup, open files were copied to the Open Files Directory (in this case - c:\Program Files\BackupAssist\OpenFiles). The file, 'Open file - shared lock.txt' was copied, and is available for restoration:

When you restore this file, you can restore it to its original location (in the Open Files Directory - C:\Program Files\BackupAssist\OpenFiles). You will then need to manually copy this file back to its original location (in this case, C:\TestDir).