Introduction
The Volume Shadow Copy Service (VSS) was originally added to Windows Server 2003. The first version of Exchange Server to support VSS was Exchange Server 2003. The primary reason for the existence of VSS is to improve the performance of backup operations. VSS is only available for NTFS volumes.
I will not cover specific implementation details here, but concepts; some of those concepts will be specific to NTFS and Exchange specific and some will be general to VSS.
VSS backups (as implemented by Windows, not including the implementations by SAN/NAS/and other third-party vendors), are based on two primary concepts:
- A disk volume’s in-use bitmap, and
- The Master File Table (MFT) for the volume.
A bitmap is a way of identifying the blocks of a disk volume which are being used vs. those blocks which are unused. In terms of implementation, it is usually an array (an ordered list of locations in a computer’s memory), where each entry in that array represents the status of 32 blocks of that disk. In C-language terms:
#define block_in_use(x) (drive_map_array [Floor((int)x / 32)] & (1 << ((int)x % 32)))
This assumes that the "element size" of the array is 32 bits per element. In the case of differing element sizes, adjust the value of 32 in the define to match the word size in bits per element.
Using this define, block_in_use() will return a non-zero value if the bit-field for a given block is set to 1. This means that the block is in use by some disk-related construct. If that bit-field is zero, block_in_use() will return a value of zero. This means that the block is not in use.
The in-use bitmap doesn't care about operating systems or file systems. It simply defines what blocks are available on a volume and what blocks are NOT available on a volume.
The MFT is both an operating system and file system specific construct. For a given disk volume, the MFT identifies what files/directories/etc. exist within that operating system/file system and what block(s) of the disk volume map to those file system constructs.
Every block contained within the MFT should be marked as in-use by the in-use bitmap.
Note that the in-use bitmap and the MFT work together. Neither provides a complete picture of a disk volume without the other. The MFT works per file system. Therefore, you can have multiple MFTs per disk volume. However, the in-use bitmap is per disk volume. You can only have a single in-use bitmap per disk volume.
When a particular file system needs to allocate a block on a disk volume, it calls a disk Application Programming Interface (API) that allows a block to be added to the MFT for the file system.
VSS Components
VSS has three main components:
- the requester - the backup application
- the provider - an operating system component that sits "on top of" the file system and mediates access to a VSS copy, while that VSS copy is active
- the writer - an application component that has registered with the provider to enable support for an application that needs extra processing prior to a VSS backup (typically to flush buffers to disk)
A VSS Backup Overview
A VSS backup consists of five separate phases. Errors can occur and cause a backup to fail in any phase. The phase names are:
- PrepareBackup
- Freeze
- ...Backup
- Thaw
- PostBackup
When the requester wants to make a backup, it makes a PrepareBackup call to the VSS API. Part of this call is identifying the writers that need to be used to make the backup (e.g., if you are making an Exchange-only backup, you can exclude the COM+DB writer and the Registry writer; in fact you can exclude all writers except for the Exchange writer).
During PrepareBackup, VSS initializes each involved writer and indicates to the writer that a backup will be starting soon. After the return from PrepareBackup, the requester will make a Freeze call to the VSS API.
During the Freeze process, all involved writers will cause their subsystems to be application consistent on disk. That is, if buffers need to be flushed; they will be. If transaction logs need to be switched, they will be. Once a writer returns success to its Freeze call, it cannot make any modifications to disk until notified by VSS. This is enforced by the writer.
Once all writers are frozen, VSS creates a snapshot. A snapshot consists of, for all practical purposes, a copy of the MFT and the in-use bitmap of the disk. The entire file system is not processed, only a copy is made. That is what provides the speed in creating snapshots. Also note that only the involved writers have application consistency at this point. All other files are crash-consistent. A snapshot is exposed to the application as a mount-point. This mount point may be persistent or it may be ephemeral. A persistent mount point continues to exist after a VSS backup is complete.
After the snapshot is created, VSS will inform each involved writer to Thaw. At this time, applications are again allowed to make updates to the disk volume, using copy-on-write (which is done by the writer - the complexity of this is hidden from the application). That is, if a block in a file contained within the snapshot changes, then that block is not over-written on disk. Instead a new block is allocated and the MFT for that file will be updated to contain the new block. This allows a snapshot to maintain an unchanged image of the disk from when the snapshot occurred.
Now, the requester can take its backup from the snapshot. The requester may be as simple as cmd.exe's copy command or it may be a third-party backup application or it may be a SAN's special disk duplicating software. The requester is required to return a status back to VSS to indicate whether the backup was successful.
When the backup phase is complete, VSS will call each involved writer indicating a status of PostBackup. At this point each writer can do whatever clean up (if any) may be necessary. The writer is also informed of the status of the requester's backup. In some cases, this may affect PostBackup handling by the writer. For example, if a successful backup of a storage group occurred, the Exchange writer will flush the transaction logs for that storage group. If the backup failed, the transaction logs will not be flushed.
If the the snapshot was persistent, it will remain after PostBackup. If it was ephemeral, it will be deleted.
So, consider the example of backing up an Exchange server and only the Exchange portion of the server. The process will go like this:
- A backup application calls VSS and says that it wants to back up Exchange and only Exchange
- VSS calls the Exchange Writer with PrepareBackup.
- Asynchronously, Exchange initiates a transaction log switch and pauses the Lazy Writer process (which is responsible for flushing updates from the memory cache to the Exchange database) and the Log Writer process (which writes updates to a log file).
- VSS calls the Exchange Writer with Freeze
- Exchange waits for the transaction log switch to be complete and for the Lazy Writer and Log Writer processes to be paused
- VSS creates the snapshot
- VSS calls the Exchange Writer with Thaw
- Exchange unpauses the Lazy Writer and the Log Writer processes
- The backup application will back up all system files (Exx.CHK for each storage group), all log files (Exx*.log for each storage group), and all database files (*.EDB for each database) and return success or failure back to VSS
- VSS calls the Exchange Writer with PostBackup with the status of the backup
- If the backup was successful, transaction log files will be flushed
- The backup application disconnects from VSS and the backup is complete
As you can see, while there are a number of steps involved in the process it really isn't that complicated. Given a little help (in terms of a programmatic interface to VSS), it shouldn't be that hard to back up an Exchange server.
We'll talk about that in our next article in this series.
Until next time...
As always, if there are items you would like me to talk about, please drop me a line and let me know!
Follow me on twitter: @EssentialExch