Blog

In my article Getting a List of Stores in a PowerShell Script you learned how to obtain a list of all the files involved for the Exchange database stores on a particular Exchange server. In the preceding article, Getting a List of Storage Groups in a PowerShell Script, you learned how to obtain a list of all the files unique to the Exchange storage groups on a particular Exchange server.

As a part of both of those articles, you learned how to create a list of the volumes used by the files in the storage groups and in the database stores.

Now that we have that information, what can we do with it?

Easy! We can generate a script that can create an offline backup of our Exchange databases. In future articles, you’ll learn how to turn this offline backup into an online backup, using VSS (the Volume Shadow Copy Service).

As a quick reminder, the following global objects are important and were introduced in the earlier articles of this series:

$volumes – a hash array containing the disk volumes used by the Exchange storage groups and database stores

$pathPattern – a hash array containing a list of all the regular-expression patterns required to back up all the files involved for all Exchange storage groups and database stores

getStores – a function populating $volumes and $pathPattern for the files used by Exchange database stores

getStorageGroups – a function populating $volumes and $pathPattern for the files used by Exchange storage groups

validateArrays – a function verifying that the $volumes and $pathPattern arrays are not empty; the function returns zero if the main program should proceed.

A utility that we have not previously discussed is robocopy. Robocopy was introduced as a part of the Windows 2000 Server Resource Kit. Among other features, it copies large files as quickly as possible, much more quickly than the cmd.exe copy and xcopy. Robocopy is a standard utility in Windows Vista and Windows Server 2008.

Two additional global variables need to be introduced:

$destination – the directory below which backups will be stored; in the case of Exchange backups, the directory structure is reproduced identically. For example, if $destination is “C:\Backups” and the file “C:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\E00.CHK” is a file to be backed up, then the destination file name will be “C:\Backups\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group\E00.CHK”.

$nl – a DOS newline

So, after all the preparation we’ve already done, an offline backup script actually turns out to be quite simple. The PowerShell script below will generate a DOS script to be executed by cmd.exe. Robocopy will copy all relevant files to the backup location specified by $destination. If all copies succeed then the script succeeds. If any copy fails, the script aborts.

A couple of things to be careful of – this script actually executes the backup! To do that, in an offline mode, the Microsoft Exchange Information Store service must be stopped. When that service is stopped, Exchange is basically down. So…don’t run this on your production system without commenting out the line that starts with cmd.exe (unless you are actually doing the offline backup).

Secondly, offline backups do not purge transaction logfiles. We’ll need to learn how to do an online backup before we can make that happen.

Finally, the out-file cmdlet uses an unusual parameter: “-encoding ascii”. This is because cmd.exe does not understand Unicode files (which is the default for out-file). Something to remember for your own scripts!

  
    $destination = "C:\backups"

    $nl = "`r`n"

    function buildRobocopyString($collection)
    {

        $str = ""

        foreach ($filepath in $collection)
        {
            $file = split-path $filepath -leaf
            $path = split-path $filepath -parent
            #
            # the destination path is the source path appended to
            # the backup folder location.
            #
            $destpath = join-path $destination $path.SubString(3, $path.Length - 3)

            $str += "echo Copying " + $file + "..." + $nl
            $str += "robocopy " + '"' + $path + '" "' + $destpath + 
                '" "' + $file + '" /copyall /ZB >nul' + $nl
            $str += "if not errorlevel 0 goto :abort" + $nl
        }

        return $str
    }

    function buildCMD
    {
        $script = "@echo off" + $nl

        $script += 'net stop "Microsoft Exchange Information Store" /y' + $nl

        $script += buildRobocopyString $pathPattern.keys
	$script += $nl
        $script += 'net start "Microsoft Exchange Information Store"' + $nl
        $script += "exit 0" + $nl
        $script += ":abort" + $nl
        $script += "exit 1" + $nl

        $script | out-file (join-path (gc env:temp) "offline-backup.cmd") -encoding ascii
        cmd.exe /c (join-path (gc env:temp) "offline-backup.cmd")
    }

    #
    # Main
    #

    if ((getStorageGroups) -eq 0)
    {
        getStores
        if ((validateArrays) -eq 0)
        {
            buildCMD
        }
    }

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Follow me on twitter: @EssentialExch

In my last post, Getting a List of Storage Groups in a PowerShell Script, you saw how to use the information from the Get-StorageGroup cmdlet to discover the particular disk volumes used by a storage group and to build a list of the files that were used by the storage group.

Now, that list of files did not contain the databases contained within the storage group. Instead, it simply contained the system file (Exx.CHK) for each storage group and the log files (Exx*.LOG) for each storage group. This is because the Get-StorageGroup cmdlet does not return us that information. Instead, we use the Get-MailboxDatabase and Get-PublicFolderDatabase cmdlets to find out the name of our databases. It’s unfortunate, but there is not a single Get-ExchangeDatabase cmdlet that combines the functionality of both,

Note that in Exchange Server 2007, each database consists of a single file, with an extension of EDB. In Exchange Server 2003, there was also a second file per database called the streaming file with an extension of STM.

To remind you of the global variables being used:

• $computername is a string that contains the name of the computer on which the script is being executed.
• $volumes is a hash array that contains a list of all the disk volumes so far detected.
• $pathpattern is a hash array that contains a list of all the fully-qualified paths to all files so far discovered.

A store is always a simple filename. However, it’s important to remember that all of the file names that are used for naming stores and storage groups in Exchange Server may contain spaces and parentheses. That can lead to a requirement for special handling of file names.

When interrogating the list of stores on a particular server, any stores present in the Recovery Storage Group are also listed. These should be ignored.

Each cmdlet we use will return a collection. Get-MailboxDatabase will return a collection of all mailbox databases present on the given server. This collection could be empty even if storage groups are present. Get-PublicFolderDatabase will return a collection of all public folder databases present on the given server. This collection could also be empty (and, in fact, is more likely to be empty). The PowerShell code needs to be prepared to handle those eventualities.

In the PowerShell code below, the getStores function obtains all the stores on the server and adds the store filenames to the $pathpattern array and adds the disk volumes used by the stores to the $volumes array. Following getStores is the validateArrays function. If either the $volumes or the $pathpattern array is empty, it returns a value of 1 and displays a message on the PowerShell host. If both have contents, validateArrays returns a value of 0 and displays the contents of those arrays.

Without further ado:

function getStores
{
	## locate the databases, both mailbox and public folder

	$colMB = get-MailboxDatabase -server $computername
	$colPF = get-PublicFolderDatabase -server $computername

	## parse them for volumes too

	foreach ($mdb in $colMB)
	{
		if ($mdb.Recovery)
		{
			write-host ("Skipping RECOVERY MDB " + $mdb.Name)
			continue
		}
		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	foreach ($mdb in $colPF)
	{
		## a PF db can never be in a recovery storage group
		## which is why the Recovery check isn't done here

		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	return
}

function validateArrays
{
	$drives = $volumes.keys
	if ($drives.Count -lt 1)
	{
		write-host "No disk volumes were found. Aborting."
		return 1
	}

	write-host ("There were " + $drives.Count.ToString() + " disk volumes for Exchange server $computername. They are:")
	foreach ($drive in $drives)
	{
		write-host "`t$drive"
	}

	write-host " "

        $paths = $pathPattern.keys
        if ($paths.Count -lt 1)
        {
                write-host "No paths were found. Aborting."
                return 1
        }

	write-host ("There are " + $pathPattern.Count.ToString() + " directories to be backed up. They are:")
	foreach ($directory in $pathPattern.keys)
	{
		write-host "`t$directory"
	}
	write-host " "

	return 0
}

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Follow me on twitter: @EssentialExch

In Getting Our Computername in a PowerShell Script, you learned how to do just that – store the running computer name (the short NetBIOS name) into a PowerShell variable. At the same time, you learned WHY and HOW it worked. From this point forward, I’ll presume that you’ve executed this particular PowerShell statement:

$global:computername = (gc env:computername)

Note that this causes the storage scope of the $computername variable to be global. It can be interrogated from any function as just $computername (as long as there are no variables that have the same name in a closer scope [such as local scope or function scope]), but for the global value to be updated requires the inclusion of the “global:” specifier.

Thankfully, when we install the Exchange Tools on a workstation or server, one of the Exchange cmdlets is get-storagegroup. To see a list of all the storage groups on all the (Exchange 2007+) servers in your Exchange organization, you just enter get-storagegroup at the prompt in an Exchange Management Shell (EMS).

So, are we done? Nope.

While that list is probably all we need if we are working from a command prompt, if we are writing a script, we are probably interested in more than just the displayed information. For example, consider these requirements of the things we may want to know:

• where the system files for the storage groups are placed
• where the log files for the storage groups are placed
• the log file prefix for the storage groups
• a list of all the disk volumes used by the storage groups

Using get-storagegroup plus a feature of PowerShell makes all of this pretty easy. The feature we will use is called an associative array. Also known as a hash array, for those of you with a Perl background. An associative array is basically a two-dimensional array that is mapped to a single dimensional array based on the index element. Unlike a normal array, an associative array can be indexed by almost anything – a string, a regular expression, an integer – anything you may want to use. In order to pull off this “magic”, an associative array actually consists of two parallel arrays – the keys array and the values array. Now, these arrays get indexed as you are used to – by integers, starting at zero. However, their contents may consist of any object.

Note: in Perl, associative arrays aren’t quite as flexible. In PowerShell, you can literally have an object as a key and an object as a value. This allows you the flexibility of storing arbitrarily complex values into an associative array. However, the search through the keys array is linear. Thus, you should probably keep the arrays fairly small.

While in the EMS, you may get the impression that get-storagegroup returns text, that would be incorrect. Instead, get-storagegroup returns a collection (which is a fancy word for an enumerable array) of objects of type Microsoft.Exchange.Data.Directory.SystemConfiguration.StorageGroup. That’s a mouthful. Let’s just say that it returns an array containing all of the storage group objects.

So, to process that array, you would do something like this:

$collection = get-storagegroup
foreach ($entry in $collection)
{
        ... do something with $entry ...
}

However, you are more likely to want to look at the storage groups on a particular server. So, that leads you to a construct like this:

$collection = get-storagegroup -server $computername
foreach ($entry in $collection)
{
        ... do something with $entry ...
}

Note the tie-in back to $computername! This will access the global $computername variable which has the name of the running computer stored within it.

Now, what does a storage group consist of? It really has only two things:

• log files
• system files

And each of those two things also has a prefix associated with them (which allows multiple storage groups to share a single directory). All log files have a particular format:

.LOG

All system files (there is only a single system file per storage group) have a particular format:

.CHK

A prefix follows this format:

[E ] | [R00]

In words, a prefix is either R00, which is exclusively for the Recovery Storage Group, of which there may be only one per server; or the letter ‘E’ followed by a two-digit number. The number represents the index of the storage group on the server, where the first storage group created is ’00’ the second is ’01’, etc. So the first storage group created on a server will have a prefix of E00.

For that same first storage group, the log files have a format of E00*.log and the system file is named E00.chk.

Note: the number of digits contained in went from five hexidecimal digits in Exchange 2003 (and all earlier versions) to eight hexidecimal digits in Exchange 2007+. Even with the change in log file size from 5 MB to 1 MB, this means that you get 80+ times as many log files before rollover in Exchange 2007+ as you did in earlier versions of Exchange Server.

Let’s see…what else? Oh yes – you want to ignore recovery storage groups. Except for recoveries from backup, you are not supposed to touch them.

Now, given all those above details, what can we do with them? This!

## $volumes will contain the volume letters used by all named
## files and directories.

$global:volumes = @{}

## any storage group will contain:
## a] a system file directory
## b] a log file directory
## c] a filename for each database within the SG
##
## $pathPattern contains the dos patterns of files in the storage group

$global:pathpattern = @{}		### Exx.chk, Exx*.log, *.edb

function getStorageGroups
{
        $count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
	if ($colSG.Count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	## parse the pathnames for each SG to determine what
	## volumes it stores data upon and what directories are used

	foreach ($sg in $colSG)
	{
		if ($sg.Recovery)
		{
			write-host ("Skipping RECOVERY STORAGE GROUP " + $sg.Name)
			continue
		}

                $count++

		$prefix  = $sg.LogFilePrefix
		$logpath = $sg.LogFolderPath.ToString()
		$syspath = $sg.SystemFolderPath.ToString()

		write-host $sg.Name.ToString() "`t" $sg.Guid.ToString()
		write-host "`tLog prefix:      $prefix"
		write-host "`tLog file path:   $logpath"
		write-host "`tSystem path:     $syspath"

		## E00*.log
		$pathpattern.(join-path $logpath ($prefix + "*.log")) = 1

		$vol = $logpath.SubString(0, 1)
		$volumes.$vol += 1

		## E00.chk
		$pathpattern.(join-path $syspath ($prefix + ".chk")) = 1

		$vol = $syspath.SubString(0, 1)
		$volumes.$vol += 1

		write-host " "
	}

	if ($count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	return 0
}

This routine stores, for each storage group, the files that are contained within that storage group. It also stores away the disk volumes used by that storage group. For the write-host output of the function, you could surround the blocks by $debug conditional statements to minimize the output of the routine (or just remove them entirely).

So, what is contained within storage groups? Databases! In our next post in this series, you’ll learn how to deal with them programatically too.

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Follow me on twitter: @EssentialExch