Mailbox Permissions: Pulling Back the Curtain…

Let’s talk about mailbox permissions. People often get a little confused between store-level mailbox permissions and Active Directory-level mailbox permissions. They are similar but not the same. For clarity it may help for us to look at them all.

Note: this post is written specifically against Exchange Server 2007. For Exchange Server 2010, storage groups disappear – mailbox stores acquire storage group attributes, they are promoted to equal status (you could see this coming, as a number of features in Exchange Server 2007 only worked when you had a single mailbox store per storage group). So, the contents of this post apply equally to Exchange Server 2010 – just where-ever you see “storage group”, replace that with “mailbox store”.

Mailbox permissions include: FullAccess, SendAs, ExternalAccount, DeleteItem, ReadPermission, ChangePermission, and ChangeOwner. This list does not include “Send on Behalf”. That’s because a user can set “Send on Behalf” for another user by defining the other user as a delegate and that’s handled separately from mailbox permissions.

Relevant Active Directory permissions include: FullControl, SendAs, ReceiveAs, Delete, and ViewInfoStoreStatus. Three of these (SendAs, ReceiveAs, and ViewInfoStoreStatus) are so-called “extended rights”, which means they are handled somewhat differently than standard access rights.

The AD permission ViewInfoStoreStatus allows a specific user or group to do just that – view the status of an information store. It doesn’t map to anything at the mailbox level. I don’t believe that it is used in Exchange 2007 and above. It had applicability in the Exchange 2000 and Exchange 2003 timeframe when administration of Exchange was handled at the “Administrative Group” level, and ViewInfoStoreStatus was assigned to an administrative group for the administrators of that administrative group (and set to inherit down through all the servers and stores in that group).

The AD permission FullControl includes Delete, SendAs, and ReceiveAs at the AD level. At the mailbox level this maps to FullAccess and SendAs.

The AD permission ReceiveAs maps to FullAccess at the mailbox level. Note that this does not include SendAs or ExternalAccount permissions. There is no way (even though some Microsoft documentation states otherwise) to provide read-only access to a mailbox via permissions.

The AD permission SendAs maps to SendAs at the mailbox level. Note that while it is possible to set Send-As on the mailbox itself, without having it set in AD, you will not be able actually Send-As using Outlook – it depends on Send-As being set within AD.

The AD permission Delete maps to DeleteItem at the mailbox level.

The store permission FullAccess includes all mailbox permissions except SendAs and ExternalAccount.

Setting the AD permission does not cause the mailbox permission(s) to be set (AD has no direct knowledge of Exchange). One must presume that the information store service is smart enough to check both. I have no idea of the official precedence map of AD permissions vs. store permissions. However, behavior indicates that AD permissions are evaluated first, and if they produce a “pass” then the store permissions are evaluated to get the final result (similar to the “share” vs. “NTFS” precedence rules).

The AD permissions can be set on a storage group or a mailbox store, and apply to all mailboxes in that storage group or mailbox store (if set for inheritance). There is no mechanism to do that within the mailbox store itself (that is, there is no cmdlet for Add-MailboxDatabasePermission or Add-StorageGroupPermission, nor do I believe that a store has a concept of a security hierarchy above the mailbox level). Also, while you can set SendAs at the storage group or mailbox store level, this just means that you can impersonate the storage group or mailbox store – it does not mean that you can Send-As for all accounts in that storage group or mailbox store. That permission must be set on a per-mailbox basis.

There is a good whitepaper for understanding store and AD permissions written against Exchange 2000 and Exchange 2003. It is still at microsoft.com/downloads : Working with Store Permissions in Microsoft Exchange 2000 Server and Exchange Server 2003. However, it is a little dated and there are a couple of errors in the document. The basics are still good, but Exchange 2007 reintroduced the idea of setting actual permissions on the mailbox (in 2000 and 2003, you could set mailbox permissions only before the mailbox was created, everything else was set against the AD user object). Exclusive of the impact of Role Based Access Control (commonly referred to as RBAC), I believe that Exchange Server 2010 continues to follow the Exchange Server 2007 rules.

While we have not discussed them here, note that the store-level permissions available to mailboxes are the same permissions available to public folders (excepting only ExternalAccount). And, with the exception of SendAs and ExternalAccount, these are the same permissions available to subfolders within a mailbox and a public folder.

Implementation note:

From a technical perspective, the AD attributes actually represent Access Control Entries set within the nTSecurityDescriptor object assigned to a user object within Active Directory.

For more information on Access Control Lists, Access Control Entries, and the nTSecurityDescriptor object, see Displaying Security on Active Directory, Exchange, and Registry Objects and Windows Permissions – Access Control Lists.

Until next time…

If there are things you would like to see written about, please let me know!

P.S. Thanks to Ross Smith, IV of Microsoft for clarifying a couple of points contained in this article.


Follow me on twitter: @EssentialExch

Mailbox Permissions: Pulling Back the Curtain…

<p>Let’s talk about mailbox permissions. People often get a little confused between store-level mailbox permissions and Active Directory-level mailbox permissions. They are similar but not the same. For clarity it may help for us to look at them all.</p>
<blockquote>
<p>Note: this post is written specifically against Exchange Server 2007. For Exchange Server 2010, storage groups disappear – mailbox stores acquire storage group attributes, they are promoted to equal status (you could see this coming, as a number of features in Exchange Server 2007 only worked when you had a single mailbox store per storage group). So, the contents of this post apply equally to Exchange Server 2010 – just where-ever you see “storage group”, replace that with “mailbox store”.</p></blockquote>
<p>Mailbox permissions include: FullAccess, SendAs, ExternalAccount, DeleteItem, ReadPermission, ChangePermission, and ChangeOwner. This list does not include “Send on Behalf”. That’s because a user can set “Send on Behalf” for another user by defining the other user as a delegate and that’s handled separately from mailbox permissions.</p>
<p>Relevant Active Directory permissions include: FullControl, SendAs, ReceiveAs, Delete, and ViewInfoStoreStatus. Three of these (SendAs, ReceiveAs, and ViewInfoStoreStatus) are so-called “extended rights”, which means they are handled somewhat differently than standard access rights.</p>
<p>The AD permission ViewInfoStoreStatus allows a specific user or group to do just that – view the status of an information store. It doesn’t map to anything at the mailbox level. I don’t believe that it is used in Exchange 2007 and above. It had applicability in the Exchange 2000 and Exchange 2003 timeframe when administration of Exchange was handled at the “Administrative Group” level, and ViewInfoStoreStatus was assigned to an administrative group for the administrators of that administrative group (and set to inherit down through all the servers and stores in that group).</p>
<p>The AD permission FullControl includes Delete, SendAs, and ReceiveAs at the AD level. At the mailbox level this maps to FullAccess and SendAs.</p>
<p>The AD permission ReceiveAs maps to FullAccess at the mailbox level. Note that this does <strong>not</strong> include SendAs or ExternalAccount permissions. There is no way (even though some Microsoft documentation states otherwise) to provide read-only access to a mailbox via permissions.</p>
<p>The AD permission SendAs maps to SendAs at the mailbox level. Note that while it is possible to set Send-As on the mailbox itself, without having it set in AD, you will not be able actually Send-As using Outlook – it depends on Send-As being set within AD.</p>
<p>The AD permission Delete maps to DeleteItem at the mailbox level.</p>
<p>The store permission FullAccess includes all mailbox permissions except SendAs and ExternalAccount.</p>
<p>Setting the AD permission does not cause the mailbox permission(s) to be set (AD has no direct knowledge of Exchange). One must presume that the information store service is smart enough to check both. I have no idea of the official precedence map of AD permissions vs. store permissions. However, behavior indicates that AD permissions are evaluated first, and if they produce a “pass” then the store permissions are evaluated to get the final result (similar to the “share” vs. “NTFS” precedence rules).</p>
<p>The AD permissions can be set on a storage group or a mailbox store, and apply to all mailboxes in that storage group or mailbox store (if set for inheritance). There is no mechanism to do that within the mailbox store itself (that is, there is no cmdlet for Add-MailboxDatabasePermission or Add-StorageGroupPermission, nor do I believe that a store has a concept of a security hierarchy above the mailbox level). Also, while you can set SendAs at the storage group or mailbox store level, this just means that you can impersonate the storage group or mailbox store – it does not mean that you can Send-As for all accounts in that storage group or mailbox store. That permission must be set on a per-mailbox basis.</p>
<p>There is a good whitepaper for understanding store and AD permissions written against Exchange 2000 and Exchange 2003. It is still at microsoft.com/downloads : <a href=”http://www.microsoft.com/downloads/details.aspx?familyid=2ae266f0-16b7-40d7-94d9-c8be0e968a57&amp;displaylang=en” target=”_blank”>Working with Store Permissions in Microsoft Exchange 2000 Server and Exchange Server 2003</a>. However, it is a little dated and there are a couple of errors in the document. The basics are still good, but Exchange 2007  reintroduced the idea of setting actual permissions on the mailbox (in 2000 and 2003, you could set mailbox permissions only before the mailbox was created, everything else was set against the AD user object). Exclusive of the impact of Role Based Access Control (commonly referred to as RBAC), I believe that Exchange Server 2010 continues to follow the Exchange Server 2007 rules.</p>
<p>While we have not discussed them here, note that the store-level permissions available to mailboxes are the same permissions available to public folders (excepting only ExternalAccount). And, with the exception of SendAs and ExternalAccount, these are the same permissions available to subfolders within a mailbox and a public folder.</p>
<p><strong>Implementation note:</strong></p>
<p>From a technical perspective, the AD attributes actually represent Access Control Entries set within the nTSecurityDescriptor object assigned to a user object within Active Directory.</p>
<p>For more information on Access Control Lists, Access Control Entries, and the nTSecurityDescriptor object, see <a href=”http://theessentialexchange.com/blogs/michael/archive/2007/11/13/displaying-security-on-active-directory-exchange-and-registry-objects.aspx” target=”_blank”>Displaying Security on Active Directory, Exchange, and Registry Objects</a> and <a href=”http://theessentialexchange.com/blogs/michael/archive/2007/11/13/windows-permissions-access-control-lists.aspx” target=”_blank”>Windows Permissions – Access Control Lists</a>.</p>
<p>Until next time…</p>
<p>If there are things you would like to see written about, please let me know!</p>
<p>P.S. Thanks to Ross Smith, IV of Microsoft for clarifying a couple of points contained in this article.</p>
<hr><p>Follow me on twitter: @EssentialExch</p>

Monitoring Exchange Server 2007 with System Center Operations Manager 2007

Just a few days ago, my new book (of the subject title!) started shipping from Amazon. If it hasn’t already, it should start appearing in your local bookstores this week.

If you want to order from Amazon, you can get it here.

Eight months in the making, this book was a labor of love. Including information on installing and configuring OpsMgr 2007, it also includes much information about operating a reliable Exchange environment. Even if you don’t have OpsMgr in your organization, you can learn from this book the key areas of Exchange that need monitoring and how to do so.

A strong PowerShell component is provided in several chapters to assist you in the generation of synthetic transactions for testing your Exchange environment.

While the book is written toward a key audience of Exchange Server 2007 administrators, much material is also provided for the Exchange Server 2003 administrator. The book uses a virtualized environment to describe a test roll-out of an OpsMgr 2007 and Exchange 2003/2007 mixed environment.

Since Exchange Server depends on the health of Windows Server, Active Directory, DNS, and IIS; tracking the health and well-being of these key services is also covered.

Go buy it. You’ll like it. 🙂

The chapter titles are:

  1. An Evolution of Server Management
  2. Monitoring Exchange Server 2007
  3. Installing and Configuring OpsMgr 2007
  4. Deplying OpsMgr 2007
  5. The First Management Pack: WIndows Server
  6. The Active Directory Management Pack
  7. The Domain Name System (DNS) Management Pack
  8. The Internet Information Services Management Pack
  9. SQL Server: An Ancillary Management Pack
  10. Exchange Server 2003
  11. Exchange Server 2007
  12. Exchange Server 2007 Redundancy
  13. Exchange Server Operations
  14. Tracking Mail Flow

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

Monitoring Exchange Server 2007 with System Center Operations Manager 2007

Just a few days ago, my new book (of the subject title!) started shipping from Amazon. If it hasn’t already, it should start appearing in your local bookstores this week.

If you want to order from Amazon, you can get it here.

Eight months in the making, this book was a labor of love. Including information on installing and configuring OpsMgr 2007, it also includes much information about operating a reliable Exchange environment. Even if you don’t have OpsMgr in your organization, you can learn from this book the key areas of Exchange that need monitoring and how to do so.

A strong PowerShell component is provided in several chapters to assist you in the generation of synthetic transactions for testing your Exchange environment.

While the book is written toward a key audience of Exchange Server 2007 administrators, much material is also provided for the Exchange Server 2003 administrator. The book uses a virtualized environment to describe a test roll-out of an OpsMgr 2007 and Exchange 2003/2007 mixed environment.

Since Exchange Server depends on the health of Windows Server, Active Directory, DNS, and IIS; tracking the health and well-being of these key services is also covered.

Go buy it. You’ll like it. 🙂

The chapter titles are:

  1. An Evolution of Server Management
  2. Monitoring Exchange Server 2007
  3. Installing and Configuring OpsMgr 2007
  4. Deplying OpsMgr 2007
  5. The First Management Pack: WIndows Server
  6. The Active Directory Management Pack
  7. The Domain Name System (DNS) Management Pack
  8. The Internet Information Services Management Pack
  9. SQL Server: An Ancillary Management Pack
  10. Exchange Server 2003
  11. Exchange Server 2007
  12. Exchange Server 2007 Redundancy
  13. Exchange Server Operations
  14. Tracking Mail Flow

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

A Small Patch: Online Exchange Backup

If you only had a single storage group, the script in Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7) would not find any storage groups. The primary reason for that I always have a RSG too! 🙂 The fix is easy. In getStoragegroups, change the beginning few lines:

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
	if ($colSG.Count -lt 1)
	else
	{
		write-host "No storage groups found on server $computername"
		return 1
	} 

to look like this:

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
###	if ($colSG.Count -lt 1)
	if (($colSG -is [Microsoft.Exchange.Data.Directory.SystemConfiguration.StorageGroup]) -or
	    ($colSG -is [System.Object[]]))
	{
		## everything is good
	}
	else
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

A Small Patch: Online Exchange Backup

If you only had a single storage group, the script in Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7) would not find any storage groups. The primary reason for that I always have a RSG too! 🙂 The fix is easy. In getStoragegroups, change the beginning few lines:

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
	if ($colSG.Count -lt 1)
	else
	{
		write-host "No storage groups found on server $computername"
		return 1
	} 

to look like this:

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
###	if ($colSG.Count -lt 1)
	if (($colSG -is [Microsoft.Exchange.Data.Directory.SystemConfiguration.StorageGroup]) -or
	    ($colSG -is [System.Object[]]))
	{
		## everything is good
	}
	else
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

In the first five parts of this series, I’ve given you the background to understand how Exchange backup works when using VSS and how to acquire the necessary information from your Exchange server to know what you should back up. Today, I present to you a full-blown working script that will generate a full-backup of your Exchange 2007 Server on Windows Server 2008, verify that the backup is good using ESEUTIL, and flush the transaction logs for the Exchange storage groups if the backup is good.

The first five parts of the series were:

Part 1: Getting a List of Storage Groups in a PowerShell Script

Part 2: Getting a List of Stores in a PowerShell Script

Part 3: Exchange 2007 and Windows 2008: Offline Exchange Backup

Part 4: Volume Shadow Copy Services (VSS) and Exchange – The Basics

Part 5: Exchange 2007 and Windows 2008: Using Diskshadow for Online Exchange Backup

Now, before you ask – is this a supported backup tool? The answer is yes and no. VSS backups are a supported way to back up an Exchange server’s databases. Diskshadow is a supported tool on Windows Server 2008. Is my script supported? No. Only so far as I find the time, energy, and effort to provide support for it. I can’t warrant that it will work in your environment. It’s worked everywhere I’ve tested it, that’s all I can tell you. If you find a problem, let me know and I’ll try to help, but there are no guarantees.

You won’t find anything new in this script (from the prior postings in this series), except that the Diskshadow script is generated within the PowerShell script. This makes it easier when you run into a situation that you are using multiple volumes in your Exchange environment (which is a best practice for performance reasons).

The script takes three parameters:

$backupLocation – This is the volume and directory (or mountpoint) where the backup should go. It defaults to C:\Backups. You will probably need to change this for your environment.

$startLetter – This is the first letter that should be used by the script for exposing shadow copies as drive letters for the backup scripts. This defaults to g.

$startScript – This is a switch parameter. When set, the PowerShell script will initiate the backup using diskshadow.exe as soon as the script is built. The switch defaults to unset.

The #1 limitation of this script is that it backs up all storage groups on an Exchange server. I have plans to address that in a future revision.

The #2 limitation of this script is that it’s “an ugly command line tool”. I have plans to address that in a future revision.

This script will create a metadata file named (join-path $backupLocation “online-backup.cab”) (That is C:\Backups\online-backup.cab by default). This file is used by diskshadow.exe for storing information required for restores. We will cover basic restores in part 7 of this series.

The cmd.exe script is stored as (join-path %TEMP% “online-backup.cmd”). The Diskshadow.exe script is stored as (join-path %TEMP% “online-backup.dsh”).

On my server, I store PowerShell scripts in the c:\scripts folder, and name this particular script MBS-online-backup.ps1. A typical invocation is:

join-path “F:\ExchangeBackups” (get-date -uFormat “%Y-%m-%d-%H-%M-%S”) |% { md $_ |% { ./MBS-online-backup.ps1 -backupLocation $_.FullName -startScript } }

This causes a unique directory to be created for each invocation of the backup script and for the script to be automatically run.

Granted, that’s a dense for a beginner to understand. You can separate that into multiple lines quite easily:

$backupSubDir = get-date -uFormat “%Y-%m-%d-%H-%M-%S”
## As specified, the uFormat string means: yyyy-mm-dd-hh-mm-ss
## where the first ‘mm’ is the month number, and
## the second ‘mm’ is the minute number.
$backupDir = join-path “F:\ExchangeBackups” $backupSubDir
md $backupDir
./MBS-online-backup.ps1 -backupLocation $backupDir -startScript

And that is much easier to understand. Without further ado, here is the script:

##
## MBS-online-backup.ps1
##
## Michael B. Smith
## January, 2009
##
## This program generates an online VSS-based backup of an Exchange server
## (Exchange related files only) to a specified remote disk location.
##
## No warranties, express or implied, are available. It works for me. If
## you find errors or have problems, please feel free to let me know, but
## I can't guarantee that I can fix them.
##
## Feel free to use this in your own scripts. I would appreciate attribution.
##
Param(
	[string]$backupLocation = "C:\Backups",
	[string]$startLetter    = "g",
	[switch]$startScript    = $false
)

## $backupLocation is where the files and metadata go.

## $startLetter will contain the first letter we use to remap
## the volume letters contained in $volumes.

## $nl is the DOS newline character string

$nl = "`r`n"

## $volumes will contain the volume letters used by all named
## files and directories.

$volumes = @{}

## any storage group will contain:
## a] a system file directory
## b] a log file directory
## c] a filename for each database within the SG
##
## $pathPattern contains the dos patterns of files in the storage group

$pathpattern = @{}		### Exx.chk, Exx*.log, *.edb

## $storeList contains the filenames of the Exchange databases that need
## to be checked.

$storeList = @{}

## $letters contains the mapping between the original drive letter
## and the exposed driver letter in the shadow copy.

$letters = @{}

## $computerName contains the local computer's name

$computerName = $env:ComputerName

function buildRobocopyString($collection)
{
	$str = ""

	foreach ($filepath in $collection)
	{
		$file = split-path $filepath -leaf
		$path = split-path $filepath -parent
		#
		# the destination path is the source path appended to
		# the backup folder location.
		#
		$destpath = join-path $backupLocation $path.SubString(3, $path.Length - 3)
		#
		# the source path is the true path modified by the
		# letter of the exposed shadow copy
		#
		$letter = $letters.($path.SubString(0, 1))
		$subpath = $path.SubString(1, $path.Length - 1)
		$srcpath = "$letter$subpath"
		$str += "echo Copying " + $filepath + "..." + $nl
		$str += "robocopy " + '"' + $srcpath + '" "' + $destpath + 
		        '" "' + $file + '" /copyall /ZB >nul' + $nl
		$str += "if not errorlevel 0 goto :abort" + $nl
	}

	return $str
}

function buildESEUTILString($collection)
{
	$str = ""

	foreach ($path in $collection)
	{
		#
		# the destination path is the source path appended to
		# the backup folder location.
		#
		$path = $path.ToString()
		$destpath = join-path $backupLocation $path.SubString(3, $path.Length - 3)

		$str += "echo Checking " + $destpath + "..." + $nl
		$str += "call :checkit " + '"' + $destpath, '"' + $nl
		$str += "if not errorlevel 0 goto :abort" + $nl
	}

	return $str
}

function buildCMD
{
	$script = "@echo off" + $nl

	$script += buildRobocopyString $pathPattern.keys
	$script += $nl
	$script += buildESEUTILString $storeList.keys
	$script += $nl
	$script += "exit 0" + $nl
	$script += ":abort" + $nl
	$script += "exit 1" + $nl
	$script += $nl
	$script += ":checkit" + $nl
##	$script += "echo Checking %1" + $nl
	$script += "eseutil /k %1 >nul" + $nl
	$script += "if not errorlevel 0 exit 1" + $nl
	$script += $nl

	$scriptFile = join-path $env:temp "online-backup.cmd"
	$script | out-file $scriptFile -encoding ascii

	return $scriptFile
}

function writerOptimizationGarbage
{
	$script  = ""

	$script += "# verify presence of Exchange Writer" + $nl
	$script += "writer verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}" + $nl
	$script += "# exclude system writer" + $nl
	$script += "writer exclude {e8132975-6f93-4464-a53e-1050253ae220}" + $nl
	$script += "# exclude IIS config writer" + $nl
	$script += "writer exclude {2a40fd15-dfca-4aa8-a654-1f8c654603f6}" + $nl
	$script += "# exclude ASR writer" + $nl
	$script += "writer exclude {be000cbe-11fe-4426-9c58-531aa6355fc4}" + $nl
	$script += "# exclude BITS writer" + $nl
	$script += "writer exclude {4969d978-be47-48b0-b100-f328f07ac1e0}" + $nl
	$script += "# exclude WMI writer" + $nl
	$script += "writer exclude {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}" + $nl
	$script += "# exclude registry writer" + $nl
	$script += "writer exclude {afbab4a2-367d-4d15-a586-71dbb18f8485}" + $nl
	$script += "# exclude iis metabase writer" + $nl
	$script += "writer exclude {59b1f0cf-90ef-465f-9609-6ca8b2938366}" + $nl
	$script += "# exclude com+ regdb writer" + $nl
	$script += "writer exclude {542da469-d3e1-473c-9f4f-7847f01fc64f}" + $nl
	$script += "# exclude shadow-copy optimization writer (does not apply to exchange)" + $nl
	$script += "writer exclude {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}" + $nl
	$script += $nl

	return $script
}

function buildDSH([string]$cmdfilename)
{
	write-host "Building backup script"

	$script  = ""
	$script += "# Diskshadow backup script." + $nl
##	$script += "set verbose on" + $nl
	$script += "set context persistent" + $nl
	$script += "set metadata " + (join-path $backupLocation "online-backup.cab") + $nl
	$script += $nl
	$script += writerOptimizationGarbage
	$script += "begin backup" + $nl
	$script += $nl

	foreach ($drive in $volumes.keys)
	{
		$script += "add volume " + $drive + ": alias shadow_" + $drive + $nl
	}

	$script += $nl + "create" + $nl + $nl

	foreach ($drive in $volumes.keys)
	{
		$script += "expose %shadow_" + $drive + "% " + $letters.$drive + ":" + $nl
	}

	$script += $nl
	$script += "exec " + $cmdfilename + $nl

	#
	# If the batch file from exec fails, diskshadow terminates without
	# executing any more commands.
	#
	$script += "end backup" + $nl

	foreach ($drive in $volumes.keys)
	{
		## remove the temporary shadow copy and unexpose the letter
		$script += "delete shadows exposed " + $letters.$drive + ":" + $nl
	}

	$script += $nl
	$Script += "exit" + $nl

	$scriptFile = join-path $env:temp "online-backup.dsh"

	$script | out-file $scriptFile -encoding ascii
	write-host "Diskshadow script file $scriptFile"
	return $scriptFile
}

function getStores
{
	## locate the databases, both mailbox and public folder

	$colMB = get-MailboxDatabase -server $computername
	$colPF = get-PublicFolderDatabase -server $computername

	## parse them for volumes too

	foreach ($mdb in $colMB)
	{
		if ($mdb.Recovery)
		{
			write-host ("Skipping RECOVERY MDB " + $mdb.Name)
			continue
		}
		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1
		$storeList.($mdb.EdbFilePath)   = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	foreach ($mdb in $colPF)
	{
		## a PF db can never be in a recovery storage group
		## which is why the Recovery check isn't done here

		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1
		$storeList.($mdb.EdbFilePath)   = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	return
}

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
	if ($colSG.Count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	## parse the pathnames for each SG to determine what
	## volumes it stores data upon and what directories are used

	foreach ($sg in $colSG)
	{
		if ($sg.Recovery)
		{
			write-host ("Skipping RECOVERY STORAGE GROUP " + $sg.Name)
			continue
		}

		$count++

		$prefix  = $sg.LogFilePrefix
		$logpath = $sg.LogFolderPath.ToString()
		$syspath = $sg.SystemFolderPath.ToString()

		write-host $sg.Name.ToString() "`t" $sg.Guid.ToString()
		write-host "`tLog prefix:      $prefix"
		write-host "`tLog file path:   $logpath"
		write-host "`tSystem path:     $syspath"

		## E00*.log
		$pathpattern.(join-path $logpath ($prefix + "*.log")) = 1

		$vol = $logpath.SubString(0, 1)
		$volumes.$vol += 1

		## E00.chk
		$pathpattern.(join-path $syspath ($prefix + ".chk")) = 1

		$vol = $syspath.SubString(0, 1)
		$volumes.$vol += 1

		write-host " "
	}

	if ($count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	return 0
}

function validateArrays
{
	$drives = $volumes.keys
	if ($drives.Count -lt 1)
	{
		write-host "No disk volumes were found. Aborting."
		return 1
	}

	write-host ("There were " + $drives.Count.ToString() + " disk volumes for Exchange server $computername. They are:")
	foreach ($drive in $drives)
	{
		write-host "`t$drive"
	}

	write-host " "

	$paths = $pathPattern.keys
	if ($paths.Count -lt 1)
	{
		write-host "No paths were found. Aborting."
		return 1
	}

	write-host ("There are " + $pathPattern.Count.ToString() + " directories to be backed up. They are:")
	foreach ($directory in $pathPattern.keys)
	{
		write-host "`t$directory"
	}
	write-host " "

	$letter = $startLetter.Chars(0)

	foreach ($drive in $volumes.keys)
	{
		$letters.$drive = $letter
		$letter = [char]([int]$letter + 1)
	}

	return 0
}

	##
	## Main
	##

	if ((getStorageGroups) -eq 0)
	{
		getStores
		if ((validateArrays) -eq 0)
		{
			$scriptFile = buildCMD
			$scriptFile = buildDSH $scriptFile
			if ($startScript -and ($scriptFile.Length -gt 0))
			{
				diskshadow.exe -s $scriptFile
			}
		}
	}

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

Exchange 2007 and Windows 2008: Using Diskshadow for Online Exchange Backup (part 5 of 7)

If you hadn’t noticed, I’ve been following a theme the last couple of months with some of the entries on this blog:

Part 1: Getting a List of Storage Groups in a PowerShell Script

Part 2: Getting a List of Stores in a PowerShell Script

Part 3: Exchange 2007 and Windows 2008: Offline Exchange Backup

Part 4: Volume Shadow Copy Services (VSS) and Exchange – The Basics

In this posting, which is part 5 of a 7-part series, I’ll talk about a command-line tool which is new for Windows Server 2008: Diskshadow. The entire purpose behind Diskshadow is to allow a system administrator to harness the power of VSS from an easy-to-use utility. This is not the first utility from Microsoft that uses VSS (BETest and VShadow were both part of the VSS Software Development Kit), but it is the first supported utility for using VSS.

Let me begin by displaying a Diskshadow input file, and then I’ll discuss it, line by line. In case it isn’t obvious, you do not include line numbers inside a Diskshadow input file. They are shown here just for discussion. Also, the typical extension used for a Diskshadow input file is DSH. So, you might call this file offline-backup.dsh.

1 # set verbose on
2 set context persistent
3 writer verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
4 begin backup
5 add volume c: alias shadow_c
6 create
7 expose %shadow_c% g:
8 # exec offline-backup.cmd
9 end backup
10 delete shadows exposed g:
11 exit

Line 1 begins with a hash-mark. This is indicative of a comment to the Diskshadow utility. Anything that occurs after the hash-mark on this line is ignored. However, if the hash-mark were not present, the command “set verbose on” would cause Diskshadow to output additional information as it determines the writers and components that will be included within the shadow copy.

Line 2 indicates that the shadow copy which is created will be persistent – that is, the shadow copy will continue to exist after the “end backup” (line 9) statement is executed. Shadow copies must be persistent in order to expose them (line 7). It is much easier to work with a shadow copy exposed as a drive letter than to use native format shadow copy names. For example, a typical shadow copy may be named \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy25 with a GUID of {e18b18b2-c8dd-4429-9996-af8d582616d8}.

Line 3 causes Diskshadow to verify that the particular writer having the named GUID is present on this computer. This GUID is the specific ID for “Microsoft Exchange Writer”. This check effectively requires that this script is executed on an Exchange server. In order to see a list off all writers and their writer IDs, you can enter “vssadmin list writers” from a command prompt or within Diskshadow you can enter the “list writers” command. Note that you can also use the “writer exclude” command to ensure that a specific writer is not called as part of this backup.

Line 4 will start the actual VSS communication process by notifying all non-excluded writers to PrepareBackup. PrepareBackup was discussed in Volume Shadow Copy Services (VSS) and Exchange – The Basics.

Line 5 is used to identify a specific volume (could be a mount-point) that must be included in this VSS snapshot. You must identify all volumes that are involved in a backup. You learned how to do that for Exchange in Getting a List of Storage Groups in a PowerShell Script and Getting a List of Stores in a PowerShell Script.

Line 6 signals VSS to initiate Freeze and to create a snapshot. When the snapshot is complete, Diskshadow will signal VSS to Thaw. Freeze and Thaw were discussed in Volume Shadow Copy Services (VSS) and Exchange – The Basics.

Line 7 tells Diskshadow to expose the snapshot of a particular volume as a different drive letter (which may also be a mount point). This is primarily for ease-of-access, as I discussed for Line 2.

Line 8 is another comment. If it were not a comment, the EXEC command would cause Diskshadow to execute an external script. THAT SCRIPT is where a copy from a snapshot is actually created. You learned about how to generate that type of script in Exchange 2007 and Windows 2008: Offline Exchange Backup. If the script returns a non-zero value, then Diskshadow aborts.

Line 9 causes Diskshadow to signal VSS to send PostBackup to all involved writers. PostBackup was discussed in Volume Shadow Copy Services (VSS) and Exchange – The Basics.

Line 10 removes the snapshot associated with the drive letter G:, and deletes the drive mapping for G:. The storage space used by the snapshot is returned to the system.

Finally, line 11 terminates Diskshadow.

It is important to realize that after the CREATE statement finishes, what you have is a snapshot. It is not a backup, just a copy of the MFT and in-use bitmap for the disk drives that were included in the snapshot. You learned about the MFT and in-use bitmaps in Volume Shadow Copy Services (VSS) and Exchange – The Basics.

The script executed by line 8 is what creates copies of the files that are your actual backup.

You should also be aware that for any writer that was excluded from the backup, the files protected by that writer are still present in the snapshot; however they are CRASH CONSISTENT (i.e., in the same condition that would’ve happened if the power plug had been pulled on a server) not APPLICATION CONSISTENT (which is what you want for maximum recoverability).

In part 6 of this series, I will put all of the pieces together, and come up with a single PowerShell script that does “everything” to create an Exchange backup for you.

In part 7 of this series, I will cover doing simple restores of Exchange databases.

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

Exchange Connections – Fall 2008

Next week, in Las Vegas, Nevada is the semiannual Connections conference. The Connections conference is a technical conference covering SQL Connections, Windows Connections, Exchange Connections, etc. There are lots of individual tracks, both for IT Pros and Devs.

I’ll be speaking next week at the conference, delivering three Exchange presentations:

EXC10: Exchange 2007 and Windows 2008: Backups the Easy Way (75 minutes)
In this presentation I’ll show you how to use the native Windows tools present in Windows 2008 to make Exchange 2007 backups AND to restore them. I’ll cover some theory, some philosophy, and lots of PowerShell.

EXC11: SMB Exchange Operations (60 minutes)
In this presentation I’ll discuss so key factors of Exchange day-to-day operations that affect the Small Business

EXC12: Building an Exchange Test Environment in a Hurry (75 minutes)
In this presentation I’ll discuss some of ways in which you can quickly generate a virtualized Exchange test environment. After all, all the time you spend building, is less time you can spend testing.

You can see the Event Schedule here and general conference information here.

Please come say “hi”. Even better – attend my presentations!

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch

It’s All About The IOPS, Silly!

In yesterday’s EMO (Exchange Messaging Outlook eZine, subscribe at http://www.slipstick.com/emo) I had an article named It’s All About The IOPS, Silly! which discussed the impact of RAID-1 and RAID-5 arrays on total IOPS calculations.

However, space didn’t allow me to talk about SANs (Storage Area Networks) in regards to IOPS.

Where Exchange is concerned (and SQL Server as well, although that isn’t our focus here), you do not want to be sharing your “disk group” or “volume group” or “virtual array” (or whatever your SAN software calls it) with any other application – unless you’ve performance tested it in the worst case.

SANs allocate storage in terms of LUNs – Logical Unit Numbers. For a given server, a LUN is allocated to that server (note that in the case of clusters, quorum disk may be visible to multiple servers at one time) and only one server may have write access to that LUN at a time. A LUN may, or may not, be tied to a particular disk array or disk set contained within the physical hardware of the SAN.

Many types of SAN software support a concept known as “LUN Stacking”. In this case, an array (be it RAID-1, RAID-5, RAID-10, whatever) has multiple logical stripes of the disk where multiple LUNs are allocated on the same array. Arguably, this is what SANs are all about – allowing you to control storage logically instead of drive-by-drive. And, for many (perhaps most) applications this makes sense.

However, while the BEST CASE SCENARIO says that the IOPS available for a LUN is the maximum IOPS available for that disk set, the WORST CASE scenario says that the IOPS available for a LUN is equal to the number of accessors (servers) that are using that physical array.

So, for example, if you have five servers hitting a RAID-1 array via different LUNS, the best case performance scenario says that you can have the maxiumum IOPS available to the hardware available to each server. The worst case is IOPS / 5.

For example, if your RAID-1 group has a calculated IOPS of 250, one server could potentially benefit from having a total IOPS of 250 available to it. However, if all the servers are hitting the disk, the performance would be more like 50 IOPS. While for many small and medium businesses, an IOPS of 250 is probably sufficient to meet their Exchange needs – only the smallest of companies can get adequate performance on 50 IOPS total.

Another reason you want to avoid sharing LUNs is due to the usage profiles of the various Exchange needs.

Database access (which includes queues, mailbox stores, and public folder stores) is completely random. The creation of log files (transaction log files, message tracking logfiles, and protocol log files) is completely sequential. If you share LUNs between different usage profiles, then your performance will suffer (especially for those needs which are optimized for sequential access).

Conclusion: Just Say No! Don’t let your SAN guys tell you what kind of storage performance requirements Exchange has. You need to be prepared to tell them!

Until next time…

As always, if there are items you would like me to talk about, please drop me a line and let me know!


Follow me on twitter: @EssentialExch