I spent eight hours over the last two days figuring out an interesting – but weird – problem. Once I figured out the problem, correcting it was a simple matter.
I’ll just give you the problem description and then the solution. Hopefully it will save you more than a little time in the future.
I have a client (one of several in this particular situation) who has an Exchange 2003 Front-End server located in their DMZ. Yes, that’s right, their DMZ. In the “long-ago time”, this was considered to be a favored practice by Microsoft. That, of course, has changed over the years. Now, in 2011, we do not want any domain member servers located in the DMZ. Why is that? Because to install a domain member in a DMZ basically means that you have to make “swiss cheese” out of your firewall. And, effectively, if a DMZ server is compromised it means that your entire Active Directory can be compromised. The entire list of potentially affected ports is shown in KB 832017, and at the end of the day – it basically says that you have to open everything above port 1024 to make it all work.
With all that being said, if you are using Exchange 2003 with Server 2003 Active Directory (by which I mean an Active Directory domain controller hosted on a Windows Server 2003 server), you could get by with a much more limited set of ports (although it still isn’t small!):
DNS (TCP and UDP 53) Kerberos (TCP and UDP 88) RPC Endpoint Mapper (TCP 135) NetBIOS Name Service (TCP 137) NetBIOS Datagram Service (TCP 138) NetBIOS Session Service (TCP 139) LDAP (TCP and UDP 389) Directory Services (TCP 445) LDAP Global Catalog (TCP 3268) Remote Desktop Protocol (TCP 3389) Active Directory RPC End-Point (AD-RPC-EP)
You should notice that all of these have defined ports except for the last! By definition, the AD-RPC-EP is a random high port.
However, a somewhat surprising fact is that when you run Active Directory on Windows Server 2003 (in other words, you have a domain controller on Server 2003), the AD-RPC-EP is always either port 1025 or 1026!
Starting with Server 2008, this is no longer true. “Port randomization” ensures that ports are allocated at truly random locations.
So… I have a particular client who, as part of their Exchange 2003 to Exchange 2010 upgrade effort decided that they first wanted to upgrade their Active Directory. That was fine by me – there are no documented restrictions regarding the operating system level of domain controllers to be used by Exchange 2003.
The day we changed the last domain controller from Windows Server 2003 to Windows Server 2008 R2, the Exchange FE server stopped working. Oh, $%&#.
Thankfully, the FE server was only being used for Outlook Web Access (OWA). SMTP was injected into the Exchange environment via a Barracuda cluster directly into the internal back-end server environment.
I’m rather hardheaded. So I wanted to figure this out. And, quite frankly, it took awhile. Eventually, it took an examination using portqry and rpcping (you can find out lots of information about these utilities by using Google or Bing and searching for “portqry site:support.microsoft.com” and “rpcping site:support.microsoft.com”). Comparing the results of those to the “access control lists” from the firewall showed me that the firewall always expected a port to be open at TCP 1025 and/or TCP 1026. Neither of these ports were open on Server 2008 R2!
I went back and investigate some other customers who were running Active Directory on Server 2003. On every single one of their servers, the process lsass.exe had either TCP 1025 or TCP 1026 (or both) open.
A google here, a bing there, and I was led to Active Directory Replication Over Firewalls and KB 224196: Restricting Active Directory replication traffic and client RPC traffic to a specific port.
These led me to understand that on Server 2008 and above AD-RPC-EP could happen at any random port – but that there is a way to specify the port that will be used. YAY!
For the following
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters Registry value: TCP/IP Port Value type: REG_DWORD Value data: (available port)
specifying a “value data” of either 1025 or 1026 ensures that Server 2008 and Server 2008 R2 operate as did Server 2003. After setting this value, it does take a reboot of the affected DC/server for the value to take effect.
Once this was done, my client was happy again! I hope this helps you in your migrations…
Until next time…
As always, if there are items you would like me to talk about, please drop me a line and let me know!
Follow me on twitter! : @EssentialExch