Wednesday, October 5, 2016

Dealing With Failed Passive Copies

Hi folks,

I would love to share with you about my recent experience which I had with mailbox DB copies. I had an issue where mailbox DBs on one of the servers in DAG which contains only passive DB copies only some of the copies couldn't replicate if they got activated on one of the servers in primary datacenter. I also saw the following event in the Application log:

Some investigations lead me to find that this server was missing static route for the replication link.

It was fairly easy to resolve as I had to add persistent static route for replication network by running route add -p command with the appropriate information for route. After adding route replication was sorted out.

Now I had to sort out the mailbox copies themselves. I used the Get-MailboxDatabaseCopyStatus command on the server where these copies were failed which without any specified parameters retrieves mailbox databases copies hosted on the server from which you run it explicitly. Since all the copies on my servers were only passive I could filter them out by running the following command:

Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}

It retrieves all the copies DB copies that don't have status of healty. If you also have active DB copies you may also get all the mounted DB copies as their status will be Mounted. In this case you will need to tweak command to cater for these DB copies as well. So after I restored replication I ran this command to first suspend and then to resume copies:

Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}| foreach {Suspend-MailboxDatabaseCopy -Identity $_.Name -Confirm:$false}

However, after trying to suspend I got an error as below because of double pipelines:

So instead I dumped all the failed copies into a variable called $FailedCopies. My code turned out something as below:

$FailedCopy = Get-MailboxDatabaseCopyStatus |Where-Object {$_.Status -ne "Healthy"}

After this I have ran commands to suspend and resume database copies against all DBs that were in variable:

$FailedCopy |foreach {Suspend-MailboxDatabaseCopy -Identity $_.Name -Confirm:$false}
$FailedCopy |foreach {Resume-MailboxDatabaseCopy -Identity $_.Name}

Finally if you decide to reseed (it may and will consume a lot of network traffic) you may run command as below:

$FailedCopy |foreach {Update-MailboxDatabaseCopy -Identity $_.Name -DeleteExistingFiles}