Exchange 2013: Resubmitting messages from the safety net using Add-resubmitrequest

Exchange 2013: Resubmitting messages from the safety net using Add-resubmitrequest

Yes, I am still blogging, it has been a while, but I am still alive, this time, I am back to Exchange blogging.
I faced a tough case last week, one of my customers faced an awkward Exchange issue when the log drive was full, and followed by a power cut that put the DB in dirty shutdown and we faced a logical/physical corruption.

So let us see what happened, I was called in after this disaster happened and several attempts to recover the DB has failed, so I started by trying to bring the DB out of the dirty shutdown state, this can be done using ESEUTIL
eseutil /r “E02” /l “m:\usersdb” /d “m:\usersdb
but that didn’t bring the DB out of the dirty shutdown state; I resorted to /P switch
Eseutil /p usersdb.edb
But that failed with error message “illegal duplicate key”, but the DB moved to clean shutdown, so I tried to mount the DB, but I got this fancy error:
“Dbtime on current page is greater than global database dbtime”
Tim McMichael has a great blog about this error here
So, I figured that the EDB file is gone, and I have to restore the DB. Luckily we had a complete full back up from last night, so I restored it, brought it out of the dirty shutdown and swapped the DB files, that went fine, and the DB was mounted, but what about the emails from the time of the backup to the time of failure (the backup completed 1 AM and the failure happened 10:40 AM).

I knew that we can get emails resubmitted from the safety net but this is done by the active manager, so the question was how to resubmit those messages manually?!

I had to revert to my own safety net, my friend Mohamed Dawy from the MS Exchange PFE team, and he advised me that I can use the Add-resubmitrequest to resubmit messages from the safety net after restore.

That was new to me and it worked perfectly, that caused duplicate resubmissions for some users, so they got the same email delivered twice, but that was fine because we recovered all the emails that were lost.

The Add-resubmitrequest was new to me and I didn’t find a lot of people speaking about it, so I hope that this command could be handy to you one day.

Till next time.

One thought on “Exchange 2013: Resubmitting messages from the safety net using Add-resubmitrequest

  1. OK I ran in to a need to fall back on this, but something isn’t working right. First some details. I had 2 database servers mb1, and mb2 and logs on a single san part of a DAG (don’t ask why, reasons). Because of san performance issues, I wanted to remove one of the database servers from the DAG in order to ultimately decomission one of the mailbox servers. So i removed the database copies on mb2 and removed mb2 from the dag. It was late and I didn’t pick the operation up until morning. Unknown to me at the time, but the receive connector on mb1 was boned and the edge tranport delivered to mb2. So message went through edsync to mb2, but didn’t land in anyones mailboxes for a period of about 12 hours. I can see them in message tracking logs on mb2, so i though easy enough, i could add-resubmit from safetynet for that time period now that the receive connector was fixed. The add-resubmit has been running since i started it. The messages in my time window are fairly important. Any input? Does the destination guild need to be a recovery database? Output is the same either way.

    Here is the results of get-resubmitrequest

    RunspaceId : 71abbdd6-da8f-4472-9695-f90e526368bd
    Identity : 15
    Server : mb2
    Destination : databaseguid
    StartTime : 3/16/2017 9:00:00 PM
    EndTime : 3/17/2017 11:30:00 AM
    DiagnosticInformation : (Primary – Running)
    CreationTime : 3/17/2017 11:06:39 PM
    State : Running
    IsValid : True
    ObjectState : New

Leave a Reply

%d bloggers like this: