If you don’t want to hear the back story, go ahead and skip down to ‘Rubber, Meet Road’ section, I tend to be windy. Is it even called windy when you type incessantly? I digress….
So, we’re in the middle of an Exchange migration. As most of you know, sometimes these migrations can be painfully slow, suffer setbacks, delays, etc. So, to that end, we had to move mailboxes off of server A and over to server B due to disk storage issues. Sadly, server A doesn’t have enough room on the drive to do an offline defrag after moving the email to server B. Needless to say, the drive is REALLY low on space, I’m talking under 1GB out of 600GB low. I monitored the disk carefully for a while, making sure that the almost 400GB of whitespace in the databases I have freed up is doing the trick keeping the drive from filling up. After a week or two, things are looking good with no substantial decrease in disk space, which was what I expected. Alas, a month later (yes, the migration was only supposed to be delayed by a few weeks, but again, we all know how that goes) I get more disk space alerts, and about 100 more MB of space had been chewed up. I filter the event log for event ID 1221 which tells me that I have ample whitespace remaining in the two databases…. So what is taking up my space? Following best practices, log files are on a different drive, so I know that isn’t the culprit. So, as you likely already figured out from the title, it was in fact the Exchange CatalogData-<GUID> folder(s). There were two folders, one for each database, and their total size was 15GB. Now, that is a lot of index for only 300GB of mail. I knew that even 10GB would likely buy me the space I needed to ride this migration delay out and not have to perform any unnecessary maintenance and downtime.
In Exchange 2007 the CatalogData-<GUID> folder is located in the same folder as the mailbox database by default. I had initially set out to move the CatalogData-<GUID> folders as I have some other drives on that server with some space. One of the first few links returned on my search sent me to this article by Vidad Cosonok. Vidad’s article was a quick ‘how to’ on freeing up space in short order on a filled up drive to buy some time. I read through it, and it looked easy enough, and it also didn’t look like it was going to cause any downtime so I gave it a go. Right around 5 p.m. (to be on the safe side) just like the article said, I stopped the Microsoft Exchange Search Indexer service, deleted two CatalogData-<GUID> folders with a hard delete (shift-delete) and restarted the service. Bing Bang, just like that, I had 15GB of free space on the drive, no errors, and the service had re-created the folders and was re-indexing the two databases. It is important to point out, that if a user tries to run a search on their email while the index is being rebuilt, it will take an extremely long time and may return a “No Items Found” when the item is actually there (false negative). It might behoove you to do this after hours, and depending on your user tolerance, you might consider doing it over a weekend as it will tax the server during the rebuilding of the index. The next morning, I checked the CatalogData folders and their combined size was 1.5GB. How about that! This leads me to believe that Microsoft Exchange 2007 does zero maintenance on catalog files, and an administrator should probably add it to their best practices and list of yearly maintenance tasks to rebuild the indexes. I would also encourage you to do this shortly after a migration on the source server, as is the case in this scenario, assuming you need the space. I monitored the index over the next couple of days and found the CatalogData-<GUID> folder did not increase much, which suggests to me that it is done with the initial indexing pass on the database and is simply keeping up with new email.
Rubber, Meet Road
Initial Problem: Free Space on Exchange 2007 Database Drive
Initial Solution: Move CatalogData-<GUID> File(s)
Actual Solution: Regenerate Exchange 2007 Database Index to flush old index files and return space to the OS.
- Stop Microsoft Exchange Indexer Service
- Delete CatalogData-<GUID> folders*
- Start Microsoft Exchange Indexer Service
- Monitor for full re-index to complete.
It is important to point out, that if a user tries to run a search on their email while the initial re-index is occurring, it will take an extremely long time and may return a “No Items Found” when the item is actually there (false negative). It might behoove you to do this after hours, and depending on your user tolerance, you might consider doing it over a weekend.
One more quick disclaimer, I bugged my Exchange Guru colleague Robert Durkin and asked about the application of this situation to Exchange 2010, and he said that while there are some differences with the distribution of the catalog when a 2010 mailbox database is in a DAG, in general the process should be the same in 2010 as mentioned by the article I linked above by Vidad Cosonok. I did not test this on 2010 and cannot speak to its applicability, so as always I’d recommend testing first.