Expiring Azure Queue Message

In my notes from Day 1 of Azure Boot Camp I briefly mentioned that my class got hung up on the 30 second expiration of reservations on queue messages. What this means is that your code has a 30 second reservation on a queue message after it gets it. In those 30 seconds you need to complete the processing and then delete the message from the queue. If your code fails to do this the message will be made available again and the next call to Get will be given a reservation to the message. This will prevent the first thread from deleting the message.

For a vast majority of situation this should even be a consideration. The usual work we do in IT land doesn’t require 30 seconds of processing. That’s like an ice age worth of time to the computer. But being programmers we must obsess about edge cases, so lets say we’ve been hired by NSA to break a bunch of 768bit RSA keys. Obviously this is going to take a little more than 30 seconds. Your ‘normal’ queue processing patter would look like:

CloudStorageAccount storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference(_MessageQueueName);

if (null == queue)
{
    return;
}

bool done = false;
while (!done)
{
    IEnumerable<CloudQueueMessage> messages = queue.GetMessages(_BlockSize);
    if (_BlockSize > messages.Count())
    {
        done = true;
    }
    foreach (CloudQueueMessage message in messages)
    {
        BreakKey(message);
        queue.DeleteMessage(message);
    }
}

Obviously I don’t know how to break RSA keys, but BreakKey is where you would do that work. Once you are finished you would then delete the message from the queue. This is where the small issue pops up.

If the process takes more than the default 30 seconds the lease on the message will have expired so the delete will fail. In addition to not being able to delete the message the message will be available to be picked up by another process. What to do?

1) Change the default setting.

This is a horrible idea. Whatever amount you come up with won’t be enough. It is sort of like the “640Kb will be plenty of memory” idea. Just leave the defaults alone.

2) Use a tracking log

Create a queue manager that first gets the message, put its id and message body in a tracking table and then delete the message from the queue and then return the message to the caller. When the client is done the manager deletes the message from the queue and the table.

If the processor fails the message will still be in the table so it can be recovered. This is similar to BizTalk’s suspended message queue.

3) Just delete the message

A lot of people don’t like this, but frankly it isn’t as risky as they make it sound. Your code is running in the cloud. The servers are guaranteed to have 99.9% so it’s more likely your code is going to fail than the server. So debug your damn code! Get the queue item out of the queue, delete it, then process it and get on with your life.

All the above is pretty irrelevant, but my class still spent a good 15 minutes talk about it. In most cases the processing will be done in less than a seconds so the ‘usual’ pattern of get, process, delete will work just fine.