Take the 2-minute tour ×

I'm designing an application using Azure Storage Blobs/Table/Queue, handling massive amount of data. One important aspect of the application, is that work will be done if a given key don't exist, and determining the existence of a key is a frequent and intensive task.

I need to optimize as much possible billable transactions from existence checks of keys. It could be either against blobs or tables.

I looked at this document Understanding Windows Azure Storage Billing – Bandwidth, Transactions, and Capacity It seems that 404 errors are not counted only from anonymous requests.

I was also thinking of using a BatchTableOperation to check 100 keys at once, maybe using a Replace or Merge, and determine in the results if the key indeed existed (haven't tried, actualy I got the idea while writing)

Any good hack are welcomed.

share|improve this question
add comment

1 Answer

You should use Windows Azure Caching:

  • Load all existing keys in the cache
  • Each time you add a record to Table Storage, also add it to cache

Once you've done that, have your application check cache first. If the item is not present there, check Table Storage just to be sure (to cover edge cases). But 99% of the time, if the item has already been processed the key will be available in the cache and you won't need to query Table Storage (this will drastically reduce transactions to Table Storage).

If using Windows Azure Caching is not an option there are alternatives, like using MemoryCache, save all keys in a file, ...

share|improve this answer
I've considered using Co-located caching. The problem is that I expect hundreds of millions of keys, and I already need to cache results of operations. However yet I haven't considered writing them in a file. –  uzul Jan 20 '13 at 2:21
add comment

Your Answer


