One Month to Rule Them All Challenge

In my company we decided to start a Mission Impossible task: creating a simple app from scratch, based on following technologies without great knowledge of some of them:

• Server:
  • 1 MVC4 RESTful WebApi on Azure
• Clients:
• 1 iOS native client
• 1 Android native client
• 1 WP7 native client
• MVC 4 Web interface running on any desktop OS
• Win8 Metro client

Let’s see what happens, in any case it’ll be a lot of fun

4ward blog post: http://blogs.4ward.it/index.php/one-month-to-rule-them-all-challenge/

NoSQL engines do not like System.Decimal

Here we are with another discovery to be aware of when you plan to go on the cloud or more in general to use some NoSQL engine: System.Decimal is not supported.

We are making a test drive of 3 NoSQL engines:

• Azure Table Storage: key – value
• Amazon SimpleDB: key – value
• MongoDB: document based

All of them do not support System.Decimal, therefore if you want to store currencies, exchange rates, large amounts with decimal digits, you’ll have to find a workaround. For the POC we have decided to go for double type, but during communications with real POS terminals ISO8583 protocol is used. In it amounts are divided in 2 separate parts:

• Amount: total amount with integer part followed by mantissa without separatore (i.e. 100,91 becomes 10091)
• MinorUnit: number of decimal digits (i.e. considering 100,91 it is 2)

Here follows a short summary table of types supported by the 3 engines:

Tipo	Azure Table Storage	Amazon SimpleDB	MongoDB
byte[]	X		X
bool	X		X
DateTime	X		X
double	X		X
Guid	X		X
Int32 o int	X		X
Int64 o long	X		X
String	X	X (max 1024 bytes)	X
regex			X

Amazon has decided to use a simpler technical implementation for its NoSQL on the cloud, but this lead to some limitations: values are all stored as strings with a maximum size limited to 1024 bytes and comparison is lexicographic, therefore for SimpleDB 10 is before 2, so developers have to do some additional work (losing a bit of performance on application code) to handle numeric values: http://aws.amazon.com/articles/1232

Note: it’s always possible to implement an event handler to be called during reading/writing operation against NoSQL above, to be used to convert Decimal to string and viceversa.

Stay tuned for the next stop!

Azure Queue Service: maximum message size is 8KB or 64KB?

Here we are with a small problem encountered: the maximum size of the Azure Queue has been increased from 8KB to 64KB from version 18.8.2011. Cool! A little more space available is great, especially on messages with lots of metadata to be sent.
But if like me you try to create a message with a size of 10KB using the SDK 1.5 (October 2011), you will receive an error.

Now to figure out exactly where the problem was I used JustDecompile (since the good old Reflector is no longer free, I tried the beta of JustDecompile from Telerik) and I decompiled the CloudQueueMessage by Microsoft.WindowsAzure.StorageClient.dll and … surprise! In the constructor there’s a nice hard-coded limit to 8192 bytes in the MaxMessageSize property:

CloudQueueMessage.MaxMessageSize = (long) 8192;

How to solve the problem? By downloading the 1.6 SDK Azure where the client libraries have been updated with the new limits and the same dll decompiled now has:

CloudQueueMessage.MaxMessageSize = (long) 65536;

Our cloud adventure has begun! Azure and then AWS

Finally we started! A nice proof of concept of a distributed architecture Cloud – OnPremise. First we’ll try Azure and then AWS.
I’ll try to post some of travel notes  of this adventure with problems, advantages, strengths and weaknesses, performance identified in order to help future adventurers.

We’ll try everything a lot of things:

• Azure Table Storage – SimpleDB: as NoSQL to save and retrieve large amounts of data
• Blob Storage – S3 to store large sets of data
• Azure Queue – SQS: for communication between the different roles within the same region
• Azure SQL – MySql: for relational data on the cloud
• SQL Data Sync: to synchronize distributed databases cloud – OnPremise
• Bus Service – SNS + SQS: to use publish/subscribe pattern for data distribution between the region and OnPremise
• Traffic Manager: to automatically balance the client calls to the nearest Azure region
• MongoDB: there will be experiments to compare a NoSQL Document based like MongoDB with the key-value solutions offered by Microsoft and Amazon
• Other: MEF, AutoMapper, AppFabric, Caching, TPL

Let’s start the journey!

Azure Table Storage and the FirstOrDefault operator

Today I’ve lost the whole afternoon fighting against this damn issue so I want to share this with everyone to save your time!

I was performing a simple query against Azure Table Storage using PartitionKey and RowKey in order to retrieve a specific entity in a specific table (i.e.: http://127.0.0.1:10002/devstoreaccount1/Spaceship(PartitionKey=’380′,RowKey=’1234′)).

During tests, the first scenario was where no entities with give PK and RK were present and I was using a simple query ending with FirstOrDefault operator like 

TableRepository.FindWhere(p => p.PartitionKey == entity.Spaceship.CountryCode.ToString() &&
        p.RowKey == entity.SerialNumber).FirstOrDefault();

and I was expecting null but in reality I started to get 404 error with detailed message below:

<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <code>ResourceNotFound</code>
  <message xml:lang="en-US">The specified resource does not exist.</message>
</error>

After several hours struggling with various tests I discovered following Pearls of Azure Wisdom:

- Internet Explorer can’t be used to query directly table storage account, it does not return results at all, because

- You have to set IgnoreResourceNotFoundException = true in the TableServiceContext in order to receive null when an entity is not found instead of error 404