WhatTheCloud.Com

Do you ever wanna check your Twitter or tweet about something that’s important to you and get the Fail Whale?  Twitter serves 19 Billion searches per month.  That’s more then Google or Bing.  Sixty five million tweets a day from over a 190 million users.  That requires some serious horsepower.  But something in that kind of demand shouldn’t fail.  Seems like a great case for the cloud, doesn’t it?  Think about it, you’re serving that much data you would want to have almost infinite power and redundancy.  You would also want some load balancing and maybe even a content distribution network.

So, who offers all of these things now,  in one integrated form?  The answer to that, and why it still doesn’t work coming up.

Amazon Web Services offers all of those services in one package.  They have load balancing; cloud servers in both Unix and Windows varieties.  Twitter uses a Unix Apache based server, so they have the stability of Linux.  What they don’t have is infinite expandability.  They would constantly need to monitor their load, and add new servers (VPS) every time their loads start to max out.  Then, they would need to add those servers to their load balance and allow all of that to propagate.  That would be more than a full time job and if you miss your window of opportunity you would have down time.

Add on to all of that, how often the Amazon Cloud fails, you have a system that has fail whales.  Here is what we would do differently:  Set up infinitely expandable nodes with the LAMP (Linux, Apache, MySQL, PHP), setup that would expand and contract as needed.  We would add in a CDN (Content Distribution Network) to handle images, and load balance all of that across the globe.  Now, none of this would be cheap but it would allow for full redundancy and expandability for something that, as of right now, is unreliable.

Let us know your thoughts.  What would you do to help Twitter not Fail Whale?