Tech Stuff

Amazon announced that it is expecting to roll out a Content Delivery Network before the end of the year.   As a user you will store your files on S3 and then make an API call and get a new domain name to use for your web pages.  This is different from the functionality already availible in S3 because when the website visitor accesses your file they will be routed to the nearest edge location.  This will allow for latency and the highest data transfer rates.  Below are Amazon’s goals for the content delivery service:

• Lets developers and businesses get started easily – there are no minimum fees and no commitments. You will only pay for what you actually use. 

• Is simple and easy to use – a single, simple API call is all that is needed to get started delivering your content. 

• Works seamlessly with Amazon S3 – this gives you durable storage for the original, definitive versions of your files while making the content delivery service easier to use. 

• Has a global presence – we use a global network of edge locations on three continents to deliver your content from the most appropriate location. 

It is good to see that Amazon released a pretty detailed statement detailing the events of the S3 outage on 7/20.  Some people have been complaining about the stability of the environment, but I think it has been overall really good.  When there are problems they are fixed extremely quickly and if there was going to be any delays they kept everybody up to date.  I think the Service Health Dashboard is great information tool.  I may not be happy when there is a problem, but atleast I don’t feel like I am in the dark like I have in the past with some other hosts/service providors.

Amazon announces that they are making another type of EC2 instances available.

These instances have proportionally more CPU resources than RAM (compared to our Standard Instances) and are well suited for compute-intensive applications such as rendering, search indexing, and computational analysis. These new instances include High-CPU Medium and Extra Large Instances – the Extra Large Instance includes 8 virtual cores to meet your computing needs.

The specs look pretty good.

High-CPU Medium Instance
* 1.7 GB of memory
* 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
* 350 GB of instance storage
* 32-bit platform
* I/O Performance: Moderate
* Price: $0.20 per instance hour

High-CPU Extra Large Instance
* 7 GB of memory
* 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
* 1690 GB of instance storage
* 64-bit platform
* I/O Performance: High
* Price: $0.80 per instance hour

Today Amazon announced that they will start offering premium support options for their web services.

AWS Customers who sign up for AWS Premium Support will receive personalized technical assistance from the Amazon Web Services team, whenever and as frequently as their business demands. The service offers support for operational issues or technical questions during development, test or integration. Customers can contact AWS developer support engineers and count on fast, predictable response times and personalized support to help bring their issues to resolution. AWS Premium Support is currently available to customers of the Amazon Simple Storage Service (Amazon S3), Amazon Elastic Compute Cloud (Amazon EC2), and Amazon Simple Queue Service (Amazon SQS).

The pricing seems reasonable for the services especially since you an open an unlimited number of cases.

Amazon announces that they will have support for persistent storage in an upcoming release. Right now it is being used privately by a handful of EC2 users.

This new feature provides reliable, persistent storage volumes, for use with Amazon EC2 instances. These volumes exist independently from any Amazon EC2 instances, and will behave like raw, unformatted hard drives or block devices, which may then be formatted and configured based on the needs of your application. The volumes will be significantly more durable than the local disks within an Amazon EC2 instance. Additionally, our persistent storage feature will enable you to automatically create snapshots of your volumes and back them up to Amazon S3 for even greater reliability.

You will be able to create volumes ranging in size from 1 GB to 1 TB, and will be able to attach multiple volumes to a single instance. Volumes are designed for high throughput, low latency access from Amazon EC2, and can be attached to any running EC2 instance where they will show up as a device inside of the instance. This feature will make it even easier to run everything from relational databases to distributed file systems to Hadoop processing clusters using Amazon EC2.

This is an exciting announcement and soon there will be no reasons why you wouldn’t use EC2 for a production system.  The other big thing was the dynamic IP addresses, but that was addressed a couple weeks ago with the Elastic IP Addresses.

Amazon announces the creation of availability zones for the EC2 system. Amazon describes them as “distinct locations that are engineered to be insulated from failures in other availability zones and provide inexpensive, low latency network connectivity to other availability zones in the same region”. With new web service calls you can choose what zone your instances are created in. In the past when you created a new instance you had no control over where the instance really lived.  Here is part of the email that I received this morning about it:

Availability Zones give you the ability to easily and inexpensively operate a highly available internet application. Each Amazon EC2 Availability Zone is a distinct location that is engineered to be insulated from failures in other Availability Zones. Previously, only very large companies had the scale to be able to distribute an application across multiple locations, but now it is as easy as changing a parameter in an API call. You can choose to run your application across multiple Availability Zones to be prepared for unexpected events such as power failures or network connectivity issues, or you can place instances in the same Availability Zone to take advantage of free data transfer and the lowest latency communication.

One of the problems with Amazon EC2 was the dynamic nature of the IP address that you received for each instance you started up.  For development it isn’t a big problem, but for a production system you can see the problems this can cause if you instance fails for some reason.  This morning Amazon announced the ability to have a static IP that is associated with your AWS account.  By default a user can have up to 5 static IP addresses.  The IP addresses are free of charge as long as the IP address is associated with a running instance.  If the IP isn’t then there is a charge of $.01/hr.  You can read more about this on the Amazon Web Services Developer site.

I came across an interesting article on the New York Times site that talks about how they used Amazon EC2 and S3 to help make their articles from 1851 -1922 available to the public online.  There was a total of 11 million articles.  They had to take sometimes several TIFF images and scale and glue them together to create one PDF version of the article.  They used 100 EC2 instances to complete the job in just under 24 hours.  They started with 4TB of data that was uploaded into S3 and through the conversion process created another 1.5TB.

In an articles on Forbes.com, “The Death of Hardware”, they discuss Zillow and their recent use of Amazon’s EC2 cloud computing platform to recalculate the values of some 67 million homes for their site.  Zillow estimated the task would take 6 months and millions of dollars before they went with the EC2 platform which was about $50,000, roughly 3 weeks and used 500 servers.

In the article Forbes also mentions that Microsoft is expected to roll out their own form of the computing cloud (Windows Live Core) this month which is aimed at large businesses. Yahoo is expected to release theirs later this year. With the talk about Microsoft purchasing Yahoo we will have to see if the cloud happens for Yahoo. I hope Yahoo does as I would think their platform would be linux based and that is what I am looking for. Would be nice to see more competition in the space which would help drive more advancements and hopefully even lower prices.

Amazon explains the S3 outage that occurred Friday morning. They got it corrected rather quickly, but with so many start ups using it as part of their hosting environment there are surely more gray hair out there now. Luckily I only use it for backups right now so not a big deal for me. One of the good things coming out of this is that they say they are working on a service health dashboard and expect to release it shortly.