Tech Stuff

I have been using Amazon’s EC2, S3 and Cloud Front a lot over about the last 9 months.  Last week I just started looking at Amazon’s Associates program for some display advertising.  While looking the info I discovered the Product Advertising API.  In my quest to improve my ASP.NET/C# skills I decided to put together a little website that was powered by the web service.  The result of a few hours is http://searchforgiftsonline.com/.  The site allows the user to search by keyword and category the Amazon inventory.  The search results are displayed on the site and then linked to the product description on the Amazon site.  There is also a list of the major categories from Amazon.  When the user clicks on a category they get the Top Selling, Most Gifted, Most Wished For and New Releases.  The API is pretty easy to work.  Now I just need to figure out new things to do with it.

In celebration of S3’s 3rd anniversary Amazon has announced the reduction of the transfer-in costs for data from $0.10/GB to $0.03GB for April – May 2009.  Now is the time to transfer all the home videos and music that isn’t backed up else where.

Amazon anounces Elastic MapReduce.

Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

If you wanted to do a large amount of data processing the past with EC2 you would have to setup the whole environment which can be time consuming for the first time user or over whelming just thinking about it. This will make it much easier for other companies to tackle tasks like the NY Times did when they converted a large amount of scanned articles in TIFF format into PDFs.

In the last couple of weeks I have been working with Amazon’s Cloudfront to deliver small javascript files.  It has turned out to be a pretty good experience.  The only problem we have had so far is that you can not invalidate the cache on the CDN servers and force them to immediately fetch new copies from Amazon S3.  You have to wait 24 hours for the content to expire.  If you plan on your files changing often you will need to find a way to version your files.  In my case the version doesn’t work too well as we can’t easily change the file references to these javascript files.

Amazon EC2 is now out of Beta after two year and now gets an SLA.

Service Commitments and Service Credits

If the Annual Uptime Percentage for a customer drops below 99.95% for the Service Year, that customer is eligible to receive a Service Credit equal to 10% of their bill for the Eligible Credit Period. To file a claim, a customer does not have to have wait 365 days from the day they started using the service or 365 days from their last successful claim. A customer can file a claim any time their Annual Uptime Percentage over the trailing 365 days drops below 99.95%.

Today Amazon anounced new tiered pricing for the S3 service.

For the United States: 

Storage — Current Pricing (thru October 31st)

• $0.15 per GB-Month of storage used

Storage — New Pricing (effective November 1st)

• $0.150 per GB – first 50 TB / month of storage used
• $0.140 per GB – next 50 TB / month of storage used
• $0.130 per GB – next 400 TB /month of storage used
• $0.120 per GB – storage used / month over 500 TB

Amazon has announced that later this fall that they will offer the ability to run Windows Server and SQL Server in the EC2 environment.

Amazon announced that it is expecting to roll out a Content Delivery Network before the end of the year.   As a user you will store your files on S3 and then make an API call and get a new domain name to use for your web pages.  This is different from the functionality already availible in S3 because when the website visitor accesses your file they will be routed to the nearest edge location.  This will allow for latency and the highest data transfer rates.  Below are Amazon’s goals for the content delivery service:

• Lets developers and businesses get started easily – there are no minimum fees and no commitments. You will only pay for what you actually use. 

• Is simple and easy to use – a single, simple API call is all that is needed to get started delivering your content. 

• Works seamlessly with Amazon S3 – this gives you durable storage for the original, definitive versions of your files while making the content delivery service easier to use. 

• Has a global presence – we use a global network of edge locations on three continents to deliver your content from the most appropriate location. 

It is good to see that Amazon released a pretty detailed statement detailing the events of the S3 outage on 7/20.  Some people have been complaining about the stability of the environment, but I think it has been overall really good.  When there are problems they are fixed extremely quickly and if there was going to be any delays they kept everybody up to date.  I think the Service Health Dashboard is great information tool.  I may not be happy when there is a problem, but atleast I don’t feel like I am in the dark like I have in the past with some other hosts/service providors.

Amazon announces that they are making another type of EC2 instances available.

These instances have proportionally more CPU resources than RAM (compared to our Standard Instances) and are well suited for compute-intensive applications such as rendering, search indexing, and computational analysis. These new instances include High-CPU Medium and Extra Large Instances – the Extra Large Instance includes 8 virtual cores to meet your computing needs.

The specs look pretty good.

High-CPU Medium Instance
* 1.7 GB of memory
* 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each)
* 350 GB of instance storage
* 32-bit platform
* I/O Performance: Moderate
* Price: $0.20 per instance hour

High-CPU Extra Large Instance
* 7 GB of memory
* 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each)
* 1690 GB of instance storage
* 64-bit platform
* I/O Performance: High
* Price: $0.80 per instance hour