Tech Stuff

I recently started using  Cloudberry’s Backup Desktop edition as my Carbonite (Online Backup) subscription is running out.  In their newest version 2.0 they have added the ability to backup to a local folder or network share.   With storage space so cheap now I would much rather backup my music and videos locally.  I just bought a 2TB external eSATA drive for $100.  The plan is to backup documents and other smaller files to Amazon S3.  So far this is working out really good.  The only thing that it is missing is version control for the local backups.  It maybe there someone in the options, but just haven’t found it yet.  I have been really happy with Cloudberry’s products and at the speed that they keep up with new functions that Amazon rolls out for S3 or Cloudfront.  I have come to rely on Clouberry S3 Explorer for managing my CloudFront distributions.

At the end of August Amazon announced that CloudFront now has an invalidation feature which can clear the files from the edge locations before the expiration period.  The other day I got to try out this new feature when I had a file I needed to update and didn’t really want to wait the 1hr TTL that I had set.  I downloaded the newest version of CloudBerry Explorer which supports the new feature and gave it a try.  It didn’t seem to be instantaneous, but the roughly 10 minutes I had to wait was much better than waiting the hour.  Not sure if this time delay is what everyone else experiences or if it was something on my computer caching it longer.

I just recently read that you can now set the TTL of an object on Amazon’s Cloudfront to 1 hr instead of the 24 hr minimum that it used to be.  I only found this out because I was reading through the discussion forums looking to see if there was an ETA of when this could be done.  Didn’t know this had been around since April.  Since Amazon didn’t include it on any of the monthly announcements of new/improved features I figured I would share.

The default for an object is still 24 hours, but you can use the HTTP Header Expires to set a different expiration.  If the time is less than an hour, Cloudfront will default to 1 hour.  I still wish there was a way to expire a given object at anytime, but this is a big step forward.  However, it would be nice if there was a tool out there that would allow me to set a default expires time period when I upload an object to S3 based on the content type.  In my case, I can usually have my images set to a 24 hour TTL, but there are some HTML, CSS and Javascript files that I would want at only an hour.  Right now I use S3Fox and Cloudberry to manage my S3 buckets.

I have been using Amazon’s EC2, S3 and Cloud Front a lot over about the last 9 months.  Last week I just started looking at Amazon’s Associates program for some display advertising.  While looking the info I discovered the Product Advertising API.  In my quest to improve my ASP.NET/C# skills I decided to put together a little website that was powered by the web service.  The result of a few hours is http://searchforgiftsonline.com/.  The site allows the user to search by keyword and category the Amazon inventory.  The search results are displayed on the site and then linked to the product description on the Amazon site.  There is also a list of the major categories from Amazon.  When the user clicks on a category they get the Top Selling, Most Gifted, Most Wished For and New Releases.  The API is pretty easy to work.  Now I just need to figure out new things to do with it.

In celebration of S3′s 3rd anniversary Amazon has announced the reduction of the transfer-in costs for data from $0.10/GB to $0.03GB for April – May 2009.  Now is the time to transfer all the home videos and music that isn’t backed up else where.

Amazon anounces Elastic MapReduce.

Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

If you wanted to do a large amount of data processing the past with EC2 you would have to setup the whole environment which can be time consuming for the first time user or over whelming just thinking about it. This will make it much easier for other companies to tackle tasks like the NY Times did when they converted a large amount of scanned articles in TIFF format into PDFs.

In the last couple of weeks I have been working with Amazon’s Cloudfront to deliver small javascript files.  It has turned out to be a pretty good experience.  The only problem we have had so far is that you can not invalidate the cache on the CDN servers and force them to immediately fetch new copies from Amazon S3.  You have to wait 24 hours for the content to expire.  If you plan on your files changing often you will need to find a way to version your files.  In my case the version doesn’t work too well as we can’t easily change the file references to these javascript files.

Amazon EC2 is now out of Beta after two year and now gets an SLA.

Service Commitments and Service Credits

If the Annual Uptime Percentage for a customer drops below 99.95% for the Service Year, that customer is eligible to receive a Service Credit equal to 10% of their bill for the Eligible Credit Period. To file a claim, a customer does not have to have wait 365 days from the day they started using the service or 365 days from their last successful claim. A customer can file a claim any time their Annual Uptime Percentage over the trailing 365 days drops below 99.95%.

Today Amazon anounced new tiered pricing for the S3 service.

For the United States: 

Storage — Current Pricing (thru October 31st)

• $0.15 per GB-Month of storage used

Storage — New Pricing (effective November 1st)

• $0.150 per GB – first 50 TB / month of storage used
• $0.140 per GB – next 50 TB / month of storage used
• $0.130 per GB – next 400 TB /month of storage used
• $0.120 per GB – storage used / month over 500 TB

Amazon has announced that later this fall that they will offer the ability to run Windows Server and SQL Server in the EC2 environment.