Joyent

Joyent Weblog

When you're really pushing traffic, Amazon S3 is more expensive than a CDN

By Jason Hoffman 16 August 07

I had largely ignored the Amazon S3 pricing hike or decrease (depending on who you are) which included the addition of a per request charge. It really popped up again when I was recently talking to a new accelerator (just a few) + CDN customer recently.

They have about 9 million images taking up ~200 GB of space (average size is around 20 kb). The images are served from the hundreds to a million in a month (a busy one can be nearly a 100,000 hits in a day). In a given month they did 8,025,705,676 requests out and 21,506,532 in requests (backups and uploads). 97.3% of the request are “out” and 2.7% are “in”, and a total of 68.2 TBs was pushed.

There’s 86400 seconds in a day, and 2,592,000 in a 30 day month.

So in month of June that meant 3096 requests/second out (roughly what you can get out of 3-4 nginx/lighttpd/mainly-tuned-apache web servers.

When I feed that into the nice AWS simple monthly calculator 

Storage                       $30.00
Data Transfer         $10,626.80
Requests                 $8,240.78
------------------------------------
Total                      $18,897.58

Now, working backwards how much is this in US$/Mbps?

68,200 GBs/month = 212.5 Mbps

How much is that per Mbps (I’ll round it up by 8 cents)?

It’s $90/Mbps.

The interesting thing here is that $90/Mbps is a bit expensive for pushing ~200 Mbps out of a single datacenter.

In fact, if you’re going to be pushing that (and growing) than it is more expensive that what you can negotiate out of CDNs from Akamai, Limelight or Level3.

With the difference being that Akamai is pushing from “25,000 servers in 69 countries” and Level3 is one of the few Tier 1 network providers and their pricing includes 14 datacenters in the US, 11 in Europe and 6 in Asia. All for less than the S3.

It’s pretty simply to see for a site that is 98% downloads, 2% uploads and a smaller file size, the per requests pricing makes Amazon not even price competitive with major CDNs.


  1. I’ve been an Akamai customer for years. The things are a little less trivial than that, as for each of their server that doesn’t have the requested resource in its cache, they will hit your “origin” server to fetch it. It becomes very interesting when your visitors keep hitting their caches for the same content rather than your origin server, so best for sites that have a lot of recurrent trafic from the same geographical locations. And although you can negotiate good deals with CDNs (I certainly did with Akamai, being one of their first clients in France), you may end up just reliefing your “first mile” trafic by a meager percentage at a cost that doesn’t justify the CDN cache/bandwidth cost. You need to have a very good knowledge of your trafic (patterns, recurrence, origin…) before jumping ship. Also, AFAIK, Amazon S3 doesn’t distribute your content in the cloud close to your visitors, as a CDN does, so the two are not comparable in that respect (this said, S3 ought to be CHEAPER than the CDNs, I agree ;-).

    — padawan    356 days ago    #

  2. You glossed over an important factor, though – with Amazon S3, you don’t need to negotiate with some salesperson. You can just sign up and start working with the API immediately. I’d love to see CDNs offer an easier entry-level service like this (Cachefly is the only thing that comes close that I know of.)

    — DG    356 days ago    #

  3. I agree with padawan and DG that Amazon offers more value than these other services in many ways. The key to keeping their costs reasonable is competition. There need to be compatible alternatives and then the world will be a happy place. I’ve suggested some radical moves to make this happen over on my blog:

    http://smoothspan.wordpress.com/2007/08/16/how-does-virtualization-impact-hosting-providers-a-secret-blueprint-for-web-hosting-world-domination/

    Cheers!

    — Bob Warfield    356 days ago    #

  4. @padawan agreed on having to know one’s patterns

    @DG, no didn’t gloss over it, I’ve said before that But would you be willing to deal with a sales person if you went from $16,000/month to $5000/month while going from 1 datacenter to 25 datacenters with geographically aware and “intelligent” traffic management? Now that said, there’s no reason why one couldn’t put a CDN service on top of what you’re doing with S3 and only get requests/sec when the cache is being refreshed.

    @ Bob Warfield, yes they offer a better interface (or rather that they offer an interface at all) for developers and a low barrier of entry. But the key is that a low barrier of entry can be a trap down the road (good for developers, bad for business). For someone like the above, a high traffic pattern, high hits per object and a small storage footprint (< 1TB), then it’s relatively easy to leave. For someone with a traffic patter, lower hits per object and a larger storage footprint (>10TB), then leaving would be tough (have you moved 10TBs over the open internet lately?). And great article btw.

    — Jason A. Hoffman    356 days ago    #

  5. So, how does BingoDisk compare … or is it even an appropriate comparison to Amazon’s S3 or a CDN?

    — Tim    356 days ago    #

  6. Is there any alternative to S3 or CDN? (with the same type and amount of traffic)

    Maybe 2-3 load-balanced SPARC accelerators? Extra transfer cost $0.20 per GB, similar to S3.

    — ragni    355 days ago    #

  7. @Tim

    I don’t believe BingoDisk is an appropriate comparison.

    BingoDisk is more about storing your private documents that won’t be accessed often.

    Whereas, Amazon S3 and CDN’s are meant to serve up LOTS of web traffic.

    — Vincent    355 days ago    #

  8. @ Jason: Instead of “is more expensive THEN a CDN” I believe the correct English spelling is THAN. ;)

    — Ryan Parman    355 days ago    #

  9. The new S3 request pricing made us stop using S3:

    http://developer.amazonwebservices.com/connect/thread.jspa?threadID=16078&tstart=0

    Due the our large number of requests of small objects – we use it for “serverless” checkpoint & state management – or costs increased by a factor of 50!

    We’re taking Nirvanix for a spin now; similar storage and bandwidth cost, but no request charge.

    — Ivo Beckers    354 days ago    #

  10. This is off-topic I know, but does it bother anyone else that the Joyent/Joyeur web site take’s soooo long to load.

    I’ve always noticed this, for as long as I have been coming to his blog. Which is about a year now.

    I really want to host with Joyent, but just their slow loading home page (for a web host) deters me from hosting here.

    And, in reference to Tim … I still haven’t figured out if BingoDisk is a good solution to use to store my static web content.

    — Chad    353 days ago    #

  11. @Chad, our pages load fine. Joyent.com and Joyeur.com for example are on their own servers, are actually in separate datacenters and one is static and one is textpattern.

    — Jason A. Hoffman    353 days ago    #

  12. @Jason,

    What about the questions relating to BingoDisk.

    Is BingoDisk meant more for personal backup storage or is BingoDisk designed for serving large amounts of small static files via HTTP (e.g. css, images, javascript).

    Wayne
    P.S. Not to sound argumentative, but I agree with Chad that Joyeur seems to load slow. Maybe it’s related to the CMS that is used.

    — Wayne    353 days ago    #

  13. man this is crap… :( I had some ideas on how to use S3 and the price was awsome. I did some numbers and S3+E3 were cheaper than my hosting account. Now with this extra charge… I doubt it.

    Thanks to people that have mentioned alternatives; I had no idea about them.

    — Rafa    352 days ago    #

  14. @Bob Warfield: I didn’t write that Amazon S3 offers more value than CDNs. My point is rather that those are different and not directly comparable. CDN are about caching and distributing content over the internet, through caches close to the visitors. Amazon S3 is a storage service that doesn’t do any kind of distributed caching (at this time AFAIK). CDNs don’t necessarily offer storage (they can, but then it’s an extra option on top of the distributed cache main feature).

    — padawan    351 days ago    #

  15. To my knowledge L3 doesn’t offer CDN services; they can offer you colocation or IP transit at those locations but don’t have a CDN service.

    Another “entry level” CDN (at least as far as pricing is concered) is PantherExpress, started by some DoubleClick alums. And aside from Akamai & Limelight, there is VitalStream (now part of Internap), MirrorImage, CacheLogic, BitGravity, Savvis, Peer1, etc. Most of these should be competitive with $90/mbps at a 200mbps commit level.

    — nottlv    350 days ago    #

  16. Seems like I’m a bit behind the times…Savvis sold it’s CDN business to L3…

    — nottlv    350 days ago    #

  17. @Jason: I’d like to know what numbers you use to compare Amazon to CDNs.

    I’m new to CDNs and their prices, but I just got a quote from Limelight and they’re charging $0.8/GB for 10GB commitment per month or more. That’s $55,869 which puts it at $263/Mbps. Sure, Limelight has worldwide datacenters from where to distribute the content, but they cost almost 3 times more. And that’s without the storage cost which is another $600, 20 times more than Amazon.

    — jacob    331 days ago    #

  18. 1 Mbps is ~320 GB, so a 10GB commitment isn’t much, and yes it will be more expensive. Hence the “when you’re really pushing traffic” part of the title.

    Try committing to 500 Mbps, and you’ll see that can be about 4 cents per GB.

    — Jason A. Hoffman    314 days ago    #

Commenting is closed for this article.