Joyent

100,000 Joyent Accelerators

We just delivered the 100,000th Joyent Accelerator to a customer. That’s a big milestone. Congratulations to the Joyent team. And congratulations to our customers who are doing such interesting things with Joyent Accelerators, everyone from Prince (the artist known as), to all the Facebook developers, to the many enterprise shops removing the barriers of IT from the innovations of smart developers. Onwards to 1,000,000.

Cloud Computing is actually Transparent Computing: there's shouldn't be anything cloudy about it

A little over a year ago Joyent was identified in a Forrester report as one of ten “Cloud Computing” companies to watch. The report was really the beginning of “Cloud Computing” being applied to those of us doing infrastructure and I think marks the point in time when the term really began to appear in marketing materials.

Our inclusion on that list was notable because we were the only privately held company on the list that operates their own server farms and develops their own software stack to instantiate and manage it all.

Since then it’s been cloud cloud cloud cloud cloud cloud cloud cloud and I recall the first time I noticed it showing up as an adjective, I overheard someone actually ask “Is your application cloudy yet?”.

Does one really want a “cloudy” application?

Our premise here has always been that an application should “Just Work” and when it does not, then we should have the means to quickly figure out why and correct it. Importantly, one should be able to figure out the issue(s) within the context of the entire architecture.

The general “gut feeling” though is that something is changing in our industry, and I would say that like how “the network” was flattened, standardized and made into a utility, what people are talking about as cloud computing is the next wave of making data and compute available as utilities or even if entirely owned, one conceptualizes and runs it as a utility. Well that’s part of it. For now, we’re all just calling it Cloud Computing.

Like anything that is a utility, there are initial concerns that fundamentally boil down to trust.

Trust does not emerge from things that are cloudy (opaque). It emerges from complete transparency and the ability to determine what’s is happening or has happened. Trust comes from letting an infrastructure know what’s important to you. I want to query an infrastructure with “This application needs to be available within 1 second to 99.99% of my users 99.99% of the time. How much is that and can you do it?” or perhaps “This 246GB of data you’re about to handle is ‘level 3’ data in our privacy policies. How much is that and can you do it?”.

The hallmark of this “Cloud Computing” needs to be complete transparency, instrumentability and while making it certain that applications just work, the interesting aspects of future APIs aren’t provisioning and self-management of machine images, it’s about stating policies and being able to make decisions that matter to my business.

Part 3, On Joyent and Accelerators as Cloud Computing "Primitives"

In the last part of this series we ended by talking about 6 “simple” utilities that software uses on “servers”. They were

1) CPU space
2) Memory space
3) Disc space
4) Memory bus IO
5) Disc IO
6) Network IO

Along with their natural minimums (zero) and maximums.

Providing compute units that do these utilities

What we’ve always wanted to do at Joyent was provide “scalable network appliances”: online servers that just worked for given functions and were capable of both handling bursts and serving as logical lego-like building blocks for new and legacy architectures. Sometimes these appliances might contain our own software, sometimes not. They would be on a network where it would be difficult for a given piece of software to saturate the immediate parts of it.

For most workloads, a ratio of 1 CPU:4GB:2 spindles works pretty well. The faster the CPU (constrained by power) the better, memory is well … memory, and the size and speed of those spindles can be varied depending on what a node is going to be doing (one end of a workload possibility or the other). In other words, 1) CPU space, 2) Memory space and 3) Disc space aren’t terribly interesting or difficult to schedule and manage in most environments (in some ways, they’re a purchasing decision), and 4) Memory bus IO is set in silicon stone by their creators.

Which ones matter?

We’re left with disc and network IO as being the key utilities and the ability to move things on-and-off disc and in-and-out of the network comes from using more CPU. So when an application experiences a surge in activity, it typically results in more use of the CPU, disc IO and network IO. We decided the best approach was to put CPU and disc IO on a fair-share system, and standardize on an overbuilt network (10 Gbps interconnects, multi-gigabit out of the back of each physical node). Our physical servers exclusively use intel NICs in a PCIe format. This way we can have multiple gigabits per node, physical segregation and failover as options, and we can standardize on a network driver and get away from on-board NICs changing from server to server.

The fair share system overall is supposed to provide for short-term (could just be minutes) bursting needed to handle spikes in demand. Spikes that occur too fast for either hardware upgrades or even to spin up new VMs on most systems (and are often too fast to be noticed in 5-10 minute monitoring pings). This allows for you to stop thinking about the servers that you pay for as being these constrained maximums, and start thinking about a “1 CPU, 4GB of RAM” server as being a guaranteed minimum allotment.

This was at least why we stopped calling them “Grid Containers” and started calling them “Accelerators”.

Moving up the stack

The next installments are going to be talking about some kernel and userland experiences and choices, our desire to use ever more “dense” hardware, 32 and 64 bit environments, and rapid reboot and recovery times.

Part 2, On Joyent and Cloud Computing "Primitives"

In the first part of this series I made a key list of some of the underlying ideas at Joyent, that we believe that a company or even a small development team should be able to:

1. Participate in a multi-tenant service
2. Have your own instantiations of this service
3. Install (and “buy”) the software to run on your own infrastructure
4. Get software and APIs from Joyent that allows for the integration of all of these based on business desires and policies.

And said

The successful future “clouds” have to be more accessible, easier to use and to operate, and every single part of the infrastructure has to be addressable via software, has to be capable of being introspected into and instrumented by software and this addressability means that one can write policies around access, performance, privacy, security and integrity. For example, most of our customer really don’t care about the details, they care in knowing what is capable of providing 99.99% of their end users some great experience 99.99% of the time. These concepts have to be bake in.

I continue to think that from a developer’s perspective the future is closer to the SMART platform where Ramin’s comment on an older Joyeur article about EC2 versus Accelerators is relevant, let me quote him:

Whoever has the fewest number of steps and the fastest build/deploy time is likely to attract the most developers. Whoever can show that the operating cost scales linearly with use will have developers casting flower petals in their path :-)

As an app developer, I don’t care that it runs on Solaris, FreeBSD, or Mac-OS. I want it to work. I want an optimized deployment workflow and a simple way to monitor and keep things running.

That all said.

In the second part to this series I wanted to start talking about “primitives”. I’m saying “start” because we’re going to be going to be covering primitives over the next couple of posts.

I’m going to loosely define “Primitives” (now with a capital P) as of all the stuff underneath your application, your language and the specific software you’re using to store your data. So yes, we’re talking about hardware and the software that runs that hardware. Even though most Primitives are supposed to eventually be hidden from a developer they’re generally important to the business people and those that have to evaluate a technology platform. They are important parts of the architecture when one is talking about “access, performance, privacy, security and integrity”.

Previously, I’ve talked about a bit about Accelerators ( On Accelerators) and that fundamentally we deal with 6 utilities in cloud computing.

The fermions are the utilities where things take up space

1) CPU space
2) Memory space
3) Disc space

The bosons are the utilities where things are moving through space and time

4) Memory bus IO
5) Disc IO
6) Network IO

All of these utilities have physical maximums dictated by the hardware, and they have a limit I’d like to call How-Likely-Are-You-To-Do-This-From-One-Machine-Or-Even-At-All.

I’ll admit at this point of a particular way of thinking. I think “what is the thing?”, “how it is going to behave?”, “what are the minimums and maximums of this behavior?” and finally “why?”.

The minimum for us is easy. It’s zero. Software using 0% of the CPUs, 0 GB of memory, doing 0 MB/sec of disc IO and 0 Gbps of network traffic.

The maximums:

1. Commercially available CPUs typically top out in the 3s of Ghz
2. “Normal” servers typically have <128 GB of memory in them and the ratio of 4GB of memory per CPU core is a common one from HPC (we use this and it would mean that a 128 GB system would have 32 cores)
3. Drives are available up to a terabyte in size but as they get larger you’re making performance trade-offs. And while you can get single namespaces into the petabyte range, even though ones >100 TB are still irritating to manage (for either the increased fragility of a larger and larger “space”, or the variation in latencies between a lot of independent “storage nodes”).
4. CPUs and memory talk at speeds set by the chip and hardware manufacturers. Numbers like 24 Gbps are common.
5. Disc IO can be in the Gbps without much of an issue
6. For a 125 kb page with 20 objects on it, 1 Gbps of traffic will give you 122,400,000 unique page views per day and that in a 30 day month this is 3,672,000,000 page views (Check my math). Depending on how much stuff you have going on, this basically puts you in as a top 100 web property. With the number of public website is ~200 million (source), being in the top 200 is what … 0.00001% of the sites?

As something to think about and as an anchor, I remember seeing a benchmark of a “Thumper JBOD” attached to a system capable of saturating the 4×10Gbps NIC cards in the back of it. Yes the software was special, yes it was in C, and yes it was written with the explicit purpose of pushing that much data off of discs; however, think about that for a minute.

Imagine having a web property doing 120 billion monthly page views coming off of a single “system” that you can buy for a reasonable price. Starting from there, expand that architecture and I wonder with the “right software” and “primitives” where you would end up. If we change it from a web property to a gaming or a trading application, where would you end up? What is the taxonomy of applications out there (common and uncommon) and do we come up with the best architectures for each branch and leaf on that tree?

Please think about that anchor and a taxonomy for a few days and then I’m going to get into some of the key differentiators of our Primitives and answer some of the “Why?”.

Speaking at the FSF Meeting, Libre Planet '09

On Saturday and Sunday I’m going to be at the FSF Meeting in Cambridge MA where I’m going to talk about The Free and Open Cloud.

Of course I’m not talking about Free-as-in-Beer here, but instead software freedom, and how the cloud is a new battleground for Free and Open Source software — something that matters a lot to me.

Most of my life I’ve benefited from Free Software. I wouldn’t have been able to acquire the knowledge, or do the jobs, and therefore, even earn what little money I have if it hadn’t been for free software.

It matters a lot.

When Joyent first approached Bryan and I at Reasonably Smart with the opportunity to join them one of my first concerns was “What does this mean for the idea that the platform software should be free?” I wouldn’t be posting on the Joyeur blog today, if the answer to that had been anything less than it was, and so when the opportunity to talk about free software and the cloud passed by my desk late last week, I jumped at the opportunity – not only because of what I feel about free software, but also how Joyent approaches free software.

For those of you not able to be there, I’ll be posting the talk here early next week – I’d post it sooner, but I’m polishing, polishing, polishing, and with a lot work and a little luck, it’ll be alright on the night.

On Joyent and "Cloud Computing", Part 1 of Many

With this post, I’m starting a series where we’re going to be much more explicit about what we’re thinking, what we’re doing, how we’re doing it and where we’re going. I’m not interested in any of it being thought of as impersonal “marketing material”, so I hope you’ll allow the occasional use of “I” and for the interweaving of my perspectives and perhaps a story or two. As the CTO and one of the founders whose lives has been this company, I’m going to go ahead, be bold and impose.

In May of this year, we’ll have been in business for five years, and looking back at it, it’s amazing that we’re still here. Looking back at it, it makes complete sense that we’re still here. Our idea was pretty simple: let’s take what it normally BIG infrastructure (i.e. 64GB of RAM, 16 CPU servers, 10 Gbps networking), virtualize the entire vertical stack and then go to people who normally buy some servers, some switches and some storage, and sell them complete architectures. All done and ready to go.

It is commonly said that businesses only exist out of necessity, that the best ones take away or care of “pain”. Makes sense, doctors only exist because there is disease.

It is a pain for most companies (“Enterprises”) to expertly own and operate infrastructure that can scale up or down depending on their business needs. Infrastructure is more a garden then a monument but rarely treated as one.

It is difficult to be innovative in the absence of observing tens of thousands of different applications doing tens of thousands of different things.

It is easy to make the same mistakes that others have made.

That final point is an important one, by the way. I’ll tell you why in a slightly roundabout way.

I’m a scientist by training and have always been a consumer of “large compute” and “big storage” not just a developer of them (I can write about why that is a good thing another time). While at UCLA as a younger man, I was lucky enough to have had a graduate physiology course where one of the lecturers was Jared Diamond and later on I went to an early reading of one of his books that was coming out, Guns, Germs, and Steel. What reached out and permanently imprinted itself in my mind from that book was the Anna Karenina principle. It comes from the Tolstoy quote, “Happy families are all alike; every unhappy family is unhappy in its own way.”, and simply put it means that success is purely the absence of failure.

Success is purely the absence of failure.

Education and “products” are meant to save people and enterprises from mistakes. Both the avoidance of “known” mistakes (education) and the surviving of “unknown” mistakes (luck, will power, hustle, et al).

Internally I commonly say,

It’s fine to make mistakes as long as 1) no one has ever made that mistake before, if they have, you need to question your education or get some, 2) we’re not going to be repeating this mistake, if we do, we need to question our processes, our metrics and how we communicate and educate and 3) the mistake does not kill us

This is all important because I believe the primary interest in “cloud computing” is not cost savings, it’s in the participation of a larger network of companies and people like you. It is the consumption of products that

1. Are more accessible and easy to use (usually from a developers perspective)
2. Are easier to operate

However, that’s not it. I can say with complete certainty that main barrier towards adoption is trust. Trust specifically around privacy, security and integrity.

The successful future “clouds” have to be more accessible, easier to use and to operate, and every single part of the infrastructure has to be addressable via software, has to be capable of being introspected into and instrumented by software and this addressability means that one can write policies around access, performance, privacy, security and integrity. For example, most of our customer really don’t care about the details, they care in knowing what is capable of providing 99.99% of their end users some great experience 99.99% of the time. These concepts have to be bake in.

These are some of the underlying ideas behind everything that we’re doing. At Joyent, we believe that a company or even a small development team should be able to

1. Participate in a multi-tenant service
2. Have your own instantiations of this service
3. Install (and “buy”) the software to run on your own infrastructure
4. Get software and APIs from Joyent that allows for the integration of all of these based on business desires and policies.

Sometimes people refer to this as the “public and private cloud”. Oooook.

I happen to believe we’re capable of doing this quite well compared to most of the large players out there. We own and operate, and we’re making those APIs and software available (often open source).

Amazon Web Services is a competent owner and operator and allows you to participate in a multi-tenant service like S3, but there are no API extensions for integrity, you cannot get your own S3 separate from other people, you cannot install “S3” in your own datacenter, and of course there is no software that allows an object to move amongst these choices based on your policies.

Question: “If you have a petabyte in S3, how do you know it’s all there without downloading it all?”, I can answer that question if I had a petabyte on Netapps in-house.

The Cisco, Sun, HPs and IBMs (I’ll toss in the AT&Ts too) of the world want to sell you more co-lo, perhaps something best called co-habitation, more hardware, more software, less innovation and no ease of operating.

Larry Ellison says that this is all a marketing term. I think he’s wrong. We think he’s wrong, and that Joyent seems to be unique in focusing on making the above a reality.

Next week, I’ll be discussing “primitives” and “architectures”.

Try Out Aptana Cloud for Free

Joyent has been working with the folks at Aptana for some time and we’re excited to mention that Aptana is now offering free trials of Aptana Cloud built on top of Joyent Accelerators.

Aptana Cloud is an excellent service that makes development, deployment and scaling of PHP, JavaScript, and Ruby on Rails applications as automagic as anything we’ve seen. Using Aptana Studio, the easy-to-use tool (based on Eclipse) available for Macintosh, Windows, and Linux, a developer literally writes code and simply deploys to the cloud by the click of a button. Scaling is simply a matter of dragging a slider to provide the application more resources. Source code control (and rollback), backup, remote edit and preview, database tools, ssh and sftp, staging environments, team tools, alerts, dashboards, stats and logs are all included and, in our opinion, beautifully implemented and designed.

And now you can try out Aptana Cloud for free for seven days. Please do.

Open, Loving, Just Workingness: The Smart Platform and Javascript

About a month ago Joyent acquired Reasonably Smart and we were happy with the breadth of the coverage (e.g. @GigaOm). Much of the feedback was positive and there were also some important questions and comments: many around “what is open?” and the current choice of Javascript as the server-side programming language for the platform.

There were a few other factors that were important to us at Joyent, and that weren’t covered. David and I, the founders, happened to get along great with Reasonably Smart’s founders and could see ourselves working together for a long time. Basically, if I started a company today, I’d give those guys a call, and try and get them involved. I’d say that we’re also pretty happy about being able to make smart, targeted acquisitions in today’s economy, considering everything that’s going on in regards to technology companies.

But.

Back to the Big Technology Things.

On Javascript

There’s a few things that have been sitting in my mind for a bit of time:

1) Our connector suite of software happens to be nearly 10% javascript (see the ohloh analysis). I’d find myself wondering why not 20%, 50% or 100%? Why aren’t we just writing all of our applications in straight Javascript? Once you begin to think of it as a server-side language, it gets pretty interesting.

2) Joel Spolsky wrote this great little piece called Can Your Programming Language Do This? and all the neat examples actually used Javascript as the programming language.

3) When talking to the guys at Sun’s Fishworks Project, I learned that they were writing their restricted shell for their storage appliance in Javascript (the CLI is 24227 lines of javascript). A shell. A command line shell. It’s in javascript. Javascript.

4) The realization that the problem with Javascript wasn’t the language, it was the DOM (Document Object Model) and how it’s implemented in different browsers.

Bruce Tate hits the nail on the head here

The beleaguered language sags under the weight of a complex programming model called the document object model (DOM), poor tools for implementation and debugging, and inconsistent browser implementations. Until recently, many developers had all but written off JavaScript as a necessary evil at best or a toy at worst.

Our own James puts it nicely “if you’ve not programmed JavaScript without all that tedious mucking about in the DOM, try it – you’ll be pleasantly surprised!”.

5) And yet … Javascript finds itself in the top 3 or 5 languages to know for the future (Red Canary’s survey and GigaOm coverage is just one example.)

6) In the same article, James points out that Javascript is a hardened language. That’s great for a service provider, nothing to strip away from a language like what had to be done with Python in Google App Engine.

7) And finally, the Javascript VM used in the Reasonably Smart platform could find itself supporting the syntax of other languages (for example, python and ruby) in the near future.

And we have the potentially interesting adoption pattern of client-side javascript to server-side javascript, and the possibility of more and more functionality being added to applications by just having the javascript that’s already there be extended and hooked directly into data stores et cetera. It was feasible to even look at our own current applications (or geez, future command line shells …) and see how they would evolve this way give the right platform, frameworks, backends.

Javascript, at least for me, emerged then as a fascinating language. And fascinating in several different ways: technically, culturally (in lack of a better word) and as a core for one of our service businesses.

On Smart Platforms

Now to that Service.

We’re calling it the Joyent Smart Platform now (don’t quite know if it’ll be the Joyent Smart, Joyent SMART or Joyent S.M.A.R.T platform — likely the first one because wouldn’t one expect S.M.A.R.T to mean something? — but I’ll let David and Bryan figure that one out).

And it’s actually a “platform”.

Joyent has been working on providing a rock, solid set of “primitives”: powerful, secure and flexible infrastructure (as a service …). These primitives have been able to crank away on respectably sized sites and problems. [Author’s note: I’m trying to limit the “as a service” trailer].

We’ve always wanted things to Just Work™, to be open, flexible, loving and happy.

Now why would one write to a platform?

You can write an application, use documented APIs and interfaces in that platform to do everything you need to do, and boom bam magic bim bam it’s deployed, up, running, instrumented and scaleable. Fast, easy, beautiful, and you can focus on your actual business.

Why wouldn’t one write to a platform?

Three reasons: vendor lock-in, portability and junk parts.

A Loving Cloud is what I want, and Simon Wardley wants Happiness. Honestly I often think of Simon’s rules when looking at our own roadmap.

Rule 1: I want to run the service on my own machine.

Rule 2: I want to easily migrate the service from my machine to a cloud provider and vice versa with a few clicks of a button.

Rule 3: I want to easily migrate the service from one cloud provider to another with a few clicks of a button.

The Smart Platform, when deployed, will be provided as an supported and open source offering, and will always strive to use the best components. This means you can make use of your own installation of the platform or our service (we think the value in software emerges when done as a service) or both. You can easily access your application code and it’s data, and when integrated with all the primitives Joyent has underneath you will be able to clone the entire platform stack (from load balancers to smart datastores). As long as you have access to raw iron you will be able to do what’s best for you, and do it in the same exact way.

We’ll be Open, you’ll be Loved and hopefully from that, you’ll be very Happy.

Scalable Traffic Direction and Load Balancing in the "Cloud" with Zeus and Joyent

Last night Zeus and Joyent announced the offering of ZXTM inside Zeus accelerators.

There are some interesting things to note about the offering, the relationship and how the Zeus accelerators work. It also marks the beginning of networked “appliances” being offered at Joyent and a new delivery mechanism and model for Zeus.

Zeus, as you may know, produced one of the better performing web servers and now has the only traffic direction solution that’s offered as software and would be considered “enterprise grade”. As an enterprise grade load-balancer/traffic manager, it’s typically sold and distributed like most enterprise software. ZXTM as offered at Joyent is a new way for software makers like Zeus to ship and sell software, and we’re excited that they trusted us with it. It’s definitely the most accessible and cost-effective way to get ZXTM.

As a fairly regular topic on this blog, we’ve noted that traffic direction is the key to any scalable architecture: HTTP is a stateless protocol (its scale is inherently horizontal) and a domain name requires a relatively small set of IP addresses in the front. There is state then in the IP address and scale requires linking those few front-end IP addresses (e.g. 2-3) with tens to thousands and perhaps back to tens of back-end application nodes. That’s where products like F5’s BIG-IPs and Zeus ZXTM come into play. One is a hardware solution and the other is software.

We’re only offering the full ZXTM, to keep things simple, and everything is basically available within normal accelerator pricing (you bought 2x $500 accelerators before and now they’re a ZXTM cluster). The clustered accelerators provide each customer with an independent and isolated ZXTM setup, complete access to the ZXTM management interface and API, and unprecedented access to network and traffic metrics.

Also a “developer version” of ZXTM will be showing up as standard package in all of our future accelerators. The developer version won’t do all the clustering, in-memory caching, failover and multi-node stuff but Zeus has been kind enough to provide all of us with what is otherwise a feature complete software stack. This will allow anyone to investigate the API, use it on single accelerators and get an idea of the kinds of information and metrics that you can get out of Zeus’s management interface.

For the paid Zeus Accelerators, they’re only available as at least redundant pairs and there are no limitations on the number of backend accelerators or clustering (except the normal practical limits). It took some work for this type of network failover to work within the Joyent cloud (and as far as I know, we’re the only ones that can securely do it). We feel that if you actually need these then they should always be VLAN’ed and clustered, and that is the only way we’re selling them. You can scale your Zeus Accelerators by just adding more memory and CPU power, and their scale is really driven by how much you have to cache in-memory.

With the ability to load-balance and direct gigabits per second now completely in the hands of our customers (and at a moment’s notice), we’re working hard to add more and more unique “functional” accelerators to the mix and in many cases, like ZXTM, they’ll be available as larger, complex and more interesting setups and packages.

So let’s welcome Zeus to the family and Happy Traffic Directing everyone.

Joyent Acquires Reasonably Smart

I’m very excited to announce that Joyent has acquired the Montreal-based Reasonably Smart platform. James Duncan and Bryan Bogensberger, founders of the company and platform, have joined Joyent.

This acquisition comes at an important point in time for Joyent. We’ve just finished our best year, ever. Revenue doubled in 2008 from 2007 even as the economy continues to tank. Customers are moving spend from cap-ex to op-ex and Joyent products are ideally situated for this trend. However, more and more customers are expecting auto-scale from their cloud vendors and this acquisition allows Joyent to respond to those demands.

Reasonably Smart is an auto-scaling platform-as-a-service; it is a direct, open-source competitor to Google App Engine. We believe the future of cloud computing is found in open, transparent platforms that allow customers to move between clouds easily, if desired. Unfortunately, that is not the case for most cloud vendors today. This acquisition underscores Joyent’s commitment to customers and their interests in cloud computing.

This acquisition also underscores a bet that Javascript will become the language of scale for the web. When I talk of scale, I don’t only mean “big” but also “very small” since most applications developed are used by a handful of people from time-to-time. The platform will support other languages in the coming months, but we feel confident that Javascript is a very competent first start.

I want to welcome James Duncan and Bryan Bogensberger to Joyent. James will continue to develop the platform. He has been a champion and contributor of open source for many years and has experience in building platforms at enormous scale. Bryan will be serving as Joyent’s VP of Marketing. Jason Hoffman and I knew they both were great fits for Joyent since they like to have fun and they are passionate about doing what is right for customers.

The Reasonably Smart platform is off-line for a couple months while we integrate it with Joyent backend services. Meanwhile, please sign up at http://reasonablysmart.com (running on Joyent Accelerators) to allow us to keep you informed of progress.