I’ve just returned from Sys-Con’s Cloud Computing Conference and Expo in beautiful Prague. A very useful networking event; it was great to meet familiar faces in the growing cloud community and to evangelize the cloud approach to a new audience!
Roman Staneck, CEO of Good Data, opened with the keynote – Building Great Companies on the Cloud. Good Data is one of the Czech Republic’s cloud success stories (although Roman is now based in San Francisco, the engineering is centered in his home country); they provide deep business analytics and really could not operate in a cost-effective way without the pay-by-use and scaling capabilities of the cloud.
As well as a session at the Cloud Bootcamp, I spoke about architectures for applications in the cloud, and why load balancing is not only a key architectural component but also a proven way to get more performance, control and manageability out of cloud applications.
Application Architecture
At first glance, cloud-based applications may not look significantly different to applications running in traditional physical or virtualized datacenters. In the Cloud, you still get servers to run things on, you still get storage (though not always persistent) and you get basic network connectivity. The Cloud Provider manages large amounts of this physical infrastructure, chops it up using virtualization techniques, then rents to you what you need on a by-use basis.
However, one of the more common reasons for using an infrastructure cloud is to utilize the massive scalability it provides. If you have this in mind, it will affect the architecture you want to use.
There are other considerations too; I wouldn’t say that the Cloud is ‘less reliable’ than other infrastructures, but the fact remains that if you plan for failure and build a fault-tolerant architecture, many tasks become easier. Scalability requires automation, as do so many routine management tasks. Finally, if you've planned for disaster recovery you’ll find it much easier to migrate your application to another provider if commercial or SLA reasons require you to do so.
Distilling this down, we want a scalable, fault-tolerant, and most importantly simple architecture for our cloud based application.
Here's a few choices:
1. Monolithic Applications
The most simple application architecture – the monolithic (all on one server) design as epitomized by the LAMP stack – is not amenable to cloud applications.
Not only does it present a single point of failure, but because it’s limited to a single server instance, it has limited scalability if traffic demands exceed the capacity of the machine.
2. Three-tier applications
In his Cloudnavigator blog, Rick Otten recently wondered aloud whether 3-tier web architecture models were too rigid for the cloud.
In his blog, he identifies the role of the load balancer in this architecture. He asks if an F5 balancer can render the webserver-part of the architecture irrelevant (Rick – an F5 can’t, but a ZXTM certainly can).
Rick compares the Infrastructure-as-a-Service (IaaS) approach with Platform-as-a-Service (PaaS) architectures, but I’d like to stay with the different architectures, you can use in IaaS clouds.
There’s nothing stopping you deploying this architecture, but it has scalability challenges and is probably more complex than is necessary for most applications. For example, when the service becomes overloaded, scaling out the web tier may only have a marginal effect if the app tier is the bottleneck, and you have much less granularity over how you scale.
3. Service Oriented Architecture (SOA)
The SOA approach to application architecture breaks an application apart into constituent, reusable services, and then wires them back together with network cables (virtually perhaps!) to create something like this:
You might be tempted to conclude, after 10 years of hard-learned experience, that the SOA approach was developed by a consortium of vendors with paid-for voting rights on various standards bodies, championing it for their own commercial gain rather than for the good of the IT industry as a whole.
This architecture can work for massively complex, distributed applications. Amazon use a home-grown SOA approach, where well over 150 SOA components are queried to generate a single web page on amazon.com, but they built their application this way because it suited the way they governed their development team as much as for scalability reasons. With low utilization of individual servers and a complex architecture that is very expensive to scale, it’s probably not the best way to begin.
4. An architecture for the Cloud?
There’s one architectural design pattern beginning to prevail in cloud applications where scalability is a key concern. It’s emerging from experiences in building highly-scalable applications across any infrastructure; I don’t know if it has a widely accepted name yet, and I would not presume to give it one:
The application part of your service is implemented in a single, self-contained unit that is designed from the outset to be scalable horizontally. Ideally, no session state is stored in that unit, so it can be scaled at will for easy performance increase.
The data is stored in a separately scalable database. There are a lot of approaches to highly scalable database implementation, and most of them dispense with the traditional monolithic relational approach and partition the data in some scalable manner. Again, there’s lots of real-world examples (for example, this one) referenced through the invaluable site highscalability.com.
Scalability becomes much easier (apart from the problem of tracking where data is if you partition it – master-slave is one approach) because your application becomes much more homogeneous. It’s relatively straightforward to spin up more ‘app’ when latency or CPU utilization increases and poke the load balancer to send traffic to it.
In this design, caching becomes an architectural solution rather than an afterthought. Memcached may be used for caching on the application tier, and web caching used on the front-end load balancing tier.
Partition the app as well as the data:
It’s not a scalable idea to put transient state in the database in a busy system (for example, the intermediate state of a shopping cart). If you need to track state, client-side cookies and/or local data on the application instances. You can use the session persistence capabilities of the load balancer to pin simple sessions (keyed by cookie for example) to the application partition that contains the more detailed data. If you follow that route, you can partition your application layer as well as your data layer, thus minimizing internal network chatter which can sometimes cost you dear.
For more information on this sort of architecture, Wille Faler’s blog is a good starting point.
Load balancing in the Cloud
None of this was the focus of the session in Prague... it was just a lead into the more interesting question of why and how you could architect advanced load balancing right into the core of your application, whether it’s physical, virtual or cloud based. There’s a lot of lessons the cloud ecosystem will inherit from experience in physical or virtual environments, and I’ll touch on a few another time.
Owen Garrett
