[i got permissions from Lachlan Donald at 99 Designs to post his email to me here - jm3]
We [at 99 Designs] actually don't have a huge number of instances, although we are steadily adding more and more.
We have at present:
2 x Caching Proxies (Large)
8 x App Servers (Large)
3 x DB Servers (High-CPU X-Large)
2 x Workers (Large)
We started out with a lot more small instances, but found that the disk performance on the small instances is extremely variable and often terrible.
We also use RightScale for management and are at present looking at using server arrays to scale up and down the App Server layer to deal with load spikes.
The biggest thing that reduced our need for lots of servers was moving as much functionality to asynchronous tasks as possible, which are queued on each appserver via Beanstalk queues and then consumed by the worker servers. Additionally we have as many pages http cache-able as we can, which are served directly from the squid cache at the front of the app.
We [at 99 Designs] actually don't have a huge number of instances, although we are steadily adding more and more.
We have at present:
We started out with a lot more small instances, but found that the disk performance on the small instances is extremely variable and often terrible.We also use RightScale for management and are at present looking at using server arrays to scale up and down the App Server layer to deal with load spikes.
The biggest thing that reduced our need for lots of servers was moving as much functionality to asynchronous tasks as possible, which are queued on each appserver via Beanstalk queues and then consumed by the worker servers. Additionally we have as many pages http cache-able as we can, which are served directly from the squid cache at the front of the app.
Hope that helps!
Cheers, Lachlan Donald CTO, 99designs