Steffen Konerow had this to say in a post on HighScalability.com entitled "6 Ways To Kill Your Servers - Learning How To Scale The Hard Way": 

 

Scaling a website is a never ending process. As soon as you fix one bottleneck you’r very likely to stumble into the next one. Never ever start thinking “that’s it, we’re done” and lean back. It will kill your servers and perhaps even your business. It’s a constant process of planning and learning. If you can’t get a job done on your own, because you have a lack of experience and/or resources – find a competent and realiable partner to work with. Never stop talking with your team and partners about the current issues and the ones that might arise in (near) future. Think ahead and be proactive!

 

Sadly, nearly 13 years after Amazon's first major Christmas collapse, and 9 years after the crush of 9/11 traffic took down CNN & Co - we're still bedeviled by manual caching and tweaking and structural issues around building our websites.  It's almost embarrassing that we don't have a drop-in toolkit that lets us roll out technologies easily.  

 

I know the adage that every application and web service is different, but really, most are not.  Most are making 80% read calls/20% write calls.  Most are making many calls to a database per page.  Some are making multiple database connections per page (really poor design).   Every application is different - Ruby, Java, PHP, Python, etc.  It is amazing that this is still voodoo magic. 

 

I believe that to be the attraction of PaaS models like Google App Engine - you're given a set of constraints to work against, a set of libraries for doing certain types of things, and you have to fit your model to the platform.  App Engine is great for web applications, but doesn't work for things that require sockets - things like jabber clients, peer-to-peer file transfer, things of that nature.  So there's an opportunity there as well.