WHY GMAIL FAILED ?

Gmail suffering some turbulence, but it appears now that it has completely crashed and disappeared. Both Apps for domain and the usual consumer Gmail service are down completely. Google seem to be going backwards on fixing the problem, this morning they sent out an alert saying:

September 1, 2009 8:18:00 AM PDT
Google Mail service has already been restored for some users, and we expect a resolution for all users in the near future. Please note this time frame is an estimate and may change.

When GMAIL WENT DOWN YESTERDAY, it caused more than a minorpanic. People, like me, who use Gmail as their primary email couldn’t get much work done. There’s nothing like an outage to make you realize how much you rely on something.

So what happened exactly? Isn’t Gmail supposed to have multiple points of failure? Well yes, Gmail has thousands and thousands of overlapping mail servers which can pic up the slack if any one fails because the data is replicated and spread all around. But there are also request servers which do nothing but route the requests for email to whichever server (with the right emails on it) happens to be available.

It tuns out that Google took down some regular email servers for routine maintenance, and because of some recent changes, that overloaded the request servers. Google engineering VP Ben Treynor explains on the Gmail Blog: