Technology & MSG

Hotmail is “452 Out of memory” … Again.

Jan 8th, 2006 by Alex

After attempting to send a brief message to a few MSN subscribers, it appears that Microsoft is having some memory issues on a few of their mail exchangers.

It appears that Microsoft utilizes round robin DNS for their email infrastructure, thus including Hotmail and MSN. No big shock there; their capacity is such that they’d require several servers just to handle the raw number of simultaneous connections, let alone capacity. So according to DNS, MSN’s mail goes to one of four mail exchangers.

> host -t mx msn.com
msn.com mail is handled (pri=5) by mx2.hotmail.com
msn.com mail is handled (pri=5) by mx3.hotmail.com
msn.com mail is handled (pri=5) by mx4.hotmail.com
msn.com mail is handled (pri=5) by mx1.hotmail.com

And then those records all resolve to four IP addresses each.

> host mx4.hotmail.com
mx4.hotmail.com has address 65.54.245.104
mx4.hotmail.com has address 65.54.190.179
mx4.hotmail.com has address 65.54.244.104
mx4.hotmail.com has address 65.54.244.232

That’s 16 possible IP addresses to connect to, assuming there isn’t anything more fancy going on behind the scenes. In actuality, each IP address probably is owned by a load balancer, responsible for doling out the connections even further to numerous backend servers.

Unfortunately — even with all that technology — the second one listed is dead entirely…

Jan  8 21:14:16 vision postfix/smtp[95439]: connect to mx4.hotmail.com[65.54.190.179]: Operation timed out (port 25)

On an immediately re-try, the first one is found to be up and listening, but not too responsive.

Jan  8 21:14:17 vision postfix/smtp[95439]: 3A283405B: to=, relay=mx4.hotmail.com[65.54.245.104], delay=32,
	status=deferred (host mx4.hotmail.com[65.54.245.104] said: 452 Out of memory (in reply to MAIL FROM command))
Jan  8 21:14:17 vision postfix/smtp[95439]: 3A283405B: to=, relay=mx4.hotmail.com[65.54.245.104], delay=32,
	status=deferred (host mx4.hotmail.com[65.54.245.104] said: 452 Out of memory (in reply to MAIL FROM command))
Jan  8 21:14:17 vision postfix/smtp[95439]: 3A283405B: to=, relay=mx4.hotmail.com[65.54.245.104], delay=32,
	status=deferred (host mx4.hotmail.com[65.54.245.104] said: 452 Out of memory (in reply to MAIL FROM command))

All right Microsoft! “Out of memory” errors from the vendor whose founder proclaimed that 640k ought to be enough for anyone. I’m amused. Of course, I’m not the first one to encounter this problem. I wonder if Microsoft actually uses IIS for their mailing infrastructure? I’d be amused if they do and it can’t handle the load, as evidenced. I’d be even more amusing if they don’t because they know it can’t handle the load.

Edit: The mail did go through on the second attempt (by way of the mx3 cluster). Just in case you were wondering. Didn’t want you to lose any sleep.