E-mail communication at Pomona College was impaired for almost a week because of an outage in Microsoft’s MyMail system. The outage, which began Nov. 8 and was not fully resolved until Wednesday, was caused by several problems within the system, which hosts all Pomona student e-mail accounts.
“Microsoft Live experienced an outage affecting North American clients. That was the first problem,” said Andrew Crawford, Director of Information Systems at Pomona’s Information Technology Services (ITS). “After that, our internal college-to-MyMail routing problem persisted even after Microsoft claimed resolution.”
Microsoft acknowledged the outage at 2 p.m. Nov. 8, but announced that all issues would be resolved within 15 hours, Crawford said. However, problems at Pomona persisted after the 15-hour time block.
“When we checked in on Saturday morning, it was still a problem, so that’s when we knew we had something else going on also,” Crawford said.
E-mail between student MyMail accounts had begun to flow, but e-mails from staff and faculty, who have internally hosted Pomona.edu addresses, had not.
“The underlying issue that was persisting was that mail from Pomona.edu, for instance when faculty is trying to e-mail you, wasn’t getting through to MyMail,” Crawford said.
The fix turned out to be receiving some updated [Domain Name System] info from MyMail,” Crawford said. “After we plugged in that information, MyMail started being delivered successfully from Pomona’s routers.”
This problem was fixed Saturday evening, but students were still not receiving e-mail updates from administrators.
“The Listserv was not sending mail,” said Steve Moss, Senior Systems Engineer at ITS, referring to the automatic mailing system. “All the stuff that goes to all students goes through Listserv, and that was still broken until Wednesday morning. We had thought that Microsoft had taken care of Listserv, but obviously they had not, so we had to change the configuration so that it would send.”
Since the e-mail outage stemmed from the e-mail hosting service, Pomona had to wait for Microsoft to fix the problem.
“There’s nothing we could really do to resolve the issue, but the primary focus for us is managing the communication with the various college constituents and then making sure the issue gets resolved quickly,” Crawford said.
ITS used alternative digital means to inform students of the hosting problems.
“We were able to utilize the campus notifications system called ConnectEd to get word out,” Crawford said. “We were able to create temporary mail accounts at Gmail. We utilized digitalized signage on campus, put announcements up on online resources like the campus portal and Sakai and utilized social media, like Facebook and Twitter, to help spread the word.”
Pomona had not previously experienced such significant problems with MyMail.
“This is the first time something like this has happened since we moved to live@edu five years ago,” Moss said. “We’ve had some one- or two- hour outages that just took care of themselves, but this is the first time we had a major, multi-day outage.”
According to Crawford, unless outages become a recurring problem with MyMail, there are no current plans to switch e-mail providers.
“Service interruptions are always a risk in IT,” Crawford said. “Whether you look to the cloud for solutions, or whether you choose to manage the service internally, things go wrong, no matter who you partner with.”
He added, “Our job is to be as prepared as possible, but there’s always risks involved when you’re dealing with the technology.”
MyMail also offers advantages over an internal hosting system, which Pomona previously employed, Crawford said.
“MyMail is your lifetime Pomona College e-mail address. It’s kind of a nice service,” Crawford said. “When you come to college, you get one address. When you graduate and go on to the rest of your life, you’re able to keep that same address. There’s no migration required. You don’t have to update your contacts with the new e-mail address, print new business cards. It will just stay with you.”