The cause of MIT’s major power loss

A look behind the scenes at MIT’s cogeneration plant

5408 power
Building 42 houses a gas turbine that can supply up to about 22 megawatts of electrical power. However, it normally supplies only a fraction of the electricity used on campus.
Elijah Mena—The Tech
5409 power 2
MIT was consuming 27 megawatts at the time of the power failure on Thursday. MIT’s power plant provided 22 megawatts, and 5 megawatts was being supplied by NSTAR. Chart based on hourly samples.

What actually happened when MIT and much of Cambridge lost power last Thursday night? Why didn’t MIT’s 20 megawatt cogeneration turbine power the campus like a lighthouse in a sea of Cantabrigian darkness? What was the root cause of the failure?

MIT’s Central Utilities Plant, in Building 42, has a gas turbine that can supply up to about 22 megawatts of electrical power to most of MIT’s campus south of Albany St. But the campus regularly draws 25 MW or more, so the cogen turbine cannot supply it all — the balance comes from NSTAR, the local utility.

While the cogen plant provides a measure of redundancy, that’s not its primary purpose. It exists to save energy and improve efficiency by generating heat and electrical power simultaneously.

(The plant is called cogeneration because it produces both electricity and steam. The plant burns fuel, usually natural gas, which mixes with air under compression and spins the gas turbine to produce electricity. The same hot gases are used to boil water and produce steam, which is used to both heat the campus and to run chillers that provide cooling.)

Outage begins

At 4:24 p.m. on Nov. 29, an automated NSTAR relay detected a disturbance and took a 115 kilovolt underground transmission line out of service, affecting 19,000 customers, according to NSTAR’s early diagnosis. NSTAR spokesman Michael Durand said that NSTAR’s analysis was preliminary and that a more detailed investigation was ongoing.

The 115 kV line was one of two parallel lines that would normally back up each other. But maintenance crews were working on the other line, so it was unavailable.

That work is part of NSTAR’s “Cambridge Cooling Line Reliability Project,” see The transmission lines connect NSTAR’s Alewife East and Putnam Stations, and serve all of Cambridge east of Harvard Square, according to NSTAR filings with the city. The work is expected to increase the lines’ load capacity by 20–40 percent. The lines date from 1988 and each consists of three copper cables in an 8 5/8" steel pipe surrounded by dielectric fluid.

Large parts of Cambridge lost power. It affected East Cambridge, Kendall Square, and MIT, as well as many sections along Massachusetts Avenue up to Harvard.

MIT was using 27 MW: 22 MW from cogen, and 5 MW from NSTAR. The cogen turbine couldn’t supply the extra load, so it shut down automatically, as designed. MIT was without power, just like much of Cambridge.

In buildings throughout the campus, generators start automatically. They power emergency lighting, elevators, life-safety equipment, critical research equipment, etc. In some buildings, computer network equipment is on emergency power.

Within the main group, this emergency power comes from a single large generator located at the Central Utilities Plant. But newer buildings are required to have their own generators, so scores of generators started all around campus. The emergency power circuits are connected to automatic transfer switches that switch them to the generator power. Generators are designed to start within a few seconds of an outage.

Central Utilities Plant

At the plant, a number of things have to happen after power fails before the turbine can start generating electricity again. The process takes hours. The first order of business is to start up a steam boiler and restore steam pressure, according to Randall D. Preston, director of utilities for Facilities.

Meanwhile, NSTAR was trying to restore power. Within 20 minutes, NSTAR was able to restore power to 5,500 customers (29 percent of those affected), using remote switching technology and powering them from other parts of the grid, Durand said. For instance, the traffic signal at Main and Vassar was running, but signals on Mass. Ave. were not. Everybody else, including MIT, would have to wait.

After steam is available, MIT’s attention can turn to the cogeneration turbine. MIT can operate the turbine in “island” mode, disconnected from NSTAR’s grid. Preston said that MIT will try to do this unless NSTAR has estimated the outage will be short — but even then they’ll prepare for the possibility of no NSTAR power. MIT does not require NSTAR’s permission to operate in island mode.

A number of services need to function before the gas turbine can operate, Preston said. They include compressed air, cooling water, and exhaust ventilation. Without those, the turbine cannot run. And, of course they need to disconnect from NSTAR to avoid trying to power all of Cambridge.

Plant operators also need to clear any alarms that the turbine control system might report. They need to make sure that neither the turbine nor any other critical component was damaged when the power failed.

The best case is “probably an hour,” Preston said, “but realistically it takes one to two hours.”

Once they’re ready to go, they start by disconnecting almost all the circuits that feed the campus from the plant, so that the turbine can start with a known low load. They then start the turbine.

Then, they slowly add campus circuits to the turbine to control its load and warm up the heat recovery steam generator that captures the heat from the turbine’s exhaust.

“We were at the point of pushing the start button,” Preston said, when NSTAR restored power.

But because MIT is such a large customer, NSTAR treats it carefully, and wants to bring back large loads slowly.

“NSTAR asked us to wait 15 minutes,” Preston said. And the 15 minutes dragged on to 20 minutes. But Facilities used that time to reconnect the campus circuits that they had disconnected in preparation for starting the cogeneration turbine. Meanwhile, the traffic signal at 77 Mass. Ave. was back on.

What if NSTAR hadn’t come back? Preston said that Facilities would have to leave some portions of the campus without power until they could get unnecessary loads removed from the sections that were powered up first.

“In general, we would power up the campus circuits serving major research buildings and critical facilities first, and then go from there,” Preston said.

NSTAR’s effort

Why did it take NSTAR two hours? Durand said the time taken is actually “expected and normal to get that kind of transmission line back in service given the detailed analysis and numerous restoration steps involved.” NSTAR was “all hands on deck,” including every available management and field crew, he said.

NSTAR had to test the 115 kV line to determine if there was a fault in it. Much of the time is spent eliminating possibilities and making sure that the situation is “what we believe it is,” Durand said.

NSTAR’s preliminary determination was that the relay was operating in error, and there was no damage to the line. “The relay sensed something that didn’t happen,” Durand said.

After concluding that the relay had misoperated, it had to be removed from service.

Durand said Friday that the parallel 115 kV transmission line which had been out of service for maintenance was expected to be back in service on Saturday. No NSTAR workers were injured in either the outage or the response.

Geoff Olynyk almost 11 years ago

This is a fantastic article, very well-researched. Kudos!

Forest Grauel over 10 years ago

Dan Holohan's e-news letter had posted this article, and like Geoff, I wanted to commend John Hawkinson for such an informative and technically accurate article. I'd like to echo another Kudos to the one posted above.