REQUEST INVITE
When Winter Storm Uri hit Texas in February of 2021, it caused the greatest loss of electricity in the state’s history. What were supposed to be rolling blackouts designed to ease the power grid ended up in days-long outages for millions of people. Approximately 70% of Texans suffered blackouts for an average of 42 hours, while roughly 50% suffered water disruptions for an average of 52 hours. Amidst all this devastation, one beacon of light sustained no disruption at all: the University of Texas’s 160-building, 20-million-square-foot campus in Austin.
This will come as no surprise to those familiar with UT-Austin’s award-winning power system. The Carl J. Eckhardt Combined Heat and Power plant is one of the biggest microgrids in the country. In operation since 1929, the plant currently produces 100% of the campus’s power with 99.9998 reliability. It accomplishes this with two combustion turbines, four boilers powering four steam turbines, eighteen electric chillers across five chilling stations, and two heat-recovery steam generators that put waste heat to work. Overseeing this complex system was one man: Associate Vice President of Utilities and Energy Management Juan Ontiveros.
With 25 years at UT-Austin (he retired in December 2021) and roughly half a century as an engineer, Ontiveros has watched his field evolve from using slide rules to the neural networks controlling UT-Austin’s energy system. He’s also learned a thing or two about how to build a facilities team, like the one that refused to go home during Winter Storm Uri. “They worked 90 hours straight. They slept on the floor. They would not go home,” he said during a recent presentation at the Higher Ed Facilities Forum. “You cannot force people to do that. They have to want to do that.”
Over the course of his session, Ontiveros explained how UT-Austin survived the storm, why you have to design paranoia into your systems, and what it takes to create a culture of high performance.
Minutes from Disaster
The power plant’s combined cycle system – using natural gas turbines to generate electricity while sending excess heat to the steam turbines – was key to UT’s survival during the storm, when solar and wind power were out of the question. But that’s not to say survival was easy. Even though it was winter, Ontiveros’s team had to keep the cooling system going; when the water stops flowing through the pipes, after all, the pipes freeze. “We were able to go from 15,000 tons down to about 4,000 tons,” he explained. “We were running the whole campus on one chiller.”
As water pressure in the cooling plants descended from the usual 100 PSI to 20 PSI, the plants were within minutes of tripping. “We were running around the campus trying to plug all the leaks and shut off the buildings that had leaks,” he said. “We got it stabilized, and we didn't have to cut the plants.”
Another risk lurked in the reduced electric output, from 25 megawatts to 18 megawatts: any lower would have tripped the system. At the same time, steam output rose from an average winter load of 200,000 pounds per hour to 350,000, a historical high. “Everything went to heating the buildings,” Ontiveros said. Then there was the gas company, which he was afraid would cut off supply to UT’s two high-pressure gas mains. “They kept saying, ‘Lower your load, lower your load,’” he recalled. “So we lowered our load. We just basically put it where we needed it to go. And they were happy with the fact that we were able to respond to that.”
Carl J. Eckhardt Combined Heating and Power Diagram
Paranoid by Design
The freeze caused minor damage in about 30 buildings, but the campus was still able to avoid shutting down. How did UT-Austin avoid catastrophe? Ontiveros attributes this outcome in part to the lessons learned after a major freeze in 2011 that tripped one of his combustion turbines. “We insulated the heck out of everything” after that experience, he said, and conducted heat tracing on critical systems. When Uri rolled around, his team leaned on its most reliable equipment and a complex web of contingencies: improvements to the high voltage system, a quench vessel dissipating excess steam production, thermal storage to optimize the cooling system, water recovery and reclamation systems, and a microgrid controller breezily managing everything in one place.
“We use paranoia to design our controls,” Ontiveros said of this interlocking system of backups. “You want to apply paranoia in your design, but you don’t want to be paranoid in your operation.”
A Culture of High Performance
From an operational standpoint, he attributed his team’s success to the culture of high performance he’s spent his tenure building. That means serious investments in people. Ontiveros is committed to employee development, using a test-based training program to help employees advance from temporary to senior roles. "We have a philosophy that if there's any promotion, we don't advertise outside – we give our existing employees the opportunity first," he said. "We also hire from below entry-level. Hire a temporary employee for a year and hand 'em the training program, then we get to test them and they get to test us. We like 'em, we hire 'em. Out of 20 people we've done that with, we've hired probably 15. And one of those guys is now a supervisor.”
Ontiveros has also learned that to foster a culture of high performance, you have to incentivize mistakes rather than punish the people who make them. “In our business, it takes seconds to make decisions before you lose it,” he said. “So if you have the skill and knowledge to do it, do it. If you made a mistake, tell us about it, we’ll fix it, and I’ll take the hit for it. It’s my job… If you punish them, they stop making decisions so that they don’t make mistakes and you lose it.”
Related is what he calls the “IQ Principle,” or incremental improvement using innovation: a process of making small innovations to existing practices, learning from failures, and building on successes. “You want to fail small. You do one thing, you can see whether its effect was good or bad, and if it was good, okay – what’s the next thing?”
People, People, People
What do you get when you put these things together? A facilities team that refuses to go home during the deadliest winter storm in almost 30 years. As Ontiveros concluded, the most impressive microgrid in the country is only as reliable as the employees making it run. “The key thing was people, people, people, people. They left their families at home and came to work to fix it, and keep it running, and they refused to go home,” he said. “They believed in it, they bought into it. That’s what allowed us to succeed.”
Comments