Russians hack Ukraine’s electricity network, turning lights off and on at will, rendering the country’s best tech hands helpless to intervene. North Korea takes over the controls of a South Korean nuclear power plant. Snipers with high-velocity rifles unleash a fusillade on a transmission station near San Jose, inflicting $15 million in damage.
It’s not the plot of the latest spy novel. Rather, it’s small sampling of actual attacks, the kind of sabotage against vulnerable energy systems that can cut off power with the click of a mouse and bring officials to their knees.
Experts say energy grids are the new front in cyber-terrorism. Although the wildfires that periodically dominate the news are a serious threat to California’s power supply, cyber-invaders are an around-the clock danger, trying to penetrate grid security every minute of every day. An all-hands-on-deck battle is being waged against them, and the network that serves nearly 40 million people’s homes, industries and public-safety agencies depends on a successful defense.
Should the grid be hijacked, the entire state could be held hostage, experts say. Can the state prevent what one utility executive likened to “a hostile takeover?”
Never has California’s aging electricity infrastructure been more vulnerable, even as the government plans to rely on it more completely with 5 million electric cars and, eventually, to fully operate the world’s fifth-largest economy. Moreover, fire and sophisticated hacking aren’t the only risks to the consistent flow of electrical power: The grid can also be undone by California’s formidable natural forces—wind, earthquakes, floods—and the most humble of creatures.
Gnawing squirrels have brought stock trading to a halt more than once by chewing electricity lines and disrupting NASDAQ computers. Rodents also interrupted some operations at Los Angeles International Airport on Thanksgiving Day in 2015, briefly stopping elevators, baggage screening and other functions.
RELATED: As California Enters A Brave New Energy World, Can It Keep The Lights On?
A widespread, sustained power outage is frightening to contemplate, with the tools we use to navigate our lives taken from us: no lights, telephone service or charging capacity; no heating or cooling; no computers, working gas pumps or ATMs.
“Think of the internet as a weapon of mass destruction,” said former news anchor Ted Koppel, whose book “Lights Out” explores threats to U.S. electricity grids.
The scale of the challenge to keep the lights on was drawn in sharp relief in November at Stanford University, where experts gathered to peer into the energy future. Hint–it’s murky and potentially dark, including ransom schemes and even grid attacks intended to sow chaos ahead of terrorist ground assaults.
“Our adversaries are advancing at a rapid clip,” said Sue Gordon, principal deputy director of national intelligence, the federal government’s leading cybersecurity agency and a conference participant. “Within a few years Russia and China will have the ability to conduct on-demand, localized disruption of service, including of control systems in multiple sectors, simultaneously.”
California’s vulnerable electricity infrastructure has the attention of lawmakers in Sacramento. Some have faulted the oversight of safety procedures at power companies, a displeasure that has brought new scrutiny to the Public Utilities Commission, which regulates those businesses.
A new state law requires utilities to undertake broader, more aggressive planning for reducing fire risks associated with utility equipment, though regulators say they have neither funding nor technology to comprehensively monitor even the measures companies already have in place.
In addition, the state operates two entities tasked with analyzing and addressing potential hacking threats, particularly to critical infrastructure such as power stations and the dams where hydroelectric power is generated, and coordinating their efforts with utility companies.
Still, state Senator Bob Hertzberg, who has an extensive background in the power sector, worries that California is underprepared.
“It’s unbelievable, oh my God; there are so many things that can go wrong,” said the Democrat from Van Nuys, who authored a law last year pinpointing threats to the grid from electromagnetic attacks. “Security is a big deal, and it’s become a big deal because we rely on electricity for everything.”
The California Independent System Operator, the private company that is the state’s grid overlord, is obsessed with security. A spokeswoman said it has significantly increased investment in cybersecurity in recent years and regularly conducts internal testing. It even relocated its fortress-like facility to cocoon its operations against harm—sited on high ground, away from known seismic faults, close (but not too close) to Sacramento and a highway.
Like every utility and power company in the state, the grid operator has marauders at its cyber-gate every minute of every day.
“We are always being attacked,” said Mark Rothleder, a CAISO vice president, gazing down at the gymnasium-sized control room where electricity supply and demand are constantly monitored. “We have mechanisms in place … but it’s constantly changing. It’s a constantly evolving threat we have to defend against.”
Security managers universally seem to adopt a “security through obscurity” approach, not wishing to discuss specific strategies other than to say they are in place. In addition, the state’s largest investor-owned utilities—Pacific Gas and Electric, Southern California Edison and San Diego Gas & Electric—all have specific departments to defend against cyber-incursions.
Attacks can come from an unknown source, anywhere in the world. Some hackers gain access to a host system and do nothing more than hide in the figurative bushes, getting to know the lay of the cyber-land and biding time. Malware lurks in dark corners.
It can take three to six months to detect a computer breach, according to David Goeckeler, Executive Vice President of San Jose-based CISCO, the computer networking giant with cybersecurity expertise.
Just as often, hackers bang on the digital front door, on the off-chance it will be mistakenly opened. Goeckeler, speaking at the Stanford energy forum, said saboteurs attempt to hack CISCO’s systems 20 billion times a day. Yes, billion.
Such threats have become a priority for the federal government. California utilities participate with those of other states in federal exercises such as GridEx, a war game that simulates grid attacks and coordinates potential responses with local and state emergency agencies, law enforcement, the Department of Defense and telecommunications and banking firms.
At SDG&E, which has 3.6 million electricity customers, “there’s always some type of an intrusion attempt daily,” said Zoraya Griffin, the company’s emergency operations manager. “It’s not a matter of if we have them, but how many.”
In addition to threats that are difficult to see, such as those posed by hackers, some vulnerabilities are literally before our eyes. Aging equipment. Poorly maintained circuits and other key equipment. Miles of wires surrounded by overgrown vegetation and tinder-dry trees that can ignite in a flash.
The state’s response to the issue has been accelerated by the wildfire threat.
Officials have ordered utilities to “harden” their equipment with such measures as replacing wooden power poles with metal or composite ones, swathing lines in more robust insulation and wrapping other equipment in metal or other fire-resistant material.
The Public Utilities Commission is awaiting companies’ specific fire-mitigation plans, due in early February. Those reports are to detail how the utilities will construct, maintain and operate their equipment to minimize its risk of causing catastrophic blazes.
The work will carry a sobering price tag. For example, a federal judge has proposed that PG&E inspect its entire transmission system–including nearly 100,000 miles of power lines–and make its equipment more fire-safe. The utility balked, saying the work would cost $150 billion.
But why expose equipment and circuits to fire danger in the first place? Money. It makes the most financial sense for utilities to traverse California’s steep mountain passes and broad deserts by erecting power towers that march with giant steps across those vast landscapes.
And although companies tend to locate power lines underground in new housing tracts, relocating above-ground lines under the earth is expensive–as much as $3 million a mile. In addition, digging new trenches in established neighborhoods and congested areas typically meets local resistance.
Utilities have stepped up their resiliency efforts, clearing brush and cutting trees, installing cameras to monitor far-flung equipment and focusing more on the potential of weather to cause problems. But one company, San Diego Gas & Electric, is ahead of the pack in elevating weather forecasting to a high priority, investing billions of dollars to respond to threats from nature.
The company’s first line of defense begins 20 feet up on power poles, with compact but complex remote weather stations. The utility has the state’s most extensive weather-monitoring network, with 177 stations across 4,000 square miles.
Every 10 minutes the stations beam real-time information about temperature, humidity and wind speed and direction to a command center in San Diego. The data is also available to the public on the company’s website.
In conditions of high wildfire risk—excessive heat or wind, for example—the information is folded into twice-daily emails shared with the state firefighting agency and local responders. The weather forecast is also included in a separate email dispatched every morning at 6:30 to company leaders.
Remote-controlled cameras provide a picture of what’s happening not only with the company’s equipment but also with the surrounding landscape. Trouble can be spotted more quickly, crews can be better positioned and power shutoffs can occur earlier when necessary. Control of the cameras can be handed off to fire officials during emergencies. In a revenge-of-the-nerds moment, meteorologists and fire forecasters now have a seat at the company table alongside the brass and the engineers when emergencies arise.
“Understanding the weather enables decision making,” said Chris Arends, who manages the utility’s meteorology program.
David Nahai is the former head of another company, the Los Angeles Department of Water and Power, the largest municipal utility in the United States. He said keeping the lights on requires investment, planning and constant vigilance.
“You think about natural disaster all the time. You think about fires all the time. It’s an everyday priority,” Nahai said. “That doesn’t mean you can withstand everything that we are vulnerable to.”
This is the second in a series of articles, published in partnership with The Sacramento Bee, to explore California’s need to modernize the electricity network that powers the state.
