Agencies are getting more data on cybersecurity incidents than ever before. But if they can't manage the deluge of information coming in, all those tools are useless.

Bringing together multiple data sources in a unified system is a lot like conducting a symphony where everyone is using different sheet music. The keys to making it work are automation and orchestration.

"We rely very, very heavily on automation and integration," said Philip Quade, special assistant to the director of NSA for cyber.

He pointed to a federal agency that was getting hit with more than a billion incidents a day. Prior to automating its security operations, the agency was only able to handle approximately 65 of those a day through manual processes. Additionally, manually reviewing an incident took anywhere from 11 minutes to 11 hours.

"They turned on the COTS [commercial off-the-shelf] secure orchestration and instead of doing 65 events a day, they started doing tens of thousands of them simultaneously," Quade said. "And they were doing that in anywhere from 0.1 seconds and 1 second. They were able to increase security and increase efficiency at the same time."

Quade offer four lessons the NSA has learned from its efforts to automate its threat identification systems.

Cyber defense means protecting networks inside and out

"We need to have active cyber defenses at the boundaries of our networks," Quade said. "However, boundary defense alone is not sufficient."

Quade equated it to only focusing on cyber hygiene, which is important but only one aspect of cybersecurity. To have a truly robust security system, defensive tools need to focus on protecting the perimeter as well as the data flowing inside the network.

Orchestrate, don't replace

Quade noted it isn't practical to expect organizations — whether federal agencies or private companies — to replace their entire enterprise architecture just to automate threat detection. Rather, organizations should be looking for orchestration products that can be bolted on to boost security monitoring on existing systems.

For NSA, that means having a "bring your own security enterprise approach," he said.

"Whatever products you already have … you need to embrace existing COTS products on your network and get them to work as a team," he said.

Products should act as one

"One-off solutions have proven unwieldy," Quade said. "You can't expect customers to buy three or four or five, six, twelve different products, each with its own dashboard and figure out a way to manage all those on the enterprise."

Instead, organizations should look to integrate as many products as possible.

"It's all about integration and automation to accommodate security at speed and scale," he said.

COTS, meet GOTS

Finally, Quade noted the immediate depreciation of products as soon as they're developed, a problem that is perhaps most stark when it comes to government off-the-shelf solutions (GOTS).

"GOTS products that the NSA or someone else might advocate … their shelf-life starts ticking down the second they're conceived," he said. "What we really need are COTS-based solutions that use standards — an open standards based approach," so they can be integrated with new tools as those become available.

"Secure orchestration is a game-changing approach," Quade said, so long as it's done correctly.

Editor's Note: A previous version of this story quoted Quade speaking about a specific agency. He actually referred to an unnamed agency CIO. The story has been updated to reflect this.