‘Big Data’ offers big progress in network security
Narus and Teradata
As they survey the frustrating current cyber security scene, experts recognize that the problem is not that we know too little about the threats that are attacking our computer networks.
“The problem is we have too much security data,” explains Microsoft’s Scott Charney, “and we don’t know what to make of it.”
Charney, a corporate vice president for trustworthy computing, delivered a keynote address at the RSA cyber security conference on Feb. 28, in which he described the current dilemma -- petabytes of seemingly useless data that cry out to be analyzed -- and suggested one promising solution: Big Data.
In essence, Charney and many other cyber gurus have begun to focus on what they call Big Data as an important new weapon that the good guys can try to perfect and deploy in the never-ending battle with cyber attackers who would steal money, intellectual property, sensitive information and anything else of value that resides on a vulnerable computer network.
What is Big Data? How is it being used? And how will Big Data evolve in the near future? Government Security News gathered some interesting insights at the recent RSA show, and has put together some preliminary answers.
Big Data is a term that has come to describe a humongous mountain of information (typically captured on a computer network from mind-numbing log files, flow data, IP traffic and other voluminous sources) that might contain worthwhile clues to the origins and behavior of a cyber attacker, if only that data could be analyzed effectively in real-time. Until recently, cyber security professionals have found it nearly impossible to cope with the vast amounts of data captured daily on their networks, exceedingly difficult to identify miniscule “clues” hiding amidst all that data that might warrant further analysis, and overwhelmingly burdensome to store all that incoming data, while waiting for one of those suspicious clues to come to the attention of an alert analyst.
The vast amount of data that has been burying network administrators in recent years is only growing worse, Charney warned. He specifically cited two new sources of information, “geo-location data,” derived from the widespread use of GPS and mapping software on mobile hand-held devices, and “user-created content,” the diaries, scribblings, videos and postings that are washing around the globe on ever-more-popular social media Websites. As this ocean of network-related data rises, the need to find a way to make some sense of it grows accordingly.
That’s where two companies – Narus and Teradata – have entered the scene. They announced at RSA a promising new partnership in which Narus would handle the analysis of vast quantities of network-related data and Teradata would handle the storage and crunching of that data. Here’s how a press release issued on Feb. 27 by both companies described the capabilities of their new partnership:
- The most scalable, real-time traffic intelligence system that captures, analyzes and correlates IP traffic in real time, and offers wide visibility across heterogeneous networks and deep insight into multiple layers of network traffic.
- Patented analytics to detect patterns and anomalies that can predict and identify security issues, misuse of network resources, suspicious or criminal activity, and other events that can compromise the integrity of the network.
- The ability to respond quickly to known and previously unknown cyber threats with effective, informed action based on business and operational policies.
To gain a better understanding of what that actually means, GSN spoke with Jay Thomas, vice president of global services at Narus, and Monica Smith, a marketing executive with Teradata. They described a hypothetical situation in which an important, but nearly-invisible, piece of network data could serve as a clue to a much larger, ongoing cyber threat. Such a clue almost begged to be discovered through the use of more-potent analytic and storage capabilities.
Suppose an evil botnet commander, who had already “captured” millions of vulnerable computers around the world, had recently penetrated your desktop computer and had lodged one of its discrete, secret “agents” inside your network. One day, this agent might release a virus into your network or instruct your computer to launch a denial of service (DOS) attack against another network. In the meantime, this botnet agent typically would lay low and attempt to avoid being noticed by your network administrator. Even so, in order to remain in contact with its remote commander, the agent would need to transmit a message periodically -- perhaps once per month -- back to its commander’s IP address at a nefarious computer sitting in Russia, China, Eastern Europe, another country or (heaven forbid) in the United States.
That once-per-month transmission of an IP address -- buried amidst millions of nondescript log entries -- generally wouldn’t tip off a network administrator that something suspicious was taking place on your network. But suppose a sharp-eyed analyst on your staff noticed that unusual occurrence and sought additional evidence. That’s where Narus might come in.
The NarusInsight traffic intelligence system, and the human analysts who use it, theoretically could spot that anomaly and request a real-time analysis of weeks, or months, or years of similar information from your network that was sitting in a massive data warehouse operated by Teradata.
Teradata’s newest database software, called Teradata 14, which was introduced last October, has the ability to search through vast quantities of data at lightning speed to find every instance in which an agent sitting inside your network apparently sent a single transmission, in a single short burst, to a single remote IP address, and then suspiciously became silent and inactive once again.
Suppose, when the results came back a few seconds later, they showed that the same agent had sent a single message to the identical IP address (located at a computer based in Russia). Such an eye-opening conclusion, which could never have been discovered through any manual process, might reveal that a botnet agent was residing dangerously on your network and that its commander appeared to be located in Russia. Those insights, in theory, could help you mitigate the botnet problem on your own network and enable you to turn over crucial evidence about the botnet commander to government investigators.
Of course, this is just one example of the potential benefits available when one stores, analyzes and derives crucial insights from Big Data.
Top executives at Narus and Teradata believe these new capabilities can help protect the nation’s infrastructure and save billions of dollars.
“Narus' leading real-time traffic intelligence system, coupled with Teradata’s blazing speed and with query performance and scalability that exceed conventional databases, provide our joint customers real-time situational awareness against cyber threats, and minimize the impact of cyber attacks better than ever,” said Greg Oslan, the chief executive officer of Narus, a wholly-owned subsidiary of The Boeing Company.
“The Teradata Database is the industry’s most intelligent high-performance database for analytics. Period,” added Scott Gnau, president of Teradata Labs, in a news release issued on Feb. 27. “Not only are we recognized by industry analysts and customers for our superior capabilities and vision, but we continue to innovate with hybrid capabilities that meet the needs of cyber security analysts and operations centers worldwide. Combined with Narus’ analytics, together we can help the cyber world strengthen its defensive position against attacks. Our combined technologies take cyber security to the next level.”
In its initial phase, the partnership between Narus and Teradata envisions both companies approaching prospective government and commercial companies as a team. “We’re not reselling each other’s products,” Thomas told GSN. “We’re working together.”
Looking for anomalies in network data is certainly a primary goal of the Narus-Teradata partnership, but there are likely to be other applications, as well, for these stepped-up data crunching capabilities. Thomas noted that the ability to wade through petabytes of shopping and personal data to determine the likes and dislikes of individual consumers has captured the imagination of marketing executives (though he acknowledged that these executives have not yet begun to open their wallets.) Thomas and Smith both cited the intelligence community as another likely user of their partnership's Big Data capabilities.
Smith, of Teradata, proudly described her company’s exceptional capability at what she called “massively parallel processing,” which is one key to Teradata’s ability to wade through huge quantities of data in the blink of an eye. She described massively parallel processing with an easy-to-understand illustration. Suppose you had a deck of 52 playing cards and you wanted to find the ace of spades as quickly as possible. If one person turned over all 52 cards in search of that specific card, it might take quite a while. But, if one card was handed to each of 52 people simultaneously, and all 52 people looked at their card at the same time, the ace of spades could be found rather quickly. That’s massively parallel processing.
Clearly, Big Data has just moved into cyberspace’s spotlight. You’ll be hearing a lot more about this innovative field in the months ahead.
In fact, in the opening keynote at the RSA conference, Art Coviello, a top executive at RSA Security (as well as its parent company, EMC Corp.) said it was time for organizations to adopt the Big Data model to spot and categorize network anomalies.
“We need to identify and respond to those anomalies in real time,” he urged.
Coviello went a step further in boosting Big Data by deriding the traditional approach of network administrators, who have been capturing boatloads of logs and network traffic, but doing precious little with all this accumulated information
Security teams should stop wasting their time and money tracking meaningless network incidents, Coviello advised, and begin looking at the more-promising strengths of Big Data.