CNP Expo: Big Data, Big Deal

May 22, 2014

CNP Expo: I’m From the Government and I’m here to Help Dealing with big data can be overwhelming, partly because “big data” is such a broad term that it’s hard to know what we’re talking about. Matthew Joseff, senior technology evangelist at RSA, started Thursday morning’s session by asking the panel to define the term. Eric Thompson of EMC defines big data as “the set of data you have access to that is not available to be analyzed by traditional means. It’s too large and too varied in structure for traditional analysis, so it requires new tools and ways of thinking to get value from it.”

When it comes to collecting the data, Ken Meiser, director of authentication and verification solutions at ID Analytics, recommends casting a wide net, at least at first, because “you may not expect correlations you discover, so the more data you have to begin with, the better.” Thompson agreed, and noted that with big data, “you need to have an exploration mindset and watch for patterns to emerge.”

In order to get the most out of big data, Thompson recommended setting up a team of data analysts who receive and store the data, then have other departments bring their questions and problems to that department. “We’re already measuring behavior on our site—I can multi-purpose the same data set for seeing which ads are effective for the marketing department, seeing when customers want help, watching how good consumers behave on the site vs. fraudsters.”

Meiser recommended that when trying to develop best practices around using big data, “a lot of that can be derived from collaboration and forming industry partnerships.” He said analysts should be spending time at industry conferences, “not necessarily sharing data, but talking with people at other companies about their methods and tools.”