What is Big Data?

Technology has made it easier than ever to compile, store and evaluate data gathered online. With the cost of storing large amounts of data dropping, more and more companies are doing so. But why?

Simply put, sifting through data can reveal amazing insights for businesses and marketers looking to develop a robust profile of their prospects and ideally, their customers.

So what is big data?

Big data is an umbrella term used to describe the exponential growth, availability and use of information.

Big data is basically any information that can’t be processed through a conventional database. Either the sheer volume of data is so massive it can’t be stored on site, or it comes in too quickly to analyze. Some data simply doesn’t fit into the existing analytics structure within a company. In these instances the information must be captured and processed using other methods.

What makes up big data?

Volume: This refers to the amount of information that’s able to be captured, tracked and stored. With data storage costs dropping, the amount of information companies are capturing will only continue to rise.

The key to dealing with volume starts with sifting through the data to determine what information is relevant, and assigning value to the relevant info.

Example:  FICO, or Falcon Credit Card Fraud Detection System, protects 2.5 billion active credit card accounts world-wide.

Variety: Data comes in many forms and only a small portion of it is numeric. That means there’s relevant data pouring in from multiple sources and much of it is difficult to quantify. Despite the challenge of identifying and measuring these sources, it still must be accounted for and considered in a company’s analysis and decision making process.

Example: In 2012, the White House announced the Big Data Research and Development Initiative, which explored how big data could be used to address important problems facing the government. The initiative was composed of 84 different big data programs spread across six departments.

Velocity: This not only refers to how fast the information is coming in, but how quickly it’s being processed in order to meet demand. Being nimble enough to react to data quickly is one of the major challenges most organizations struggle with.

Example:  Walmart handles more than 1 million customer transactions every hour, which is imported into databases estimated to contain more than 2.5 petabytes (2560 terabytes) of data. That’s the equivalent of 167 times the information contained in all the books in the US Library of Congress.

In part two of this series, we’ll take a look at some of the advantages big data offers for businesses, especially those in the retail and service industries.