What Happens When Business Drives the Big Data Train

The big data train has pulled out of the station. But where is it going?

Often when a company wants to start growing their big data capabilities, the first question they ask is, "Which technology should I invest in?" They take the "If you build it, they will come" approach.

The company buys a vast array of new technology, deploys it in their enterprise and then decides which business problems it should solve. Outside consultants are paid handsomely for their opinions on which software is best, and teams of data engineers and scientists are hired to start using this new technology to take the company into the 21^st century.

Here's what a typical Big Data approach looks like:

The first step on the big data ride is figuring out where you're going. In other words, to get value from big data, you need to start by identifying the business outcome you want to get from a big data implementation. Then next question is: what big data use cases will get you there?

It's only after you've answered these questions that you can think about what kind of data you need; explore which advanced analytical techniques make the most sense for your organization; and deploy the right technology for your big data solution.

When we take a look at this approach, we see three major benefits.

1. Customized technology selection reduces the risk of stranded assets:The various features of each platform — and there are a lot of them — can be put under the microscope and examined with a tangible goal in mind.

Your company can choose which features are critical and which are not, and can direct investments accordingly. Questions like "Do we need real-time capabilities?" or "What types of analytical methods will we be using?" can be answered upfront so you can make the most out of your investment dollars.

2. Optimize resource selection to speed time to value: If you know the business problem(s) you are trying to solve from the get-go, you can handpick the right resources and not waste time trying to figure out how to get value from your newly acquired big data capabilities.  People with experience in solving a given problem can pinpoint the critical areas that need to be improved and identify what data and technology platform have the highest probability of getting you where you want to be in the shortest amount of time.

3. Capture all critical data upfront to ensure the highest quality predictive models: Your model is only as good as the data that is used to build it (garbage in, garbage out).  With a clear business goal in mind, the right data can be captured in the right way to allow data scientists to be more confident in the models they build and deploy. And as time goes on and your big data capabilities spread across the enterprise, you can easily create consistent methodologies and cross-functional capabilities.

The big data ride will be bumpy no matter what approach you take. Just remember, if you start with the business rather than the technology, you'll never get lost.