On to the Next Step: How to Move Through the Phases of Big Data

While the concept of big data analytics may seem easy enough, many businesses experience obstacles as they work to move through the phases involved in this critical enterprise process.

After all, it's not just about hoarding informational assets, inputting them into a technological solution and waiting to see what pops out. Organizations across every sector can benefit from big data, so long as they have the right tools in place. and are able to put the resulting insights to work for their business.

Overall, there are a few main phases that companies must move through with their big data. However, industry analysts recently pointed to a few hurdles that are preventing businesses from reaping the most benefits from their analytics initiatives.

If your organization is experiencing difficulties with its big data analytics, fear not – you're not alone. It can be difficult to take the next step when it comes to a big data project. Thankfully, there are a few best practices your company can harness to ensure that it's in the best position to move to the next phase and derive the most value from its analytics.

IDC forecasts significant investments and global growth

Channelnomics editor Sam Trendall recently reported on an anomaly with industry analysts. Where big-name research firms like IDC and Gartner typically agree – or offer considerably close predictions – it seems these organizations have found something they disagree on.

Trendall noted that while IDC is predicting explosive expansion within the big data technology market, Gartner is painting a darker picture of this sector. IDC noted that the global big data and analytics market is poised to grow by more than 11 percent year over year in 2016, and will continue to expand at this rate through 2020. In fact, IDC researchers forecasted $203 billion in annual revenues for the big data and analytics industry by 2020. What's more, the firm predicted that 76 percent of global organizations with 500 or more employees would be making considerable investments in their big data initiatives, contributing $154 billion to the market within the next four years.

"This year and over the life of the forecast, we're expecting to see healthy growth in spending on big data and analytics technologies from nearly all industries, including banking and telecommunications," said IDC program director Jessica Goepfert. "In our end-user research, respondents from organizations in these industries are placing a high priority on big data and analytics initiatives over other technology investments."

Gartner analysts highlight big data struggles

Gartner, on the other hand, isn't so confident about the direction this market will take in the next few years. In fact, analysts wrote that "big data investments continue to rise, but are showing signs of contracting," according to Trendall.

What's the main reason behind this stunted market growth? Gartner analysts point to the inability of many businesses to move past the beginning stages of big data projects, preventing them from putting their insights to work for their organizations. Gartner supported this view with research showing that only 15 percent of enterprises surveyed had achieved the production stage of their big data initiatives. This is only a small improvement over the 14 percent of businesses at this level in 2015.

"Investment in big data is up, but the survey is showing signs of slowing growth with fewer companies having a future intent to invest," said Gartner research director Nick Heudecker. "The big issue is not so much big data itself, but rather how it is used. While organizations have understood that big data is not just about a specific technology, they need to avoid thinking about big data as a separate effort."

Despite these predictions, this isn't a death sentence for big data – there are strategies businesses can use to improve their use of analytics results, thus enhancing the overall industry.

Defining the 4 phases of big data

Before we discuss the ways to address this big data issue, it's important to unpack the four main stages of a big data initiative, and better understand where an organization might fall within these phases of its analytics efforts.

CIOReview contributor and MapR Technologies marketing solutions director George Demarest noted that the operational phases of big data adoption include:

  1. Experimentation: During the first phase, businesses begin with the basics. They have the data, but require a technological platform to help them analyze, draw insights and make sense of the information. This first stage sees companies experimenting with Hadoop, or another big data solution, while working to integrate it into their existing infrastructure.
  2. Implementation: After experimenting, enterprises are ready to move on to the implementation phase, where they begin developing tangible use cases for their data analytics and the resulting insights. Businesses at this stage are more confident in their use of their big data analytics technology and can begin broader analytics that help to hone their informational processes. Toward the end of this second phase, the organization has completed smaller, more targeted big data use cases that begin to offer actual value for the company.
  3. Expansion: After moving past the implementation phase, firms are now prepared to accelerate and expand their use of big data analytics. This takes place in the form of illustrating the value of big data through the successes of early use cases, Demarest noted. From here, companies often look to leverage analytics for use cases that move beyond their individual organization and to the overall industry vertical they operate within. Here, enterprises are better able to address pain points and struggles common to not only their business, but any organization in their industry. At this stage, financial service companies, for example, utilize analytics for risk management or fraud detection, and health care providers may seek to reduce readmission or inform their use of smart medical devices via big data results.
  4. Optimization: Once a company is able to leverage its big data analytics for wider, more impactful purposes, it can achieve the final phase: optimization. Here, businesses experience the actual advantages big data has to offer in all departments across the company and can utilize these to drive their market position.

"You should leverage this capability to improve business operational processes, which will reshape the business and give you a competitive edge," Demarest wrote of the final phase. "The goal is to be able to be predictive about most aspects of the business, and be able to respond and change operations in real time across more than one line of business."

Best practices for big data development

As Gartner noted, though, many businesses experience complications and challenges after reaching the first or second phase. In order to move past these levels and keep the development and advancement of analytics strong within an enterprise, there are a few best practices that have proven successful:

  • Look to solve problems outside the company: As Demarest pointed out, a hallmark of the expansion phase is the ability of an organization to look outside its own business, and to its overarching industry. Once big data has provided solutions for unique, company-specific problems, it's time to expand and look to address larger pain points.
  • Have a specific focus: Building upon this, it's important for enterprises to go into their big data initiatives with a specific plan. What defined problem will a certain project seek to solve, or what question will it look to answer? WIRED contributor Olly Downs noted that having to wide of a scope is akin to boiling the ocean, and will not provide such impactful, actionable results.
  • Make analytics a company-wide initiative: While it's important to ensure that there aren't too many cooks in the kitchen during the analytics process, it's also critical to make the results of these projects accessible to all company departments. This will help drive the tangible value that big data initiatives can provide. After all, what good are insightful results if they exist in a vacuum?
  • Look to the experts for assistance: The value of a big data partnership with an expert solution and analytics provider simply cannot be understated. Leveraging the skilled knowledge of experts can be just what a company needs to help it excel and move past the beginning phases of big data. Such a partnership guarantees that an organization not only has the right tools for the job, but can also support a successful scope of work that results in the best insights possible. 

To find out more, contact the industry-leading experts at Data Realty today.