Big Data Needs People

Last month I wrote about how we are not quite ready to take advantage of big data. In my post I mentioned that we must first be able to identify new knowledge and have the ability to make changes to capitalize on any new findings. I still stand by these assertions but I want to expand on this further.

Our goal with big data is to find nuggets of new knowledge buried deep within large volumes of high velocity data. We can use big data technologies to better manage the volume and velocity issues and perhaps develop means to filter data to find what we area looking for. For example, we can look for trends in data to discover new product opportunities, detect fraud, determine product anomolies, or predict component failures. All of these possibilities, and many more, represent the payback for big data investments.

The problem is that technology alone is not the solution. As with other forms of knowledge management systems, big data must also rely on people. Organizations must engage knowledgable people to interpret data, determine the patterns to look for in the data, and develop ways to take advantage of any knowledge gained from the data. These intelligent individuals may use big data to discover new information or test a hypotheis about some cause and effect scenario. Regardless of how big data is used, it requires people to realize the benefits. The technology provides the tools and manages the data but it is the people that turn the information into knowledge and enable the organization to capitalize on the new knowledge.