A brief big data touch point. In a conversation with a colleague, he asked a question about how big data is changing everyday activities. I noted a conclusion made by Alessandro Mantelero, in his paper “Masters of Big Data: concentration of power over digital information.” He stated, in the context of big data, that:
Examination of the data flows in the evolving datasets shows trends and information without the need of prior working hypothesis, changing the traditional paradigm of social analysis in which the design of the study sample represents the first step which is than followed by the analysis of raw data.
The key point is that big data is a catalyst for the exploration of level three knowledge: those things we don’t know we don’t know. Traditional discovery methods, used in level 1(know what we know) and 2 knowledge (know what we don’t know) knowledge acquisition, are limited because they infer, and often require, the relevant aspects of the questions under study. In contrast to the traditional deductive approach of knowledge acquisition, the big data is self-explanatory and can be based on inductive knowledge acquisition, which fundamentally requires large amounts of information.
Therefore, if one believes that transformative business insights can be found in understanding more of what we don’t know we don’t know (level 3), then the use of big data is one of the proven means to achieve this.