We live in a digital society which generates an increasing quantity of information every day. We talk about “generating” instead of “producing” or other terms, because in some cases information seems like some kind of “residue”, a waste created by our everyday processes (phone calls, bank wires, watching TV, or even commuting through the city). This information is generated unwittingly and lost without even being acknowledged. But as we see in many detective movies in which a character digs through someone´s thrash to know more about them, we can also obtain information from the data trail we generate every day.
The question is, can someone obtain any value from the analysis of that information? And of course the answer is affirmative in most cases, because some fundamental needs haven’t changed at all. The needs for a worker at a grocery store (to know their clients, supply, how to set the right price for each item…) are not very different from the needs Amazon has, with the only difference being that Amazon faces a base of millions of clients worldwide.
And it is here where the tittle of this post makes sense. We trust a “machine” on the interpretation of datasets which would be nearly impossible for a human being to digest. Could we know the names of each one of the millions of clients that enter my store? Can I guess which is the product with more chances of being purchased by a male client, single, 36 year-old from Valencia, who has already bought a pair of blue jeans on a Friday night at 10 after clicking an Instagram ad?
The challenges of analyzing all that information and extracting value are not negligible, and big data is here to answer those challenges. We define big data as the set of technologies, procedures, methodologies, algorithms and tools aimed at giving solutions to big issues related to data that arise in business environments.
It is at this point where we should stop before embarking on a journey with no clear direction. Instead, we should invest part of our effort and time in defining the questions that we would like to be answered by big data´s magic lamp. And I like the idea that big data can be as useful or as useless as the magic lamp. We can waste the privilege of formulating three wishes if we are not sure about what our three big priorities are. Once we clear the unknown, we will have covered part of the path, since something so apparently innocent as determining what we want to achieve with a big data project can be the cornerstone to success. This goes from the proper definition of the more suitable model of data, to the architecture, and the analysis models that we will use.
Post written by Carlos Arciniega, New Business Director, Smartup.