There seems to be a divide between Big Data sources when it comes to trust.  Big Data efforts based on internal sources (e.g. log data, transactions, etc.) are trusted.  But apparently Big Data efforts based on external sources in the form of social media etc is not.  See this IBM study for more details.

But this may be due to one of two things.  The first is simple experience with the external data.  Yes it comes from the wild and wholly world of the internet but like the internal data that does not mean there isn't gold in there. 

Which leads to the second point - technique.  There is a wild array of techniques you can use to analyze your Big Data.  Finding the right size, shape and analytical technique in order to find that gold is not easy.  It takes trial and error and great skill in interpreting the results.  

Like all marketing efforts, Big Data is now suffering from "the gloss over effect".  We are given the impression that Big Data value is as simple as hooking a pipe of Big Data into a magic analytical box and out pops the gold you were told is hiding there.  Sorry - not that easy.  

So the realism that sets in after giving your Big Data this initial try might also contribute to the skepticism.  Big Data should be thought of more like a science project - embarking on a journey of discovery.  It takes many dead ends and blind alleys before you reach the promised land.  But it is there.    

Comment