Joyce Hostyn argues that Better Human Understanding, Not Big Data, Is the Future of Business. Some excerpts (with my emphasis):
Despite the best of intentions, we’re not data driven, we’re hypothesis driven. Our stories (our mental models) are merely hypotheses of how the world works.
Small conversational data such as tweets or text messages are a goldmine of individual interests. Millions of people everyday tweet about their favourite food, send a Kik or Facebook message about a recent TV episode, or take a photo on Instagram while attending a sporting event or concert.
We know what personalization means and the compromises it imposes on our individual privacy.
Or do we?
This is perhaps the most insidious myth among the technorati: In order for people to benefit from advanced and personalized technologies, they need to compromise their individual privacy.
Do you share this opinion?
So given that, what can we say about the eventual development of something we can call “The Star Trek Computer”? Right now, I’d say that we can say at least two things: It will be Open Source , and licensed under the Apache Software License v2.
The fervor around big data continues to grow. The World Economic Forum and The New York Times are jumping on the bandwagon. While we share their enthusiasm for the potential, big data needs a reality check.
Here are just a few of the how-do-you-get-there-from-here questions for anyone considering big data projects.
Data-driven technologies are plagued with small data problems. Their performance suffers in markets that aggregate a large number of unique interests. Some of the largest markets share these small data characteristics, including local ecommerce, personalized media, and interest networking. New approaches are needed that are far less sensitive to the cost and complexity of the data.