Human Decisions: the real reason for Big Data

It is true and well stated by Joe Hellerstein, CEO of Trifacta. We find ourselves often in love with technology and the development of something and lose site of its real intent. Same is happening with Big Data and the statement that some are making that technology will eventually eliminate the need for a Data Scientist.

Let's remember that humans make qualitative decisions out of the quantitative data that is provided to them, ever faster and more complete by Big Data technology (review article written on this blog). Within the data research process that ends with enough data to make a decision, there is a constant analytical process that asks how can I get to what I want to know using the data I have. The data scientist will always be in control of that process, no matter how much we automate it.

This was well described in Joe's article.

Integrity in Business - a thing of the past? Or do we all aspire to be a "Wolf of Wall Street"?

It's hard not to want what the Wolf has. He has it all. He has money, friends, drugs, wild parties where prostitutes are used like the salsa bowl. And, as the movie painted for us, he never really had to feel bad - not even a hangover the next day! He never felt bad about the investments he misappropriated, of the lies he told to those who were trying to invest their money. Most of his clients lost everything, but the Wolf didn't lose a thing.

And when we hoped our laws would put such scum where he belonged, he ratted out his closest associates and only got three years in a celebrity jail cell. The last scene shows that he got out and is a celebrated author, speaker and probably got a % of the $12 of my movie ticket. What a hero!

My values are different. I am a consultant who does not lie to his clients. I don't lie to my employer. I don't lie while I am selling something. I don't lie to my wife of 26 years and have always chosen to come back to my family. Martin Scorsese would have me believe I screwed up and have missed out in life. This movie is heralded by my society as a great movie that will win many awards.

Am I wrong?
Do we have to lie to clients to get ahead as a consultant?
In order to sell something do I have to lie?

If I do, then I quit and accept my success as the little that those get who believe integrity has value...and I will be happy with it.