The data scientist fallacy

February 26, 2013

Listening to the panel discussion on analytics and data skills [today at Gartner] (/blog/2013/02/26/Gartner-BI-Summit-Day-2/) it occurred to me that the huge explosion in demand for data scientists is the result of:

failure to adapt to change

The myth of the data scientist is the result of executives seeking a solution to the failure of their organisations to adapt; to be agile and responsive.

while I call myself a data-scientist…

I’m pleased to call my self a data scientist because:

  • I have a 4 year degree in Economics, with honours during which time I studied advanced econometrics, statistics as well as traditional micro-economics, behavioural economics, and game theory, plus a host of other economics subjects. I also took electives in [advanced math] (http://www.maths.adelaide.edu.au/courses/level.html#1), philosophy and logic.

  • I’ve spent four years working as an economist and performing [micro-economic research] (http://www.pc.gov.au).

  • I’m more than handy in python and a whole other host of programming languages. I taught myself C++ as a teenager by writing an advanced ircbot.

  • I’ve been working as a data-miner for more than two years in an internal consulting role in a large enterprise (more than 20k staff). My role involves, at times, very technical machine learning work, and more recently high-level strategic thinking, and constant effective communication.

So, despite my dislike for the hype around the term, I embrace the concept of a data scientist. But,

####… if your organisational strategy revolves around the mythical concept of a jack of all trades single handedly coming in to absolve your existing executives of their responsibility to adapt to and understand information

… then I don’t want to be the one to have to tell you that’s not going to work.

You don’t want a data scientist; you actually want to embrace change

Easier said than done. You could try:

Change is difficult to embrace

Here’s two of my favourite quotes.

The first is from the inimitable Ed Yourdon and his interview with American Airlines Monte Ford in CIOs at work

(imagine if your executives said) “ugh numbers, financials, budgets,… I don’t understand that budget stuff. I’ll leave that up to the financial guys” Nobody is accepting of that… but in some places it is still acceptable for somebody not to have a thorough background and training and understanding of technology.

And of course, Paul Graham’s famous quote from Hackers and Painters

“There is a great temptation to work on problems you can treat formally, rather than problems that are, say, important.”