a sql query by any other name

I recently (August 2012) had the experience of being interviewed for a “Big Data” technical role at Suncorp.

As an aside - the job-spec mentioned a particular big data appliance - which got me thinking about the understanding of big data. During the interview I may have said - “you know you’ve got a big data problem when you can no longer assume your appliance will automatically be able to optimize your queries”.

In hindsight I was wrong there - that is not actually a problem specific to big data ;)

I was initially impressed by a number of things:

• the fact that the role was explicitly about big data
• the company had recently implemented a BYO device policy

It quickly emerged, however, that apparently what they were looking for was a SQL tuning expert to convert SQL from existing datawarehouse to big data appliance.

In my view, this concept completely misses the potential posed by big data technology.

Aside from my disappointment that the role was not what I had hoped for, it made me wonder about the broader understanding of big data technology; if that was the big data strategy in a demonstrably innovative business.

Because, when I think about big data technology, I don’t see it as a replacement or enhancement of existing business intelligence technology. I see it as a whole new way of looking at data, opening up possibilities which are completely new.

but if all you’ve got is a hammer, then….

Has anyone else had a similar experience?