Finding the Unicorn Data Scientist

Finding the Unicorn Data Scientist

The dichotomy is that there is a difference between mathematicians and data scientists , although statistics both deterministic and probabilistic and other schools of Maths are an integral part of data science. Data scientists or Data Engineers as they exist today are more programmers who are experts at manipulating large, complex data and know very little statistics. They are adept at Big Data technologies like Hadoop, Spark, Scalding, MapR and their equivalents. They also might know the R packages, the Python Libraries, Spark ML and Mahout Algorithms, but they seldom know the interplay of data, the non linear structures, the formula behind the Collaborative Filtering and matrix factorization packages they use and how to figure out which to use where. A typical statistician is the opposite... They seldom know how to store and manage huge sets of data,structured or unstructured,  but what they do know, is to unearth the true interplay between data but at a much smaller scale.

Experiments are always done in beakers inside a lab, never at a large scale. However the implementation can be at a large scale. So as a business if you are interested in  unearthing insights into data, create that lab for the Scientists/Statisticians and let them work in tandem with Data Engineers who will ingest,manage and implement the findings of the lab at a large scale.

It is this combination that is most likely to succeed in achieving your business goals. Searching for that one Unicorn Data Scientist who will solve all your problems might be as elusive and futile exercise as searching for a Unicorn in real life. Their presence is at best Mythical.

Vikram Khurana

Vice President -Analytics ,Gen AI @ Delhivery | Ex.|| Mobile Premier League |Gaana | Fractal

10y

This is awesome Eron. However its still hard to find a good statistician with sound practical knowledge. Struggling with one such problem of my own :-)

Like
Reply
Prahlad D.N.

Non Executive Chairperson at Surya Software Systems Private Limited

10y

This is like the search in old days for programmers who also had intense domain knowledge as well as management knowledge and abel to fine tune all processes besides knowing finance. AKA silver bullet

Like
Reply
Dr. A. Tierno

Dottore Di Ricerca, Insegnante di Ruolo di Fisica

10y

GOD

Like
Reply
Eron Kar

AI Business and thought Leader -Converting AI capability to Outcomes

10y

Alexander Linden thought leaders like yourself do articulate the differenciation very well. However when I interact with business leaders I see a lack of understanding and willingness whom I was addressing in my article. A lot of my thoughts are inspired by you and I wish more people subscribe to and are led by your pathbreaking research.

Like
Reply

To view or add a comment, sign in

More articles by Eron Kar

Explore content categories