This post is for DATA SCIENTIST enthusiasts with general questions about the field and their fit in this field.
I see posts across Facebook groups where beginners with enthusiasm to work in the domain of data science ask questions like,
“Do I need to learn statistics?”, “Do I need to learn to program”, “I come from a different domain, do I need to know mathematics for machine learning”, etc.
Seeing various posts like this, I thought of sharing my experience of what level of programming expertise is needed? and what kind of programming we do in the real world? (I am writing with an experience of working as an intern in two french companies each for a period of 6 months, one as a data scientist and another as a deep learning engineer).
“Its not wrong to be enthusiastic or willing to switch domains”.
“ Flow like a river rather than being like a standstill water”.
If I had not switched domain from being a teacher, I would not write this post.
I am now a data scientist, While entering the first year of my studies (Masters in data science, Big data) I was under the impression that very basic skills in programming are enough. To abet my thoughts, the courses I took in udemy before I entered into masters made me substantially believe that good knowledge of pandas and machine learning is enough. Guess what, I was totally wrong.
On Academic Side:
Yes for “statistics” which is the fundamentals of data science- excel/R/python were used in class works, similarly for machine learning, NLP, deep learning we used R/python, For classes and experiments on Optimization, heuristics largely Matlab was used to visualize convergence of different optimization algorithms. For Big data courses, Java/Scala/Pyspark was used. Having said the important languages used, My favorite programming language are Python, pyspark.
In the companies:
Mostly python is used as it comprises various packages to perform statistics and various other tasks such as data preprocessing, exploratory data analysis, modeling, deployment. Name a package, there is a great certainty that it is available in python. Hence mastering python will surely help in getting a job.
So what is the level of programming skill needed?
If you are from a background with no prior programming knowledge. It's better to get at least fundamental knowledge of Programming so that you can write a program for any given logic. Once you are familiar with writing logic into codes, you are good to go as just the syntax differs from language to language. you can easily adapt the syntax of different programming languages to frame your logic. Having said that, I feel comfortable working with Java, Scala as well although I don’t remember the syntax completely. If I am told to write work with scala, With few references to syntaxes used in scala, the logic starts to shape into code.
so what level of mathematics is needed?
you need to have a fundamental understanding of linear algebra, (remember the matrix, vector calculations?), a good understanding of probability and statistics, and last but not least “Differential calculus” (very important in deep learning)
First experience as a Data scientist!!!
In the first company, we developed an end-to-end product to serve clients. Remember the companies are generally product-oriented, Hence being a data scientist, I had to take control of all the logic that happens to give end clients precise recommendations (this is just an abstract idea of my experience, not about the project, hence I will keep the details of the project low). The idea is basically the clients connect to our website using their credentials through API endpoints, For this, we needed to build an interface using RestApi Frameworks. Once the clients connect and requests for certain service, the codes in the backdrop (python used for data retrieval, data storage, and machine learning) works and sends back the precise recommendations, on the front end i.e RestFramework does the job of showing these recommendations in terms of graphs and attractive visual contents.
So did I alone complete this entire product? The answer is No, The front end was done by a team of web developers experienced in RestFrameworks, Nodejs, Html, CSS, javascript. Backend tasks were done by the data scientist. So as a data scientist to contribute to such big projects, BASIC PROGRAMMING SKILLS ALONE IS NOT SUFFICIENT, you need to master OOPS so that it is easy to write different modules and tie these modules together so that the codes could be reused. It was easier to contribute more to the project as I had sufficient knowledge of RestFrameworks (Django, FastApi) other than good skills in programming. So it's better not to stop with just knowing python, Learn a bit about RestFrameworks as well. It will be an added advantage. Once you are confident about your python skills, Evaluate your skills in python by building some simple applications like a calculator, Pomodoro app, music player, etc.
Second experience as a Deep learning engineer:
In the next span of 6 months as a deep learning engineer which was completely different from my first experience, I found that thorough knowledge of computer vision tools like OpenCV, PIL was demanded in addition to skills to work with NumPy library, Keras (the easiest), PyTorch(little tough but with practice can be mastered).
To conclude:
Having tasted data scientist career for 1 year, I strongly suggest that one should possess the ability to work with OOPS concepts, NumPy, regex, pandas, reading/writing files from the operating system directories. These are the very basic skills that need to be mastered at a higher level. once you are convenient with these, then learning other libraries like OpenCV, PIL, Keras, PyTorch are all a subsequent process that you will naturally do without much stress or hard work. Gain good knowledge of the mathematical concepts listed above. you will eventually feel the data scientist career to be very interesting.
Feel confident about the path in which you step, success will follow you
Other useful articles you might want to read before your venture!
For any help, Reach out to me @ Linkedin. I would be happy to help you.
