Two years as a Data Scientist

What I learned working for two years as a data scientist coming in straight from university.

• The speed at which things move in most companies is appallingly slow compared to university. Testing, architecture, data cleaning, documentation, audits, admin, ……. eat into productive time.
• Your project will move at the speed of trust. The most important thing for speed is to have a team which trusts each other. The next most important thing is trust with stakeholders (customers and suppliers). Most projects move slowly because Person A needs a meeting with Person B who needs a meeting with Person C to take each decision.
• Software is agile. Architecture is not necessarily agile. In order to leverage the true power of software, you need an agile team working with a stable architecture. Some teams lose too much time due to constant architecture changes each year.
• Your manager does not have much power. Ever wonder why that useless colleague of yours makes more money than you? Its probably cause he has been around longer and your manager cant do much to get rid of him.
• New technologies are fun to read about but hard to deploy. Imagine if it made sense to migrate your database to a blockchain in a company where nobody knew what a blockchain was. How many people would be willing to learn it? Even if they did learn it, is the ecosystem behind the technology mature enough for you to actually do something with it? Think about doing data analysis on a blockchain vs on a normal database, and the difference between the ecosystems.
• The impulse to re-do the whole project at time of joining is strong. If you are the new guy, you should only suggest 20% of the changes you wish to make. If you are the incumbent developer, it is important to justify previous decisions. THis is why decision logs are super important, esp on projects with high attrition rates. Some great resources : https://adr.github.io/
• Have a good playlist for coding. Currently listening to this: https://open.spotify.com/playlist/0yw4N3Mt1EleKHM7ZamjFK?si=a6d4cd65589c4a04
• Being bad at your job might actually be good for your career. Essentially your manager might want to promote you cause you suck at your job and your manager has no other way of getting rid of you. (https://en.wikipedia.org/wiki/Dilbert_principle)
• It is hard to be bad at your job. You probably have an impulse to get people’s approval and that is a tough one to suppress
• VScode is awesome. You can link spotify to VScode. Whoo!
• Read books. Probably the investment ever.
• Scrolling reddit at work is fun. Especially for coding memes.
• Writing documentation is painful. Keeping it up to date is impossible. Mostly people update the code but forget to update the documentation. Seriously, if somebody can build something for this, it would be awesome.
• Testing data science code is tough. You need this test dataset and then you need to maintain it and more often than not, you edit the test after each change you make. You spend 10 mins writing the code and an hour editing the test.
• OOP is garbage for data science more often than not.
• Machine learning is a very tiny part of most data science projects if there at all.
• Try not to waste too much time finding the perfect VScode theme.
• Having friends at work can be nice. But I find myself to be happier in jobs where I do not have friends.
• Use emojis in all commit messages. https://gitmoji.dev/
• Operations is hard and boring and adds little value to your CV. Big projects can afford to hire a separate operations team. But for small projects, operations start eating into the time and resources of the dev team. Eventually the best people leave and then the project just gets restarted with a brand new team.
• Large meetings suck. They might as well be happening in German and it makes no difference.
• Someone needs to make a VScode extenstion which allows me to run https://www.kaleidosync.com/ inside VScode. Seriously. I just want to look at cool visuals while coding. Also while doing LSD.
• Side-projects are fun to fantasize about.
• Writing is an awesome tool to sharpen your thinking. Clear writing forces clear thinking and not the other way around.
• Hire interns. Seriously, those guys are awesome and will work their ass off for almost no money. I have picked up some awesome new skills and made some awesome friends thanks to this.
• Use memes to say offensive things in presentations.
• Write documentation after getting drunk and put in some funny stuff. Wait and watch how long it takes someone to notice. More often than not, nobody reads that stuff anyway.
• Work from home is awesome but we need to be around each other. Digital connectivity is awesome but it makes it hard to generate empathy. Its also difficult to show people you care over digital mediums. Your boss has a very different digital body language as compared to you.
• Have a life outside work. Pickup some hobbies and not just reading and working out. Actually create something physical. It could be cooking or just doing those 3000 piece puzzles.