Working Efficiently vs. Effectively with Python and Data Science

Yesterday 9 a.m. to 4 p.m. I worked on my data science project with complete focus.

No distractions.

Deep work.

Just as one wishes to work when reaching for high efficiency. And indeed, I churned out ~300 lines of Python code to crack a very challenging NLP classification problem. The result was good: the model achieved about 90% accuracy. That’s pretty good, actually! However, for this specific task, I needed closer to 99%.

So I sat down again, later the same day, around 11 p.m.

And I realized that there is a waaay better solution to the issue! I completely deleted my original script and replaced it with ~50 lines of Python — which I wrote in roughly an hour.

Now, the model reaches the desired 99% accuracy!

9am-4pm: I worked efficiently.
11pm-midnight: I worked effectively.
There is a difference.

P.S. The funny part is that it felt painful to delete my 9-to-4 code, even though I knew the second solution would be far superior. I wish I had thought of the second solution at 9 a.m., finished it by 10 a.m., and taken the rest of the day off. I guess I need to remind myself that mistakes are just stepping stones. ¯_(ツ)_/¯

Tomi Mester

The Junior Data Scientist's First Month
A 100% practical online course. A 6-week simulation of being a junior data scientist at a true-to-life startup.