Advice for a Bad Career in Data Science

You nailed the interview and landed the hottest job in America; go you, now you’re a data scientist! As you come into work every day to incorporate data science into product development, or apply machine learning to solve business problems, it’s blue skies everywhere. There are so many possibilities it’s hard to know where to begin and how to spend your time.

Well, here are a few suggestions, broken down into the stages of a typical data science project, from my experience practically guaranteed to work every time.

Disclaimer: No data scientists were intentionally harmed. Any resemblance to your coworkers, past or present, or the project you’re working on is purely coincidental, although expected.

Picking your problem
Start working on whatever problem you want.
— Try to pick the least defined problem, hopefully one you vaguely understand. That way you’ll have plenty of room to explore, and you don’t have to worry about running out of things to try.
—If someone comes to you with a problem, use it as an opportunity to reframe it into something you want to work on.
— Don’t bother wasting your time asking anyone what would be useful or valuable for them, you know better anyway! They’re probably not going to have the answers. And really, the fewer people that know about your work the better, you don’t want them interrupting you.
— You shouldn’t have to explain yourself either, the value of data science is self-evident; everybody wants it! But if someone does ask you what you’re working on, a good rule of thumb is that the more you have to convince them that it’s useful, the more you’re on the right track and can rub it in their face later.
— The business will eventually find value in whatever you do, don’t worry about when or how yet.

Start working on any and every problem someone comes to you with.
— Best not to ask too many questions, they’re the expert after all. They should know exactly what they need, and they’ve already identified the right question to ask and the true causes of the problem. Anytime someone comes to you it means there’s a data science solution that’s worth building. You don’t want them thinking you’re not up to it. They may start doubting your commitment.
— You have to show them that with data science you can do anything. They’ve likely read the news recently, so reassure them AI can do whatever they want, probably more and better and faster and stronger and 24/7/365 with a smile.

Continue reading “Advice for a Bad Career in Data Science”

An Experimental Development Process for Making an Impact with Machine Learning

Originally published on Towards Data Science.

It’s really hard to build product features and internal operations tools that use machine learning to provide a tangible user value. Not just because it’s hard to work with data (it is), or because there are many frivolous uses of AI that are neat but aren’t that useful (there are), but because it’s almost never the case that you’ll have a clearly defined and circumscribed problem handed to you, and there are many unknowns outside of the purely technical aspects that could derail your project at any point. I’ve seen a lot of great articles written on the technology side providing code and advice on how to work with data and build a machine learning model, and the people side of how to hire engineers and scientists, but that’s only one part. The other part is how to steer the technology and people through the hurdles of getting this kind of work to have an impact.

Fortunately, I’ve failed to deploy AI many times and for many reasons to provide business and user value, and watched friends and colleagues from startups to Fortune 500 data science and research groups struggle with the same. Almost invariably the technology could have been valuable, and the people were competent, but what made the difference was how people were working together and what technology they were working on. In other words, I trust you can hire good technically-able people that can apply their tools well, but unless it’s the right people building the right things at the right time for the right business problem, it’s not going to matter. (Yeah, no duh, right, but it’s harder to do that than you may think).

In this post I’ve tried to consolidate learnings, reference existing articles I’ve found useful (apologies to the references I’m sure to have missed), and add some color to why building experimental products is hard, how it’s different from other engineering, what your process could look like, and where you’re likely to encounter failure points.

Continue reading “An Experimental Development Process for Making an Impact with Machine Learning”