The ML Mindset

"Machine Learning changes the way you think about a problem. The focus shifts from a mathematical science to a natural science, running experiments and using statistics, not logic, to analyse its results." - Peter Norvig - Google Research Director

In traditional software engineering, you can reason from requirements to a workable design, but with machine learning, it will be necessary to experiment to find a workable model.

Many machine learning systems produce models that encode knowledge and intelligence by interpreting signals differently than humans do. A neural network might interpret a word via an embedding, so "tree" is understood as something like, [0.37, 0.24, 0.2] and "car" as [0.1, 0.78, 0.9]. The neural network might use these representations to do accurate translations or sentiment analysis, but a human looking at the embeddings would find them very hard to understand. This can make machine intelligence difficult, but not impossible, for humans to understand and evaluate.

Models will make mistakes that are difficult to debug, due to anything from skewed training data to unexpected interpretations of data during training. Furthermore, when machine-learned models are incorporated into products, the interactions can be complicated, making it difficult to predict and test all possible situations. These challenges require product teams to spend a lot of time figuring out what their machine learning systems are doing and how to improve them.

Look at the Example of Google Photos

The video in this section dives into how ML powers Google Photos: ML powers the search behind Google Photos to classify people, places, and things.

This example demonstrates that we can teach a model to recognize cats in photos, but it is difficult to know what features the model uses to determine something is in fact a cat. This uncertainty can feel a little uncomfortable at times if you are used to determining every detail of your code's behavior.

Check out the links below for more information on the progress and impact of Google Photos.

Extra Resources
Wired Article
Google Blog at launch
Google Blog 1 year later
Google Blog 2 years later

Experimental Design Primer

Get Comfortable with Some Uncertainty

Beyond simply thinking about problems differently, implementing ML is different than traditional programming. In traditional programming, you have set parameters and you understand how everything should behave. With ML, the non-coding work can be very complicated, but you'll usually write far less code.

Will you end-up with a usable model? You don't really know at the start.

Scientific Method

To address the challenges of transitioning to ML, it is helpful to think of the ML process as an experiment where we run test after test after test to converge on a workable model. Like an experiment, the process can be exciting, challenging, and ultimately worthwhile.

1. Set the research goal.I want to predict how heavy traffic will be on a given day.
2. Make a hypothesis.I think the weather forecast is an informative signal.
3. Collect the data.Collect historical traffic data and weather on each day.
4. Test your hypothesis.Train a model using this data.
5. Analyze your results.Is this model better than existing systems?
6. Reach a conclusion.I should (not) use this model to make predictions, because of X, Y, and Z.
7. Refine hypothesis and repeat.Time of year could be a helpful signal.