Stay organized with collections
Save and categorize content based on your preferences.
Broadly speaking, there are two ways to train a model:
A static model is trained offline. That is, we train the model exactly
once and then use that trained model for a while.
A dynamic model is trained online. That is, data is continually
entering the system and we're incorporating that data into the model through
continuous updates.
Static vs. Dynamic Training
ML System Paradigms: Training
Static Model -- Trained Offline
ML System Paradigms: Training
Static Model -- Trained Offline
Dynamic Model -- Trained Online
ML System Paradigms: Training
Static Model -- Trained Offline
Easy to build and test -- use batch train & test, iterate until good.
Dynamic Model -- Trained Online
ML System Paradigms: Training
Static Model -- Trained Offline
Easy to build and test -- use batch train & test, iterate until good.
Still requires monitoring of inputs
Dynamic Model -- Trained Online
ML System Paradigms: Training
Static Model -- Trained Offline
Easy to build and test -- use batch train & test, iterate until good.
Still requires monitoring of inputs
Easy to let this grow stale
Dynamic Model -- Trained Online
ML System Paradigms: Training
Static Model -- Trained Offline
Easy to build and test -- use batch train & test, iterate until good.
Still requires monitoring of inputs
Easy to let this grow stale
Dynamic Model -- Trained Online
Continue to feed in training data over time, regularly sync out updated version.
Use progressive validation rather than batch training & test.
ML System Paradigms: Training
Static Model -- Trained Offline
Easy to build and test -- use batch train & test, iterate until good.
Still requires monitoring of inputs
Easy to let this grow stale
Dynamic Model -- Trained Online
Continue to feed in training data over time, regularly sync out updated version.
Use progressive validation rather than batch training & test
Needs monitoring, model rollback & data quarantine capabilities
Will adapt to changes, staleness issues avoided
Video Lecture Summary
Broadly speaking, the following points dominate the static vs. dynamic
training decision:
Static models are easier to build and test.
Dynamic models adapt to changing data.
The world is a highly changeable place. Sales predictions built from
last year's data are unlikely to successfully predict next year's
results.
If your data set truly isn't changing over time, choose static training
because it is cheaper to create and maintain than dynamic training.
However, many information sources really do change over time, even those with
features that you think are as constant as, say, sea level.
The moral: even with static training, you must still monitor your input
data for change.
For example, consider a model trained to predict the probability that users
will buy flowers. Because of time pressure, the model is trained only once
using a dataset of flower buying behavior during July and August.
The model is then shipped off to serve predictions in production,
but is never updated. The model works fine for several months, but then
makes terrible predictions around
Valentine's Day because
user behavior during that holiday period changes dramatically.