Stay organized with collections
Save and categorize content based on your preferences.
You can choose either of the following inference strategies:
offline inference, meaning that you make all possible predictions
in a batch, using a MapReduce or something similar. You then
write the predictions to an SSTable or Bigtable, and then feed these
to a cache/lookup table.
online inference, meaning that you predict on demand, using a server.
Learn more about static vs. dynamic inference in the following video (2 min).
Static vs. Dynamic Inference
ML System Paradigms: Inference
Offline Inference
Make all possible predictions in a batch, using a mapreduce or similar.
Write to a table, then feed these to a cache/lookup table.
Online Inference
Predict on demand, using a server.
ML System Paradigms: Inference
Offline Inference
Make all possible predictions in a batch, using a mapreduce or similar.
Write to a table, then feed these to a cache/lookup table.
Upside: don't need to worry much about cost of inference.
Upside: can likely use batch quota.
Upside: can do post-verification of predictions on data before pushing.
ML System Paradigms: Inference
Offline Inference
Make all possible predictions in a batch, using a mapreduce or similar.
Write to a table, then feed these to a cache/lookup table.
Upside: don't need to worry much about cost of inference.
Upside: can likely use batch quota.
Upside: can do post-verification on predictions on data before pushing.
Downside: can only predict things we know about -- bad for long tail.
Downside: update latency likely measured in hours or days.
ML System Paradigms: Inference
Online Inference
Predict on demand, using a server.
Upside: can predict any new item as it comes in -- great for long tail.
ML System Paradigms: Inference
Online Inference
Predict on demand, using a server.
Upside: can predict any new item as it comes in -- great for long tail.
Downside: compute intensive, latency sensitive -- may limit model complexity.
Downside: monitoring needs are more intensive.
Video Lecture Summary
Here are the pros and cons of offline inference:
Pro: Don’t need to worry much about cost of inference.
Pro: Can likely use batch quota or some giant MapReduce.
Pro: Can do post-verification of predictions before pushing.
Con: Can only predict things we know about — bad for long tail.
Con: Update latency is likely measured in hours or days.
Here are the pros and cons of online inference:
Pro: Can make a prediction on any new item as it comes in — great for long
tail.
Con: Compute intensive, latency sensitive—may limit model complexity.