Stay organized with collections
Save and categorize content based on your preferences.
This section consists of six exercises to complete at the beginning of an ML
project to start framing your problem and set you up for success.
If you have a problem of your own you would like to work on, please do.
Otherwise, try to build a model to identify whether an email in Gmail
is “important.”
You can enter your response to the prompt into the textbox. You can
then print or store your answers as a .pdf using the button at the
bottom of the page.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-08-25 UTC."],[],[],null,["# Try It Yourself: Framing\n\n\u003cbr /\u003e\n\nThis section consists of six exercises to complete at the beginning of an ML\nproject to start framing your problem and set you up for success.\nIf you have a problem of your own you would like to work on, please do.\nOtherwise, try to build a model to identify whether an email in Gmail\nis \"important.\"\n\nYou can enter your response to the prompt into the textbox. You can\nthen print or store your answers as a .pdf using the button at the\nbottom of the page.\n\nExercise 1: Start Clearly and Simply\n------------------------------------\n\nWrite what you'd like the machine learned model to do.\n\nWe want the machine learned model to:\n\n| Tips for Success |\n|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| At this point, the statement can be qualitative, but make sure this captures your real goal, not an indirect goal. If you are unsure, take a look at the previous lessons, and see what machine learned models can possibly do. |\n\nExercise 2: Your Ideal Outcome\n------------------------------\n\nAdding your ML model to your system should produce a desirable outcome. What is this outcome, independent of the model itself? Note that this outcome may be quite different from how you assess the model and its quality.\n\nOur ideal outcome is:\n\n| Tips for Success |\n|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| You don't need to limit yourself to metrics for which your product has already been optimizing (those will be covered in the next exercise). Instead, try to focus on the larger objective of your product or service. |\n\nExercise 3: Your Success Metrics\n--------------------------------\n\nWrite down your metrics for success and failure with the ML system. The failure metrics are important, that is, how will you know whether the ML system has failed? Both the success and failure metrics should be phrased independently of evaluation metrics for the model. For example, don't talk about precision, recall, or AUC; talk about the anticipated outcomes, instead. Frequently these metrics will be tied to the ideal outcome you specified above.\n\nOur success metrics are:\n\nOur ML model is deemed a failure if:\n\n| Tips for Success |\n|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| - Are the metrics measurable? - How will you measure them? - When will you be able to measure them? - How long will it take for you to know whether your new ML system is a success or failure? - Consider engineering and maintenance costs over the long-term. - Failure may not only be caused simply by non achievement of a success metric. |\n\nExercise 4: Your Output\n-----------------------\n\nWrite the output that you want your ML model to produce.\n\nThe output from our ML model will be:\n\nThe output is defined as one of the following:\n\n- Unidimensional regression\n- Multidimensional regression\n- Binary classification\n- Multiclass classification\n- Generate text, image, audio, video, or some combination (multimodal)\n\n\u003cbr /\u003e\n\n| Tips for Success |\n|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| - The output must be quantifiable with a definition that a machine can produce. - If you're using predictive ML, are you able to obtain example outputs to use for training data? - How and from what source? - Your output examples may need to be engineered, such as turning watch time into a percentile. - If it is difficult to obtain example outputs to use for training, you may need to revisit your responses to past exercises to reformulate your problem and goals into ones that will allow you to train a model on your data. |\n\nExercise 5: Using the Output\n----------------------------\n\nWrite when your output must be obtained from the ML model, and how it is used in your product.\n\nThe output from the ML model will be obtained when:\n\nThe outcome will be used for:\n\n| Tips for Success |\n|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Consider what data you'll need to make the prediction. For instance, if you're using ML to predict travel time in a maps app, this might be written as, \"The output from the model will be obtained when the user selects origin, destination, and mode of transportation.\" |\n| Consider how you will use the predicted outcome in your product. - Will it be presented immediately to the user in a UI? - Will it be consumed by subsequent business logic? - What latency requirements do you have? Those requirements (serving requirement of the ML model) can impact what information can be used to make predictions. For example: \u003c!-- --\u003e - The latency of using data from remote services may make them infeasible to use. - If data sources lag in making new information available: - Processed logs may be generated only once a day. - Certain information is not known until it actually happens (such as conversion events). The Oracle Test: assume you always had the correct answer. How would you use that in your product? |\n\nExercise 6: Your Heuristics\n---------------------------\n\nWrite how you would solve the problem if you didn't use ML. For example, what heuristics you might use.\n\nIf we didn't use ML, we would:\n\n| Tips for Success |\n|-----------------------------------------------------------------------------------------------------------------------------------------|\n| Think about a scenario where you need to deliver the product tomorrow, and you can only hardcode the business logic. What would you do? |\n\nClick the button below to either print or save your responses as a .pdf.\n\n[Print Page](#buttons)"]]