Check Your Understanding: Model Metrics

Mel and you work on metrics for your unicorn appearance predictor. Mel comes to you with a few questions. For the following questions, click on your selection to expand and check your answer.

Regression

You are predicting the timing of unicorn appearances in minutes. Your model's validation MSE is 15. Is this an improvement on a baseline that has a MSE of 25 minutes?
Yes
Correct. A lower MSE indicates higher quality.
No
Incorrect. A lower MSE indicates higher quality.

Real-World Metrics

For your unicorn appearance predictor, which two of the following real-world metrics could complement your model metrics?
Rose production because unicorns eat a lot of roses.
Incorrect. Many non-unicorn factors could explain changes in rose production.
Sightings of shooting stars because unicorns cause shooting stars.
Incorrect. Even if unicorns cause shooting stars, seeing a shooting star in the sky does not tell you exactly where or when the unicorn appeared.
Reports of unicorns posted online.
Correct. Using online reports, you can check whether your predictor predicted an appearance at the time and location of the report.
If a user sees a prediction on your app, and then uses your app another time.
Correct. If a user sees a prediction and subsequently uses your app again, then that prediction was probably accurate enough.