View source on GitHub
|
Meridian module for analyzing model fit in a Meridian model.
This module provides a ModelFitProcessor, which assesses the goodness of fit
of a trained Meridian model. It compares the model's predictions against the
actual observed data, generating key performance metrics.
Key metrics generated include R-squared, MAPE, and Weighted MAPE. The output also includes timeseries data of actual values versus predicted values (with confidence intervals) and the predicted baseline.
The results are structured into a ModelFit protobuf message.
Key Classes:
ModelFitSpec: Dataclass to specify parameters for the model fit analysis, such as whether to split by train/test sets and the confidence level for intervals.ModelFitProcessor: The processor class that performs the fit analysis.
Example Usage:
from schema.processors import model_fit_processor
from schema.processors import model_processor
# Assuming 'mmm' is a trained Meridian model object
trained_model = model_processor.TrainedModel(mmm)
# Default spec: split results by train/test if holdout ID exists
spec = model_fit_processor.ModelFitSpec()
processor = model_fit_processor.ModelFitProcessor(trained_model)
# result is a model_fit_pb2.ModelFit proto
result = processor.execute([spec])
print("Model Fit Analysis Results:")
for res in result.results:
print(f" Dataset: {res.name}")
print(f" R-squared: {res.performance.r_squared:.3f}")
print(f" MAPE: {res.performance.mape:.3f}")
print(f" Weighted MAPE: {res.performance.weighted_mape:.3f}")
# Prediction data is available in res.predictions
# Each element in res.predictions corresponds to a time point.
# e.g., res.predictions[0].actual_value
# e.g., res.predictions[0].predicted_outcome.value
Classes
class ModelFitProcessor: Generates a ModelFit proto for a given trained Meridian model.
class ModelFitSpec: Stores parameters needed for generating ModelFit protos.