Regularization for Sparsity

This module focuses on the special requirements for models learned on feature vectors that have many dimensions.

Regularization for Sparsity

Let's Go Back to Feature Crosses

Caveat: Sparse feature crosses may significantly increase feature space
Possible issues:

Model size (RAM) may become huge
"Noise" coefficients (causes overfitting)

L₁ Regularization

Would like to penalize L₀ norm of weights

Non-convex optimization; NP-hard

L₁ Regularization

Would like to penalize L₀ norm of weights

Non-convex optimization; NP-hard

Relax to L₁ regularization:

Penalize sum of abs(weights)
Convex problem
Encourage sparsity unlike L₂

Programming Exercise

L1 Regularization

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2022-07-18 UTC.