Google is committed to advancing racial equity for Black communities. See how.

Regularization for Sparsity

This module focuses on the special requirements for models learned on feature vectors that have many dimensions.

Regularization for Sparsity

  • Caveat: Sparse feature crosses may significantly increase feature space
  • Possible issues:
    • Model size (RAM) may become huge
    • "Noise" coefficients (causes overfitting)
  • Would like to penalize L0 norm of weights
    • Non-convex optimization; NP-hard
  • Would like to penalize L0 norm of weights
    • Non-convex optimization; NP-hard
  • Relax to L1 regularization:
    • Penalize sum of abs(weights)
    • Convex problem
    • Encourage sparsity unlike L2
Previous
Programming Exercise
Next
L1 Regularization