Regularization for Sparsity

This module focuses on the special requirements for models learned on feature vectors that have many dimensions.

Estimated Time: 2 minutes

Learning Objectives

• Learn how to drive uninformative coefficient values to exactly 0, in order to save RAM.
• Learn about other kinds of regularization besides L₂.

Regularization for Sparsity

Let's Go Back to Feature Crosses

• Caveat: Sparse feature crosses may significantly increase feature space
• Possible issues:
• Model size (RAM) may become huge
• "Noise" coefficients (causes overfitting)

L₁ Regularization

• Would like to penalize L₀ norm of weights
• Non-convex optimization; NP-hard

L₁ Regularization

• Would like to penalize L₀ norm of weights
• Non-convex optimization; NP-hard
• Relax to L₁ regularization:
• Penalize sum of abs(weights)
• Convex problem
• Encourage sparsity unlike L₂

Video not displaying?

Please see the community page for troubleshooting assistance.

Help Center