Feature Crosses

A feature cross is a synthetic feature formed by multiplying (crossing) two or more features. Crossing combinations of features can provide predictive abilities beyond what those features can provide individually.

Feature Crosses

Feature Crosses

  • Feature crosses is the name of this approach
  • Define templates of the form [A x B]
  • Can be complex: [A x B x C x D x E]
  • When A and B represent boolean features, such as bins, the resulting crosses can be extremely sparse

Feature Crosses: Some Examples

  • Housing market price predictor:

    [latitude X num_bedrooms]

Feature Crosses: Some Examples

  • Housing market price predictor:

    [latitude X num_bedrooms]

  • Tic-Tac-Toe predictor:

    [pos1 x pos2 x ... x pos9]

Feature Crosses: Why would we do this?

  • Linear learners use linear models
  • Such learners scale well to massive data e.g., vowpal-wabit, sofia-ml
  • But without feature crosses, the expressivity of these models would be limited
  • Using feature crosses + massive data is one efficient strategy for learning highly complex models
    • Foreshadowing: neural nets provide another

Send feedback about...

Machine Learning Crash Course