Feature Crosses: Playground Exercises

Introducing Feature Crosses

Can a feature cross truly enable a model to fit nonlinear data? To find out, try this exercise.

Task: Try to create a model that separates the blue dots from the orange dots by manually changing the weights of the following three input features:

  • x1
  • x2
  • x1 x2 (a feature cross)

To manually change a weight:

  1. Click on a line that connects FEATURES to OUTPUT. An input form will appear.
  2. Type a floating-point value into that input form.
  3. Press Enter.

Note that the interface for this exercise does not contain a Step button. That's because this exercise does not iteratively train a model. Rather, you will manually enter the "final" weights for the model.

(Answers appear just below the exercise.)

More Complex Feature Crosses

Now let's play with some advanced feature cross combinations. The data set in this Playground exercise looks a bit like a noisy bullseye from a game of darts, with the blue dots in the middle and the orange dots in an outer ring.

Task 1: Run this linear model as given. Spend a minute or two (but no longer) trying different learning rate settings to see if you can find any improvements. Can a linear model produce effective results for this data set?

Task 2: Now try adding in cross-product features, such as x1x2, trying to optimize performance.

  • Which features help most?
  • What is the best performance that you can get?

Task 3: When you have a good model, examine the model output surface (shown by the background color).

  1. Does it look like a linear model?
  2. How would you describe the model?

(Answers appear just below the exercise.)