Different cities in California have markedly different housing prices.
Suppose you must create a model to predict housing prices. Which of the
following sets of features or feature crosses could learn
city-specific relationships between roomsPerPerson and housing
Three separate binned features: [binned latitude],
[binned longitude], [binned roomsPerPerson]
Binning is good because it enables the model to learn nonlinear
relationships within a single feature. However, a city exists in
more than one dimension, so learning city-specific relationships
requires crossing latitude and longitude.
One feature cross: [latitude X longitude X
In this example, crossing real-valued features is not a good idea.
Crossing the real value of, say, latitude with
roomsPerPerson enables a 10% change in one feature (say, latitude)
to be equivalent to a 10% change in the other feature (say,
One feature cross: [binned latitude X binned longitude X binned
Crossing binned latitude with binned longitude enables the
model to learn city-specific effects of roomsPerPerson.
Binning prevents a change in latitude producing the same result
as a change in longitude. Depending on the granularity of
the bins, this feature cross could learn city-specific or
neighborhood-specific or even block-specific effects.
Two feature crosses: [binned latitude X binned roomsPerPerson]
and [binned longitude X binned roomsPerPerson]
Binning is a good idea; however, a city is the conjunction of
latitude and longitude, so separate feature crosses prevent the
model from learning city-specific prices.