You now have embeddings for any pair of examples. A similarity measure takes these embeddings and returns a number measuring their similarity. Remember that embeddings are simply vectors of numbers. To find the similarity between two vectors \(A = [a_1,a_2,...,a_n]\) and \(B = [b_1,b_2,...,b_n]\), you have three similarity measures to choose from, as listed in the table below.
| Measure | Meaning | Formula | Relationship to increasing similarity |
|---|---|---|---|
| Euclidean distance | Distance between ends of vectors | \(\sqrt{(a_1-b_1)^2+(a_2-b_2)^2+...+(a_N-b_N)^2}\) | Decreases |
| Cosine | Cosine of angle \(\theta\) between vectors | \(\frac{a^T b}{|a| \cdot |b|}\) | Increases |
| Dot Product | Cosine multiplied by lengths of both vectors | \(a_1b_1+a_2b_2+...+a_nb_n\) \(=|a||b|cos(\theta)\) | Increases. Also increases with length of vectors. |
Choosing a Similarity Measure
In contrast to the cosine, the dot product is proportional to the vector length. This is important because examples that appear very frequently in the training set (for example, popular YouTube videos) tend to have embedding vectors with large lengths. If you want to capture popularity, then choose dot product. However, the risk is that popular examples may skew the similarity metric. To balance this skew, you can raise the length to an exponent \(\alpha\ < 1\) to calculate the dot product as \(|a|^{\alpha}|b|^{\alpha}\cos(\theta)\).
To better understand how vector length changes the similarity measure, normalize the vector lengths to 1 and notice that the three measures become proportional to each other.
- Euclidean distance = \(||a-b|| = \sqrt{||a||^2 + ||b||^2 - 2a^{T}b} = \sqrt{2-2\cos(\theta_{ab})}\).
- Dot product = \( |a||b| \cos(\theta_{ab}) = 1\cdot1\cdot \cos(\theta_{ab}) = cos(\theta_{ab})\).
- Cosine = \(\cos(\theta_{ab})\).