Deep Learning

Embeddings & Cosine Similarity

An embedding turns a word into an arrow, and two words are similar when their arrows point the same way. Rank the same words by distance instead and the answers change.

the querythe word compared with itthe rest of the vocabulary

Drag anywhere to move the query · click a word to compare with it

Similarity with its best match—

—

Rank the words by

Compare the query with

Force the query to length 1Show the shadow the query casts

Values

angle θ—

cos θ—

dot product—

distance ‖q − w‖—

query length—

What to observe

Lay the query on puppy: cos θ reads 1.00, dog and thing come next, car near 0 and king below zero. Drag it longer or shorter and nothing changes: the cosine measures direction alone.
The shadow is the query dropped onto the word's direction. Its length is |q| cos θ, the dot product divided by the word's length.
Rank by dot product and the long generic word thing takes the top spot from puppy. Raw dot products reward long vectors, which is why retrieval normalises first.
Rank by distance with a short query between cat and kitten:kitten, a short arrow, wins although cat points closer to the query's direction.

Shortcuts:f fullscreen