embeddings · vector search · quantization · lossless codecs

Close Enough

A visual primer on how machines turn meaning into points in space, how they find the nearest points among billions, and how few bits a point really needs.

49 short pagesinteractive figures throughoutreal embeddings and cached attention valuesnumpy sketches throughout

Fig 1Nearest neighbours of a drifting query

The whole field in one picture A query (the star) drifts through a cloud of embedded items, always linked to its six nearest neighbours. Part 1 is about where the points come from, Part 2 about finding the neighbours quickly, and Parts 3 to 6 about storing the points in far fewer bits than they arrive in.

1How this book works

Small pages, one idea each Every page answers one question and ends with a checkpoint. Most take three to six minutes. The bar at the top and the ticks in the contents remember what you have read, on this device.
Show, then tell Wherever a claim can be tried, there is a playground: drag a point, move a slider, watch the number. Figures marked measured use real embeddings, not toy data.
No term left hanging Dotted-underlined words such as HNSW or residual quantization open a one-line reminder. Press t for the contents, g for the glossary, ← → to turn pages.

Colours carry meaning, and they mean the same thing in the text, the equations, the figures and the code:

data vector x query, search time codebook, index structure code: the stored integers reconstruction x̂ residual, error encoder work bits, entropy, models

2The parts

3Three ways through

The bird’s-eye tour (about an hour). Parts 1 and 2, then 7.4 The big picture and 7.5 The whole thing, and where to go next. You will be able to place every term you meet in a vector-database changelog.
The compression track. Part 3 for the frame, Part 4 for codebooks, then 4.8 Measured on real embeddings for the numbers. This is the route if you need to shrink an index.
The deep end. Part 5 on hashing and Part 6 on building a lossless codec assume you are comfortable with entropy and probability models. They are self-contained after Part 3.

4Where the numbers come from

The paper club adds model-specific checks. Our experiments collects results from kvdiff, starting with five GSM8K sequences from Qwen 3.5 2B. Those pages name their own data and evaluation splits.

This book grew out of three long conversations: a field guide to embeddings and search, a deep dive on additive codebooks, and a note on where PCA fits. It keeps their examples and adds two chapters they only hinted at: hashing, and exact compression.