Embeddings

The 30-second version

That list of numbers is called an EmbeddingEmbeddingA list of numbers (a vector, often hundreds long) that an embedding model produces for a piece of text, such that texts with similar meaning get vectors that point in similar directions.. Once text is numbers, a computer can measure "how close in meaning are these two sentences?" with simple arithmetic. No exact words need to match.

Here is the idea with only two numbers per word. These numbers are made up by hand to show the shape of the idea; real models produce hundreds of numbers, and we'll use real ones from §05 on.

The words about animals point one way, the weather words another, food a third. To ask "is kitten related to cat?", you compare the directions of their arrows. The standard score for that is Cosine similarityCosine similarityA score from −1 to 1 for how closely two vectors point in the same direction, ignoring their length. For embeddings, higher means more similar meaning. For unit-length vectors it equals the dot product (multiply matching numbers and add them up). (§03).

What you can build with that

The catch, which this page shows with real data: embeddings capture the topic of a text much better than its details. "I love this phone" and "I don't love this phone" get very similar vectors (§07).

Vectors: lists of numbers you can draw as arrows

A VectorVectorAn ordered list of numbers, like [3, 1]. With two or three numbers you can draw it as an arrow from the origin (0, 0) to that point; with more numbers you can't draw it, but the maths works the same way. is just an ordered list of numbers, like [3, 1]. In code it's an array: []float32{3, 1} in Go, [3, 1] in JavaScript. The number of entries is its DimensionDimensionHow many numbers a vector has. [3, 1] is 2-dimensional; an all-MiniLM-L6-v2 embedding is 384-dimensional. Each position is one dimension.: [3, 1] has 2 dimensions, and a real embedding from the model on this page has 384.

Drawing a vector

With two numbers you can draw a vector as an arrow from the origin (the point 0, 0): go 3 to the right and 1 up, and put the arrow tip there.

With 384 numbers you can't draw the arrow any more, but every formula below works the same way, just with a longer loop. That's the nice part: you can build intuition in 2-D and trust it in 384-D.

Length

The lengthLength (magnitude, norm)How long a vector's arrow is: the square root of the sum of its squared numbers. The length of [3, 4] is √(9 + 16) = 5. Also written |a|. of a vector is how long its arrow is. It's Pythagoras: square every number, add them up, take the square root.

• |a| = √(3² + 1²) = √10 ≈ 3.16
• |b| = √(2² + 2²) = √8 ≈ 2.83

The bars |a| are just notation for "the length of a".

The dot product

The Dot productDot productMultiply two vectors' matching numbers and add the results: [3, 1] · [2, 2] = 3×2 + 1×2 = 8. Large when the vectors point the same way and are long; 0 when they are at right angles. of two vectors: multiply the matching numbers, then add the results.

a · b = 3×2 + 1×2 = 6 + 2 = 8

That one number mixes two things together: how much the arrows point the same way, and how long they are. Double the length of b to [4, 4] and the dot product doubles to 16, even though the direction hasn't changed. The next section separates the two.

Cosine similarity: comparing directions

cosine(a, b) = (a · b) / (|a| × |b|)

Worked example

Using the numbers from §02, a = [3, 1] and b = [2, 2]:

1. Dot product: a · b = 3×2 + 1×2 = 8
2. Lengths: |a| = √10 ≈ 3.162 and |b| = √8 ≈ 2.828
3. Multiply the lengths: 3.162 × 2.828 ≈ 8.944
4. Divide: 8 ÷ 8.944 ≈ 0.894

So the Cosine similarityCosine similarityA score from −1 to 1 for how closely two vectors point in the same direction, ignoring their length. For embeddings, higher means more similar meaning. For unit-length vectors it equals the dot product (multiply matching numbers and add them up). is about 0.894. (It's called cosine because the result is exactly the cosine of the angle between the arrows, here about 27°. You don't need trigonometry to use it; the formula does the work.)

Now double b to [4, 4]. The dot product doubles to 16, |b| doubles to 5.657, and the cosine is 16 ÷ (3.162 × 5.657) ≈ 0.894 again. Same direction, same score.

What the numbers mean

Treat the middle rows as rough. What counts as a "high" score depends on the model, and §07 shows real scores where the best match is only 0.49.

Drag the arrows and watch every step of the formula update:

Why ignore length?

For text embeddings, the length of a vector often reflects things that have nothing to do with the topic, and many models (including the one on this page) throw it away by normalizing to length 1. You want "Is this paragraph about password resets?" to get the same answer for a short and a long paragraph. Comparing directions does that.

Normalize once, then it's just a dot product

Computing cosine means three running sums per comparison: the dot product and two lengths. When you compare one query with a million stored vectors, the stored vectors' lengths get recomputed a million times, even though they never change. There's a simple trick to skip it.

Normalizing: make every length 1

To normalizeUnit vector / normalizingA unit vector has length exactly 1. Normalizing means dividing every number by the vector's length, which keeps the direction and makes the length 1. a vector, divide each number by the vector's length. The direction stays the same and the length becomes exactly 1.

a = [3, 1],  |a| = √10 ≈ 3.162
a ÷ |a| = [3/3.162, 1/3.162] ≈ [0.949, 0.316]
check: 0.949² + 0.316² ≈ 0.900 + 0.100 = 1.000 ✓

Do the same for b = [2, 2]: b ÷ 2.828 ≈ [0.707, 0.707].

Now the cosine formula has 1 × 1 underneath, so cosine = dot product:

[0.949, 0.316] · [0.707, 0.707] = 0.671 + 0.224 = 0.895

That's the 0.894 from §03 (the last digit differs only because we rounded to three decimals along the way). Normalize every vector once when you store it, normalize the query once, and from then on cosine is a single multiply-and-add loop. Many embedding models and APIs already return vectors of length 1; check the documentation or measure one (§05 does).

Cosine vs Euclidean distance

The other common way to compare vectors is the Euclidean distanceEuclidean distanceThe straight-line distance between the tips of two vectors: subtract them number by number, square, add up, take the square root. Smaller means closer.: the straight-line distance between the two arrow tips. For unit vectors, the two are tied together by one equation:

distance² = 2 − 2 × cosine

Check it with our unit vectors: the difference is [0.949 − 0.707, 0.316 − 0.707] = [0.242, −0.391], so distance² = 0.0586 + 0.1529 ≈ 0.211. And 2 − 2 × 0.895 = 0.210. ✓

So for normalized vectors, sorting by "highest cosine" and sorting by "smallest Euclidean distance" give the same ranking. That's why a Vector databaseVector databaseA database that stores embeddings and answers “which stored vectors are closest to this one?” quickly, usually with an approximate index. It can be a dedicated product or an extension of an ordinary database, such as pgvector for PostgreSQL. may offer cosine, dot product and Euclidean options: with unit vectors, any of them returns the same top results. They only disagree when the vectors have different lengths, and then the dot product starts rewarding long vectors instead of matching directions.

Real embeddings: 384 numbers per sentence

Time to leave the toy behind. Every number from here on comes from a real embedding modelEmbedding modelA neural network trained to turn a piece of text (or an image) into one fixed-length vector, so that texts with similar meaning get vectors that point in similar directions. It doesn't generate text; it only produces numbers., all-MiniLM-L6-v2: a small, free, widely used model that turns English text into 384 numbers. We ran it offline while building this page and shipped the results with it (each widget has a "How this data was made" note). Nothing on this page calls a live model.

Three things to notice:

• Every sentence gets exactly 384 numbers, whether it has 3 words or 30. That fixed size is what makes comparison possible.
• The length is 1.000. This model's vectors are normalized (§04), so cosine similarity is a plain dot product.
• You can't read meaning from single positions. The numbers are small, positive and negative, and spread out. Meaning lives in the direction of the whole vector.

How does a sentence become one vector?

A model like this first produces one vector per token, and each token's vector depends on the words around it. To get one vector for the whole sentence, this model simply averages them, a step called Mean poolingMean poolingHow many embedding models get one vector for a whole sentence: the model first produces one vector per token, then averages them number by number., and then normalizes the result.

(Other models use different final steps, but the output is always one fixed-length vector per input. How the per-token vectors are computed is covered in How LLMs Work.)

A map of 66 real sentences

Below are 66 short sentences across six themes, plus deliberate traps: pairs that differ only by a negation, a number, a name or word order. To draw 384-dimensional points on a flat screen we squash them to 2-D with PCAPCA (principal component analysis)A standard way to squash many-dimensional points down to 2 or 3 dimensions for a picture. It keeps the directions along which the points vary the most and drops the rest, so distances in the picture are only approximate.. The flat picture keeps only a small part of the information, so the widget always lists the true nearest neighborsNearest neighbors (top-k)The k stored vectors most similar to a query vector, for example the 5 with the highest cosine similarity. Finding them by comparing with every stored vector is called brute-force or exact search., computed from all 384 numbers.

Things to try:

• It starts on How do I reset my password? All five true neighbors are account-help sentences, scoring 0.29 to 0.57.
• Click The cat is sleeping on the sofa. The kitten (0.687) and the rescue cat (0.283) come first. After that the scores drop to 0.14 and below, and the "neighbors" are a password question and a restaurant. A top-5 list always has 5 entries, even when only 2 are relevant, so look at the scores, not just the ranks.
• Turn on the traps filter and click I don't love this phone. Its neighbors are "I hate this phone" and "I love this phone", both scoring above 0.8.
• Under the list, the widget counts how many of the 5 dots that look closest are real top-5 neighbors. Click around: for about two thirds of the sentences it's 2 or fewer.

Cosine similarity and search in Go and JavaScript

All of this is a few lines of code. The examples below run on real vectors: we saved three all-MiniLM-L6-v2 embeddings from §05 to a file called vectors.json, each as { "text": …, "vector": [384 numbers] }. (Go syntax is explained in the Go primer.)

Cosine similarity

This version doesn't assume the vectors are normalized, so it works for any model.

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"math"
	"os"
)

type Item struct {
	Text   string    `json:"text"`
	Vector []float32 `json:"vector"` // 384 float32s = 1,536 bytes
}

// cosine returns dot(a, b) / (|a| * |b|).
func cosine(a, b []float32) float64 {
	var dot, na, nb float64
	for i := range a {
		x, y := float64(a[i]), float64(b[i])
		dot += x * y // ← multiply matching numbers, add them up
		na += x * x
		nb += y * y
	}
	return dot / (math.Sqrt(na) * math.Sqrt(nb))
}

func main() {
	raw, err := os.ReadFile("vectors.json")
	if err != nil {
		log.Fatal(err)
	}
	var items []Item
	if err := json.Unmarshal(raw, &items); err != nil {
		log.Fatal(err)
	}
	cat, kitten, rain := items[0], items[1], items[2]
	fmt.Println(len(cat.Vector), "numbers per vector")
	fmt.Printf("%.3f %s ↔ %s\n", cosine(cat.Vector, kitten.Vector), cat.Text, kitten.Text)
	fmt.Printf("%.3f %s ↔ %s\n", cosine(cat.Vector, rain.Vector), cat.Text, rain.Text)
}

Real output (Go 1.24 and Node 22 give the same):

384 numbers per vector
0.687 The cat is sleeping on the sofa. ↔ A kitten naps on the couch.
-0.166 The cat is sleeping on the sofa. ↔ It is raining heavily outside.

The cat and kitten sentences share almost no words and score 0.687. The rain sentence scores slightly below zero: unrelated.

Searching: compare the query with everything

Semantic searchSemantic searchSearch by meaning instead of by exact words: embed the query, then return the stored texts whose embeddings are closest to it. “I can't log in” can find “Trouble signing in to my account”. is "embed the query, score every stored vector, keep the best k". With normalized vectors, the score is just the dot product (§04). This time the JavaScript version also creates the embeddings, using the same model locally through the transformers.js library; the Go version reads the vectors that the JS script saved.

package main

import (
	"cmp"
	"encoding/json"
	"fmt"
	"log"
	"os"
	"slices"
)

type Doc struct {
	Text   string    `json:"text"`
	Vector []float32 `json:"vector"` // already normalized to length 1
}

type Hit struct {
	Text  string
	Score float64
}

// dot of two unit-length vectors = their cosine similarity
func dot(a, b []float32) float64 {
	var s float64 // add up in float64, as in cosine.go
	for i := range a {
		s += float64(a[i]) * float64(b[i])
	}
	return s
}

// topK compares the query with every doc (brute force) and keeps the k best.
func topK(query []float32, docs []Doc, k int) []Hit {
	hits := make([]Hit, 0, len(docs))
	for _, d := range docs {
		hits = append(hits, Hit{d.Text, dot(query, d.Vector)})
	}
	slices.SortFunc(hits, func(a, b Hit) int {
		return cmp.Compare(b.Score, a.Score) // b before a → highest score first
	})
	return hits[:min(k, len(hits))]
}

func main() {
	raw, err := os.ReadFile("search.json") // vectors saved by the JS script, same model
	if err != nil {
		log.Fatal(err)
	}
	var in struct {
		Query []float32 `json:"query"`
		Docs  []Doc     `json:"docs"`
	}
	if err := json.Unmarshal(raw, &in); err != nil {
		log.Fatal(err)
	}
	for _, h := range topK(in.Query, in.Docs, 3) {
		fmt.Printf("%.3f %s\n", h.Score, h.Text)
	}
}

Real output (both). The widgets on this page show 0.535 for the first pair, not 0.530: they embedded every sentence on its own, while this script embeds the five documents as one batch. §10 explains why batching moves the third decimal.

0.530 How do I reset my password?
0.415 How do I delete my account?
0.356 My verification code never arrives.

Comparing the query with every stored vector is called brute-force or exact search. It's perfectly fine for thousands of vectors. At 100,000 vectors it's 100,000 × 384 ≈ 38 million multiply-adds per query. Simple loops like the ones above take roughly 50–150 ms for that on one CPU core (machine-dependent; on our test machines Go was at the low end and Node somewhat slower): fine for a batch job, slow for every keystroke of a search box. For millions of vectors you'll want an Approximate nearest neighbor (ANN) searchApproximate nearest neighbor (ANN) searchFinding vectors that are very probably among the closest to a query, without comparing against every stored vector. It trades a little accuracy (it may miss a few of the true closest vectors; the share it finds is called recall) for a lot of speed. index; that's the next topic, Vector Databases.

Where vectors come from in production

There are two options. Run a model yourself, as the JS example does (on a server, or even in the browser). Or call a hosted embeddings API, which is usually how Go services do it. Here is the shape of one such API, OpenAI's embeddings endpoint, as of October 2026:

// Not run (needs an API key). OpenAI embeddings endpoint, as of October 2026.
func embedTexts(ctx context.Context, texts []string) ([][]float32, error) {
	body, err := json.Marshal(map[string]any{
		"model": "text-embedding-3-small",
		"input": texts, // several texts in one request
	})
	if err != nil {
		return nil, err
	}
	req, err := http.NewRequestWithContext(ctx, "POST", "https://api.openai.com/v1/embeddings", bytes.NewReader(body))
	if err != nil {
		return nil, err
	}
	req.Header.Set("Authorization", "Bearer "+os.Getenv("OPENAI_API_KEY")) // ← key from the environment, never in code
	req.Header.Set("Content-Type", "application/json")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("embeddings API: %s", resp.Status)
	}
	var out struct {
		Data []struct {
			Index     int       `json:"index"`
			Embedding []float32 `json:"embedding"`
		} `json:"data"`
	}
	if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
		return nil, err
	}
	vecs := make([][]float32, len(texts))
	for _, d := range out.Data {
		vecs[d.Index] = d.Embedding // ← index says which input this vector belongs to
	}
	return vecs, nil
}

Other providers (and self-hosted model servers) have similar shapes: send a batch of texts, get back one vector per text. Whatever you use, keep the API keyAPI keyA long random string a service gives you so your program can call its API. Whoever sends the key is treated as you, so it is a password for a program. on the server, send texts in batches, and plan for rate limitsRate limitA cap on how many requests are allowed per unit of time — e.g. 100 per second. It says nothing about how many may be running at once. and retries when you embed a large corpus.

What embeddings capture, and what they miss

That's not a bug in one model. An embedding squeezes a whole sentence into one fixed-size vector, and models are trained mostly to put texts on the same topic close together. Small words that flip the meaning ("not") and tokens that change one detail ("#48213" vs "#91577") are a small part of that average, so they move the vector only a little. Names are the odd one out: swapping one person's name for another can move the vector a lot (the explorer's last query) or hardly at all, and you can't tell in advance.

Explore it with real scores. Queries in italics show a failure or a trap.

What the real numbers show

Every wrong pair scores higher than cat ↔ kitten, and all but one also beat "can't log in ↔ can't sign in". No threshold can separate them: a cutoff at 0.8 would keep the opposites and throw away "cat ↔ kitten".

Four lessons

1. Scores are relative, not absolute. In this set, the typical (median) score between two unrelated sentences is 0.07, and a correct paraphrase can score 0.49. Another model will have a completely different spread. Compare candidates with each other, and if you need a cutoff, pick it by testing on your own examples.
2. Exact things need exact matching. For order numbers, error codes, product SKUs and people's names, extract them and filter with a normal database query, or use Keyword (full-text) searchKeyword (full-text) searchSearch that matches the words themselves, usually ranked with a formula such as BM25 that rewards rare words appearing often. Good at exact names, IDs and codes; blind to synonyms.. Combining keyword and vector search (hybrid searchHybrid searchRunning keyword search and vector search for the same query and merging the two ranked lists, so exact names and IDs are found by keywords and paraphrases by meaning.) is standard in Retrieval-Augmented Generation.
3. Similar is not the same as true or relevant. Retrieval hands you the closest text, even when it says the opposite of what you need. Whatever reads the results next (a person, or an LLMLLM (large language model)A program trained on huge amounts of text to predict the next piece of text. Given a prompt, it generates a reply one token at a time. It has no database of facts: it produces likely-sounding text, which is usually but not always true.) has to check the details.
4. Short queries are vague. "it's pouring" carries almost no context, so a soup recipe nearly ties with the rain. Longer, more specific queries give a clearer direction.

One model per index: compatibility, dimensions and choice

Never mix models

Number #17 in one model's vector has nothing to do with number #17 in another's, even if both happen to have 384 numbers. We only have one model here, so we made a stand-in "second model" by rotating every MiniLM vector with the same random rotation. That keeps every distance within the second model the same, exactly like a different model that learned the same geometry in its own coordinates. Real measurements:

A real second model is even less compatible, because it also has a different geometry. The rules that follow from this:

• Embed queries with the same model (and the same version) as the documents.
• Store the model name next to every vector, e.g. a model column. Then mixing is a query you can check for, not a silent bug.
• Changing models means re-embedding everything (§10 covers the cost).

Dimensions: bigger isn't automatically better

Models produce anything from a few hundred to a few thousand numbers per text. More dimensions can hold finer distinctions, but every vector costs more to store and to compare: a 1,536-number vector is 4× the size of a 384-number one and takes about 4× as long to score. Whether the extra numbers help your data is something you measure, not assume.

Some models are trained so that the first part of the vector works on its own, and some APIs let you ask for fewer dimensions. As of October 2026, for example, OpenAI's embeddings API accepts a dimensions parameter for its text-embedding-3 models and returns the shortened vector still normalized to length 1. Only shorten vectors this way when the model's documentation says it was trained for it; cutting an ordinary model's vector in half just throws information away.

Choosing a model: the questions that matter

This page deliberately doesn't rank models; rankings change every few months. Ask these instead:

Then test two or three candidates on your own data: take 50–100 real queries, write down which documents should come back, and measure how often each model puts them in the top 5. That small table beats any public leaderboard for your use case. (Building that kind of test set is covered in Evaluating LLM Apps.)

What you build with embeddings

Almost every use of embeddings is one idea: close vectors = related items. What changes is what you do with "close".

Semantic search

Embed every document (or every ChunkChunkA piece of a longer document (a paragraph, a section, a few hundred tokens) that gets its own embedding. Long documents are split into chunks because one vector can't represent many topics well. of a long document) once, store the vectors, embed each query, return the nearest. §06 showed the code. Pair it with keyword search for exact terms (§07). Feeding the results to an LLM so it can answer from them is RAG.

Deduplication

To find near-duplicate bug reports, product listings or support tickets, look for pairs with a very high cosine. In this page's data, "Error E1027 in the payment service." and "Error E4410 in the payment service." score 0.908. Are those duplicates? Same service, different error codes: probably not. So treat a high score as "a human (or a stricter check) should look at this pair", not as an automatic merge. Exact deduplication (same bytes → same hash → same item) is a different, simpler job.

Clustering

ClusteringClusteringGrouping items automatically so that items in the same group are similar to each other, without labels given in advance. With embeddings, “similar” means close vectors. groups items without labels: "what are customers writing to us about this week?". Algorithms such as k-meansk-meansA clustering algorithm: pick k centre points, put each vector in the group of its nearest centre, move every centre to the average of its group, and repeat until the groups stop changing. take the vectors and return groups: pick k centre points, put each vector with its nearest centre, move each centre to the middle of its group, repeat. The output is only groups of numbers; a person (or an LLM) still has to read a few items from each group and name it.

Classification by nearest example

Need to route tickets into categories but don't have thousands of labelled examples? Embed a handful of labelled examples per category. For a new ticket, find its 3 nearest examples and take the most common label. This is called k-nearest-neighbors (kNN) classification. The similarity explorer in §07 does exactly this under each result list. Real example: "the app keeps crashing" has the server-crash sentence as its nearest neighbor, but its top 3 are code, account, code. The vote still picks code & bugs, 2 to 1. Adding examples fixes most mistakes, with no training run.

Recommendations

Embed item descriptions (articles, products, songs' tags). For a user, average the vectors of things they liked and show the nearest items they haven't seen. Simple, and good for the "cold start" problem: a brand-new item can be recommended from its description before anyone has clicked it.

Chunk length, truncation and re-embedding

Dilution: real measurements

Here one sentence answers the query "How do I reset my password?". We embed it alone, then glued together with more and more unrelated office announcements, and measure the cosine each time.

The answer never leaves the chunk, yet the score falls from 0.788 (alone) to about 0.37 with 8 filler sentences and 0.17 with 32. Meanwhile, a short chunk that merely mentions passwords could easily score higher and win. That's why long documents are split into chunksChunkA piece of a longer document (a paragraph, a section, a few hundred tokens) that gets its own embedding. Long documents are split into chunks because one vector can't represent many topics well. of a paragraph or a few hundred tokens, each with its own vector. How to split well (by headings, with overlap, keeping tables whole) is a big part of RAG.

Silent truncation

Every model has a maximum input length in tokens, and text past it is usually cut off without an error (TruncationTruncation (input limit)Every embedding model has a maximum input length in tokens. Text past the limit is usually cut off silently, so it has no effect on the vector at all.). Switch the widget to answer last and pick 64: the chunk is 602 tokens, the tool we used stops reading at 512, and the answer at the end simply disappears. The score drops to −0.003, exactly what the filler alone scores.

The limit also depends on the software, not just the model. The same all-MiniLM-L6-v2 is cut at 512 tokens by transformers.js (which reads the tokenizer's settings) and at 256 word piecesWord pieceThe tokens BERT-style models such as MiniLM use: common words stay whole, rarer words are split into pieces, e.g. “unbelievable” → “un”, “##bel”, “##ievable” (## marks a piece that continues a word). A model's length limit is counted in these pieces. (this model's tokens) by the sentence-transformers library, which follows the model card. Check the limit in the library you actually run, and count tokens before embedding long text.

The same text doesn't always give the same vector

Embedding is deterministic for the same model build, the same software and the same input, but small changes leak in. The 8-bit build we used rounds its internal numbers per batch: "The cat is sleeping on the sofa." embedded alone vs inside a batch of 66 sentences gave vectors with a cosine of 0.992, not 1. That's enough to move scores: cat ↔ kitten scores 0.687 when each sentence is embedded alone (the way all data on this page was made) but 0.714 when both are embedded inside that batch. Close results can swap places. Different hardware, library versions or numeric precision cause similar drift. Two practical consequences:

• Don't use embeddings as cache keys or equality checks. "Same text?" is a string comparison or a hash, not a cosine of 1.0.
• In tests, compare scores with a tolerance (as you would for any Floating-point numberFloating-point numberThe way computers store fractional numbers: a sign, a fraction and an exponent, like scientific notation in binary. Fast and wide-ranging, but most decimal fractions (like 0.1) can only be stored approximately. value, see Numbers in Computers), and expect the third decimal to move.

Changing models costs a full re-embed

Because vectors from different models are incompatible (§08), a new model means re-embedding every stored text. Do the arithmetic before you switch. With 10 million chunks, sending 100 texts per request and running 20 requests at a time, at a hypothetical 0.5 s per request: 10,000,000 ÷ 100 = 100,000 requests; ÷ 20 in parallel = 5,000 rounds; × 0.5 s ≈ 42 minutes, if you never hit a rate limit. Real runs take longer: rate limits, retries, and the time to rebuild the index.

The safe migration looks like a database migration:

1. Add a new vector column (or a new index) for model B. Keep serving from model A.
2. Backfill model-B vectors in a background job; new writes get both.
3. Compare search quality on your test queries (§08).
4. Switch queries to model B, keep A for quick rollback, then drop it.

Storing vectors: bytes, quantization and bias

How big is a vector?

Each number is a 32-bit Floating-point numberFloating-point numberThe way computers store fractional numbers: a sign, a fraction and an exponent, like scientific notation in binary. Fast and wide-ranging, but most decimal fractions (like 0.1) can only be stored approximately. value: 4 bytesByte8 bits — enough to store a number from 0 to 255, or one ASCII character. Network speeds are usually quoted in bits per second, file sizes in bytes.. (Why floats are approximate, and why float32 is enough here, is explained in Numbers in Computers.)

384 numbers × 4 bytes        = 1,536 bytes per vector  (1.5 KB)
1,536 bytes × 1,000,000 docs = 1.536 GB of raw vectors

That's before the index, the text itself and any metadata. A 1,536-dimension model needs 6 KB per vector and about 6 GB per million. Vector indexes are fastest when they fit in RAM, so this number decides how big a machine you need.

Quantization: fewer bits per number

QuantizationQuantizationStoring numbers with fewer bits to save space, for example rounding each 4-byte float to a 1-byte integer, or keeping only its sign (1 bit). Similarities become slightly less precise. stores each number with fewer bits. Two common levels, measured on this page's 66 sentences (all 2,145 pairs):

int8 is nearly free for this data: a quarter of the space, and the neighbor lists barely change. Keeping only the sign of each number is 32× smaller but loses more than a quarter of the true top-5 neighbors here. So 1-bit vectors are used as a fast first pass (find 100 candidates quickly), followed by re-scoring those candidates with the full vectors. Your data will give different numbers; measure before you commit. (The model we used is itself 8-bit quantized internally, which is a different trick: it shrinks the model's weights, not the stored vectors.)

Bias is part of the geometry

Embedding models learn from huge amounts of human-written text, so they also learn its stereotypes. Published research on older word embeddings found, for example, occupation words sitting closer to one gender than the other, and similar associations for names linked to different ethnic groups (Bolukbasi et al., 2016; Caliskan et al., Science, 2017). Newer sentence models have been shown to carry such associations too, to varying degrees. In a product this shows up quietly: a résumé search that ranks some names lower, or a recommender that steers groups apart. Embeddings don't make decisions on their own, but anything you build on "nearest" inherits these patterns. If your system ranks or filters people, test it with matched examples that differ only in a name or a pronoun, compare the scores, and keep a human in the loop for decisions that matter.

Check yourself

Work out each answer before you click. Q1–Q2 are covered in §03, Q3 in §04, Q4 in §07, Q5 in §08, Q6 in §11.

Cheat sheet