AI Embedding Migration: Reindex Before You Cut Over

Changing an embedding model looks like a dependency update. In production it is a data migration, an index migration and a retrieval-policy change at the same time. Every stored document vector was created inside the old model's coordinate system; every live query must enter the same system before distance has meaning. Replace only the query encoder and the service may still return neighbours, but their scores no longer express a relationship the model was trained to produce.

The safe release question is not “has the new model finished indexing?” It is: does one complete new retrieval path beat or match the incumbent on the queries and failure slices that matter, while preserving every source update and a tested rollback?

Build the new path beside the old one. Keep source identity stable, dual-write changes, backfill reproducibly, run both retrievers on a labelled query set and switch one versioned route only when coverage, relevance, latency, cost and policy filters pass. Never mix old and new embeddings in one unnamed vector field.

Treat an embedding as a coordinate-system [contract](/industries/legal)

Vector length is the visible compatibility rule. AWS requires the vector index dimension to match the selected embedding model and documents supported models with different output sizes in its Bedrock vector-store prerequisites. Google exposes model, task type and output dimensionality as embedding inputs in its current embedding documentation. Those choices describe more than storage shape: they influence what proximity represents.

Two models can both emit 768 numbers and still define unrelated spaces. The same applies when a provider changes a model revision, a task instruction changes from document retrieval to semantic similarity, normalisation changes, or the query and document sides use different prefixes. Equal dimensions make vectors storable together; they do not make their distances comparable. That conclusion is an engineering implication of the model-bound encodings, not a promise made by the cloud documentation.

Record an immutable retrieval release identity containing:

• embedding provider, model and revision or digest;
• document and query task modes, prefixes and normalisation;
• output dimensions, numeric type and distance function;
• parser, chunking rule, overlap and metadata enrichment versions;
• source snapshot or high-water mark and deletion policy;
• index engine, approximate-nearest-neighbour settings and reranker;
• filter policy, top-k values and application assembly version.

This is the retrieval counterpart to the AI model change-control baseline. A model name alone cannot reconstruct the system that produced a result.

Choose the migration shape before the model

The storage design determines whether the change can be isolated and rolled back.

Qdrant's embedding migration tutorial describes both blue-green collections and side-by-side named vectors. Its blue-green flow searches the old collection, dual-writes updates, re-embeds the corpus in the background and then changes both collection and query model. Weaviate's vectorizer migration guide likewise treats recalculating stored vectors as explicit work rather than a metadata toggle. AWS is stricter in one managed path: its managed knowledge-base documentation says switching between managed and custom embedding types requires a new knowledge base.

Prefer a parallel collection when index settings, filters, chunking or the embedding API are changing together. Named vectors reduce duplicated payload storage, but demand an explicit completeness flag so a new-vector query cannot silently search a half-migrated corpus.

Separate source truth from generated vectors

Re-embedding must start from canonical chunks or, better, the canonical document plus the versioned chunker. Do not migrate by exporting the old vectors: they contain no text from which the new model can derive its coordinates. Do not make a vector database the sole copy of source text, access metadata, legal holds or deletion state.

Give each logical chunk a stable identity derived from the source object, source revision and chunk position or semantic boundary. Store the embedding release separately. That lets the old and new indices refer to the same evidence even if their vector dimensions differ.

Establish a source high-water mark before backfill. Then process change events in order:

• create the empty target with its own release identity;
• begin idempotent dual writes for creates, updates and deletes;
• scan the source snapshot into resumable backfill batches;
• reject an older backfill write when a newer live revision already exists;
• reconcile source counts, revisions, tombstones and failed items;
• keep dual writes active through evaluation and cutover.

This detail is not optional. Qdrant notes that a simple two-collection example needs extra handling for deletes and partial updates, while its insert-only backfill protects a newer target write from being overwritten. The same race exists in any store. A successful job count can conceal resurrected deletions or stale chunks.

Evaluate the whole retrieval path on local work

A public leaderboard is a candidate filter, not a release gate. The original MTEB paper evaluated more than 30 models across eight task families and found no single best solution; different models led different tasks. The BEIR study found that in-domain performance did not reliably predict zero-shot generalisation, that BM25 remained a strong baseline, and that similarity choice could materially change results by dataset.

Build a fixed, versioned evaluation set from the real information needs of the product. Include frequent queries, high-consequence queries, known misses, short ambiguous requests, long specific requests, acronyms, negation, multilingual inputs, recently changed documents and cases where metadata filters must exclude a semantically attractive result. Our retrieval quality audit explains how to label source-backed queries and separate retrieval from answer failure. Keep assessors blind to which retriever produced each candidate, following the principles in our blind release-test guide.

Measure at least:

• recall at the application's actual retrieval depth;
• nDCG or graded relevance where order and partial usefulness matter;
• first-relevant rank and the rate of no relevant result;
• policy-filter escapes and missing entitled evidence;
• source freshness and duplicate-version collisions;
• p50 and tail latency, timeout rate and reranker load;
• embedding, storage, indexing and per-query cost.

Top-k overlap between old and new is diagnostic only. A better retriever should disagree with the incumbent on some queries. The question is whether judged evidence improves, not whether two systems return the same IDs.

Shadow both paths without duplicating consequences

Send a representative sample of production queries to both retrieval paths, but let only the incumbent feed the user-visible answer at first. Preserve query category, both ranked result lists, filter decisions, latency and release identities. Minimise or hash personal text where analysis does not require it, and apply the same retention and access rules as ordinary traces.

Do not run the downstream agent twice if it can call tools or create side effects. Compare retrieval results offline, or run the candidate answer path with all commit authority disabled. Our safe-action validation guide explains why a shadow model must still be outside the action boundary.

Use shadow traffic to find capacity problems that a static set cannot expose: provider throttling during a backfill, hot-filter latency, target-index warm-up, larger vectors increasing memory pressure, or reranking cost rising because the candidate needs a larger top-k. Maintain a lexical or hybrid fallback where exact identifiers, product codes or legal references matter; dense retrieval is not automatically the correct route for every query.

Worked cutover: a supplier-policy assistant

Consider an assistant that retrieves approved procurement policies and drafts answers with citations. The old index uses one embedding model over 400-token chunks. The candidate promises better multilingual retrieval and uses a task-specific document mode.

The team creates a new collection rather than overwriting the old vectors. Both collections share stable chunk IDs, source revision, effective date, supplier scope and access labels. New and revised policies are embedded into both paths; a tombstone event removes the same chunk revision from both. Backfill reads a frozen source manifest and cannot overwrite a higher revision already produced by the live pipeline.

The release set contains labelled English and Arabic queries, exact policy codes, superseded clauses, conflicting regional policies and requests from roles with different access. The candidate must meet the incumbent's recall floor overall, improve the declared multilingual slice, produce zero access-filter escapes, contain every current source revision, stay inside the tail-latency and cost budgets, and support a successful rollback drill.

Only then does one versioned search alias move to the candidate. Elasticsearch documents that multiple alias actions can execute atomically, so a route can swap without pointing at both indices. That atomicity prevents a routing gap. It does not prove vector coverage, source correctness or retrieval quality; those gates must already be green.

Make the cutover small and the rollback complete

Change the query encoder and index route as one configuration release. If the application can combine an old query encoder with the new index during partial rollout, bind both values behind a single retrieval-release ID and reject unknown pairs. Purge or namespace result caches so old ranked lists do not mask the change.

Start with a small traffic fraction or one low-consequence tenant when architecture permits. Compare live no-result rate, citation support, overrides, latency, cost and slice behaviour with the shadow baseline. Stop on a filter escape, unexplained corpus deficit, material relevance regression, stale-source retrieval or inability to reconstruct which release answered.

Rollback must restore the old encoder, index route, filters, reranker and caches—not merely repoint an alias. Keep dual writes running until the rollback window closes. Retain the old index and its source manifest long enough to investigate regressions, then retire it under the normal data-retention and deletion process.

Approve evidence, not migration activity

A useful release record answers four questions:

• Compatibility: Are document and query vectors produced by one declared coordinate-system contract?
• Completeness: Does every current source revision and deletion appear in the candidate?
• Quality: Does the candidate pass labelled retrieval and policy slices against an appropriate baseline?
• Operability: Can the service meet latency and cost budgets, identify the active release and roll back without losing updates?

Revisit the decision after corpus mix, language mix, chunking, filters, model revision, distance function, index parameters or user behaviour changes. The migration is finished when the evidence supports one complete path and the old path can be retired safely—not when the re-embedding progress bar reaches 100%.