Filtered Vector Search: Empty Is Not Absent

An assistant says it cannot find a policy that you know exists. The document is indexed, its embedding is present, and the user is entitled to read it. Removing the department filter makes it appear. That is not necessarily an embedding failure: the search plan may have discarded the document before it ever competed inside the permitted result set.

Filtered vector search combines two different questions: which records satisfy an exact condition, and which vectors are closest to a query? Their execution order matters. Retrieving a short global list and then removing ineligible records is not equivalent to finding the nearest records among eligible ones.

The reader decision is practical: when should a filtered query use approximate search, spend more effort finding candidates, or switch to exact search over a bounded subset? Answer it with a filtered reference result, not an unfiltered benchmark.

This guide concerns retrieval mechanics, not a new embedding model or a product ranking. The documentation below establishes engine-specific behaviour; the diagnostic method and worked example are AIEngine's engineering synthesis. Living documentation was checked on 15 September 2026. No local database benchmark or customer result is claimed.

Define the result set before tuning the index

Start with the eligible set: records matching the trusted tenant, access, validity and business predicates for this request. Within that set, define the nearest k vectors under the declared distance function. That is the reference target for testing the search algorithm.

Keep three meanings separate. An eligible document passes the filter. A nearest neighbour scores highly under the embedding's distance function. A relevant document actually helps answer the user's question. None of these terms implies the other two.

Exact vector search can identify the nearest eligible vectors without proving that their content is useful. A weak embedding can rank irrelevant documents perfectly accurately. Conversely, an approximate index can miss a useful neighbour even when the embedding placed it correctly. Our broader retrieval quality audit separates those retrieval errors from corpus, ranking and answer failures.

Write down whether k means chunks or distinct source documents. Five near-identical chunks from one policy may fill five result slots while supplying only one piece of evidence. Define version deduplication, minimum similarity and any grouping rule before establishing the baseline. Otherwise a later transformation can make a correct nearest-neighbour result look like an index defect.

Locate the filter in the execution plan

Do not infer execution order from where a filter appears in application code. Inspect the deployed engine, API mode and actual query plan.

Microsoft's Azure AI Search filter documentation, updated in April 2026, distinguishes filtering during HNSW traversal, filtering each shard's unfiltered candidates, and filtering the final global top-k list. The latter two can miss qualifying records outside their candidate windows. Its strict global postfilter mode remains preview when checked.

Those are not interchangeable implementations of the same guarantee. A postfilter can remove the wrong candidates correctly and still fail to discover the right ones. Increasing the displayed result count does nothing if the earlier search stage still examines the same restricted candidate pool.

The pgvector filtering documentation describes filtering after an approximate index scan. Iterative scans, available from version 0.8.0, can continue searching until enough results are found or configured scan limits are reached. Strict ordering controls the order of returned distances; it does not turn an approximate search into an exhaustive one.

Distinguish the output limit, the internal search-effort setting and any downstream reranking window. Names such as k, candidate count and exploration budget do not have universal semantics across engines. Record each separately. A larger internal budget can improve discovery without increasing the number of chunks eventually sent to the language model.

Work through a miss with no approximation at all

Consider a synthetic corpus whose first seven records are already sorted by exact vector distance. Smaller distance is better. The request needs the nearest three records belonging to its permitted department. The remaining corpus records are farther away than those shown.

The correct filtered top three are A, B and C. Taking the global top three and then filtering returns nothing. Taking five global candidates returns only A: one-third recall against the filtered reference. A six-record candidate window finds A and B; seven finds all three.

Every distance and rank in this example is exact. The failure comes entirely from truncating the wrong population. An approximate search adds another possible source of misses, but is not necessary to produce the problem.

This also explains why “we doubled k” is not a general repair. Seven happens to work for this constructed ordering. A different query may place the first eligible record at rank hundreds or thousands. A multiplier selected from one successful query cannot guarantee discovery for another.

Do not use the corpus-wide eligible fraction as a deterministic budget formula. If one in ten records passes the filter, that does not mean one in ten records near this particular query will pass. The filter may exclude almost the entire semantic neighbourhood the search visits first.

Filter selectivity is only half the workload

Two filters can admit the same number of records and produce very different search behaviour. One selects a topic tightly aligned with the query. Another selects a department whose relevant material occupies a distant part of the vector space. Combined predicates can also leave a small intersection even when each individual condition is broad.

The mechanism has a long history. Qdrant's 2019 exploration of filterable HNSW shows how naively removing disallowed nodes from graph traversal can fragment connectivity, and explores additional connections for filtered categories. Treat it as an explanation of the graph problem, not a universal performance guarantee for present-day products.

Modern implementations make different choices. Weaviate's filtering documentation describes an eligible-ID list passed to graph search, and an ACORN-inspired implementation intended to improve traversal when the filter excludes much of the query's neighbourhood. It explicitly distinguishes its implementation from the research algorithm. A filter-strategy name is therefore insufficient to reproduce behaviour across products.

For workload design, vary both eligible-set size and the relationship between filters and query meaning. Include rare departments, overlapping groups, multiple conditions and uneven shard distributions. A benchmark that assigns random labels uniformly may conceal the hard cases generated by real organisational structure.

Graph navigation also needs a precise security interpretation. Traversing an internal index node is not the same as returning its document text to an application. Establish which components may access the shared index, and ensure unauthorised payloads never reach model context, external rerankers or user-visible results. A product's use of the word prefilter does not prove your entire authorisation boundary.

Make exact search a measured option

A large corpus does not imply a large eligible subset. If a request can access only a small set of current records, directly scoring that set may be competitive with searching a broad graph for enough qualifying neighbours.

Elastic's kNN documentation describes both explicit exact scoring over matching documents and automatic per-segment fallbacks in filtered HNSW. Lucene can bypass the graph when the filtered count is no greater than the candidate count, or switch when traversal exceeds the number of matching documents. These are implementation details, not a portable threshold to copy into another engine.

Choose a plan through measurements on the intended workload:

• Use filtered ANN where its reference recall, useful-evidence retrieval and tail latency satisfy the application contract.
• Increase exploration or enable bounded iterative scanning when missed neighbours improve with additional search effort.
• Compare exact scoring when the eligible subset is small enough to fit the latency and resource budget.
• Consider partitions or separate indices for stable isolation boundaries, while accounting for updates, uneven sizes and operational overhead.

There is no universal percentage below which exact search wins. Vector dimensions, memory placement, filter-evaluation cost, index state, concurrency and hardware affect the crossover. Measure the full request, including the work required to identify eligible IDs; timing only the distance loop understates the plan's cost.

Record what the engine actually executed. A fast filtered query may already have used an exact fallback. Without plan or profile evidence, it is easy to attribute that success to the wrong algorithm.

Evaluate discovery, usefulness and fullness separately

Build an offline reference with the same query vector, corpus snapshot, distance function and complete eligibility predicate. Use an explicitly exhaustive path and inspect its plan. Calling the same approximate endpoint with a larger budget is not automatically an exact reference.

Run the raw-neighbour comparison without optional score thresholds or document grouping, then test those transformations separately against their declared rules. For each nonempty eligible set, let the reference contain the nearest minimum of k and the eligible count. Compare the approximate result's IDs with those reference IDs. The overlap divided by reference size measures filtered nearest-neighbour recall. Decide in advance how distance ties are handled so equally valid neighbours are not counted as unexplained regressions.

Measure semantic relevance separately with labelled evidence. A candidate can achieve perfect neighbour recall while missing the policy exception a reader needs. Measure result fullness separately too: returning k eligible records does not prove they are the right k. For an empty eligible set, report the case separately rather than assigning an arbitrary recall score.

Each evaluation record should retain:

• The query and embedding revision, with permitted storage or a recoverable fixture.
• The corpus snapshot and exact predicate, including identity and validity context.
• Eligible count, requested output depth and reference IDs.
• Returned IDs, distance scores and downstream deduplication decisions.
• Engine version, index configuration and observed execution plan.
• Search limits, completion status, latency and resource measurements.

Keep the embedding and index release fixed while comparing filter strategies. If the encoder also changes, use the separate embedding-migration procedure; otherwise two different coordinate systems confound the experiment.

Give fallback a budget, not wider permissions

A short result list can trigger more exploration, exact scoring or a separately evaluated lexical path. None of those transitions should remove a tenant boundary, broaden an access predicate or silently include expired material.

Treat mandatory restrictions differently from optional preferences. A user might explicitly accept a wider product category, but a search service must not reinterpret an access rule as a suggestion because results are sparse. Derive security predicates from trusted identity state, not solely from text generated by a model.

Apply the same restrictions to every branch of a hybrid query and to any fallback. Check the assembled result after merging, while preventing restricted payloads from entering downstream components in the first place. A final check is defence in depth, not permission to send forbidden text to a reranker earlier.

Bound total effort across retries. If the first search consumes most of the request deadline, an expensive exact fallback may only convert a partial result into a timeout. Prefer an explicit incomplete-search outcome when the budget cannot support the next plan. The response should not assert that no evidence exists merely because the search stopped.

Nor can reranking recover an item absent from its input set. Keep the candidate-discovery budget separate from the model's context working-set budget: finding more eligible candidates need not mean forwarding all of them into the prompt.

Watch for silent misses after release

Track filtered reference recall on a controlled sample, relevant-evidence retrieval on labelled cases, and short-result rates conditional on enough eligible records existing. Pair those with p95 and p99 latency, timeout frequency, exact-fallback frequency and resource use by query class.

Where the engine does not expose eligible counts or termination reasons, mark the gap. Use controlled diagnostic queries to establish what can be observed; do not manufacture a confidence signal from the number of returned records.

Inspect disagreement traces before changing the model. Did the predicate exclude the document correctly? Did the index contain its current vector? Did candidate truncation remove it? Did search stop early? Did deduplication or reranking discard it afterwards? Each answer points to a different repair.

Use read-only shadow comparisons to test representative filter combinations under real concurrency without running downstream actions twice. Reopen the evidence after index rebuilds, engine upgrades, shard changes, major corpus growth or changes in access-group structure.

The operating promise should be modest and testable: this plan retrieves the required eligible evidence within the declared budget, and reports uncertainty when it cannot complete. “No results” is an observation about a particular search. It becomes a claim about absence only when the search scope and completeness justify it.