What is a vector database?

You can compute distances between a query vector and ten document vectors in a short loop. Why would you need a vector database? The answer includes scale, but also the less visible work of storing, filtering, updating, and retrieving the right records.

Before you begin: Understand embeddings and the RAG evidence flow.

Name the record

A searchable record usually includes an ID, a vector, source text or a reference to it, and metadata. Metadata might identify the document, version, language, collection, and access scope. The vector is a learned representation used for ranking; it is not a replacement for the source text.

A vector index is a data structure for similarity search. A vector database or database extension adds a broader set of storage and operational features. Faiss, for example, is a similarity-search library; it should not be described as providing every database capability by itself.

Traditional databases are not inherently unable to work with vectors. Some offer vector extensions or built-in vector search. The choice is whether the complete system meets your retrieval and operational requirements.

What else must travel with a vector?

Establish exact search first

For a small collection, calculate a distance to every eligible vector and sort the results. This gives an exact baseline under the chosen metric. It helps you measure the quality lost by an approximate index later.

This complete Python 3 program uses invented coordinates and filters the collection before ranking. The owner argument represents an identity already verified by the application, not a user-asserted permission claim.

# Runnable: Python 3, standard library.
records = [
    {'id': 'a', 'owner': 'u1', 'vector': [1.0, 0.0]},
    {'id': 'b', 'owner': 'u1', 'vector': [0.0, 1.0]},
    {'id': 'private', 'owner': 'u2', 'vector': [0.99, 0.01]},
]

def search(query, owner, k=2):
    if len(query) != 2 or k <= 0:
        raise ValueError('Use a two-coordinate query and positive k')
    eligible = [r for r in records if r['owner'] == owner]
    ranked = sorted(eligible, key=lambda r: (
        sum((x-y)**2 for x, y in zip(query, r['vector'])), r['id']))
    return [r['id'] for r in ranked[:k]]

assert search([1, 0], 'u1') == ['a', 'b']
assert search([1, 0], 'missing') == []
print(search([1, 0], 'u1'))

The nearest vector overall could belong to another user. Similarity never grants access. In a real service, apply authorization in the trusted retrieval layer and test the database's filter semantics.

Understand approximate search

Approximate nearest-neighbor methods trade some exactness for speed or memory efficiency. Graph-based indexes and quantized indexes use different structures and tuning parameters. Measure their recall against the exact baseline on representative queries.

Recall at k for an approximate index can mean how many exact top-k neighbors it recovers. That is different from task retrieval recall, which measures whether human-judged relevant evidence was found. A system can perfectly reproduce an embedding ranking that is poor for the task.

Test filters and updates

Some retrieval implementations apply filters before, during, or after candidate generation. Post-filtering a small candidate set can return too few eligible results even when relevant permitted records exist. Check behavior under selective filters, not only on an unfiltered benchmark.

Test insert, update, deletion, and index rebuild behavior. Record which embedding model and preprocessing produced the vectors. A new model generally requires re-embedding the collection and a controlled migration rather than mixing incompatible vector spaces.

Choose by workload

Measure collection size, query rate, latency distribution, filtered retrieval quality, memory, durability, and operational complexity. Do not choose from invented vendor speed tables. A database already used by your application may be adequate; a dedicated vector service may fit a different workload better.

Practice with feedback

import numpy as np

def exact_search(qv, chunks, k, allowed):
    pool = [c for c in chunks if c.visibility in allowed]   # filter first
    scored = sorted(pool, key=lambda c: -float(np.dot(qv, c.vector)))
    return scored[:k]

def upsert_document(store, doc_id, new_chunks):
    store.delete(where={'doc_id': doc_id})   # remove old chunks first
    store.add(new_chunks)                    # otherwise last week's text
                                             # keeps being retrieved forever

def delete_document(store, doc_id):
    store.delete(where={'doc_id': doc_id})
    assert not store.query(where={'doc_id': doc_id}), 'chunks survived delete'

def ann_recall(index, chunks, queries, k=10):
    hits = 0
    for qv in queries:
        exact = {c.id for c in exact_search(qv, chunks, k, {'public', 'members'})}
        approx = {c.id for c in index.search(qv, k)}
        hits += len(exact & approx) / k
    return hits / len(queries)

# On 200 sample queries my default HNSW settings gave recall 0.86. Raising
# ef_search from 40 to 128 took it to 0.97 for 11ms extra. I would not have
# known either number without the exact baseline, and 0.86 would have shown
# up much later as 'the assistant sometimes cannot find things'.

Next, we will calculate the similarity measures themselves and see why their score scales should not be mixed casually.

References

Faiss documents efficient similarity search and index choices. pgvector is an example of vector search integrated into an existing database system.