Vector Databases in .NET: Semantic Search with Qdrant and pgvector

What is a Vector Database?

Traditional databases answer exact queries: WHERE name = 'Widget'. Vector databases answer semantic queries: "find products similar to what I described".

Every piece of content — text, image, code — can be converted to a high-dimensional vector (embedding) using an AI model. Similar content produces similar vectors. A vector database indexes these vectors and finds the nearest neighbours fast.

"lightweight running shoes"    → [0.21, -0.45, 0.88, ...]  (1536 dimensions)
"breathable marathon trainers" → [0.19, -0.47, 0.85, ...]  ← similar!
"winter boots"                 → [-0.34, 0.12, -0.67, ...]  ← different

Use cases:

Semantic product search ("I need something for hiking")
RAG (Retrieval-Augmented Generation) — find relevant docs for an LLM
Duplicate detection — find similar support tickets
Recommendation — "users who bought X also liked Y"
Code search — find functions that do similar things

Generating Embeddings

// Using Azure OpenAI embedding model
builder.Services.AddAzureOpenAITextEmbeddingGeneration(
    deploymentName: "text-embedding-3-small",
    endpoint:       builder.Configuration["AzureOpenAI:Endpoint"]!,
    apiKey:         builder.Configuration["AzureOpenAI:ApiKey"]!);

// Or OpenAI directly
builder.Services.AddOpenAITextEmbeddingGeneration(
    modelId: "text-embedding-3-small",
    apiKey:  builder.Configuration["OpenAI:ApiKey"]!);

// Generate embeddings
public class EmbeddingService
{
    private readonly ITextEmbeddingGenerationService _embedder;

    public EmbeddingService(ITextEmbeddingGenerationService embedder) => _embedder = embedder;

    public async Task<ReadOnlyMemory<float>> EmbedAsync(string text, CancellationToken ct)
        => await _embedder.GenerateEmbeddingAsync(text, cancellationToken: ct);

    public async Task<IList<ReadOnlyMemory<float>>> EmbedBatchAsync(
        IList<string> texts, CancellationToken ct)
        => await _embedder.GenerateEmbeddingsAsync(texts, cancellationToken: ct);
}

Qdrant (Dedicated Vector Database)

Qdrant is a purpose-built vector database with filtering, payload storage, and multiple index types.

Bash

# Run locally
docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant

dotnet add package Qdrant.Client
dotnet add package Microsoft.SemanticKernel.Connectors.Qdrant

Create Collection and Index

using Qdrant.Client;
using Qdrant.Client.Grpc;

var qdrant = new QdrantClient("localhost", 6334);

// Create collection
await qdrant.CreateCollectionAsync("products", new VectorParams
{
    Size     = 1536,           // must match embedding model output size
    Distance = Distance.Cosine
});

// Create payload index for fast filtering
await qdrant.CreatePayloadIndexAsync(
    "products",
    "category",
    PayloadSchemaType.Keyword);

await qdrant.CreatePayloadIndexAsync(
    "products",
    "price",
    PayloadSchemaType.Float);

Upsert Vectors

public class ProductVectorRepository
{
    private readonly QdrantClient _qdrant;
    private readonly EmbeddingService _embedder;

    public async Task UpsertProductAsync(Product product, CancellationToken ct)
    {
        // Create embedding from product text
        var text = $"{product.Name}. {product.Description}. Category: {product.Category}";
        var embedding = await _embedder.EmbedAsync(text, ct);

        await _qdrant.UpsertAsync("products", new[]
        {
            new PointStruct
            {
                Id      = new PointId { Uuid = product.Id.ToString() },
                Vectors = embedding.ToArray(),
                Payload =
                {
                    ["name"]     = product.Name,
                    ["category"] = product.Category,
                    ["price"]    = product.Price,
                    ["sku"]      = product.Sku,
                    ["inStock"]  = product.StockQuantity > 0
                }
            }
        }, cancellationToken: ct);
    }

    public async Task DeleteProductAsync(Guid productId, CancellationToken ct)
    {
        await _qdrant.DeleteAsync("products",
            new PointId { Uuid = productId.ToString() },
            cancellationToken: ct);
    }
}

Semantic Search

public async Task<List<ProductSearchResult>> SearchAsync(
    string query,
    string? categoryFilter = null,
    decimal? maxPrice      = null,
    bool onlyInStock       = false,
    int limit              = 10,
    CancellationToken ct   = default)
{
    var queryVector = await _embedder.EmbedAsync(query, ct);

    // Build filter
    Filter? filter = null;
    var conditions = new List<Condition>();

    if (categoryFilter is not null)
        conditions.Add(new Condition
        {
            Field = new FieldCondition
            {
                Key   = "category",
                Match = new Match { Keyword = categoryFilter }
            }
        });

    if (maxPrice.HasValue)
        conditions.Add(new Condition
        {
            Field = new FieldCondition
            {
                Key   = "price",
                Range = new Range { Lte = (double)maxPrice.Value }
            }
        });

    if (onlyInStock)
        conditions.Add(new Condition
        {
            Field = new FieldCondition
            {
                Key   = "inStock",
                Match = new Match { Boolean = true }
            }
        });

    if (conditions.Any())
        filter = new Filter { Must = { conditions } };

    var results = await _qdrant.SearchAsync(
        "products",
        queryVector.ToArray(),
        filter:     filter,
        limit:      (ulong)limit,
        withPayload: true,
        cancellationToken: ct);

    return results.Select(r => new ProductSearchResult(
        Id:       Guid.Parse(r.Id.Uuid),
        Name:     r.Payload["name"].StringValue,
        Category: r.Payload["category"].StringValue,
        Price:    (decimal)r.Payload["price"].DoubleValue,
        Score:    r.Score)).ToList();
}

pgvector (PostgreSQL Extension)

If you're already on PostgreSQL, pgvector adds vector similarity search without a new database.

Bash

dotnet add package Pgvector
dotnet add package Npgsql.EntityFrameworkCore.PostgreSQL

SQL

-- Enable the extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Table with vector column
CREATE TABLE products (
    id          UUID PRIMARY KEY,
    name        TEXT NOT NULL,
    category    TEXT,
    price       DECIMAL(10,2),
    description TEXT,
    embedding   vector(1536)        -- 1536 dimensions for text-embedding-3-small
);

-- HNSW index for fast approximate search
CREATE INDEX ON products USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

// EF Core entity
public class ProductEntity
{
    public Guid    Id          { get; set; }
    public string  Name        { get; set; } = "";
    public string  Category    { get; set; } = "";
    public decimal Price       { get; set; }
    public Vector? Embedding   { get; set; }  // Pgvector.Vector type
}

// Configure
modelBuilder.Entity<ProductEntity>()
    .HasIndex(p => p.Embedding)
    .HasMethod("hnsw")
    .HasOperators("vector_cosine_ops");

// Search with EF Core
public async Task<List<ProductEntity>> SearchAsync(float[] queryVector, int limit = 10)
{
    var vector = new Vector(queryVector);

    return await _db.Products
        .OrderBy(p => p.Embedding!.CosineDistance(vector))
        .Take(limit)
        .ToListAsync();
}

Hybrid Search (Vector + Keyword)

Pure semantic search misses exact keyword matches. Hybrid search combines both using Reciprocal Rank Fusion (RRF):

public async Task<List<ProductSearchResult>> HybridSearchAsync(
    string query,
    int limit = 10,
    CancellationToken ct = default)
{
    // Run both in parallel
    var vectorTask  = VectorSearchAsync(query, limit * 2, ct);
    var keywordTask = KeywordSearchAsync(query, limit * 2, ct);

    await Task.WhenAll(vectorTask, keywordTask);

    var vectorResults  = vectorTask.Result;
    var keywordResults = keywordTask.Result;

    // Reciprocal Rank Fusion
    var scores = new Dictionary<Guid, double>();
    const int k = 60;

    for (int i = 0; i < vectorResults.Count; i++)
        scores[vectorResults[i].Id] = scores.GetValueOrDefault(vectorResults[i].Id) + 1.0 / (k + i + 1);

    for (int i = 0; i < keywordResults.Count; i++)
        scores[keywordResults[i].Id] = scores.GetValueOrDefault(keywordResults[i].Id) + 1.0 / (k + i + 1);

    // Return top results by combined score
    return scores
        .OrderByDescending(kv => kv.Value)
        .Take(limit)
        .Select(kv => vectorResults.Concat(keywordResults)
            .First(r => r.Id == kv.Key) with { Score = (float)kv.Value })
        .ToList();
}

Indexing Strategy

| Algorithm | Speed | Accuracy | Memory | Use when | |---|---|---|---|---| | Flat (brute force) | Slow | Exact | Low | < 100K vectors | | HNSW | Fast | ~99% | High | Most production use cases | | IVF | Medium | ~95% | Medium | Very large collections (>1M) |

For most .NET applications, HNSW is the right choice.

Production Patterns

Batch indexing — embed and index in bulk during off-peak:

var batchSize = 100;
var products  = await _db.Products.Where(p => p.EmbeddingUpdatedAt < p.UpdatedAt).ToListAsync();

foreach (var batch in products.Chunk(batchSize))
{
    var texts    = batch.Select(p => $"{p.Name} {p.Description}").ToList();
    var embeddings = await _embedder.EmbedBatchAsync(texts, ct);

    await _qdrant.UpsertAsync("products",
        batch.Zip(embeddings).Select(/* create points */).ToList(), ct);
}

Cache embeddings — embedding an identical string twice is wasteful:

var cacheKey = $"embedding:{SHA256.HashData(Encoding.UTF8.GetBytes(text)).ToHexString()}";
if (!_cache.TryGetValue(cacheKey, out float[]? cached))
{
    var embedding = await _embedder.EmbedAsync(text, ct);
    _cache.Set(cacheKey, embedding.ToArray(), TimeSpan.FromHours(24));
    return embedding;
}
return cached!;

Interview Questions

Q: What is an embedding and how does it enable semantic search? An embedding is a fixed-length vector of numbers that represents the semantic meaning of text. Similar texts produce similar vectors (close in high-dimensional space). A vector database finds the nearest neighbours to a query vector — returning semantically similar content, not just keyword matches.

Q: What is the difference between Qdrant and pgvector? Qdrant is a purpose-built vector database optimised for high-dimensional search with advanced filtering, multiple vector types, and horizontal scalability. pgvector is a PostgreSQL extension adding vector types and indexes to a familiar relational DB. Use pgvector if you're already on Postgres and have moderate scale; use Qdrant for dedicated vector workloads, very large collections, or when you need advanced features.

Q: What is hybrid search and why is it often better than pure vector search? Hybrid search combines vector similarity (semantic) and keyword search (lexical), merging results using Reciprocal Rank Fusion or similar. Pure vector search can miss exact keyword matches (a product called "iPhone 15 Pro" won't always rank top for the query "iPhone 15 Pro"). Hybrid search handles both cases.

Q: What is HNSW and why is it the default vector index? Hierarchical Navigable Small World — a graph-based approximate nearest neighbour index. It builds a multi-layer graph where higher layers are sparser, enabling fast traversal. It achieves ~99% recall at 10-100x the speed of brute-force search, with logarithmic query complexity. The tradeoff is higher memory usage and build time vs flat indexes.