OrderFlow: Adding AI Features — Semantic Search, Summaries, and a Support Chatbot

This is part 7 of the OrderFlow series. The full backend is built and tested. Now we add three AI-powered features that transform OrderFlow from a standard CRUD API into a genuinely intelligent system.

What we're adding:

Semantic product search (pgvector)
AI-generated order summaries with streaming
Customer support chatbot with tools and conversation history

Step 1: Setup AI Services

// Program.cs
builder.Services.AddChatClient(services =>
    new OpenAIClient(builder.Configuration["OpenAI:ApiKey"]!)
        .AsChatClient("gpt-4o"))
    .UseFunctionInvocation()
    .UseLogging()
    .UseDistributedCache();

builder.Services.AddEmbeddingGenerator<string, Embedding<float>>(services =>
    new OpenAIClient(builder.Configuration["OpenAI:ApiKey"]!)
        .AsEmbeddingGenerator("text-embedding-3-small"));

XML

<!-- Enable pgvector -->
<PackageReference Include="Pgvector.EntityFrameworkCore" Version="0.*" />

Feature 1: Semantic Product Search

Update the Product Entity

// src/OrderFlow.Core/Entities/Product.cs
public class Product
{
    public int      Id          { get; set; }
    public string   Name        { get; set; } = "";
    public string   Description { get; set; } = "";
    public string   Category    { get; set; } = "";
    public decimal  Price       { get; set; }
    public int      StockLevel  { get; set; }
    public bool     IsActive    { get; set; }

    // AI embedding — null until generated
    public Vector?  Embedding   { get; set; }
}

// DbContext — add vector column + HNSW index
model.HasPostgresExtension("vector");

model.Entity<Product>(e =>
{
    e.Property(p => p.Embedding).HasColumnType("vector(1536)");
    e.HasIndex(p => p.Embedding)
     .HasMethod("hnsw")
     .HasOperators("vector_cosine_ops");
});

Embedding Generation

// src/OrderFlow.Application/Products/Commands/GenerateProductEmbeddingsCommand.cs
public class GenerateProductEmbeddingsCommandHandler(
    OrderFlowDbContext db,
    IEmbeddingGenerator<string, Embedding<float>> embedder)
    : IRequestHandler<GenerateProductEmbeddingsCommand, int>
{
    public async Task<int> Handle(
        GenerateProductEmbeddingsCommand cmd,
        CancellationToken ct)
    {
        var products = await db.Products
            .Where(p => p.IsActive && p.Embedding == null)
            .Take(100)   // batch
            .ToListAsync(ct);

        if (products.Count == 0) return 0;

        // Build rich text representations for embedding
        var texts = products.Select(p =>
            $"{p.Name}. {p.Description}. Category: {p.Category}. Price: {p.Price:C}.").ToList();

        var results = await embedder.GenerateAsync(texts, cancellationToken: ct);

        for (int i = 0; i < products.Count; i++)
            products[i].Embedding = new Vector(results[i].Vector.ToArray());

        await db.SaveChangesAsync(ct);
        return products.Count;
    }
}

// Background job to keep embeddings current
public class EmbeddingGenerationJob(ISender mediator, ILogger<EmbeddingGenerationJob> logger)
    : IHostedService
{
    public async Task StartAsync(CancellationToken ct)
    {
        // Run once at startup; Quartz/Hangfire would handle scheduling in production
        var count = await mediator.Send(new GenerateProductEmbeddingsCommand(), ct);
        logger.LogInformation("Generated {Count} product embeddings on startup", count);
    }

    public Task StopAsync(CancellationToken ct) => Task.CompletedTask;
}

Semantic Search Query

public class SemanticProductSearchQueryHandler(
    OrderFlowDbContext db,
    IEmbeddingGenerator<string, Embedding<float>> embedder)
    : IRequestHandler<SemanticProductSearchQuery, List<ProductSearchResult>>
{
    public async Task<List<ProductSearchResult>> Handle(
        SemanticProductSearchQuery query,
        CancellationToken ct)
    {
        // Embed the user's search query
        var result      = await embedder.GenerateAsync([query.SearchText], cancellationToken: ct);
        var queryVector = new Vector(result[0].Vector.ToArray());

        // Vector similarity search
        var products = await db.Products
            .Where(p => p.IsActive && p.Embedding != null)
            .OrderBy(p => p.Embedding!.CosineDistance(queryVector))
            .Take(query.TopK)
            .Select(p => new ProductSearchResult(
                p.Id,
                p.Name,
                p.Category,
                p.Price,
                p.StockLevel,
                // Similarity score 0-1 (1 = identical)
                1 - p.Embedding!.CosineDistance(queryVector)))
            .ToListAsync(ct);

        // Filter out results below relevance threshold
        return products.Where(p => p.Similarity > 0.7).ToList();
    }
}

public record SemanticProductSearchQuery(string SearchText, int TopK = 10)
    : IRequest<List<ProductSearchResult>>;

public record ProductSearchResult(
    int    Id,
    string Name,
    string Category,
    decimal Price,
    int    StockLevel,
    double Similarity);

// Endpoint
app.MapGet("/api/products/search", async (
    string q,
    ISender mediator,
    CancellationToken ct) =>
{
    var results = await mediator.Send(new SemanticProductSearchQuery(q), ct);
    return Results.Ok(results);
})
.RequireAuthorization()
.WithName("SemanticProductSearch");

Feature 2: AI Order Summary with Streaming

// Streaming endpoint — tokens arrive in real time
app.MapGet("/api/orders/{id}/ai-summary", async (
    int id,
    ClaimsPrincipal user,
    ISender mediator,
    IChatClient chatClient,
    HttpContext ctx,
    CancellationToken ct) =>
{
    // Load the order
    var order = await mediator.Send(
        new GetOrderByIdQuery(id, user.GetUserId(), user.GetRole()), ct);

    if (order is null)
    {
        ctx.Response.StatusCode = 404;
        return;
    }

    // Stream the AI summary
    ctx.Response.Headers.ContentType  = "text/event-stream";
    ctx.Response.Headers.CacheControl = "no-cache";

    var prompt = $"""
        You are an OrderFlow customer service assistant.
        Write a friendly, concise summary of this order for the customer.
        Include status, items ordered, total, and any relevant next steps.

        Order details:
        - Order #{order.Id}
        - Status: {order.Status}
        - Total: {order.Total:C}
        - Items: {string.Join(", ", order.Lines.Select(l => $"{l.Quantity}x {l.ProductName}"))}
        - Placed: {order.CreatedAt:MMM d, yyyy}
        """;

    try
    {
        await foreach (var update in chatClient.CompleteStreamingAsync(
            [new ChatMessage(ChatRole.User, prompt)],
            cancellationToken: ct))
        {
            if (update.Text is { Length: > 0 } text)
            {
                await ctx.Response.WriteAsync(
                    $"data: {JsonSerializer.Serialize(text)}\n\n", ct);
                await ctx.Response.Body.FlushAsync(ct);
            }
        }

        await ctx.Response.WriteAsync("data: [DONE]\n\n", ct);
        await ctx.Response.Body.FlushAsync(ct);
    }
    catch (OperationCanceledException)
    {
        // Client disconnected — normal
    }
})
.RequireAuthorization();

Feature 3: Customer Support Chatbot

// src/OrderFlow.Application/Support/SupportChatService.cs
public class SupportChatService(
    IChatClient chatClient,
    IOrderRepository orders,
    IProductRepository products)
{
    // Tools available to the chatbot
    [Description("Get the status of a customer's order")]
    private async Task<string> GetOrderStatus(
        [Description("The order ID")] int orderId,
        [Description("The customer ID making the request")] int customerId,
        CancellationToken ct = default)
    {
        var order = await orders.GetByIdAsync(orderId, ct);
        if (order is null) return $"Order {orderId} not found.";
        if (order.CustomerId != customerId)
            return "You are not authorised to view this order.";

        return $"Order {orderId}: Status={order.Status}, Total={order.Total:C}, " +
               $"Items={order.Lines.Count}, Created={order.CreatedAt:MMM d}";
    }

    [Description("Search products for the customer")]
    private async Task<string> SearchProducts(
        [Description("What the customer is looking for")] string query,
        CancellationToken ct = default)
    {
        var found = await products.SearchAsync(query, 5, ct);
        if (!found.Any()) return "No products found matching that description.";

        return string.Join("\n", found.Select(p =>
            $"- {p.Name} ({p.Category}): {p.Price:C} — {(p.StockLevel > 0 ? "In stock" : "Out of stock")}"));
    }

    public async IAsyncEnumerable<string> ChatAsync(
        int customerId,
        List<ChatMessage> conversationHistory,
        string userMessage,
        [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken ct = default)
    {
        var messages = new List<ChatMessage>
        {
            new(ChatRole.System, $"""
                You are a helpful customer service agent for OrderFlow.
                You are assisting customer ID {customerId}.
                Be concise, friendly, and always verify the customer owns any order they ask about.
                Use tools to look up real-time order status and product availability.
                """),
        };

        messages.AddRange(conversationHistory);
        messages.Add(new ChatMessage(ChatRole.User, userMessage));

        var options = new ChatOptions
        {
            Tools =
            [
                AIFunctionFactory.Create(
                    (int orderId, CancellationToken innerCt) =>
                        GetOrderStatus(orderId, customerId, innerCt)),
                AIFunctionFactory.Create(
                    (string query, CancellationToken innerCt) =>
                        SearchProducts(query, innerCt)),
            ],
            ToolMode = ChatToolMode.Auto,
        };

        await foreach (var update in chatClient.CompleteStreamingAsync(messages, options, ct))
        {
            if (update.Text is { Length: > 0 } text)
                yield return text;
        }
    }
}

// Conversation session storage in Redis
public class ConversationStore(IDistributedCache cache)
{
    private static readonly TimeSpan SessionTtl = TimeSpan.FromHours(1);

    public async Task<List<ChatMessage>> GetHistoryAsync(string sessionId, CancellationToken ct)
    {
        var json = await cache.GetStringAsync($"chat:{sessionId}", ct);
        return json is null
            ? []
            : JsonSerializer.Deserialize<List<ChatMessageDto>>(json)!
                .Select(d => new ChatMessage(
                    d.Role == "assistant" ? ChatRole.Assistant : ChatRole.User,
                    d.Content))
                .ToList();
    }

    public async Task AppendAsync(
        string sessionId, string role, string content, CancellationToken ct)
    {
        var history = await GetHistoryAsync(sessionId, ct);
        history.Add(new ChatMessage(
            role == "assistant" ? ChatRole.Assistant : ChatRole.User,
            content));

        // Keep last 20 messages (token management)
        if (history.Count > 20)
            history.RemoveRange(0, history.Count - 20);

        await cache.SetStringAsync(
            $"chat:{sessionId}",
            JsonSerializer.Serialize(history.Select(m => new ChatMessageDto(
                m.Role == ChatRole.Assistant ? "assistant" : "user",
                m.Text ?? ""))),
            new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = SessionTtl },
            ct);
    }
}

public record ChatMessageDto(string Role, string Content);

// Streaming chatbot endpoint
app.MapPost("/api/support/chat", async (
    SupportChatRequest req,
    ClaimsPrincipal user,
    SupportChatService chat,
    ConversationStore conversations,
    HttpContext ctx,
    CancellationToken ct) =>
{
    var customerId = user.GetUserId();
    var sessionId  = req.SessionId ?? Guid.NewGuid().ToString();
    var history    = await conversations.GetHistoryAsync(sessionId, ct);

    ctx.Response.Headers.ContentType  = "text/event-stream";
    ctx.Response.Headers.CacheControl = "no-cache";

    // Send session ID so client can persist it
    await ctx.Response.WriteAsync(
        $"data: {JsonSerializer.Serialize(new { sessionId })}\n\n", ct);
    await ctx.Response.Body.FlushAsync(ct);

    var accumulated = new System.Text.StringBuilder();

    try
    {
        await foreach (var token in chat.ChatAsync(customerId, history, req.Message, ct))
        {
            accumulated.Append(token);
            await ctx.Response.WriteAsync(
                $"data: {JsonSerializer.Serialize(new { token })}\n\n", ct);
            await ctx.Response.Body.FlushAsync(ct);
        }
    }
    catch (OperationCanceledException) { }

    // Save both turns to conversation history
    await conversations.AppendAsync(sessionId, "user",      req.Message,             ct);
    await conversations.AppendAsync(sessionId, "assistant", accumulated.ToString(),   ct);

    await ctx.Response.WriteAsync("data: [DONE]\n\n", ct);
    await ctx.Response.Body.FlushAsync(ct);
})
.RequireAuthorization();

public record SupportChatRequest(string Message, string? SessionId = null);

What OrderFlow Can Now Do

Semantic search:
  GET /api/products/search?q=wireless noise cancelling headphones
  → returns headphones, earbuds, speakers sorted by semantic relevance
  → works even if the user said "earphones" and products say "earbuds"

AI summary:
  GET /api/orders/42/ai-summary
  → "Your order #42 for 2x Widget and 1x Gadget totalling £44.97 is currently
     Pending. It was placed on 25 May and should be dispatched within 1-2
     business days. You'll receive an email when it ships."

Chatbot:
  POST /api/support/chat
  → { "message": "Where is my order 42?", "sessionId": "abc123" }
  → Streams: "I can check that for you! Let me look up order 42..."
     [tool call: GetOrderStatus(42)]
  → Streams: "Order 42 is currently Paid and awaiting dispatch. Expected..."

OrderFlow: Complete

You've now built a production-ready .NET API with:

Clean Architecture (Core / Application / Infrastructure / API layers)
JWT authentication with refresh token rotation
CQRS with MediatR and validation pipeline
Domain events with the Outbox Pattern
Redis caching with automatic invalidation
A full test suite (unit + integration + API)
AI-powered semantic search, streaming summaries, and a support chatbot

Continue to the AI project guides for standalone AI application walkthroughs.

OrderFlow: Adding AI Features — Semantic Search, Summaries, and a Support Chatbot

OrderFlow: Adding AI Features — Semantic Search, Summaries, and a Support Chatbot

Step 1: Setup AI Services

Feature 1: Semantic Product Search

Update the Product Entity

Embedding Generation

Semantic Search Query

Feature 2: AI Order Summary with Streaming

Feature 3: Customer Support Chatbot

What OrderFlow Can Now Do

OrderFlow: Complete

Enjoyed this article?

Leave a comment