AI Agent Planning — Decomposing Goals into Actions

Without planning:
  User: "Review patient MRN-001's warfarin therapy and flag any concerns"
  Agent: calls one function, returns partial answer, stops

With planning:
  Agent decomposes the goal into steps:
    1. Look up patient MRN-001
    2. Get their active prescriptions — identify warfarin
    3. Get their INR history (last 3 readings)
    4. Get their last INR check date
    5. Check if INR is within therapeutic range (2.0–3.0)
    6. Check if INR check is current (within 7 days)
    7. Synthesise findings — flag any concerns

  The agent decides this sequence itself, based on the goal and available tools.
  This is goal-directed planning: reason → act → observe → reason again.

Planning enables agents to:
  → Decompose multi-step clinical goals into concrete tool calls
  → Handle conditional paths (if INR expired, flag urgency)
  → Synthesise data from multiple sources into a coherent assessment
  → Recover from missing data ("patient not found — stopping here")

// AutoInvokeKernelFunctions enables the agent to plan and execute multi-step sequences
// The agent calls tools in the order it decides — not a fixed sequence you write

public sealed class WarfarinReviewAgent
{
    private readonly Kernel                _kernel;
    private readonly IChatCompletionService _chat;

    public async Task<string> ReviewWarfarinTherapyAsync(
        string mrn,
        CancellationToken ct)
    {
        var history = new ChatHistory("""
            You are a clinical pharmacist assistant reviewing warfarin therapy.
            When asked to review a patient:
            1. Look up the patient to confirm they exist
            2. List their active prescriptions — identify warfarin if present
            3. Get detailed prescription info including INR history
            4. Assess whether INR is within the therapeutic range (2.0–3.0 for most indications)
            5. Flag any concerns: out-of-range INR, expired INR check, or no warfarin found
            Always state that the prescriber must make the final clinical decision.
            If the patient does not exist, stop and report that.
            """);

        history.AddUserMessage(
            $"Please review the warfarin therapy for patient {mrn} and flag any concerns.");

        var settings = new OpenAIPromptExecutionSettings
        {
            ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
            Temperature      = 0.1,   // low temperature for clinical reasoning
            MaxTokens        = 800
        };

        var response = await _chat.GetChatMessageContentAsync(
            history, settings, _kernel, ct);

        return response.Content ?? "No assessment generated.";
    }
}

// The agent's actual execution trace for MRN-001:
// → calls find_patient("MRN-001") → patient found
// → calls list_active_prescriptions("MRN-001") → [Warfarin 5mg, Ramipril 2.5mg]
// → calls get_prescription_detail(warfarin_id) → INR 1.7, checked 9 days ago
// → synthesises: "INR is sub-therapeutic (1.7, target 2.0–3.0). INR check is overdue
//    (9 days, should be within 7). Recommend urgent INR recheck and dose review."

// Agents naturally handle conditional paths through reasoning
// The system prompt defines the decision rules; tools provide the data

var systemPrompt = """
    You are a clinical discharge planning assistant.

    When reviewing a patient for discharge readiness:
    1. Get their current prescriptions
    2. For each anticoagulant (Warfarin, Apixaban, Rivaroxaban), check INR/coagulation status
    3. Check if any prescriptions are pending approval — discharge is blocked if any are pending
    4. Check if a GP follow-up has been scheduled using get_followup_appointment

    Decision rules:
    - If any anticoagulant INR is out of range: flag as NOT READY, explain why
    - If any prescription is pending approval: flag as BLOCKED, list the pending items
    - If no GP follow-up is scheduled: flag as INCOMPLETE, request scheduling
    - If all checks pass: confirm READY FOR DISCHARGE with a brief summary

    Always explain your reasoning for each check.
    """;

// The agent follows this conditional logic itself:
// → finds warfarin prescription
// → INR = 2.4 → within range → continue
// → finds no pending prescriptions → continue
// → calls get_followup_appointment → no appointment found
// → output: "INCOMPLETE: Discharge is not yet ready. No GP follow-up appointment
//    has been scheduled. Please arrange a follow-up within 7 days of discharge."

// For transparency and auditing, capture the agent's reasoning steps

public sealed class AuditedPlanningAgent
{
    private readonly Kernel                _kernel;
    private readonly IChatCompletionService _chat;
    private readonly ILogger               _logger;

    public async Task<AgentReviewResult> ReviewWithAuditTrailAsync(
        string patientMrn,
        CancellationToken ct)
    {
        var history = new ChatHistory(ClinicalSystemPrompt);
        history.AddUserMessage($"Review warfarin therapy for patient {patientMrn}.");

        var settings = new OpenAIPromptExecutionSettings
        {
            ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
            Temperature      = 0.1
        };

        // Log each function invocation for audit trail
        var functionCalls = new List<AgentFunctionCall>();
        _kernel.FunctionInvocationFilters.Add(new AuditFunctionFilter(functionCalls, _logger));

        var response = await _chat.GetChatMessageContentAsync(
            history, settings, _kernel, ct);

        return new AgentReviewResult(
            Assessment:   response.Content ?? string.Empty,
            FunctionCalls: functionCalls,
            ReviewedAt:   DateTime.UtcNow,
            PatientMrn:   patientMrn);
    }
}

public sealed class AuditFunctionFilter(
    List<AgentFunctionCall> calls,
    ILogger logger) : IFunctionInvocationFilter
{
    public async Task OnFunctionInvocationAsync(
        FunctionInvocationContext context,
        Func<FunctionInvocationContext, Task> next)
    {
        logger.LogInformation(
            "Agent calling: {Function} with args: {Args}",
            context.Function.Name,
            JsonSerializer.Serialize(context.Arguments));

        await next(context);

        var call = new AgentFunctionCall(
            FunctionName: context.Function.Name,
            Arguments:    JsonSerializer.Serialize(context.Arguments),
            Result:       context.Result?.ToString() ?? "null",
            CalledAt:     DateTime.UtcNow);

        calls.Add(call);
    }
}

public sealed record AgentFunctionCall(
    string   FunctionName,
    string   Arguments,
    string   Result,
    DateTime CalledAt);

public sealed record AgentReviewResult(
    string                   Assessment,
    IReadOnlyList<AgentFunctionCall> FunctionCalls,
    DateTime                 ReviewedAt,
    string                   PatientMrn);

// Clinical agents must not plan around safety gates
// Use system prompt constraints AND code-level guards

// System prompt constraints:
const string SafetyConstrainedPrompt = """
    You are a pharmacist assistant. You MUST follow these constraints:

    NEVER:
    - Make dosage recommendations without checking the current INR
    - Suggest stopping a medication without checking for interactions
    - Perform more than 10 tool calls per review (stop and ask for clarification)
    - Proceed if find_patient returns null (stop and report "patient not found")

    ALWAYS:
    - State that prescribers make final clinical decisions
    - Include the data source for any clinical values you report
    - Report your uncertainty when data is missing or ambiguous
    """;

// Code-level guard — limit maximum tool calls:
public sealed class MaxCallsFilter(int maxCalls) : IFunctionInvocationFilter
{
    private int _callCount;

    public async Task OnFunctionInvocationAsync(
        FunctionInvocationContext context,
        Func<FunctionInvocationContext, Task> next)
    {
        if (Interlocked.Increment(ref _callCount) > maxCalls)
            throw new InvalidOperationException(
                $"Agent exceeded maximum allowed tool calls ({maxCalls}). " +
                "Review the agent goal — it may be too broad.");

        await next(context);
    }
}

// Register:
_kernel.FunctionInvocationFilters.Add(new MaxCallsFilter(maxCalls: 10));

Common planning failures:

1. Infinite loops
   Agent calls tool A → result triggers calling A again → loop
   Fix: MaxCallsFilter + system prompt: "Do not call the same function
        twice with identical arguments"

2. Goal misinterpretation
   User: "Check MRN-001" — agent checks everything, takes 2 minutes
   Fix: Be specific in user messages; restrict tool set per agent

3. Hallucinated planning steps
   Agent claims it called a function but didn't — reports fabricated data
   Fix: AutoInvokeKernelFunctions (the framework calls real functions)
        Never use manual function-call patterns where agent writes results

4. Plan abandonment on first null
   Agent calls find_patient → null → stops with no output
   Fix: System prompt: "If patient not found, clearly state this and stop."
        Tool description: "Returns null if no patient exists with that MRN."

5. Over-planning for simple tasks
   User: "What is the INR for MRN-001?"
   Agent: calls find_patient, then list_prescriptions, then get_detail, then...
   Fix: Design focused agents with narrow tool sets for narrow tasks
        Don't give a single-question agent access to all clinical tools