TestForge Blog
← All Posts
#ai #agent #llm #service #backend

AI Agent Service Design Patterns — Tool Calling, State Management, and Guardrails

A practical guide to turning AI Agents into real services. Covers Tool Calling, Planner/Executor separation, session state management, human-in-the-loop workflows, failure handling, and cost control.

TestForge Team ·

Demo Agents and Production Agents Are Different

An Agent demo only needs to work once. A production Agent needs to be stable, observable, and cost-aware.

The problems that appear during service rollout are usually these:

  • Too many tool calls
  • Inconsistent results for similar questions
  • Broken session state and wrong context reuse
  • External API failures causing full response failures
  • Rapidly growing inference cost

That is why Agent design is more about system structure than prompt wording.

A Practical Baseline Architecture

In production, it helps to separate responsibilities like this:

User Request
 -> Router
 -> Planner
 -> Tool Executor
 -> Memory / State Store
 -> LLM Response Composer
 -> Output Guardrail

Trying to make a single model call do everything usually makes debugging and reliability much worse.

Tool Calling Works Best When It Is Explicit

More tools do not automatically make an Agent better.

Good tools have these properties:

  • Clear names
  • Explicit input schemas
  • Stable output shapes
  • Well-defined failure behavior

Example:

{
  "name": "search_knowledge_base",
  "description": "Search internal technical documents",
  "input_schema": {
    "type": "object",
    "properties": {
      "query": { "type": "string" },
      "top_k": { "type": "integer", "minimum": 1, "maximum": 10 }
    },
    "required": ["query"]
  }
}

If tool definitions are vague, model behavior becomes unstable very quickly.

Should Planner and Executor Be Separate?

For simple FAQ-style Agents, maybe not.

For multi-step automation, separating them is usually worth it:

  • Planner decides what should happen
  • Executor actually calls tools

Benefits:

  • Easier trace inspection
  • Better retry control
  • Clearer permission boundaries
  • Better control over unnecessary repeated calls

State Management Matters More Than It Looks

Most Agent services need three levels of state:

1. Request State

Short-lived state for one request:

  • User input
  • Intermediate tool results
  • Temporary reasoning artifacts

2. Session State

State shared during a conversation:

  • Conversation history
  • User preferences
  • Recent task context

3. Long-term Memory

Persistent reusable information:

  • User profile
  • Repeated workflows
  • Previously solved cases

Saving every message forever is rarely a good default. Structured memory is usually cheaper and safer.

Human-in-the-Loop Is a Product Feature

Agents should not autonomously execute every action.

A human approval step is especially useful for:

  • Deployments
  • Permission changes
  • Payments or refunds
  • Data deletion
  • Customer-facing announcements

A safe flow looks like this:

1. Agent prepares a plan
2. User sees summary and impact
3. User approves
4. System executes and logs the result

That approval step often reduces risk far more than another prompt tweak ever will.

Failure Handling Must Be Designed Up Front

Agent systems break whenever one dependency breaks.

So every serious Agent service needs:

  • Tool-level timeouts
  • Retry limits
  • Fallback responses
  • Partial failure rules
  • Circuit breakers

Examples:

  • If document search fails, respond with limited confidence instead of pretending certainty
  • If an external API fails, use cache or explicitly ask the user to retry

Cost Control Is Part of the Architecture

The more useful an Agent becomes, the easier it is for cost to spiral.

Practical control levers:

  • Route simple questions to smaller models
  • Compress long history aggressively
  • Cap tool call count
  • Cache repeated questions
  • Limit response size

Without cost control, the system can become operationally unsustainable even if it works technically.

Observability Is Essential

In production, you need to answer: why did the Agent behave like this?

Useful logs include:

  • User input
  • Routing decision
  • Model and token usage
  • Tool call sequence
  • Intermediate state transitions
  • Final response
  • Error and fallback events

Trace-based logging is especially valuable for multi-step Agents.

Guardrails Should Exist at Three Layers

Input Layer

  • Prompt injection detection
  • Sensitive data masking
  • Content policy filtering

Execution Layer

  • Allow-listed tools only
  • Approval for sensitive actions
  • Restricted outbound domains

Output Layer

  • Unsafe response blocking
  • Strong claims softened when evidence is weak
  • Structured output validation

A good first production Agent often looks like this:

  • Single-purpose Agent
  • 2 to 3 core tools
  • Minimal session memory
  • Explicit approval step
  • Detailed execution logs

You usually get farther by making one Agent reliable than by building a complicated multi-Agent system too early.

Closing Thoughts

The core of Agent service design is not giving the system maximum autonomy. It is deciding where autonomy should stop.

Good Agent systems have:

  • Clear tool boundaries
  • Traceable state
  • Safe failure behavior
  • Human approval where risk is real
  • Ongoing quality and cost control

That is what turns an Agent from a demo into a dependable service.