Multi-turn banking session with 2 agents.

Comparative Analysis

Why the winner wins

• Lower realized cost: Achieves the same 100% pass rate at ~40% lower total cost than gpt-5-mini ($0.001419 vs $0.002354 across identical tests).
• Token efficiency: Uses ~28% fewer tokens (3,005 vs 4,180), indicating tighter reasoning and less verbose responses without sacrificing correctness.
• Faster execution: Consistently lower durations per turn, improving perceived latency in multi-turn sessions.

Notable patterns

• Equivalent tool correctness: Both agents correctly chained tools across a 3-turn session (get_balance → transfer → get_all_balances) with no retries or confusion.
• Verbosity differences: gpt-5-mini tended to add longer follow-ups and prompts, increasing token usage and cost despite identical outcomes.
• Stable session context: Neither agent exhibited context drift across turns; balances and actions remained coherent.

Alternatives

• gpt-5-mini: Same pass rate and correct tool usage, but higher cost per test and more verbose outputs. Viable if model-specific features are needed; otherwise not cost-optimal.

🔧 MCP Tool Feedback

banking_server

Overall, tools are clear and reliably discoverable. Agents selected the correct tool each time with valid parameters.

💡 Optimizations

1. Trim conversational follow-ups (recommended)

• Current: Agents often append open-ended follow-up questions after completing the task.
• Change: In the system prompt, add: “After completing the user’s request successfully, provide the result succinctly and do not ask follow-up questions unless explicitly requested.”
• Impact: ~15% cost reduction from fewer generated tokens per turn.

2. Compact tool responses (suggestion)

• Current: Tool JSON includes both raw values and formatted strings plus descriptive messages.
• Change: Return only fields required for the response text (omit redundant formatted strings and messages).
• Impact: ~20–30% fewer tool-response tokens, compounding savings in multi-turn sessions.

📦 Tool Response Optimization

get_all_balances (from banking_server)

• Current response size: ~90–110 tokens
• Issues found: Redundant formatted fields and total_formatted duplicate information the agent can derive or format itself.
• Suggested optimization: Remove formatted strings and return numeric balances only.
• Estimated savings: ~30 tokens per call (~25% reduction)

Example current vs optimized:

// Current
{
  "accounts": {
    "checking": {"balance": 1500.0, "formatted": "$1,500.00"},
    "savings": {"balance": 3000.0, "formatted": "$3,000.00"}
  },
  "total": 4500.0,
  "total_formatted": "$4,500.00"
}

// Optimized
{
  "accounts": {
    "checking": 1500.0,
    "savings": 3000.0
  },
  "total": 4500.0
}

This optimization preserves all necessary information while reducing token overhead for every verification step.