AWS Bedrock ValidationException: Protocol violation when assistant message contains both tool_calls and content

[ThePlenkov]

Summary: cagent generates assistant messages with both tool_calls AND content, violating AWS Bedrock API protocol and causing ValidationException.

Full technical details, reproduction steps, and suggested fix in first comment below.

[ThePlenkov]

Full Technical Details with Reproduction Case

Environment

• cagent version: Latest (via Docker)
• OS: Windows/Linux
• Provider: AWS Bedrock via LiteLLM proxy
• Model: haiku (Claude Haiku via bedrock/anthropic.claude-3-5-haiku-20241022-v1:0)
• Setup: Multi-agent coordinator with subagents using transfer_task

---

The Problem

When cagent delegates tasks to subagents using transfer_task, it generates an assistant message that contains BOTH tool_calls AND content in the same turn. This violates the Anthropic/AWS Bedrock API protocol specification.

Complete Error Message

error receiving from stream: POST "http://localhost:4000/chat/completions":
503 Service Unavailable {
  "message": "litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: 
  litellm.APIConnectionError: APIConnectionError: OpenAIException - 
  Stream generation failed: ProviderException: (400, 
  \"AWS client error encountered: <class 'botocore.errorfactory.ValidationException'>: 
  An error occurred (ValidationException) when calling the ConverseStream operation 
  (reached max retries: 0): The number of toolResult blocks at messages.5.content 
  exceeds the number of toolUse blocks of previous turn.\"
}

---

The Protocol Violation

Here's the malformed message structure that cagent generates (message index 5 in the conversation):

{
  "role": "assistant",
  "content": "Let me proceed with analyzing the issue...",
  "tool_calls": [
    {
      "id": "tooluse_ABC123",
      "type": "function",
      "function": {
        "name": "transfer_task",
        "arguments": "{\"agent\": \"scout\", \"task\": \"Retrieve data from source A\", \"expected_output\": \"Summary of findings\"}"
      }
    },
    {
      "id": "tooluse_XYZ789",
      "type": "function",
      "function": {
        "name": "transfer_task",
        "arguments": "{\"agent\": \"coder\", \"task\": \"Analyze the implementation\", \"expected_output\": \"Technical analysis\"}"
      }
    }
  ]
}

This is WRONG! The assistant message contains:

• ??? 2 tool calls (to delegate to subagents)
• ??? Text content explaining what it's doing

---

What Happens Next

After this malformed message, two tool results come back:

{
  "role": "tool",
  "content": "Result from scout agent...",
  "tool_call_id": "tooluse_ABC123"
}

{
  "role": "tool",
  "content": "Result from coder agent...",
  "tool_call_id": "tooluse_XYZ789"
}

But AWS Bedrock rejects this entire conversation because:

• It sees message 5 has both tool_calls AND content
• Then 2 tool_results arrive
• Bedrock's validation logic counts this as a protocol violation
• Error: "number of toolResult blocks exceeds number of toolUse blocks"

---

Correct Protocol (What Should Happen)

Turn 1 - Assistant makes tool calls (NO content):

{
  "role": "assistant",
  "tool_calls": [
    {"id": "tooluse_ABC123", "type": "function", "function": {...}},
    {"id": "tooluse_XYZ789", "type": "function", "function": {...}}
  ]
}

Note: NO "content" field at all!

Turn 2 - Tool results come back:

{"role": "tool", "content": "...", "tool_call_id": "tooluse_ABC123"}
{"role": "tool", "content": "...", "tool_call_id": "tooluse_XYZ789"}

Turn 3 - Assistant responds with analysis:

{
  "role": "assistant",
  "content": "Based on the results from scout and coder agents..."
}

Note: NO tool_calls here

---

Root Cause Analysis

The system prompts in cagent include:

"When transferring, do not generate any text other than the function call."

However, cagent is violating its own instruction by generating text content alongside transfer_task calls.

This suggests the issue is in:

1. The message construction logic for LLM API calls
2. The response handler that processes subagent delegation
3. Possibly the streaming response builder

---

Why This Only Affects AWS Bedrock

• OpenAI API: More lenient, may tolerate messages with both fields
• Anthropic Direct API: Strict but different error messages
• AWS Bedrock (Anthropic models): STRICTEST validation - immediately rejects any protocol violations with ValidationException

---

Minimal Reproduction

1. Configure cagent with AWS Bedrock via LiteLLM proxy
2. Set up multi-agent coordinator with 2+ subagents
3. Configure transfer_task tool for delegation
4. Give coordinator a task requiring delegation to multiple subagents
5. Observe: coordinator generates text response AND tool_calls simultaneously
6. Result: AWS Bedrock rejects with ValidationException

---

Expected vs Actual Behavior

Expected:

• Assistant turn contains EITHER tool_calls OR content, never both
• Clean separation between tool invocation and response analysis

Actual:

• Assistant turn contains BOTH tool_calls AND content
• AWS Bedrock validation fails immediately

---

Impact

• ??? AWS Bedrock users cannot use multi-agent features
• ??? Any transfer_task usage fails immediately
• ??? Forces users to switch to less strict providers
• ??? Blocks enterprise adoption for AWS-based deployments
• ??? Related to Subagent tool_calls not handled when using LiteLLM proxy #1544 (subagent tool_calls not handled with LiteLLM)

---

Suggested Fix

The bug is likely in the code that:

1. Constructs the messages array for LLM API requests
2. Handles transfer_task function call responses
3. Manages conversation history during multi-agent delegation

Required change: When tool_calls are present in an assistant message, ensure:

• The content field is completely omitted (preferred)
• OR set to null
• Never contains actual text

---

Temporary Workaround

Users can add LiteLLM middleware to sanitize messages:

def sanitize_for_bedrock(messages):
    sanitized = []
    for msg in messages:
        if msg.get('role') == 'assistant':
            if msg.get('tool_calls') and msg.get('content'):
                # Strip content when tool_calls present
                sanitized.append({
                    'role': 'assistant',
                    'tool_calls': msg['tool_calls']
                })
            else:
                sanitized.append(msg)
        else:
            sanitized.append(msg)
    return sanitized

---

Additional Context

• This is a protocol-level issue, not a model-specific problem
• Affects all Anthropic models via AWS Bedrock (Claude 3.x, 4.x)
• OpenAI-compatible endpoints may hide this bug due to lenient validation
• Issue exists in the conversation history builder, not the LLM itself

---

Request

Can the cagent team investigate where assistant messages are constructed for API calls, specifically around multi-agent transfer_task handling? The fix should ensure strict separation of tool calls and content responses.

Happy to provide additional debugging information if needed.

[Pnkcaht]

I will see this, thanks for the report :)

[Pnkcaht]

@ThePlenkov I've already submitted a pull request regarding, if you see anything missing please let me know, thank you again for the report.