Skip to content

fix: place cache point on last user message instead of assistant#1821

Open
kevmyung wants to merge 2 commits intostrands-agents:mainfrom
kevmyung:fix/cache-point-user-message
Open

fix: place cache point on last user message instead of assistant#1821
kevmyung wants to merge 2 commits intostrands-agents:mainfrom
kevmyung:fix/cache-point-user-message

Conversation

@kevmyung
Copy link
Contributor

@kevmyung kevmyung commented Mar 5, 2026

Move cache point injection in _inject_cache_point from the last assistant message to the last user message (including toolResult messages) for the Anthropic cache strategy. Closes #1820

Description

The current implementation appends cachePoint to the last assistant message. This PR changes it to append to the last user message instead, for two reasons:

  1. Avoids potential issues with assistant message generation — Assistant messages are generated via streaming and may be subject to unexpected interruptions or content block ordering issues (see [BUG] Invalid Cache Point Injection in BedrockModel #1820). Placing the cache point on user messages avoids this entirely since user messages are always complete and well-structured.

  2. Aligns with Anthropic's official caching guide — Anthropic's automatic caching documentation (released Feb 2026, not yet supported on Bedrock) places the cache breakpoint on the last user message, moving it forward as the conversation grows.

In testing, this change showed improved cache hit rates in both multi-turn conversations and agent loops with tool calls.

Related Issues

#1820

Type of Change

Bug fix

Testing

  • All existing cache point tests updated and passing (7 tests)

  • hatch run prepare passed (2110 tests)

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Move cache point injection to last user message to improve cache hit
rates in agent loops and avoid reasoningContent block conflicts.
@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/strands/models/bedrock.py 83.33% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Invalid Cache Point Injection in BedrockModel

2 participants