Skip to content

feat(models): add configurable TTL for Bedrock cache points#2042

Open
mattdai01 wants to merge 1 commit intostrands-agents:mainfrom
mattdai01:feat/cache-ttl
Open

feat(models): add configurable TTL for Bedrock cache points#2042
mattdai01 wants to merge 1 commit intostrands-agents:mainfrom
mattdai01:feat/cache-ttl

Conversation

@mattdai01
Copy link
Copy Markdown

Support 1-hour and 5-minute TTL options for Bedrock prompt caching. This allows users to extend cache lifetime beyond the default 5 minutes for supported models (Claude Opus 4.5, Haiku 4.5, Sonnet 4.5).

  • Add ttl field to CacheConfig for messages cache point
  • Add cache_tools_ttl field to BedrockConfig for tools cache point
  • Add optional ttl field to CachePoint TypedDict
  • Pass through TTL on user-supplied cachePoint blocks

Description

https://aws.amazon.com/about-aws/whats-new/2026/01/amazon-bedrock-one-hour-duration-prompt-caching/
Support 1-hour and 5-minute TTL options for Bedrock prompt caching. This allows users to extend cache lifetime beyond the default 5 minutes for supported models (Claude Opus 4.5, Haiku 4.5, Sonnet 4.5).

Previously, all cache points used Bedrock's default 5-minute TTL. With this change, users can configure extended TTL through:

  • CacheConfig(strategy="auto", ttl="1h") for messages cache points
  • cache_tools_ttl="1h" for tools cache points
  • Manual {"cachePoint": {"type": "default", "ttl": "1h"}} in SystemContentBlock (previously ttl was silently stripped)

Changes

  • Add ttl field to CacheConfig for auto-injected messages cache point
  • Add cache_tools_ttl field to BedrockConfig for tools cache point
  • Add optional ttl field to CachePoint TypedDict for type safety
  • Fix _format_request_message_content to pass through ttl on user-supplied cachePoint blocks (was previously dropped)
  • Refactor _inject_cache_point to accept ttl parameter from caller instead of re-reading config
  • Related Issues

Documentation PR

Type of Change

New feature

Testing

Unit tests passed, did a quick test checking cacheReadInputTokens in bedrock response was > 0 even after 5 min ttl when 1 hr was enabled.

  • [ X ] I ran hatch run prepare

Checklist

  • [ X ] I have read the CONTRIBUTING document
  • [ X ] I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • [ X ] My changes generate no new warnings
  • [ X ] Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Support 1-hour and 5-minute TTL options for Bedrock prompt caching.
This allows users to extend cache lifetime beyond the default 5 minutes
for supported models (Claude Opus 4.5, Haiku 4.5, Sonnet 4.5).

- Add `ttl` field to `CacheConfig` for messages cache point
- Add `cache_tools_ttl` field to `BedrockConfig` for tools cache point
- Add optional `ttl` field to `CachePoint` TypedDict
- Pass through TTL on user-supplied cachePoint blocks
@JackYPCOnline
Copy link
Copy Markdown
Contributor

JackYPCOnline commented Apr 3, 2026

Is this PR related to #1219

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mattdai01
Copy link
Copy Markdown
Author

I think we no longer have pass through on the cachePoint right? If you look at the _format_request_message_content function in the PR, it seems that it only passes through type currently, and drops ttl. now the behavior is that it drops everything except type and ttl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants