Bedrock Prompt Caching support in Amplify Gen2

[JustinHaut]

Is this related to a new or existing framework?

No response

Is this related to a new or existing API?

No response

Is this related to another service?

Bedrock via Amplify Gen2

Describe the feature you'd like to request

Add the ability to configure Prompt Caching when using the conversation endpoint in Amplify Gen2 schemas.

Currently, the @aws-amplify/backend-ai/conversation/runtime uses an older version of @aws-sdk/client-bedrock-runtime that doesn't support the new Prompt Caching feature introduced in v3.779.0.

Describe the solution you'd like

const conversation = a.conversation({
  aiModel: {
    resourcePath: modelId,
  },
  systemPrompt: 'system prompt here',
  promptCaching: {
    enabled: true,
    cacheSystemPrompt: true,
    cacheCheckpoints: ['system', 'messages'] // Which parts to cache
  },
  handler: chatHandlerFunction,
});

Describe alternatives you've considered

1. Custom Lambda functions: Direct Bedrock SDK calls in individual Lambda functions, but this bypasses Amplify's conversation runtime and loses integration benefits.

2. Manual SDK updates: Attempted to force newer SDK versions via package resolutions, but this caused compatibility issues with Amplify's internal dependencies.

3. Waiting for automatic updates: The conversation runtime doesn't automatically pick up newer Bedrock SDK versions, so manual intervention is needed.

The ideal solution is native Amplify support for Prompt Caching configuration.

Additional context

Prompt Caching in conversation streams was introduced in @aws-sdk/client-bedrock-runtime in v3.779.0. This feature allows caching of system prompts and common conversation patterns, which would significantly improve cost and performance for amplify gen2 conversations.

Amplify currently uses @aws-sdk/client-bedrock-runtime@3.622.0

Here's the cache update commit in v3.779.0 – https://github.com/aws/aws-sdk-js-v3/commit/e48bc0834f743c979152b7d16bd29cd211342136

Compatibility Issue:
Attempting to manually update @aws-sdk/client-bedrock-runtime to newer versions causes "API name is invalid" errors with Amplify's API Gateway integration (Amplify.configure modification to import an existing API from a CDK stack), forcing a revert to older versions without Prompt Caching support.

Is this something that you'd be interested in working on?

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

[ahmedhamouda78]

Hello @JustinHaut, thank you for opening this issue. We will take a look and come back to you once we have any updates.

[ahmedhamouda78]

This is a feature request for amplify-backend to support prompt caching; transferring this to the backend repository.

[pahud]

Hi @JustinHaut,

I've investigated the current implementation and confirmed the core issue: Amplify currently uses @aws-sdk/client-bedrock-runtime@3.622.0, while prompt caching requires version 3.779.0 or later.

Implementation Requirements:

1. SDK Update: Upgrade to @aws-sdk/client-bedrock-runtime@3.779.0+
2. API Extension: Extend the conversation configuration to support prompt caching options
3. Runtime Integration: Update the BedrockConverseAdapter to pass caching parameters

Proposed API Design (based on your suggestion):

const conversation = a.conversation({
 aiModel: { resourcePath: modelId },
 systemPrompt: 'system prompt here',
 promptCaching: {
   enabled: true,
   cacheSystemPrompt: true,
   cacheCheckpoints: ['system', 'messages']
 },
 handler: chatHandlerFunction,
});

Next Steps:
This feature would require updates to multiple components including type definitions in types.ts and the core conversation runtime. The team will need to evaluate the SDK upgrade impact and design the optimal API surface.

We appreciate you providing the specific version requirements and AWS SDK commit reference - this helps significantly with implementation planning. The feature would benefit many users looking to optimize their conversation costs and performance.