Skip to content

Add AWS BYOK with naive approach only for anthropic models #548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
lambertjosh wants to merge 2 commits into
base: main
Choose a base branch
from
+67 −20
Draft

Add AWS BYOK with naive approach only for anthropic models #548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b3924fd
Add AWS BYOK with naive approach only for anthropic models
lambertjosh Feb 25, 2026
216d213
Merge branch 'main' into aws-byok
chrarnoldus Feb 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 48 additions & 16 deletions src/components/organizations/byok/BYOKKeysManager.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ import {
// Hardcoded BYOK providers list
const BYOK_PROVIDERS = [
{ id: VercelUserByokInferenceProviderIdSchema.enum.anthropic, name: 'Anthropic' },
{ id: VercelUserByokInferenceProviderIdSchema.enum.bedrock, name: 'AWS Bedrock' },
{ id: VercelUserByokInferenceProviderIdSchema.enum.openai, name: 'OpenAI' },
{ id: VercelUserByokInferenceProviderIdSchema.enum.google, name: 'Google AI Studio' },
{ id: VercelUserByokInferenceProviderIdSchema.enum.minimax, name: 'MiniMax' },
Expand Down Expand Up @@ -390,26 +391,57 @@ export function BYOKKeysManager({ organizationId }: BYOKKeysManagerProps) {
</div>

<div className="space-y-2">
<Label htmlFor="apiKey">API Key</Label>
<div className="relative">
<Input
<Label htmlFor="apiKey">
{selectedProvider === VercelUserByokInferenceProviderIdSchema.enum.bedrock
? 'AWS Credentials'
: 'API Key'}
</Label>
{selectedProvider === VercelUserByokInferenceProviderIdSchema.enum.bedrock ? (
<textarea
id="apiKey"
type={showApiKey ? 'text' : 'password'}
value={apiKey}
onChange={e => setApiKey(e.target.value)}
placeholder="Enter API key"
className="pr-10"
placeholder='{"accessKeyId": "...", "secretAccessKey": "...", "region": "us-east-1"}'
className="border-input bg-background placeholder:text-muted-foreground focus-visible:ring-ring flex min-h-[80px] w-full rounded-md border px-3 py-2 text-sm focus-visible:ring-2 focus-visible:ring-offset-2 focus-visible:outline-none disabled:cursor-not-allowed disabled:opacity-50"
rows={4}
/>
<Button
type="button"
variant="secondary"
size="sm"
className="absolute top-0 right-0 h-full px-3"
onClick={() => setShowApiKey(!showApiKey)}
>
{showApiKey ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
</Button>
</div>
) : (
<div className="relative">
<Input
id="apiKey"
type={showApiKey ? 'text' : 'password'}
value={apiKey}
onChange={e => setApiKey(e.target.value)}
placeholder="Enter API key"
className="pr-10"
/>
<Button
type="button"
variant="secondary"
size="sm"
className="absolute top-0 right-0 h-full px-3"
onClick={() => setShowApiKey(!showApiKey)}
>
{showApiKey ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
</Button>
</div>
)}
{selectedProvider === VercelUserByokInferenceProviderIdSchema.enum.bedrock && (
<Alert>
<Info className="h-4 w-4" />
<AlertDescription>
<p>Enter your AWS credentials as JSON:</p>
<code className="mt-1 block text-xs break-all">
{'{"accessKeyId": "...", "secretAccessKey": "...", "region": "us-east-1"}'}
</code>
<p className="mt-1">
Your IAM user needs <code className="text-xs">bedrock:InvokeModel</code> and{' '}
<code className="text-xs">bedrock:InvokeModelWithResponseStream</code>{' '}
permissions.
</p>
</AlertDescription>
</Alert>
)}
{editingKeyId ? (
<Alert>
<Lock className="h-4 w-4" />
Expand Down
5 changes: 3 additions & 2 deletions src/lib/providers/openrouter/inference-provider-id.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ export const OpenRouterInferenceProviderIdSchema = z.enum([

export const VercelUserByokInferenceProviderIdSchema = z.enum([
'anthropic',
'bedrock',
'google', // Google AI Studio
'openai',
'minimax',
Expand All @@ -47,7 +48,7 @@ export const UserByokProviderIdSchema = VercelUserByokInferenceProviderIdSchema.

export type UserByokProviderId = z.infer<typeof UserByokProviderIdSchema>;

export const VercelNonUserByokInferenceProviderIdSchema = z.enum(['alibaba', 'bedrock', 'vertex']);
export const VercelNonUserByokInferenceProviderIdSchema = z.enum(['alibaba', 'vertex']);

export const VercelInferenceProviderIdSchema = VercelUserByokInferenceProviderIdSchema.or(
VercelNonUserByokInferenceProviderIdSchema
Expand All @@ -59,7 +60,7 @@ export type VercelInferenceProviderId = z.infer<typeof VercelInferenceProviderId

const openRouterToVercelInferenceProviderMapping = {
[OpenRouterInferenceProviderIdSchema.enum['amazon-bedrock']]:
VercelNonUserByokInferenceProviderIdSchema.enum.bedrock,
VercelUserByokInferenceProviderIdSchema.enum.bedrock,
[OpenRouterInferenceProviderIdSchema.enum['google-ai-studio']]:
VercelUserByokInferenceProviderIdSchema.enum.google,
[OpenRouterInferenceProviderIdSchema.enum['google-vertex']]:
Expand Down
4 changes: 3 additions & 1 deletion src/lib/providers/openrouter/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ export type OpenRouterProviderConfig = {
zdr?: boolean;
};

export type VercelInferenceProviderConfig = { apiKey: string; baseURL?: string };
export type VercelInferenceProviderConfig =
| { apiKey: string; baseURL?: string }
| { accessKeyId: string; secretAccessKey: string; region?: string };
Copy link
Contributor

@kiloconnect kiloconnect bot Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Consider making the union discriminated

This union type has no discriminant property, which means TypeScript can't narrow it without manual checks. If Vercel's gateway API accepts it, consider adding a discriminant (e.g. type: 'apiKey' | 'bedrock') or at minimum adding a comment explaining how consumers should distinguish between the two shapes. As-is, code consuming VercelInferenceProviderConfig will need to use 'accessKeyId' in config checks to narrow.


export type VercelProviderConfig = {
gateway?: GatewayProviderOptions & {
Expand Down
14 changes: 13 additions & 1 deletion src/lib/providers/vercel/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,14 +128,26 @@ export function applyVercelSettings(
? VercelUserByokInferenceProviderIdSchema.enum.mistral
: provider.providerId;
const list = new Array<VercelInferenceProviderConfig>();

if (key === VercelUserByokInferenceProviderIdSchema.enum.zai) {
// Z.AI Coding Plan support
list.push({
apiKey: provider.decryptedAPIKey,
baseURL: 'https://api.z.ai/api/coding/paas/v4',
});
}
list.push({ apiKey: provider.decryptedAPIKey });

if (key === VercelUserByokInferenceProviderIdSchema.enum.bedrock) {
const { accessKeyId, secretAccessKey, region } = JSON.parse(provider.decryptedAPIKey) as {
Copy link
Contributor

@kiloconnect kiloconnect bot Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Unvalidated JSON.parse can crash request handling and produce invalid credential objects

This parses user-provided BYOK credentials without a guarded parse or runtime schema validation. Malformed JSON will throw at runtime, and missing fields can flow through as undefined despite the type assertion, causing hard-to-debug failures when Bedrock calls are attempted.

accessKeyId: string;
secretAccessKey: string;
region?: string;
};
list.push({ accessKeyId, secretAccessKey, region });
} else {
list.push({ apiKey: provider.decryptedAPIKey });
}

byokProviders[key] = [...(byokProviders[key] ?? []), ...list];
}

Expand Down