How to Integrate Bifrost with Existing AI SDKs

Bifrost (by Maxim AI) is designed as a drop-in replacement for the AI SDKs you already use: you change one line of code — typically the base URL or client initialization — and keep the rest of your application logic intact. It works with OpenAI, Anthropic, Vercel AI SDK, LangChain, and other SDKs, and you can start it locally with npx @maximhq/bifrost in about 30 seconds. The approach below applies if you're running Bifrost yourself (open source) or pointing at a Bifrost Enterprise deployment.

What "drop-in replacement" means in practice

Bifrost sits between your application and your model providers. Instead of your SDK calling OpenAI or Anthropic directly, it calls Bifrost, which handles routing, governance, guardrails, and observability, then forwards the request to the model. Because the request and response shapes stay compatible with the SDK you're using, your existing code — message formatting, streaming handlers, tool calls — doesn't need to change.

The site states Bifrost is compatible with all model SDKs and that outputs can be sent anywhere you need. The integration surface is therefore small: point the client at Bifrost instead of the provider.

Step-by-step integration

1. Start Bifrost

npx @maximhq/bifrost

This launches the gateway locally. You'll need it running before your app can route through it.

2. Change the client initialization

For an OpenAI-style client, the only change is the base URL (and using your Bifrost key if configured):

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY"),
    base_url="http://localhost:<bifrost-port>/v1",  # the one line you change
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello world"}],
)

The site's own example keeps the OpenAI(...) constructor and the chat.completions.create call unchanged — the gateway is what moves, not your code.

3. Repeat for other SDKs

The same pattern applies to Anthropic, Vercel AI SDK, LangChain, and others: keep the SDK, redirect the endpoint. The site shows an Anthropic example using Anthropic(...) and anthropic.messages.create(...) with the model claude-3-5-sonnet-20241022, again with the client pointed at Bifrost.

4. Verify

Send a simple request (like the "Hello world" call above) and confirm you get a normal completion back. If it succeeds, your routing through Bifrost is working. Then check that the request appears in Bifrost's observability layer, which is where you'll see latency, usage, and any guardrail activity.

Common sticking points

• Base URL and port. The one-line change is only correct if the URL matches where Bifrost is actually listening. Confirm the port from the startup output.
• API keys. Your provider key still needs to be available to Bifrost so it can forward requests. Don't assume the gateway removes the need for provider credentials.
• Model names. Keep using the provider's model identifiers (e.g. gpt-4o-mini, claude-3-5-sonnet-20241022) unless your Bifrost configuration maps them to something else.
• Streaming and tool calls. These pass through the same compatible interface, but test them explicitly if your app depends on them — a basic chat completion succeeding doesn't prove streaming works.

Why route through Bifrost at all

Once the one-line change is in place, you get capabilities that would otherwise require separate integration work:

The site reports added latency of 20 microseconds, throughput of 5,000 req/s, and peak memory of 3.3 GB, and claims it is 50x faster than LiteLLM at P99 latency under a 500 RPS benchmark on identical hardware. Treat those as vendor-reported figures and validate against your own workload before committing.

When this approach fits

Choose the drop-in route if you want to keep your current SDK and application code and only add a gateway layer. If you need per-team budgets, SSO-based access control, or centralized guardrails across multiple providers, the one-line change is the cheapest way to get there. If you're evaluating Bifrost against LiteLLM specifically, run both under your own traffic — the published benchmark is a starting point, not a substitute for your own measurements.

Bifrost Enterprise is offered as a 14-day free trial, with a demo request option; pricing details are on the site's pricing page.