Should You Use Manifest Cloud or Self-Host It?

Updated

Choose self-hosting if you need full control over where LLM traffic and keys live, or if you want to run the router on your own infrastructure at no license cost. Choose the hosted/cloud option if you want the gateway managed for you and are prepared to evaluate subscription plans. Manifest describes itself as an open source LLM gateway, and its site lists both a self-host path ("Free and open source") and a pricing page with subscription and enterprise contact options — but the public page does not spell out cloud plan limits, SLAs, or exactly what each tier includes, so confirm those with the team before committing.

What each option is, based on the site

Dimension Self-host Hosted / Cloud
What it is "Run the whole LLM router on your infrastructure or your machine." Managed access to the gateway; pricing page lists subscription plans and a "Talk to sales" enterprise path
Cost signal Stated as "Free and open source" Subscription pricing exists; exact amounts and limits are not on the page reviewed
Control You run the router yourself; decide what goes to the cloud vs. stays local Managed for you; operational control sits with the provider
Provider keys Bring your own keys for every provider; OAuth tokens for subscriptions supported Same BYOK/OAuth capability is described for the gateway generally — confirm how keys are stored in the hosted setup
Local models Route to Ollama, LM Studio, llama.cpp, or any local server Local-model routing is a gateway feature; whether it applies to the hosted tier needs confirmation

What you get either way

The gateway's core feature set is described as part of the product, not tied to one deployment mode:

  • One endpoint to reach all your models and providers.
  • Fallbacks that retry on another provider when one goes down.
  • Autofix, which repairs broken requests before your agent sees the error. Manifest cites Datadog's State of AI Engineering for the claim that 5% of LLM API calls fail in production and most are avoidable; Autofix is available through the gateway or standalone.
  • Observability: full request/response body logs and cost visualization broken down by agent, key, and provider.
  • Parameter setup per route, set visually rather than in code.
  • OpenAI-compatible interface, custom providers, and local models.

How to decide

Lean self-host if:

  • You have infrastructure (or a capable machine) and want the router inside your own network.
  • Data residency or key custody rules mean provider keys and request bodies can't leave your environment.
  • You want to mix cloud and local models and control that boundary yourself.
  • You're comfortable operating and upgrading the service.

Lean hosted if:

  • You'd rather not run and maintain the router.
  • You want the same routing, fallback, Autofix, and cost-tracking features without owning uptime.
  • Your budget can absorb a subscription, and you're willing to compare tiers on the pricing page.

Verify before you commit

The page reviewed doesn't answer these, so ask directly:

  1. Cloud plan limits — request volume, token caps, retention of full body logs, and number of keys/agents per tier.
  2. Support and SLA — what response times and uptime commitments come with each plan, and what "Talk to sales" enterprise terms add.
  3. Key handling in the hosted setup — where BYOK keys and OAuth subscription tokens are stored and who can see them.
  4. Self-host scope — which components the self-hosted package includes (gateway, Autofix, dashboard/observability) and how upgrades are delivered.
  5. Autofix behavior — which error classes it repairs, and whether repairs are logged for audit.

A practical path: start with the free self-hosted route to validate routing, fallbacks, and Autofix against your own traffic, then move to a paid plan only if operating the gateway becomes the bottleneck.