How TavernAI 2 Connects to Local Models and AI Providers

TavernAI 2 supports two connection paths for generation: load GGUF language and vision models directly inside the app, or point the same chat at one of the built-in or custom API integrations. The local route runs on your own hardware with CPU, CUDA, or Vulkan; the API route sends generation to an external provider. You can switch between them inside the same chat, so the choice is per-chat rather than permanent.

Local models: GGUF loaded inside TavernAI

TavernAI 2 loads GGUF language and vision models directly, without a separate inference server. The website lists CPU, CUDA, and Vulkan as the available runtimes.

• CPU works on any machine but is the slowest option for larger models.
• CUDA uses NVIDIA GPUs and is the usual choice when you have one.
• Vulkan covers a wider range of GPUs, including AMD and Intel hardware.

The practical constraint is memory. A GGUF model has to fit in your available RAM or VRAM alongside the app itself, so the model size you can run locally is set by your hardware, not by TavernAI. Vision models follow the same loading path, which means image understanding runs locally too.

Because nothing leaves your machine on this path, local generation is the option to pick when privacy or offline use matters more than speed.

API providers: built-in and custom integrations

The same chat can switch to an API integration instead. The site states TavernAI 2 ships with more than a dozen built-in and custom API integrations, and that you bring your own provider — there is no bundled account or included credits.

For any provider you connect, expect to supply the credentials and identifiers that provider requires. In practice that means:

1. API key — issued by the provider's dashboard.
2. Endpoint or base URL — the address requests are sent to, which matters for self-hosted or proxy setups.
3. Model name — the specific model identifier you want that provider to run.

The site does not publish a provider list, per-provider setup screens, or pricing, so treat the exact fields as provider-dependent and check the provider's own documentation for the values. TavernAI 2 does not require an account to use the app itself, but your API provider almost certainly will.

Local vs. API: what actually differs

Choose local when you have the hardware and want prompts, attached files, and chat history to stay on your machine. Choose an API provider when your hardware can't run the model size you want, or when you'd rather not manage model files at all.

What stays the same either way

The connection choice doesn't change how the chat is built. Prompt Manager folders, activation rules, roles, placement, presets, token counters, and structural nodes assemble the prompt the same way regardless of backend, and you can open the final assembled prompt before generation to see every role, source, token, attachment, and the raw request. Files attached to prompt items — images, PDFs, documents, audio, video, text — join the same context system as characters, scenes, and chat history.

That means switching from local to API mid-project doesn't require rebuilding your prompt structure; it changes who generates the reply, not what you send.

Common snags

• Model won't load locally. Almost always a size problem — the GGUF exceeds available RAM or VRAM. Drop to a smaller quantization or a smaller model.
• API requests fail. Check the three values above in order: key, endpoint, model name. A valid key with the wrong model identifier fails the same way as a bad key.
• Slow local generation. Confirm which runtime is actually active; falling back to CPU when a GPU is present is a frequent cause.
• Unexpected cost. The site lists no pricing, so any spend comes from your provider's terms, not from TavernAI 2.

For installation itself — Windows 10/11 download or the Linux VPS terminal script — see the separate install walkthrough.