Which Local Models and Developer Tools Work with PrivateGPT?

PrivateGPT works with any local model server that exposes an OpenAI-compatible API, and with any developer tool that can point at an OpenAI-compatible provider. That means you don't have to swap out your existing local stack to use it — Ollama, llama.cpp, vLLM, LocalAI, and similar servers are all listed as compatible, and tools like Claude Code, OpenCode, VS Code, Cline, and n8n can connect through the same interface.

The practical condition is compatibility, not a specific vendor list: if your model server or tool speaks the OpenAI-compatible protocol, PrivateGPT's API layer can sit in front of it.

What PrivateGPT actually is

PrivateGPT is described as an open-source API layer that turns local models into production AI applications. It follows the Claude API model, so the API — not the UI — is the product. A built-in /ui workbench exists for testing, demos, internal pilots, and inspecting API requests, but the intended use is programmatic.

It's built by Zylon.ai and reports 57K+ GitHub stars, 7.5k+ derived projects, and 5k+ Discord members.

Supported local model servers

PrivateGPT runs your models through servers you already use. The site names these explicitly:

The rule stated on the site is broad: any tool that works with a local OpenAI-compatible provider will also work with PrivateGPT. So the list above is illustrative, not exhaustive.

Developer tools that connect

PrivateGPT can serve as the private backend for Claude-style local workflows and API patterns. The site lists these tools as connectable:

• Claude Code — coding assistant
• OpenCode — coding assistant
• VS Code — editor
• Cline — coding assistant
• Other local AI tools — anything that can target an OpenAI-compatible provider

Because PrivateGPT follows the Claude API model, Claude-compatible workflows are a first-class use case: you can keep the API patterns you already use and point them at a local backend instead of a cloud one.

Automation platforms

PrivateGPT also powers private AI workflows through automation tools:

• n8n — connect PrivateGPT to automation pipelines
• Internal automation systems — the same OpenAI-compatible interface applies

Custom tools and MCP support are built in, so workflows and agents can run on top of local MCPs or connect to online providers you trust.

What you get beyond model routing

Connecting a model server is only the first step. PrivateGPT supplies the higher-level building blocks so you don't rebuild backend primitives:

• Messages API — streaming, async processing, token counting
• File & artifact ingestion — upload and use documents as context
• Retrieval with citations — RAG with source-grounded answers
• Structured data access — databases, CSVs, tabular data through the same API
• Embeddings & orchestration — coordinate embeddings, retrieval, tools, and model calls behind one local application API
• Workbench UI — the built-in /ui for testing and inspecting requests

How to decide

If your goal is to avoid reconfiguring your local stack, the check is simple:

1. Confirm your model server is OpenAI-compatible. If it is (Ollama, llama.cpp, vLLM, LocalAI, or similar), it can back PrivateGPT.
2. Confirm your tool can target an OpenAI-compatible provider. If it can (Claude Code, OpenCode, VS Code, Cline, n8n, or internal systems), it can connect.
3. Decide whether you need the higher-level layer. If you only need raw inference, your model server alone may suffice. If you need ingestion, retrieval with citations, structured data access, and orchestration behind one API, PrivateGPT is the layer that provides them.

The main caveat: the compatibility guarantee is protocol-based. A tool or server that doesn't speak the OpenAI-compatible interface isn't covered by the "any tool" statement, so verify that before committing.