What Is Ollama and How Does It Enable Local AI for Private Text Rewriting?

Updated

Ollama is a local runtime that lets you download and run open-source large language models (LLMs) directly on your own Windows, macOS, or Linux machine, without sending your text to a cloud provider. It matters for text rewriting when privacy is the priority: apps like CapyBro can connect to Ollama instead of a cloud API, so the selected text never leaves your computer. Choose this route if you handle confidential material and can accept the hardware and model-quality trade-offs; choose a cloud service if you want the strongest models with no local setup.

What Ollama actually does

Ollama packages two things that are normally painful to manage yourself:

  • A model runner — it downloads model weights (Llama, Mistral, Gemma, and similar open-weight families) and executes them on your CPU or GPU.
  • A local HTTP endpoint — it exposes an API on your own machine (by default on a local port) that other programs can call, the same way they would call a cloud AI service.

So "using Ollama" means: you install it once, pull a model, and then any compatible app points at that local endpoint instead of a remote one. The model runs on your hardware; the network is not involved in generating the answer.

How local Ollama differs from cloud AI

The practical difference is where the computation and the text live.

Dimension Local via Ollama Cloud (OpenRouter, GPT-4o, Claude, Gemini)
Where text goes Stays on your machine Sent to a remote server
Works offline Yes, once the model is downloaded No
Model quality Depends on the open model you run Typically the strongest available models
Speed Depends on your CPU/GPU and model size Depends on network and provider load
Cost model Your own hardware and electricity Provider pricing/account terms
Setup effort Install runtime + pull a model API key and account

The privacy point is the headline: in a local setup, the text you rewrite is never transmitted. That is exactly the mode CapyBro advertises — "in Ollama mode, text never goes to the network at all."

Typical setup: install, pull, point

The general workflow is the same regardless of which client app you use:

  1. Install the Ollama runtime on your machine.
  2. Pull a model — you choose a model by name, and Ollama downloads it locally. Smaller models run faster on modest hardware; larger ones are slower but generally more capable.
  3. Confirm the local endpoint is running — Ollama serves an API on your machine that client apps connect to.
  4. Point your client at that endpoint — in an app that supports a local mode, you select Ollama as the provider rather than a cloud provider.

Expected result: when you trigger a rewrite, the request goes to the model running on your own computer, and the response comes back without any external network call for the generation itself.

The CapyBro example

CapyBro is a Windows 10/11 text assistant that works in any app: you select text, press a hotkey, and it rewrites, fixes grammar, or translates in place. It offers two AI paths:

  • Cloud via OpenRouter, giving access to GPT, Claude, and Gemini through one account.
  • Local via Ollama, where, per the product description, your text never leaves the computer.

This is the concrete use case for Ollama in a rewriting tool: instead of pasting sensitive text into a web chat, you select it in Word, Outlook, or an IDE, hit the hotkey, and the rewrite is produced by a model on your own machine. CapyBro also notes that API keys are encrypted locally (Windows DPAPI) and history is stored only on your computer, with the free version's core open-sourced under MIT — which matters if you want to verify what the program does with confidential text.

Trade-offs to weigh

Local AI is not free of cost — it just moves the cost to your hardware and your time.

  • Hardware requirements — running a capable model locally needs adequate RAM and, ideally, a GPU. Weaker machines will be limited to smaller models.
  • Model quality — open models you can run locally may not match the top cloud models for nuanced rewriting.
  • Speed — generation speed depends on your machine; a large model on a CPU-only setup can be slow.
  • Setup and maintenance — you install the runtime and manage models yourself, versus just adding an API key.

When to choose local Ollama vs cloud

  • Choose local Ollama when the text is confidential, when you need to work offline, or when you want to avoid sending content to a third party at all.
  • Choose cloud models when you want the best available quality, the fastest results on modest hardware, or zero local setup.

A reasonable middle path, and the one CapyBro is built around, is to keep both configured: use cloud models for routine text and switch to Ollama when the content is sensitive.