How VapourSynth Handles Frames and Resource Management

Updated

VapourSynth processes video as a stream of immutable frames, and each frame carries its own reference-counted resources (memory, GPU buffers, and so on). The practical consequence: a filter or script never "owns" a frame in the traditional sense — it holds a reference, and memory is released when the last reference disappears. The site's own section headings — "Resource management and performance quirks" and "The internal frame structure" — signal that this is the core mental model you need before writing filters or debugging memory behavior.

The internal frame structure in plain terms

A frame in VapourSynth is a container, not a raw pixel buffer. It bundles:

  • One or more planes (luma, chroma, alpha, or arbitrary planes for RGB/GRAY formats), each with its own dimensions and stride.
  • Format and property metadata describing pixel type, color family, and frame-level properties that travel with the frame through the filter chain.
  • A reference count that tracks how many consumers still need the frame alive.

Because frames are immutable, filters cannot edit a frame in place. A filter that changes pixels allocates a new frame and returns it. This is why a long chain of filters does not corrupt earlier results — but it also means naive chains can hold many frames in memory at once if something keeps references alive.

Why immutability matters for scripts

When you request a frame at a given frame number, the framework asks each filter in the chain for that frame. Filters may cache or request neighboring frames (temporal filters do this), and the reference counting ensures a frame is only freed when no filter still needs it. If you write a script that accidentally retains a frame reference — for example, storing frames in a Python list — memory grows until you release them.

Resource management and performance quirks

The site explicitly flags "performance quirks" as a topic, and the R81 release note gives a concrete, checkable example of how resource handling differs by platform. The note describes Vulkan exportable memory:

  • On Windows, the same memory can only be exported once; exporting it again produces "weird behavior."
  • On *nix systems, the same memory can be exported an unlimited number of times.
  • On Windows, the same exported memory counts twice toward a process's VRAM usage when imported.

R81 is described as fixing "most issues and Vulkan API misunderstandings from R80," yielding working exportable memory and faster GPU uploads. The takeaway for filter authors: GPU resource lifetimes are not uniform across operating systems, so a filter that works on Linux may hit export or accounting limits on Windows.

Practical implications for filter and script authors

Concern What to do
Frame lifetime Hold references only as long as needed; avoid caching frames in unbounded structures.
GPU memory Expect platform-specific export limits; test on the target OS, not just one.
VRAM accounting On Windows, imported exportable memory may appear to cost double — budget accordingly.
Version behavior The R81 note shows GPU paths changed between R80 and R81; pin and verify your VapourSynth version.

What this means when writing filters

A filter's job is to accept input frames, produce output frames, and let the framework manage lifetimes. The site's "Writing filters and Vulkan usage" section indicates that Vulkan-based filters are a supported path, which is where the exportable-memory quirks become relevant. If you are writing a CPU-only filter, the main discipline is simply not to retain frames beyond what your filter needs. If you are writing a GPU filter, the platform differences above are the first thing to verify.

For users rather than filter authors, the same model explains common symptoms: a script that runs out of memory usually has a reference leak or an oversized cache, not a bug in VapourSynth itself. Reducing cache size or breaking a chain into stages typically resolves it.