Which Inference Backends MMDeploy Supports and How to Choose One

MMDeploy 1.3.1 supports a range of inference backends, including ONNX Runtime, TensorRT, ncnn, OpenVINO, PPLNN, SNPE, RKNN, TVM, and Core ML. To choose one, start with your deployment hardware and runtime constraints, then confirm that the backend supports the operators your model needs. The documentation organizes backend information into separate feature-support pages and provides benchmark results, including tests on embedded devices and TVM, so you can compare options before committing.

Supported Backends at a Glance

The documentation lists the following backend support pages:

Each backend has its own feature-support page, and several have dedicated operator (Ops) pages. Those Ops pages are the practical place to check whether a specific model's operators are implemented for that backend.

How to Choose a Backend

Use the following dimensions to narrow the list. The documentation supports each of these checks directly.

1. Match the backend to your target hardware

• NVIDIA GPUs: TensorRT is the natural candidate, with a dedicated support page and TensorRT Ops list.
• Intel CPUs and related hardware: OpenVINO has its own support page.
• Mobile and embedded ARM devices: ncnn and RKNN both target this space; the docs include a "Test on embedded device" benchmark page.
• Qualcomm Snapdragon platforms: SNPE has a feature-support page.
• Apple platforms: Core ML has a feature-support page.
• General cross-platform CPU/GPU inference: ONNX Runtime is listed with both a support page and an Ops page.
• Compiler-based deployment: TVM has a feature-support page and a dedicated benchmark test page.

2. Check operator coverage for your model

Feature support is not uniform across backends. Before choosing, open the backend's feature-support page and its Ops page (where available) and verify that the operators your model uses are covered. The documentation also includes a "Custom Ops" section with separate pages for ONNX Runtime Ops, TensorRT Ops, and ncnn Ops, which is useful when a required operator is missing and needs to be added.

3. Compare performance with the published benchmarks

The documentation provides a Benchmark section, including:

• General benchmark results
• Benchmark test on embedded device
• Benchmark test on TVM
• Quantization test result

These pages let you compare backends on the hardware class you care about rather than guessing. Quantization results are listed separately, so if you plan to quantize your model, check that page as well.

4. Confirm the model conversion path works

Backend choice interacts with model conversion. The docs cover "How to convert model," "How to write config," and "How to evaluate model," plus an ONNX export and Optimizer section under experimental features. A backend is only usable in practice if your model can be converted to a form it accepts and then evaluated successfully.

Practical Selection Flow

1. Identify your deployment hardware (GPU, Intel CPU, ARM embedded, Snapdragon, Apple, or general-purpose).
2. Shortlist backends from the table above that target that hardware.
3. Open each shortlisted backend's feature-support page and Ops page; eliminate any backend missing operators your model requires.
4. Check the benchmark pages, including the embedded-device and TVM tests if relevant, to compare performance.
5. Convert and evaluate the model following the "How to convert model" and "How to evaluate model" guides to validate the choice end to end.

Where to Go Next

The documentation's Get Started section covers introduction, prerequisites, installation, model conversion, inference, and evaluation. The Build section covers building from source, Docker images, build scripts, and CMake options. SDK usage and SDK documentation are separate sections, and there are per-codebase deployment guides for MMPretrain, MMDetection, MMSegmentation, MMagic, MMOCR, MMPose, MMDetection3d, MMRotate, and MMAction2. The site also notes that you can switch between Chinese and English documents in the lower-left corner of the layout.