What is QDP (Quantum Data Plane) in Apache Mahout and what problem does it solve?

Updated

QDP (Quantum Data Plane) is Apache Mahout's GPU-accelerated data plane that removes the data-loading bottleneck in quantum machine learning. Instead of simulating a state-preparation circuit, it constructs the quantum state vector directly in GPU memory and hands it to your training or kernel pipeline. It is the right tool when your bottleneck is getting classical data into quantum states at scale, and you want to avoid the cost of circuit-based state preparation.

The problem it solves

Quantum machine learning pipelines need classical data (images, vectors, kernel inputs) encoded as quantum states. The conventional route is to build a state-preparation circuit and simulate it. That simulation is the bottleneck: it adds latency before any training or kernel computation begins, and it scales poorly as data volume grows.

QDP takes a different route. Rather than simulating a circuit, it constructs the state vector directly in GPU memory. The encoded state is then passed to your training or kernel pipeline, so the data-loading step no longer dominates the pipeline.

How it works

  • Direct state construction: QDP builds the state vector in GPU memory instead of simulating a state-preparation circuit.
  • Six encodings: amplitude, angle, basis, phase, IQP, and IQP-Z, with the same coverage on every GPU backend.
  • GPU backends: native CUDA kernels for NVIDIA and hand-written Triton kernels for AMD ROCm, selectable from the same API.
  • Zero-copy interop: DLPack handoff to and from PyTorch, NumPy, and TensorFlow, plus GPU-pointer paths that skip the host round trip.

The zero-copy path matters in practice: if your data already lives on the GPU in a PyTorch or TensorFlow tensor, QDP can consume it without copying back to host memory and out again.

When to use QDP

Situation QDP fits
Data loading or state preparation dominates pipeline time Yes — this is the core use case
You need amplitude, angle, basis, phase, IQP, or IQP-Z encoding Yes — all six are supported with backend parity
Your data is already in PyTorch, NumPy, or TensorFlow Yes — DLPack zero-copy interop
You run on NVIDIA CUDA or AMD ROCm Yes — native kernels for both
You want to write circuits once and run on Qiskit, Cirq, or Amazon Braket No — that is Qumat's role

Qumat and QDP are complementary: Qumat is the high-level Python library for building and executing circuits across backends, while QDP handles the data-encoding layer beneath it. Qumat requires Python 3.10+ and installs from PyPI with pip install qumat.

Verifying it in your pipeline

The project ships benchmarks on real workloads — SVHN IQP training, quantum kernel SVM, and data-to-state latency benchmarks — so you can compare encoding latency against a circuit-simulation baseline before committing. A practical check: time your current state-preparation step, then time the same encoding through QDP on your target GPU backend; the difference is the bottleneck QDP removes.

Current status

Qumat 0.6 is the current release, adding first-class AMD ROCm support, six QDP encodings with parity across CUDA and ROCm, and faster zero-copy GPU paths. The earlier Mahout Classic codebase (Samsara and MapReduce) is in maintenance mode; the community's active work is on Qumat and QDP.