How Bodywork Simplifies DevOps for Machine Learning

Updated

Bodywork is a command-line tool that deploys machine learning pipelines to Kubernetes, and it simplifies ML DevOps by removing container image management from the workflow entirely. Instead of building and maintaining a separate container image for every pipeline stage, Bodywork runs all stages using its own custom container image. Each stage pulls the required files directly from your project's Git repository, pip-installs dependencies, and then executes the stage's designated Python module. This approach suits teams that want to run ML pipelines and deploy models on Kubernetes without building a platform around a general-purpose orchestrator.

The Core Mechanism: One Image, Many Stages

The key design decision is that you never build a container image for your pipeline. Bodywork supplies a custom container image, and every stage runs inside it. At runtime, each stage:

  1. Pulls the required files from your project's Git repository (for example, GitHub).
  2. Pip-installs any dependencies the stage needs.
  3. Runs the stage's designated Python module.

Because the image is shared, the operational surface you manage shrinks to your Git repository and your bodywork.yaml configuration. There are no Dockerfiles per stage, no image registries to maintain for each pipeline component, and no image rebuild cycle tied to code changes.

What a Pipeline Is Made Of

Bodywork treats a pipeline as a combination of two stage types:

Stage type What it does Example use
Batch job Runs a task to completion Prepare features, train models, compute batch predictions
Long-running process Starts a persistent process Serve model predictions via HTTP with a Flask app

These two building blocks compose into common ML use cases, from serving pre-trained models to running continuous training on a schedule.

Configuration and Code Requirements

Stages are assembled into DAGs that define the pipeline's workflow. This and other key configuration live in a single bodywork.yaml file.

Defining a stage requires no new APIs. You develop an executable Python module or Jupyter notebook that performs the required tasks, then commit it to your project's Git repository. You can structure your codebase however you wish.

Where the DevOps Savings Come From

The reduction in operational burden comes from three places:

  • No per-stage container images. You skip building and managing images for each pipeline stage.
  • No orchestration plumbing. Bodywork handles containers and orchestration, so you don't build that layer yourself.
  • Configuration in one file. Pipeline structure and key settings are centralized in bodywork.yaml rather than spread across infrastructure code.

How It Compares to Alternatives

Bodywork positions itself as a lightweight, simpler alternative to Kubeflow, or to building your own platform around a workflow orchestration tool such as Apache Airflow, Argo Workflows, or Dagster. The relevant choice condition: if your team wants to run ML pipelines and deploy models on Kubernetes and would rather not manage container images or assemble an orchestration stack, Bodywork's shared-image, Git-driven model removes that work. If you need deep customization of the orchestration layer itself, a general-purpose orchestrator may fit better.

Getting Started

The documented path is: install Bodywork, set up Kubernetes, then work through the key concepts, user guide, CLI reference, and quickstart tutorials. Deployment templates and CI/CD for ML are also covered in the documentation.