How does a Bodywork pipeline work?

Updated

A Bodywork pipeline is a directed acyclic graph (DAG) of stages, where every stage is either a Job or a Service. Jobs run batch work to completion — feature preparation, model training, batch prediction — while Services start long-running processes, such as a Flask app that serves predictions over HTTP. You define the stages and their dependencies in a single bodywork.yaml file, and Bodywork deploys the whole thing to Kubernetes.

The two stage types

Every stage in a pipeline does exactly one of two things:

Stage type What it does Typical use
Job Runs a batch task and finishes Prepare features, train a model, compute batch predictions
Service Starts a long-running process Serve model predictions via HTTP with a Flask app

Because you can chain these freely, the same building blocks cover a wide range of ML use cases — from serving a pre-trained model to running continuous training on a schedule.

How stages become a pipeline

Stages are assembled into a DAG that defines the workflow. The DAG and other key configuration live in one bodywork.yaml file, so the pipeline's structure is declared in a single place rather than spread across orchestration code.

A stage itself is just an executable Python module or Jupyter notebook that performs the required tasks, committed to your project's Git repository. There are no new APIs to learn, and you can structure your codebase however you like.

What Bodywork handles for you

Bodywork removes the need to build and manage container images for any stage. It runs all stages using Bodywork's own custom container image. Each stage starts by:

  1. Pulling the required files directly from your project's Git repository (for example, GitHub).
  2. pip-installing any required dependencies.
  3. Running the stage's designated Python module.

This is the mechanism behind the "no boilerplate" claim: you supply code and configuration, and Bodywork takes care of containers and orchestration.

Where it fits

Bodywork is aimed at teams that want to run ML pipelines and deploy models to Kubernetes. It positions itself as a lightweight, simpler alternative to Kubeflow, or to building your own platform around a workflow orchestrator like Apache Airflow, Argo Workflows, or Dagster.

If your goal is to run batch ML stages and long-running model services on Kubernetes without writing container or orchestration plumbing yourself, the Job + Service + DAG model is the core of how Bodywork works.