diffsharp.github.io

DiffSharp is a tensor library built for differentiable programming, and its machine learning and optimization features fall into four practical areas: common optimizers and model elements, differentiable probability distributions, GPU/CUDA tensor computation through a LibTorch backend, and interactive notebooks for Jupyter and Visual Studio Code. It is aimed at machine learning, probabilistic programming, and optimization work, and it is 100% open source under a BSD license. The features below are the ones the project documents on its home page; for exact API signatures you would check the API reference.

Optimizers and model elements

DiffSharp ships with "common optimizers, model elements," which means you don't have to hand-write the standard training loop plumbing. In practice this covers the pieces you'd expect when building and training a model: parameter containers, layers or model building blocks, and optimizer routines that update parameters from computed gradients.

The important mechanism here is that these elements sit on top of DiffSharp's own computation graph and differentiation engine, not on LibTorch's autograd. That design is what lets optimizers participate in differentiation themselves — for example, differentiating through an optimization step rather than only using the optimizer to train a model.

Differentiable probability distributions

DiffSharp includes "differentiable probability distributions," which is the feature that makes probabilistic programming viable in the library. A distribution whose sampling and log-density are differentiable can be composed with the rest of the tensor code and differentiated end to end.

A concrete case: you want to fit a model by maximizing a likelihood or minimizing a variational objective. With differentiable distributions you can write the log-density directly in your loss and let DiffSharp compute gradients through it, instead of deriving gradients by hand. This is also where DiffSharp's nested differentiation matters — you can differentiate a function that internally performs gradient-based optimization.

Automatic differentiation depth

The differentiation capability is the core of the optimization story. DiffSharp provides composable gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products over arbitrary user code. It supports nested forward and reverse differentiation to any level.

The project explicitly positions this as going beyond conventional tensor libraries such as PyTorch and TensorFlow, because those typically give you first-order reverse-mode gradients as the primary path. If your optimization method needs second-order information — a Hessian-vector product inside a solver, for instance — DiffSharp expresses it directly rather than through finite differences or manual derivation.

GPU and CUDA support via LibTorch

DiffSharp uses a LibTorch backend, running the same C++ and CUDA tensor implementations that power PyTorch, with LibTorch's ATen tensors underneath and DiffSharp's own computation graph on top. It supports CUDA and GPUs, and is tested on Linux, macOS, and Windows.

Two consequences worth knowing before you commit:

• Performance profile: raw tensor operations inherit LibTorch's optimized kernels, so heavy tensor math is not a weak point.
• Portability: the same code is expected to run across the three supported desktop platforms, which matters if your optimization job moves between a laptop and a GPU machine.

Interactive notebooks

All documentation pages on the site are interactive notebooks you can execute directly in the browser without installing anything locally. For local work, DiffSharp supports interactive notebooks in Jupyter and Visual Studio Code. This is the fastest way to test whether a particular optimizer or distribution behaves as you expect before wiring it into a larger project.

API familiarity and language

The API is deliberately modeled on the PyTorch Python API, using similar naming and idioms, so where elements share names the PyTorch documentation can generally serve as a guide. The implementation language is F#, chosen for robust functional programming; the project describes F# code as generally faster and more robust than equivalent Python while remaining succinct.

Choosing based on your situation

The main adoption cost is F#: the PyTorch-like naming lowers the API learning curve, but not the language one. If your optimization work genuinely needs nested or higher-order differentiation, that cost buys capability you won't easily replicate elsewhere. If it doesn't, a conventional tensor library will likely be the simpler path.

DiffSharp's official site (diffsharp.github.io) is the central hub for all three: the Install and Getting Started / Quickstart pages cover setup, the Tutorials and More Examples pages provide runnable notebooks, and the API Documentation section holds the full API reference plus extension docs. The GitHub repository and BSD license details are linked from the same top-level navigation, so you can reach everything from the homepage without searching elsewhere.

Installation and first steps

Start from the homepage navigation, which lists Install, Getting Started, Quickstart, and Tutorials as separate entries.

• Install — the entry point for adding DiffSharp to a project.
• Getting Started and Quickstart — short walkthroughs that take you from a fresh setup to running tensor code.
• Tutorials — longer guided material that builds on the quickstart.

The site states DiffSharp is tested on Linux, macOS, and Windows, and that it supports CUDA and GPUs through its LibTorch backend. Those platform notes are the main compatibility facts to check before you begin; confirm current package and runtime requirements on the Install page itself rather than assuming them.

Tutorials and example notebooks

Two navigation entries serve this purpose: Tutorials and More Examples.

A practical detail from the site: all documentation pages are interactive notebooks you can execute directly in your browser without installing anything locally. That means you can try the tutorial code before committing to a local setup — useful if you want to evaluate the API first. The site also notes support for Jupyter and Visual Studio Code notebooks for local work.

The homepage includes an Example section, so the shortest path to seeing real DiffSharp code is the homepage example, then Tutorials, then More Examples.

API reference and extensions

The API Documentation section contains:

• API Reference — the full reference for the library's functions and types.
• Extensions — documentation for extension packages.

One navigation aid worth knowing: the DiffSharp API is designed to be similar to the PyTorch Python API through similar naming and idioms, and the site says that where elements share names, the PyTorch documentation can generally be used as a guide. So when the DiffSharp reference is thin on a particular operation, the equivalent PyTorch page is a reasonable companion.

GitHub, license, and source

The top navigation links GitHub and License (BSD). The project describes itself as 100% open source under the BSD license, so the repository is where you go for source code, issues, and release history. The site does not state pricing or login requirements for any of these resources; treat access terms as something to confirm on the pages themselves.

Which resource to open first

If you are new to the library, the efficient order is: run the homepage example in the browser, read Quickstart, then set up locally via Install. If you already have a working environment and need a specific operation, skip straight to the API Reference and use the PyTorch docs as a cross-reference for similarly named elements.

DiffSharp supports nested and mixed-mode differentiation to arbitrary depth, including composable gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products over arbitrary user code. That is the core difference from PyTorch and TensorFlow: DiffSharp is built to differentiate functions that themselves perform differentiation or gradient-based optimization, which conventional tensor libraries generally do not handle as a first-class case. The trade-off is that DiffSharp is an F# library on a LibTorch backend, so it fits teams already working in .NET/F# rather than those standardized on Python.

The mechanism: its own graph on top of LibTorch

DiffSharp uses LibTorch's ATen tensors — the same C++ and CUDA tensor implementations that power PyTorch — but deliberately excludes LibTorch's autograd. On top of those raw tensors, DiffSharp implements its own computation graph and differentiation capabilities.

This matters because the differentiation machinery is DiffSharp's own layer, not a wrapper around an existing engine. That is what makes nesting possible: a derivative operation is itself differentiable code, so forward and reverse modes can be composed and stacked to any level.

What you can compute

Per the project's documentation, DiffSharp provides:

• Composable gradients
• Hessians
• Jacobians
• Directional derivatives
• Matrix-free Hessian-vector and Jacobian-vector products
• Nested forward and reverse differentiation up to any level
• Differentiation of functions that internally use differentiation and gradient-based optimization

The last item is the practical dividing line. If your objective involves an inner optimization loop — for example, a loss that requires solving or optimizing something before it can be evaluated — DiffSharp is designed for that pattern directly.

Side-by-side on the dimensions that matter

On API familiarity, the documentation states that where elements share names, the PyTorch documentation can generally be used as a guide. That lowers the cost of moving PyTorch habits into F#.

Where the comparison actually bites

Choose DiffSharp when your work needs higher-order derivatives as a routine operation, or when the function you want to differentiate contains an optimization or differentiation step of its own. The F# basis is also a stated advantage: the project argues F# code is generally faster and more robust than equivalent Python while staying succinct.

Stay with PyTorch or TensorFlow when your team and tooling are Python-centric and your models need standard first-order gradients. DiffSharp's differentiators are specifically about differentiation depth and composability, not about replacing a mainstream training stack for ordinary workloads.

A concrete case where the distinction shows up: computing a Hessian-vector product without materializing the Hessian. DiffSharp lists matrix-free Hessian-vector products as a built-in capability, so the operation is expressed directly rather than assembled from lower-level pieces.

Trying it without commitment

Every documentation page on the site is an interactive notebook that runs directly in the browser with nothing installed locally. That is the lowest-friction way to check whether the nesting behavior matches your problem before committing to an F# environment. For local work, the site provides Install, Quickstart, Tutorials, and an API Reference, and the library is available for Jupyter and Visual Studio Code notebooks.

DiffSharp uses F# because its authors wanted a language that is concise like Python but generally faster and more robust for tensor programming, and it mirrors PyTorch's API closely enough that PyTorch documentation can often serve as a guide where element names match. The two choices are linked: F# gives you functional, strongly typed code, while the PyTorch-style naming keeps the learning curve low for anyone coming from Python. This makes DiffSharp a reasonable fit if you want differentiable tensor programming in a functional language without relearning tensor idioms from scratch.

Why F# instead of Python

DiffSharp's site describes F# as "an ideal modern AI and machine learning implementation language" and makes three specific claims:

• Robustness through functional programming. F# is a functional-first language, and the project frames this as a source of more robust code.
• Conciseness. F# is described as "still succinct and compact like Python," so you don't trade readability for type safety.
• Speed. The site states that "F# code is generally faster and more robust than equivalent Python code."

The practical implication is that DiffSharp is aimed at people who are comfortable with, or willing to learn, F#. If your team writes Python and has no appetite for a new language, the PyTorch-style API softens the transition but doesn't remove the F# requirement.

How close the API is to PyTorch

The site is explicit about the design goal: the DiffSharp API "is designed to be similar to the PyTorch Python API through very similar naming and idioms." Two consequences follow directly from the documentation:

1. Naming carries over. Where DiffSharp and PyTorch elements share a name, "the PyTorch documentation can generally be used as a guide." In practice, if you know the PyTorch operation you want, you can often guess the DiffSharp call.
2. The underlying tensors are the same. DiffSharp uses a LibTorch backend, "using the same C++ and CUDA implementations for tensor computations that power PyTorch." So the numerical behavior of raw tensor operations comes from the same ATen code, minus autograd.

Where the similarity stops

The shared backend covers raw tensors only. DiffSharp implements "its own computation graph and differentiation capabilities" on top of LibTorch's ATen, excluding autograd. That means:

• Tensor operations should feel familiar if you know PyTorch.
• Differentiation is DiffSharp's own layer, and it goes further than PyTorch's autograd (see below), so PyTorch docs won't describe DiffSharp's differentiation behavior.

A useful mental model: treat PyTorch documentation as a reference for tensor-level calls, and DiffSharp's own API reference for anything involving gradients.

What you gain over PyTorch-style differentiation

The reason the F# and PyTorch-API choices matter together is that DiffSharp's differentiation is its differentiator. The site claims "world-leading automatic differentiation capabilities for tensor code," including:

• Composable gradients, Hessians, Jacobians, and directional derivatives
• Matrix-free Hessian- and Jacobian-vector products over arbitrary user code
• Nested forward and reverse differentiation "up to any level"

The site frames this as going "beyond conventional tensor libraries such as PyTorch and TensorFlow," specifically enabling higher-order derivatives and differentiating functions that internally use differentiation and gradient-based optimization. For example, if you need the gradient of a training loop that itself computes gradients—a meta-learning or hyperparameter-optimization style task—that is the scenario DiffSharp is built for.

Should you adopt it?

Use this as a decision guide rather than a verdict:

Two practical notes before committing. First, the site says DiffSharp is tested on Linux, macOS, and Windows with CUDA/GPU support, so cross-platform use is covered. Second, all documentation pages are interactive notebooks you can run in the browser without installing anything—the fastest way to test whether the F#/PyTorch blend feels natural is to run the examples there before setting up a local environment.

DiffSharp is an open-source tensor library for differentiable programming, written in F# and built on a LibTorch (PyTorch C++/CUDA) backend. It is designed for machine learning, probabilistic programming, optimization, and other domains that need gradients over tensor code. Choose it if you want PyTorch-style tensor idioms in a statically typed functional language, need nested or higher-order derivatives, or want to run on Linux, macOS, or Windows with optional CUDA/GPU support.

What DiffSharp provides

DiffSharp combines raw tensor computation with its own computation graph and automatic differentiation layer. According to the project site, its key features are:

• Nested and mixed-mode differentiation
• Common optimizers, model elements, and differentiable probability distributions
• F# for robust functional programming
• PyTorch-familiar naming and idioms, with efficient LibTorch CUDA/C++ tensors and GPU support
• Linux, macOS, and Windows support
• Interactive notebooks in Jupyter and Visual Studio Code
• 100% open source (BSD license)

How its differentiation differs from typical tensor libraries

The site describes DiffSharp as offering composable gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products over arbitrary user code. It supports nested forward and reverse differentiation to any level, which the project positions as going beyond conventional libraries such as PyTorch and TensorFlow. In practice, this means you can compute higher-order derivatives efficiently and differentiate functions that internally use differentiation or gradient-based optimization.

A concrete case: if you are optimizing a model whose loss itself contains a gradient term (for example, a meta-learning or bi-level optimization setup), DiffSharp's nested differentiation lets you express that directly rather than hand-deriving the outer derivative.

Backend and API design

DiffSharp uses LibTorch's ATen tensors (excluding autograd) for the underlying computation, then implements its own computation graph and differentiation on top. Because the API mirrors the PyTorch Python API in naming and idioms, the PyTorch documentation can generally serve as a guide where element names match. The language layer is F#, which the project argues is faster and more robust than equivalent Python while remaining succinct.

When DiffSharp is a good fit

Trying it without installing anything

The documentation pages on the site are interactive notebooks that run directly in the browser, so you can execute examples before setting up a local environment. For local use, the site links to Getting Started, Install, Quickstart, Tutorials, Examples, and full API reference sections. The site does not state pricing or login requirements for these resources; treat access terms as unconfirmed until you check the linked pages.