Most deep-learning frameworks compute higher derivatives by repeatedly applying automatic differentiation (AD). For a first-order gradient that is exactly what you want. But scientific machine learning is full of operators that are not first-order gradients: the Laplacian in a physics-informed network, the local kinetic energy in variational Monte Carlo, the trace-of-Jacobian in a continuous normalizing flow, high-order Stein operators in score-based diffusion.
For those operators, nested AD is the wrong default. It grows in cost as you iterate, and it accumulates round-off across an ever-deeper graph.
The core idea: a derivative tower from one forward pass
Derivon evaluates the activation derivative tower σ⁽ⁿ⁾(z) in closed form. For the Riccati class of activations - sigmoid, tanh, softplus, plus the eigenfunction exp and the Hermite-generated gaussian - every derivative order is a polynomial recurrence on a single evaluation of the activation.
Onetanhcall gives youtanh,tanh',tanh'', ..., to any order, at machine precision - no nested graph, no finite differences.
The Laplacian of a one-layer field then collapses to a contraction:
∇²f(x) = Σ_h c_h · σ''(z_h) · ‖W_h‖², z_h = W_h · x + β_hThe ‖W_h‖² term is computed once and reused across the whole batch, which is why the derivative overhead is O(1) in the input dimension D.
What the numbers look like
On a single data-center GPU at D = 240 (float64, H = 256, B = 4096), the closed-form Laplacian is 67.7x faster than jax.hessian and 198.7x faster than torch.func.hessian, using 63-108x less memory. Every method agrees to ≤ 1e-15 - the speedups are bit-for-bit, not an accuracy trade.
The gap widens for iterated Laplacians. At Δ³ the closed form is roughly 480x faster than a nested forward-Laplacian, which then runs out of memory at Δ⁴ while the closed form stays at ~0.1 ms.
When you should not reach for this
If your network is a general architecture with no closed-form activation tower, a sparsity-aware forward-Laplacian library is the right tool. Derivon wins precisely when the field is built from Riccati-class activations - which covers the overwhelming majority of PINN, VMC, and operator-learning ansätze.
The honest scope: closed-form means exact-by-construction where the math permits, autodiff-exact where we differentiate an analytic expression, and clearly-labelled numerical where a grid or sample is involved. We never blur those three.
Try it
The fastest way to see the win is to drop the closed-form Laplacian into your existing field and compare against jax.hessian on your own problem sizes. The documentation has copy-paste quickstarts for both PyTorch and JAX.