Lowering modifiers to dynamical systems

Introduction

LoweredDistributions.jl lowers a distribution to a backend-agnostic dynamical-systems representation: lower(dist) returns an AbstractLowering, either a CTMC (a continuous-time Markov chain generator) or an AbstractChainTrick phase-type (ErlangChain, Coxian, PhaseType). From there a backend extension can build a Catalyst reaction network, an ODE, a Petri net, or a jump process. When LoweredDistributions is loaded alongside ModifiedDistributions a package extension defines lower for the modifier leaves.

The bridge is partial by mathematical necessity. A modifier carries either dynamics, which lower, or observation, which is refused rather than silently approximated (the same contract as LoweredDistributions' throw-on-non-integer-Erlang).

• Lowers: an affine rescaling (shift = 0) divides every rate; a modify hazard modification on a constant-hazard Exponential leaf stays an Exponential.

• Refused: an affine shift (a deterministic delay is not phase-type), a weight (a likelihood weight has no dynamics), a forward transform (thin/cumulative/series_transform), and a modify on a non-Exponential leaf or under a non-analytic link.

What are we going to do in this exercise

2. Lower a plain Exponential and Gamma leaf to set the scene.

3. Lower the modifiers that carry dynamics (affine rescaling and modify on an Exponential).

4. See the explicit refusals for the observation-only modifiers.

5. Lower a modifier applied to a ComposedDistributions chain.

What might I need to know before starting

This tutorial builds on the Getting started overview and the modifier pipeline tutorial. It additionally needs LoweredDistributions.jl and ComposedDistributions.jl, both pinned in the docs environment.

Packages used

Loading LoweredDistributions activates the lowering extension; loading ComposedDistributions activates the composed-chain extension.

using ModifiedDistributions, Distributions
using LoweredDistributions
using ComposedDistributions

Lowering a plain leaf

An Exponential is memoryless, so it lowers to a two-state CTMC: an on state leaving to absorbed at the exponential's rate.

lower(Exponential(2.0))

CTMC continuous-time Markov chain
  states: on, absorbed
  on -> absorbed @ rate 0.5

An integer-shape Gamma (an Erlang) lowers to a linear chain of exponential sub-compartments, the linear-chain trick.

lower(Gamma(3.0, 1.5))

ErlangChain(3 compartments)

An affine rescaling divides every rate

A phase-type is closed under positive time-scaling: stretching every holding time by scale divides every transition rate by scale. So affine(d; scale) with shift = 0 lowers exactly. Scaling an Exponential(2) by three is an Exponential(6), and the lowered generators match.

rescaled = affine(Exponential(2.0); scale = 3.0)
lower(rescaled)

CTMC continuous-time Markov chain
  states: on, absorbed
  on -> absorbed @ rate 0.1667

The rescaled generator is the plain Exponential(6) generator: the rate has been divided by the scale (0.5 / 3 = 1/6).

lower(rescaled).Q ≈ lower(Exponential(6.0)).Q

true

The same holds for a Gamma chain: each per-stage rate is divided by the scale.

lower(affine(Gamma(3.0, 1.5); scale = 2.0))

ErlangChain(3 compartments)

A hazard modification on an Exponential leaf

A hazard modification is exact rate-scaling only when the base hazard is constant, i.e. on an Exponential. Under the log link (proportional hazards) the rate is multiplied by exp(effect), so halving the hazard of an Exponential(2) (rate 0.5) gives rate 0.25, an Exponential(4).

prop = modify(Exponential(2.0), -log(2.0); link = log)
lower(prop)

CTMC continuous-time Markov chain
  states: on, absorbed
  on -> absorbed @ rate 0.25

Under the identity link (additive hazards) the constant effect is added to the rate; Exponential(2) (rate 0.5) with effect = 0.5 gives rate 1.0, an Exponential(1).

add = modify(Exponential(2.0), 0.5; link = identity)
lower(add).Q ≈ lower(Exponential(1.0)).Q

true

The refusals

The observation-only modifiers throw an explicit ArgumentError rather than lower to a wrong dynamical system. We catch and print each message; in real code these propagate as errors.

An affine shift adds a deterministic delay, which no finite Markov chain reproduces.

try
    lower(affine(Exponential(2.0); shift = 1.0))
catch e
    println(e.msg)
end

lowering an affine transform is exact only for a pure positive rescaling (shift = 0). A deterministic shift adds a fixed delay, which is not phase-type — no finite Markov chain has a deterministic holding time. Got shift = 1.0. Erlang-approximate the shift explicitly if an approximation is acceptable.

A weight is a likelihood weight with no dynamics content.

try
    lower(weight(Exponential(2.0), 3.0))
catch e
    println(e.msg)
end

a Weighted distribution carries a likelihood weight, not dynamics: it rescales logpdf and has no rate-structure image, so it does not lower to a dynamical system. Apply the weight at inference time, outside the lowered model.

A thin does have a dynamical reading — a competing-risk split, a fraction p into the event chain and 1 - p into a NoEvent absorbing sink — but LoweredDistributions' wave-1 API exposes no such sink, so it is refused until that lands rather than dropping the competing risk.

try
    lower(thin(Exponential(2.0), 0.3))
catch e
    println(e.msg)
end

lowering thin(d, p = 0.3) is not yet supported: it needs a NoEvent competing-risk sink in LoweredDistributions. The dynamical reading of thinning is a competing-risk split — probability p into the event chain, 1 - p into a NoEvent absorbing sink — but LoweredDistributions' wave-1 API has no NoEvent / competing-risk (MAPH) sink type (PhaseType requires the initial distribution to place all mass on event-reaching phases). Refusing rather than dropping the 1 - p competing risk.

A cumulative is a running sum on a count series, with no rate-structure image.

try
    lower(cumulative(Exponential(2.0)))
catch e
    println(e.msg)
end

cumulative() accumulates a downstream count series; a running sum is a series operation with no rate-structure image, so it does not lower to a dynamical system.

A modify on a non-Exponential leaf has a time-varying hazard, so the modified object is not phase-type.

try
    lower(modify(Gamma(2.0, 1.0), 0.3; link = log))
catch e
    println(e.msg)
end

lowering a Modified (hazard-modified) distribution is exact only on a constant-hazard Exponential leaf: proportional/additive hazards are exact rate-scaling only when the base hazard is constant. Got a Distributions.Gamma{Float64} leaf, whose hazard varies with time, so the modified object is not phase-type (it would need time-varying rates). Refusing rather than mis-lowering.

Lowering across a composed chain

A ComposedDistributions Sequential chain collapses to the one scalar it observes, exposed by observed_distribution (see the Modifiers across composed chains tutorial for why there is exactly one). That total is a distribution like any other, so it lowers: with no exact closed form it takes the adaptive two-moment phase_type fit.

chain = sequential(:onset_admit => Gamma(2.0, 1.0),
    :admit_death => Exponential(1.5))
observed = observed_distribution(chain)
lower(observed)

PhaseType(3 phases)

The modifier bridge composes with the chain. Applying affine to the chain collapses it to its observed total and wraps that in an Affine (the composed-chain extension), and lowering that rescales the fitted phase-type: doubling the chain doubles the observed mean, so the fitted rates halve.

scaled_chain = affine(chain; scale = 2.0)
lower(scaled_chain)

PhaseType(3 phases)

The rescaled fit is the observed total's fit with every rate divided by the scale, so its mean is doubled — the modifier bridge and the composed-chain bridge stack cleanly.

(chain = mean(observed), affine_2x = 2.0 * mean(observed))

(chain = 3.5, affine_2x = 7.0)

Summary

• lower maps a distribution to a CTMC or a phase-type dynamical-systems representation.

• The modifiers that carry dynamics lower: an affine rescaling divides every rate, and a modify on a constant-hazard Exponential stays an Exponential.

• The observation-only modifiers (affine shift, weight, thin/cumulative/series_transform, and modify off a constant-hazard leaf) are refused with an explicit error rather than silently mis-lowered.

• The bridge composes with ComposedDistributions: a modifier applied to a chain lowers through the chain's observed total.