Runtime, results, and persistence

Solving¶

model.solve(params=..., log_level=...) returns one immutable SolutionResult for every built-in or external solver. Its values store is indexed as period -> regime -> value array; replay and collective-dissolution data stay in the addressed artifact stores instead of changing the return type.

Optional arguments:

• max_compilation_workers caps parallel XLA compilation;

• log_path and log_keep_n_latest control diagnostic snapshots;

• retention selects which post-solve artifacts remain available.

There are no flag-selected tuple returns. Pass the complete result to model.simulate(solution=...); omitting solution asks simulation to solve first.

Solution results¶

model.solve(...) keeps values, metadata, replay artifacts, diagnostics, and explicit omission reasons in one SolutionResult:

from lcm.solver_api import ResultRetention

solution = model.solve(
    params=params,
    log_level="debug",
    retention=ResultRetention.VALUES_AND_REPLAY,
)

V_working = solution.value(period=0, regime="working")

The result and its supporting types live in the lightweight lcm.solver_api submodule; they are not re-exported from the top-level lcm namespace. The retention modes are:

Values live in a ValueStore; artifacts are addressed by an ArtifactRef(period=..., regime=..., key=...) and kept in immutable ArtifactStore instances. Both stores expose the same eager values after a solve and independently lazy entries after restoration. Inspecting coordinates, metadata, omissions, or load_state(...) does not load numerical data. After restoration, solution.value(...) and solution.values[period][regime] load only the requested value entry and verify its checksum first. Whole-store numerical traversal is explicit through solution.values.materialize(); per-coordinate access preserves independent laziness. For ordinary array artifacts, store[ref] does the same. A plugin-defined PyTree requires store.materialize(ref, template=...) (as used by model replay) so its non-executable archive leaves can be rebuilt safely. LoadState.UNLOADED is storage state, not an omission reason.

Both stores, and the omissions mapping of SolutionResult, admit a public mapping through exactly one item traversal. Every raw coordinate is checked before it is inserted anywhere: a ValueStore period must be an exact int and a regime name an exact str (so True is refused rather than merged into 1), an artifact or omission address must be an exact ArtifactRef, and a logical address that the traversal emits twice is refused instead of contracted. A mapping’s key view or length is never consulted, so a mapping whose keys disagree with its items cannot smuggle a payload past the checks. Nested period -> regime -> value and flat (period, regime) -> value forms are recognized from the same traversal.

Built-in key identities include SIMULATION_POLICY, DISSOLUTION_FLAG, EGM_CONTINUATION, and SOLVER_DIAGNOSTICS. The omissions mapping distinguishes an artifact that is not applicable, not requested, unsupported, or not persisted.

Metadata carries a durable SHA-256 model fingerprint, the exact digest of canonical parameters that can affect the solution, solver and replay-route identities, artifact descriptors, and the solution and solver-interface schema versions. The fingerprint covers the mathematical model and the facts needed to interpret stored arrays, including period/regime topology, state and action names, grid support and category order, solver/replay/artifact versions, numerical conventions, and those solution-relevant parameter values. It excludes execution-only details such as devices, JIT selection, tiling, sharding, and compiler versions, as well as parameters and callable semantics used exclusively by simulation-phase transitions. An array a declaration references — through a closure cell, a default, or a solution-relevant parameter — is hashed with the rank and shape it actually has, so a scalar array and a length-one vector holding the same bytes are different identities; its memory order is storage rather than identity and does not enter the digest. A model function wrapped by a beartype guard — pylcm’s own or one a downstream package’s claw installed — is fingerprinted through the guard: the guard is accepted as transparent only when beartype regenerates its code from the bound callee with the guard’s own configuration, and the callee is what enters the identity. A result that nothing references any more releases its value and artifact arrays: no per-call closure inside pylcm keeps them alive, which matters because the package’s beartype claw decorates every function definition and beartype retains each decorated function object for the life of the process. Separately, an in-memory result keeps its model-instance token; that same-instance check is not applied to a restored archive. metadata.source records that distinction as IN_MEMORY or PERSISTED.

Each (period, regime) value has a lightweight ValueArraySchema recording its exact shape, dtype, and canonical named axes. Artifact descriptors play the corresponding descriptive role for retained payloads. Neither authenticates returned data. Simulation rebuilds immutable authority from the canonical model, canonical parameters, and the installed consuming route, then checks values, repeated metadata, and materialized artifacts independently.

Solver diagnostics follow log_level, independently of retention. Each retained diagnostics payload is described by a model-verifiable descriptor the solve generates from the payload itself, so it is saved with the result and reads back from an archive without a model; a consuming model admits the descriptor after checking that it names only the published fields with the dtypes they carry. A continuation is always available to the backward graph that requires it, regardless of result retention. It remains in the returned result only under ALL_PERSISTABLE_ARTIFACTS and only when its model-built authority declares MODEL_VERIFIABLE; otherwise the result records NOT_REQUESTED or NOT_PERSISTED.

A retention also selects what a solve computes. Every built-in kernel publishes its programs with a scope and, for replay or additive artifact programs, exact retained_artifact_keys and an exact retained_artifact_payload_types entry for every key. All programs retaining the same key must agree on its type. The type describes the final artifact in the period kernel’s KernelOutput, after any composite or adapter transformation; it does not assert that the artifact is present in every invocation. This lets even an inapplicable omitted cell carry an exact descriptor. The solve compiles and runs only the programs selected for that period/regime cell. Each replay program explicitly names the values-only program it replaces, so selecting one replay artifact cannot suppress an unrelated values program in the same multi-core graph. VALUES runs the values-only programs everywhere, so DCEGM and NB-EGM publish their values and carries without assembling a policy, and a nested NNBEGM solve folds its candidates without building replay banks or the adaptive nested policy. A replay-retaining solve runs replay programs only where a declared replay route consumes them. ALL_PERSISTABLE_ARTIFACTS selects replay alternatives and additive artifact-only programs per exact model-authoritative artifact address; it does not widen a regime-level boolean. A standalone case-piece NB-EGM regime has no replay consumer, so it runs its values-only program under every retention and its policy is recorded as NOT_APPLICABLE. Values and carries agree across retentions to the working format’s spacing.

A values-only solve is the cheap way to obtain value functions from a case-piece or piecewise-affine budget model, for example to compare solvers or to sweep parameters:

from lcm.solver_api import ResultRetention

values_only = model.solve(
    params=params,
    log_level="warning",
    retention=ResultRetention.VALUES,
)

V_alive = values_only.value(period=0, regime="alive")

Such a result simulates only where every decision is recoverable from values; a model whose simulation reads an NB-EGM or NNBEGM policy needs the default retention before it can be simulated.

ALL_PERSISTABLE_ARTIFACTS keeps only artifacts whose model-built authority declares PersistencePolicy.MODEL_VERIFIABLE. The NNBEGM replay policy of an AdaptiveOuterMesh search is replayed against the exact mesh the solve generated. Those nodes are solution-owned data: the result carries them as the candidate axis of the policy’s descriptor, and a consuming model admits them after checking what a shared mesh must satisfy (exact finite floats, strictly increasing, within the search’s node budget and the outer state’s domain for that period) before it compares the rest of the descriptor against its own authority. The adaptive policy and the finite candidate bank of a FiniteOuterGrid search are therefore both retained under both modes and written to the complete archive. A built-in EGMCarry continuation is also model-verifiable and is retained only by ALL_PERSISTABLE_ARTIFACTS, making that mode strictly broader than replay-only retention for an EGM regime.

save_solution(solution=..., path=...) atomically writes the complete labelled result to a versioned archive; solution.save(path=...) is the equivalent convenience method. The archive contains JSON metadata and independently addressed numerical datasets with SHA-256 checksums. It contains no model, Python class, callable, pickle, or executable code. A result restored by load_solution can be saved again: its payloads are re-read from the archive it came from, verified against their checksums and descriptors, and written to the new archive without a model. See Standalone persistence for loading and version compatibility.

Simulation¶

model.simulate(...) accepts parameters, initial conditions, an optional complete SolutionResult as solution=..., and a required log_level. Omitting solution solves first. Bare value mappings and separate policy or dissolution-flag inputs are not accepted.

How a SolutionResult is consumed follows its provenance. A result the same model instance solved in this process, for the same canonical parameters, is consumed by reference: the engine reads the arrays the solve allocated without copying or re-validating them, and only checks the replay payloads and dissolution flags it is about to read. Every other result — restored from an archive, unpickled, or produced by another model instance — is validated in full and materialized into private buffers once per consuming model and parameter vector; a second simulation from the same result reuses that validated view. The consumed result stays reachable as SimulationResult.solution until the simulation result is saved.

Before consuming such a result, simulation checks its durable model and solution-parameter fingerprints, exact solver/plugin and replay-route identities, schema versions, period count, regime order, solver types, and exact active period/regime value coverage. An in-memory result additionally has to come from that model instance. It unconditionally checks every required value and its descriptive schema independently against the model-owned shape, canonical dtype, and named axes, including at log_level="off".

The model’s side of that identity is fixed when the model is built. Model(...) digests the structure once — topology, names, identities, and every declared callable’s semantics — and records each global and closure binding those callables read. Rebinding one of them afterwards, for instance by reassigning a module-level parameter a utility function closes over, would make the model run code its identity no longer describes, so solve() and simulate() refuse with ModelSealError naming the binding. Build a new model instead. A pickled model is resealed against its stored identity when it is loaded.

All artifact stores and omission records must address active result cells with the exact key version and channel; one reference cannot appear in multiple stores or be both present and omitted. Required lazy entries are materialized during preflight and checksum verified. Numerical leaves are copied into private owned buffers; a plugin-defined PyTree is rebuilt as a fresh exact tuple or structurally closed dataclass record from the model authority’s sealed construction plan, without another plugin flatten or unflatten callback. PyTree-represented static metadata is checked against the plan, while callback-injected instance state is replaced by its declared canonical value. The consuming route receives an owned replay snapshot rather than the caller’s container or array objects. Built-in EGM/NNBEGM policies and collective dissolution flags retain their specialized validation. An installed external route additionally validates solver-specific invariants and builds a JAX-transformable replay reader; this does not sandbox installed plugin code. No forward-simulation step runs until all required entries pass.

A values-only result is therefore sufficient for a model whose decisions are fully recoverable from values, but fails closed before forward simulation when a required replay artifact is absent or invalid. Use the default ResultRetention.VALUES_AND_REPLAY when the model may require such artifacts.

subject_batch_size streams subjects without changing results. seed controls random draws. A collective model may require an addressed dissolution replay artifact and own_stakeholder; see Collective regimes.

Initial conditions are a mapping of state names plus regime_id to equal-length arrays, or a DataFrame with a regime_name column.

Validation and logging¶

log_level controls both output and runtime validation:

Start model development at "debug". Reduce validation only after the model is trusted and the cost matters.

pylcm enables a persistent JAX compilation cache by default. Set JAX_COMPILATION_CACHE_DIR to choose the full directory or LCM_COMPILATION_CACHE_NAME to choose the project-specific leaf. Set XLA_PYTHON_CLIENT_PREALLOCATE=true before importing pylcm to restore JAX’s device preallocation; pylcm otherwise requests on-demand allocation.

SimulationResult¶

to_dataframe(additional_targets=None, use_labels=True, terminal_rows="first") materializes a flat DataFrame. additional_targets accepts selected DAG outputs or "all". terminal_rows="all" retains every frozen absorbing row; the default keeps only terminal entry.

Inspection properties include regime_names, state_names, action_names, n_periods, n_subjects, available_targets, raw_results, flat_params, and period_to_regime_to_V_arr.

SimulationResult.save(directory=...) writes array checkpoints, value functions, metadata, and a Feather table. SimulationResult.load(directory=...) restores it.

Standalone persistence¶

• save_solution(solution=solution, path=path) writes a complete SolutionResult atomically. A failed write leaves no partially published replacement.

• load_solution(path=path) reads metadata and constructs independently lazy value and artifact stores. Every entry is checksum verified when materialized.

• load_solution(path=path, verify_checksums=True) verifies every payload without changing any entry from UNLOADED to LOADED.

• load_legacy_solution(path=path) explicitly reads the old value-only HDF5 format for migration. Legacy values have no durable model fingerprint, descriptors, omissions, or checksums and are not a complete replay result.

• SolveSnapshot and SimulateSnapshot describe diagnostic snapshots.

• load_snapshot(path, exclude=...) loads a snapshot, with optional components omitted.

The solution archive requires exact matches for its format, solution-schema, and public solver-interface versions. Replay additionally requires the matching plugin, route, and artifact-schema versions. pylcm reports incompatible versions rather than guessing a migration. Without an external plugin installed, load_solution can still read standard metadata and omissions, verify all checksums, and inspect values or array payloads lazily. A plugin-defined PyTree cannot be interpreted or replayed until the matching route supplies its model-authoritative template.

Workflow: Solving and simulating, DataFrame interoperability, and Debugging.