build-system.md

Build system

How the pieces are built and wired, and the open design questions. The entry point is the root justfile; subsystem justfiles are exposed as modules (just <module>::<recipe>, see just --list).

The pieces

1. Emulator (machine/emulator, git submodule): C++ Cartesi Machine. Any set LIBCARTESI_PATH selects a prebuilt static archive, and the path must be absolute; the nix devshell exports one from its immutable package. INCLUDECARTESI_PATH selects its C API headers, or the build infers the conventional sibling include/cartesi-machine directory. With LIBCARTESI_PATH unset, the prepared submodule is the source provider and Cargo runs incremental Make with slirp=no. The machine-local Just module owns the generated release sources and verified Boost headers that Make consumes. The sys crate build script is network-free and owns no acquisition policy.
2. Solidity step (machine/step, git submodule): the auto-generated on-chain uarch state transition, consumed by prt/contracts tests and sources via foundry remappings.
3. Contracts (prt/contracts, cartesi-rollups/contracts): foundry projects; dependencies via soldeer (just <mod>::install-deps).
4. Rust bindings (*/bindings-rs): generated by forge bind from compiled contracts, filtered to the supported production contract surface. They are gitignored: generated code, regenerated by just bind.
5. Rust workspace: the node, the two generated contract-binding crates, and the two machine-binding crates. Its Cargo recipes do not choose a source-acquisition feature: the environment selects an external library, or the explicit machine preparation lifecycle readies the source fallback before Cargo starts.
6. Test programs (test/programs): machine images built with the cartesi-machine CLI from downloaded kernel/rootfs artifacts; the honeypot image additionally needs docker.
7. Devnet bundle (cartesi-rollups/contracts/{state.json,state.fingerprint} plus deployments/31337): an anvil state dump with everything deployed, its deployment records, and the marker that binds those outputs to the production and deployment inputs that produced them. build-devnet produces the complete bundle and the e2e harness refuses an incomplete, stale, or mixed one. Release archives preserve the same three-part unit.

Consequence of (1) and (4): raw Cargo works after the gitignored Solidity bindings exist and either a valid external machine provider is selected or the source checkout is prepared. just setup and just build are the supported fresh-clone path; just bind is enough before raw Cargo when an external provider is already available.

Incremental bind

just bind regenerates bindings only when the contract sources or the binding configuration actually changed (a content-hash stamp at <contracts>/bindings-rs/src/.bind-stamp, gitignored). just bind-force bypasses the stamp. Historically, bind cleaned and regenerated unconditionally as a dependency of every cargo recipe, which forced binding-crate rebuilds on every build and made pure-Rust iteration needlessly slow; do not reintroduce that.

One shared generator owns both modules' filters and stamp contract. The digest binds the generator and exact forge bind arguments, the Forge executable and version, relevant effective compiler configuration, the module's Soldeer lock, its local sources, configured dependency roots, and imported PRT and step source roots where applicable. It deliberately excludes Just orchestration, build caches, compiler output, and the generated bindings themselves.

Binding generation disables the Foundry test and script roots explicitly, so an unrelated fixture or deployment-script edit neither recompiles contracts nor invalidates Rust bindings. The digest hashes the local production source root and imported production content, but retains every effective remapping string: even an unused remapping changes the metadata bytecode embedded by Forge. The Rollups-only prt-contracts-test target content is therefore excluded while its remapping remains part of the effective configuration.

The filters expose only production contract types: CartesiStateTransition, Tournament, and MultiLevelTournamentFactory for PRT, and the concrete and interface forms of DaveConsensus and DaveAppFactory for Rollups. Test contracts and fixture factories are not part of the generated Rust API.

Doctor and artifact receipts

Component doctors use one exit contract: 0 means healthy, 1 means a missing or stale setup artifact was diagnosed, and 2 means the checker could not determine the result. The root doctor invokes the component scripts directly and aggregates their statuses without parsing their output. This keeps script/doctor.sh usable even when Just itself is the suspected failure; the module recipes are discoverable aliases. The two contract modules share one parameterized dependency-and-binding checker because their checks are identical apart from paths and labels.

just doctor covers build and pre-commit-check readiness, including the echo and yield images consumed by the standard Rust suite. The devnet bundle, Honeypot image, and retained E2E state belong to just doctor-e2e; just doctor-all runs both scopes. A checkout can therefore be healthy for ordinary development without constructing the expensive E2E fixtures.

The devnet receipt is deliberately narrower than the contract worktrees. Its input digest covers production and deployment Solidity, installed production dependency Solidity and lockfiles, the production machine/step sources, effective Forge compiler and deployment configuration, the build and deploy drivers, and the Forge and Anvil versions. It excludes documentation, tests, measurements, compiler output, broadcasts, and prior deployments. Those files cannot change the deployed bundle, and treating them as inputs made doctor report false staleness. The receipt separately hashes state.json and every deployment record, so copying or interrupting a bundle still fails closed.

Each persistent test image likewise records the inputs that produced it and its semantic stored-machine root. The input digest names one producer script for that image rather than the shared programs Justfile. Editing the Honeypot producer therefore does not invalidate echo, yield, or stress. The v2 receipt format intentionally makes the old shared-recipe receipts stale once; rebuild the image with the fix printed by the relevant doctor scope.

Deployment generations

The public function, event, and error ABI and the ERC-1167 clone-argument encoding are stable within one deployment generation. An intentional wire break starts a new generation: deploy a fresh Tournament implementation, MultiLevelTournamentFactory, and dependent Dave bundle; regenerate bindings and deployment artifacts; and ship matching Rust and Lua clients. Existing clones remain bound to their old implementation. No live dispute or persisted event stream may cross this boundary, so the current line carries no dual decoder. Binding stamps prevent accidental local staleness, but they are not a protocol negotiation mechanism.

Storage layout is internal while the contracts have no upgrade or state migration path and no supported raw-storage client. Test-only probes follow intentional layout changes. Storage and bytecode hashes report implementation and deployment impact rather than promising equality. Changed bytecode and CREATE2-derived addresses identify a new deployment bundle and must be regenerated together.

The node database follows the same clean-slate policy: storage/sql/schema.sql is the only schema definition and has no upgrade steps. An empty database is created from that file and atomically stamped with the node package version and the Keccak hash of the exact schema bytes. An existing database is never given DDL at startup; its two identity values must match the running binary. A mismatch requires deleting the state directory and rebuilding it from the chain and machine image.

Open design questions

Bindings generation remains open. The resolved emulator policy is retained in this section so later changes start from its reasons instead of re-litigating the discarded acquisition paths.

Bindings generation

Current: gitignored, stamp-guarded forge bind orchestrated by just. Considered alternatives:

• Commit the generated bindings (+ CI freshness check). Pro: reviewable diffs, no foundry needed for Rust-only work. Con: generated-file churn in the repo on every foundry release and contract change. Rejected for now for that reason.
• Generate from build.rs of the bindings crates. Pro: cargo tracks staleness natively (rerun-if-changed on contract sources), no just involvement, works for cargo invocations that bypass just. Con: makes foundry (and soldeer-fetched deps) a build-time requirement of the workspace; build.rs network access for soldeer is unpleasant.
• Publish bindings as versioned crates on contract releases. Pro: clean consumer story. Con: painful while contracts and node co-evolve in one repo.

The stamp approach forecloses none of these.

The emulator dependency (resolved for v0.21)

The provider policy has two paths and no implicit acquisition inside Cargo:

• External: any set LIBCARTESI_PATH selects the absolute directory containing libcartesi.a. An empty or invalid value fails instead of falling back. INCLUDECARTESI_PATH explicitly selects the absolute directory containing cm.h and cm-version.h; when it is unset, the conventional sibling include/cartesi-machine directory is used. The Nix devshell is the default development instance of this path and remains the immutable owner of its package.
• Source: with LIBCARTESI_PATH unset, just machine::setup initializes the emulator submodule, verifies and publishes the pinned v0.21 generated-file artifact and Boost headers, and performs one native incremental Make build. This tagged fallback needs no Docker. An intermediary commit instead runs just machine::generate-sources, normally through the upstream Docker toolchain, followed by prepare-boost and build.

Release preparation refuses a checkout other than the exact pinned emulator commit. Downloads are SHA-256 checked and cached under target/machine-source; generated files are validated before publication. just machine::clean removes source-provider outputs but retains that verified cache. just machine::doctor checks the pinned step checkout and selected provider; the root just doctor aggregates that result with the other subsystems.

cartesi-machine-sys/build.rs selects the provider, generates bindings, runs incremental Make for an already prepared source checkout, stages archives, and links them. It performs no network access or source generation. Cargo watches mutable external archives and headers at their selected paths. Source mode also watches the prepared inputs, the submodule gitfile, and the resolved Git index. A moved checkout therefore rechecks Make, while an unchanged second Cargo invocation does not invoke it.

Dave's Nix and fallback libraries use slirp=no; the node does not use VirtIO net-user. The sys crate still links libslirp for an external upstream release package because that package's static archive references it. This keeps full system installations compatible without making slirp part of the source-build requirement.

CI mirrors the policy. Package-backed jobs export /usr/lib and /usr/include/cartesi-machine; source-fallback jobs prepare the checkout explicitly before Cargo; and the Docker image builds the emulator once, then uses its installed archive as the final-stage external provider. Docker setup prepares its build context without first building an unused host archive.

Release acquisition and image-generation pins remain distributed across the external devshell flake, the machine preparation script, CI, and the per-image producers under test/programs/script/. Semantic version guards also live in the safe Rust wrapper, node configuration, and computation-corpus gate. An emulator bump must update every matching guard and artifact together with both machine submodules and regenerated program images; search for the current version rather than treating the acquisition list as exhaustive.

just test-computation-hash-corpus is the explicit emulator release gate. It downloads the pinned v0.21 corpus and checks its SHA-256. One test replays the complete mcycle and uarch manifest through the release CLI; a separate test compares every mcycle case that Dave supports directly with the published answer. The split keeps CLI packaging conformance distinct from Dave's collector conformance. Acquisition stays outside both setup and just check, and this gate does not opt the node into the new bulk collection API.

macOS + nix devshell note

The cartesi-dev flake devshell (outside this repo) provides all tools and exports LIBCARTESI_PATH, so cargo builds work pure-nix with no toolchain overrides. Two historical traps, fixed in the flake on 2026-07-02 but worth remembering:

• The nixpkgs default apple-sdk (14.4, and 15.5 as well) ships a broken hfs_format.h (uuid_string_t used without its uuid/uuid.h include), which failed the C builds of cargo deps like aws-lc-sys. Apple fixed the header in the SDK 26 generation; the flake now lists apple-sdk_26 explicitly (in buildInputs, not packages: only the host role sets SDKROOT, and SDK selection is per-role by version).
• Escaping to CC=/usr/bin/clang does not work under the devshell: the macOS clang trampoline resolves the real toolchain through DEVELOPER_DIR, which the devshell points back into nix.

Also note RUSTFLAGS set in the environment replaces (not merges with) .cargo/config.toml rustflags. The config file in question is the developer's own global one - this repo commits none; homebrew LLVM paths in such a file assume a non-nix setup and are dead weight under the flake.

Fresh worktree bootstrap

A new git worktree starts without the gitignored artifacts. Run just bootstrap-worktree [SOURCE]: it runs the provider-aware machine setup, installs soldeer deps in both contract dirs, regenerates bindings, optionally copies machine images and devnet state from SOURCE (a green sibling worktree), and ends with just doctor-all when SOURCE was supplied or the base just doctor otherwise. A copied machine image is accepted only when its sidecar proves that the current inputs match and that the stored machine root is unchanged. Devnet state, deployments, and state.fingerprint move as one bundle; the marker binds the production contract and deployment sources, installed production dependencies, effective compiler configuration and tool versions, state dump, and deployment files. Any mismatch forces a rebuild. Without a SOURCE, rebuild the artifacts with just setup-local.

These checks prevent the 2026-07-14 failure mode: a devnet deployed from older contract sources fails e2e with a misleading consensus assert. Fingerprints are published only after successful image construction or after Anvil has dumped a complete state, so interrupted builds cannot bless their old output.

Shell and tooling traps

Collected from real incidents; agents especially, read before debugging the environment:

• direnv exec resolves its command argument itself: a nonexistent path reports "not found on PATH" even when given absolutely - check that the file exists before suspecting the environment.
• Pipelines swallow exit codes: cargo build ... | grep ... | head reports the last command's status, not cargo's. Two real incidents of "successful" builds that never produced a binary. Verify the artifact exists, or use pipefail and inspect PIPESTATUS.
• cartesi-machine --store=DIR does not create DIR's parent directory.
• Background shells inherit the foreground shell's current directory; cd explicitly in anything that may run from elsewhere.
• Piped tail buffers until process exit; do not wait on it for live progress.

Validation

just check is the pre-commit gate: fmt checks (Rust workspace and both contract dirs), luacheck over the Lua client and harness, clippy with warnings denied, the build-tooling regressions, the provider-free contract suites, and the Rust and Lua unit suites. just doctor diagnoses build/check inputs, including the standard echo and yield images; just doctor-e2e diagnoses the E2E image set, devnet artifacts, and E2E litter; just doctor-all aggregates both. Every failed check prints the command that fixes it. Run the relevant scope before debugging a mysterious failure, especially in a fresh worktree. The e2e test recipe runs a preflight with the same spirit: missing artifacts fail with a named fix, not a cryptic jq or Lua error.

Formatter versions matter: forge changes wrapping heuristics across releases (observed live: 1.4.3 and 1.5.1-dev disagree about an 80-column function header in Match.sol). When local check and CI disagree about contract formatting, align the root justfile's FOUNDRY_VERSION pin with the devshell forge instead of hand-formatting around either. The setup-tools action reads that pin through just print-foundry-version.

CI

The per-PR and tag workflow (.github/workflows/build.yml) has jobs for PRT contracts (disputes, structured STF tests, and structured STF fuzz), consensus contracts, the honeypot e2e smoke (test-rollups-honeypot-ci) plus batched-kill, chaos, stf_all, and yield stf_revert scenarios, a Rust workspace job (fmt, check, clippy, Lua lint and unit tests, Rust tests, and explicit image-backed machine differentials), and the release pipeline (node binaries per arch, contract artifacts, deployment simulations, devnet state). The e2e lane also runs both halves of the computation-hash release corpus gate. Actions are pinned by digest. When renaming just recipes, grep the workflow first; CI calls them by name. Shared acquisition and provider policy belongs in actions or Just targets, not copied shell programs in the workflow. In particular, the emulator release commit and artifact digests live in the machine preparation script; CI only selects a lane and installs or consumes its result.

The manual Full E2E Battery workflow runs the complete E2E matrix plus two extra chaos seeds on a deliberately selected ref. Per-PR CI retains the smaller deterministic subset above. Its design goals are to exercise every maintained scenario in one isolated environment, explore a small reproducible seed set, retain useful failure logs, and bound hosted-runner storage by removing each scenario's machine state after it finishes. It is not a performance benchmark, a required pull-request gate, or currently a scheduled monitor.

GitHub exposes manual dispatch only after the workflow file exists on the repository's default branch. Until this new workflow reaches that branch, its hosted dispatch is intentionally unavailable; use the per-PR subset or run the full battery locally. Once available, a dispatch may select another ref that contains the workflow.

Manual-only is deliberate while its cost and signal have not been measured for the current 25-case suite on the selected hosted runner. One invocation also builds the devnet, three machine images, and the validator, then runs 25 battery scenarios and two additional chaos seeds. The battery is serial during this calibration phase: one scenario may use about 5 GB, while GitHub documents only 14 GB of SSD on a standard ubuntu-24.04 runner. State is removed after each scenario, logs are retained for seven days, and the job has a 150-minute cap. The older local 21-case battery measured about 42 aggregate scenario-minutes; it is useful context, not a hosted-runner forecast for the current suite.

GitHub permits repeated dispatches of the same commit. The fixed concurrency group allows one running and one pending invocation; a newer dispatch replaces the older pending one. It prevents overlap but does not deduplicate completed runs. Promote the workflow to a schedule only after representative runs record wall time and establish a safe disk bound, failures prove actionable rather than flaky, an owner exists for triage, and the scheduled form avoids rerunning an unchanged default-branch commit. Until then, use it before a release or after changes to E2E orchestration, storage/recovery, machine images, or dispute wiring.

Machine provider coverage is intentional: package lanes exercise the external archive, while release-node builds prepare and execute a wrapper test against the source fallback. The full corpus is explicit locally because it downloads release evidence, but CI owns one required execution so release conformance cannot silently rot.