# Version-resolution algorithm Status: in progress (Phases 1–2 of 6 done). This doc fixes the contract for **`(package, version) → nixpkgs commit_hash`** discovery and the flake-codegen pipeline that consumes it. It overrides `SPEC.md` §10's single-shared-rev model with a per-dep-rev model (user-directed; SPEC amendment is Phase 6). ## Overview ``` cargoxx add @ │ ▼ ┌──────────────────────┐ │ resolve_version(name,│ │ version) │ └──────────────────────┘ │ │ primary HTTP │ │ offline fallback ▼ ▼ ┌──────────────────┐ ┌──────────────────────┐ │ devbox_resolve │ │ nixpkgs_git_resolve │ │ search.devbox.sh │ │ ~/.cache/cargoxx/ │ │ /v1/resolve │ │ nixpkgs/ (lazy) │ └──────────────────┘ └──────────────────────┘ │ │ └──┬───┘ ▼ Result │ ▼ cmd_add writes nixpkgs_rev into Cargoxx.lock │ ▼ (later) cargoxx build │ ▼ codegen::flake_nix reads lockfile emits per-pinned-dep nixpkgs input ``` ## When does resolution run? | Trigger | What gets resolved | | --- | --- | | `cargoxx add @` | `(pkg, ver)` is resolved exactly once. The resulting commit is written to `Cargoxx.lock` next to the dep entry. | | `cargoxx add ` (no `@`) | **Not** resolved. Lockfile entry's `nixpkgs_rev` stays `nullopt`. The generated flake.nix uses only the shared `nixpkgs.url = github:NixOS/nixpkgs/nixos-unstable`. | | `cargoxx build` (lockfile already has rev) | **Not re-resolved.** `cargoxx build` reads existing lockfile entries and preserves `nixpkgs_rev`. Re-resolution would require an explicit `cargoxx update` (deferred to v0.3). | | `cargoxx build` (lockfile missing the rev for a dep) | Synthesized as null — same as the wildcard path. (Future: also call `resolve_version` here when manifest spec is concrete.) | `cargoxx build` is **idempotent with respect to the lockfile** — running it twice produces byte-identical `flake.nix` + `Cargoxx.lock` provided the manifest hasn't changed. This is the property the "lockfile merge" change in Phase 4 enforces. ## resolve_version ``` auto resolve_version(name: string, version: string) -> Result: if r := devbox_resolve(name, version); r.has_value(): return r->commit_hash if r := nixpkgs_git_resolve(name, version); r.has_value(): return *r return std::unexpected(ResolutionVersionNotFound) ``` Implementation point: this orchestrator lives in `src/resolver/resolver.cppm` (declaration) + `src/resolver/version_resolve.cpp` (definition). Both probes are already implemented — Phase 3 just wires them into the orchestrator and into `cmd_add`. ### Probe A — devbox_resolve (primary, HTTP) **File:** `src/resolver/search_devbox.cpp` (committed `df2c25b`) **URL pattern:** ``` GET https://search.devbox.sh/v1/resolve?name=&version= ``` This is the same endpoint devbox itself uses (`devbox/internal/searcher/client.go` `Resolve()`). Behind the URL is the same Jetify backend that powers nixhub.io. **Response shape (real, abbreviated for `fmt 10.2.1`):** ```json { "commit_hash": "f4b140d5b253f5e2a1ff4e5506edbf8267724bde", "version": "10.2.1", "name": "fmt", "attr_paths": ["fmt"], "systems": { "x86_64-linux": { "commit_hash": "f4b140d5b253f5e2a1ff4e5506edbf8267724bde", "attr_paths": ["fmt"], ... }, ... } } ``` **Parser contract** (`parse_devbox_resolve`): - `commit_hash` is mandatory. If the top-level field is missing, fall back to the first non-empty `systems..commit_hash`. - `name`, `version`, `attr_paths` are best-effort; absence leaves them blank. - 404 / curl exit 22 → `ResolutionUnknownPackage`. - Empty `commit_hash` after fallback → `ResolutionVersionNotFound`. - Other curl exits, JSON parse errors → `ResolutionNetworkError`. **Timeout:** 10 s on `--max-time`, 15 s wrapping `ExecOptions.timeout`. ### Probe B — nixpkgs_git_resolve (offline fallback) **File:** `src/resolver/nixpkgs_git.cpp` (committed in Phase 2 series) **Setup:** lazy clone of `https://github.com/NixOS/nixpkgs.git` into `$XDG_CACHE_HOME/cargoxx/nixpkgs/` (or `$HOME/.cache/...`) on first use. ~9 GB and slow (5–15 min); subsequent calls are fast and offline. **Search:** ``` git -C log --all \ -S 'version = ""' \ --pretty='%H %ct' \ -- pkgs/ ``` `-S ''` returns commits that *introduced or removed* the literal string. `--pretty='%H %ct'` emits ` ` per line. We restrict to `pkgs/` to keep noise down (out-of-tree match sites in `lib/`, `nixos/`, etc. don't matter). **Pick:** youngest committer-time (`%ct` highest) wins. The pure helper `pick_youngest_commit(text)` does this; it tolerates malformed lines (skips them). **Errors:** - `pick_youngest_commit` returns `nullopt` → `ResolutionVersionNotFound`. - Clone failure → `ResolutionNetworkError`. - Subsequent `git log` failure → `ResolutionNetworkError`. **Test fixture trick:** instead of cloning real nixpkgs in tests, the unit test builds a tiny throwaway repo with `pkgs/development/libraries//default.nix` files at two versions and asserts introducing-commit detection works. ### Heuristic limits `-S 'version = ""'` is fuzzy — it matches **any** file in `pkgs/` that has that literal. Two real-world failure modes: 1. **Unrelated package match.** `version = "1.0.0"` appears in many nix derivations. The youngest-commit tiebreaker biases toward "the most recent thing that touched this string", which usually *is* the package's bump commit, but not guaranteed. 2. **Non-string-formed versions.** Some derivations build the version via `lib.removeSuffix`, interpolation, or an inherited `pname`/`finalAttrs.version`. `-S` won't see those. For those packages, only the devbox HTTP path can answer. Both are accepted as known limits — the HTTP path is primary and fast when reachable; the git fallback exists only for offline determinism. ## Lockfile interaction `Cargoxx.lock` already carries `LockfilePackage.nixpkgs_rev` (`std::optional`). No schema change. ### Add path `cmd_add fmt@10.2.1`: 1. existing manifest validation, duplicate check, linkdb resolve / discover (separate auto-resolution feature, already shipped). 2. **NEW:** call `resolve_version("fmt", "10.2.1")`. On success, capture `commit_hash`. 3. existing manifest write of `[dependencies] fmt = "10.2.1"`. 4. **NEW:** load lockfile (or initialize empty), find/insert the `LockfilePackage{ name="fmt", version="10.2.1" }` entry, set `nixpkgs_rev = ""`, write lockfile back. `cmd_add fmt` (wildcard) skips step 2 and step 4's `nixpkgs_rev` assignment. ### Build path (Phase 4 fix) Today, `synthesize_lockfile` overwrites the lockfile every time. With per-dep revs in scope this would erase pinned revs on every build. The fix: ``` build_lockfile(manifest, recipes): let prior = parse(project_root / "Cargoxx.lock") or empty for each dep in manifest.dependencies: let prior_entry = prior.find(dep.name, dep.version_spec) new_entry = LockfilePackage{ name, version=dep.version_spec, ... } if prior_entry: new_entry.nixpkgs_rev = prior_entry.nixpkgs_rev emit new_entry ``` The lookup key is `(name, version)`. If the user changes the version, the prior rev is dropped (correct — the rev was for the old version). If the user neither edited nor `cargoxx update`d, the rev survives. ### Update path (deferred to v0.3) `cargoxx update ` would call `resolve_version` again with the existing manifest version_spec, possibly upgrading the rev to a newer one, even when the user-visible version string is unchanged. Out of scope for this milestone. ## Flake codegen — per-dep inputs **Phase 5.** Today's `flake.nix` template has a single `@@NIXPKGS_REV@@` placeholder. The new template emits: ### Inputs block ```nix inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; # one line per dep with non-null nixpkgs_rev: nixpkgs-fmt-10_2_1.url = "github:NixOS/nixpkgs/f4b140d5b..."; nixpkgs-spdlog-1_13_0.url = "github:NixOS/nixpkgs/abcdef0..."; flake-utils.url = "github:numtide/flake-utils"; }; ``` ### Outputs lambda ```nix outputs = { self, nixpkgs, nixpkgs-fmt-10_2_1, nixpkgs-spdlog-1_13_0, flake-utils }: ... ``` ### Let bindings ```nix let pkgs = import nixpkgs { inherit system; }; pkgs_fmt_10_2_1 = import nixpkgs-fmt-10_2_1 { inherit system; }; pkgs_spdlog_1_13_0 = import nixpkgs-spdlog-1_13_0 { inherit system; }; llvmPkgs = pkgs.llvmPackages; in {...} ``` ### buildInputs ```nix buildInputs = [ pkgs_fmt_10_2_1.fmt # pinned dep pkgs_spdlog_1_13_0.spdlog # pinned dep pkgs.zlib # unpinned: uses default nixpkgs ]; ``` Unpinned deps (where `nixpkgs_rev` is null) reference the shared `pkgs` set as today. ### Sanitization Helper in `src/codegen/flake.cpp`: ```cpp auto sanitize_input_attr(std::string_view name, std::string_view version) -> std::string; ``` Steps: 1. Concatenate `nixpkgs--`. 2. Replace every char outside `[a-zA-Z0-9_-]` with `_`. Mostly converts dots in versions: `10.2.1` → `10_2_1`. 3. Use the sanitized form in **all three** places: `inputs.`, the `outputs = { …, , … }` parameter list, and the `pkgs_` `let` binding. Examples: - `fmt` + `10.2.1` → input attr `nixpkgs-fmt-10_2_1`, `let` binding `pkgs_fmt_10_2_1` - `range-v3` + `0.12.0` → `nixpkgs-range-v3-0_12_0`, `pkgs_range_v3_0_12_0` - `boost_filesystem` + `1.84.0` → `nixpkgs-boost_filesystem-1_84_0` The `let`-binding name needs **all** non-alpha-num replaced with `_` (hyphens included) because nix variable names disallow hyphens. The **input** attr keeps hyphens (allowed in input names). Two derived forms. ### Collision detection Two pinned deps with the same `(sanitized_name, sanitized_version)` collide. With the version stored fully (e.g. `10.2.1`, never the manifest spec `10.2`) and dep names being unique within a manifest, collisions are pathologically rare. If a real one is ever reported, mitigation is to append `-` to the input attr. ## Phase status | Phase | Status | Commit | | --- | --- | --- | | 1. devbox_resolve + parser | ✅ | `df2c25b` | | 2. nixpkgs_git_resolve fallback | ✅ | `cb82e91` | | 3. resolve_version + cmd_add wire-up | ✅ | `6f8e9c4` | | 4. cmd_build lockfile merge | ✅ | (this commit) | | 5. flake codegen for per-dep inputs | pending | — | | 6. SPEC §7/§10 amendment + smoke | pending | — | ## End-to-end verification (Phase 6) ```sh cd /tmp && rm -rf demo && mkdir demo && cd demo cargoxx new app && cd app cargoxx add fmt@10.2.1 grep "nixpkgs-fmt-10_2_1" flake.nix # input present grep "f4b140d5" flake.nix # commit_hash substituted cargoxx build && ./build/debug/app # binary builds + runs cargoxx build # second run is no-op diff flake.nix # byte-identical ``` A second `cargoxx build` regenerates byte-identical `Cargoxx.lock` + `flake.nix` — proves the merge path preserves the rev, not re-resolves it. ## ABI note Mixing nixpkgs revisions across pinned deps trades the single-rev ABI guarantee (SPEC §10) for flexibility. Two pinned deps may have been compiled against different glibc / libc++ majors and fail to link cleanly. v0.2 silently accepts the risk; surfacing a compatibility warning is a future polish item.