[M5+] add resolver::nixpkgs_git_resolve fallback
This commit is contained in:
12
CHANGELOG.md
12
CHANGELOG.md
@@ -91,6 +91,18 @@ All notable changes to cargoxx will be documented in this file.
|
|||||||
6 cases against fixtures derived from a real fmt 10.2.1 response;
|
6 cases against fixtures derived from a real fmt 10.2.1 response;
|
||||||
`tests/devbox_resolve_live.cpp` (gated by `CARGOXX_NETWORK_TESTS=1`)
|
`tests/devbox_resolve_live.cpp` (gated by `CARGOXX_NETWORK_TESTS=1`)
|
||||||
hits the live API.
|
hits the live API.
|
||||||
|
- `cargoxx.resolver::nixpkgs_git_resolve(name, version, repo_path?)` —
|
||||||
|
fallback for when search.devbox.sh is unreachable. Lazily clones
|
||||||
|
`https://github.com/NixOS/nixpkgs.git` into
|
||||||
|
`$XDG_CACHE_HOME/cargoxx/nixpkgs/` (or `$HOME/.cache/...`) on first
|
||||||
|
use, then runs
|
||||||
|
`git -C <repo> log --all -S 'version = "<v>"' --pretty='%H %ct' -- pkgs/`
|
||||||
|
and returns the youngest matching commit. Pure helper
|
||||||
|
`pick_youngest_commit(text)` parses the `%H %ct` lines. The unit
|
||||||
|
test builds a tiny throwaway git fixture that mirrors the
|
||||||
|
`pkgs/development/libraries/<pkg>/default.nix` layout (avoiding the
|
||||||
|
real multi-GB clone) and verifies introducing-commit detection plus
|
||||||
|
the not-found path; 5 cases.
|
||||||
- `cargoxx add <pkg>` now auto-resolves packages outside the curated
|
- `cargoxx add <pkg>` now auto-resolves packages outside the curated
|
||||||
linkdb. On `LinkdbUnknownPackage`, `cmd_add` invokes
|
linkdb. On `LinkdbUnknownPackage`, `cmd_add` invokes
|
||||||
`resolver::discover` which: probes `nixpkgs#<pkg>` to confirm the
|
`resolver::discover` which: probes `nixpkgs#<pkg>` to confirm the
|
||||||
|
|||||||
@@ -59,6 +59,7 @@ target_sources(cargoxx
|
|||||||
src/resolver/verify_link.cpp
|
src/resolver/verify_link.cpp
|
||||||
src/resolver/discover.cpp
|
src/resolver/discover.cpp
|
||||||
src/resolver/search_devbox.cpp
|
src/resolver/search_devbox.cpp
|
||||||
|
src/resolver/nixpkgs_git.cpp
|
||||||
src/cli/cmd_new.cpp
|
src/cli/cmd_new.cpp
|
||||||
src/cli/cmd_build.cpp
|
src/cli/cmd_build.cpp
|
||||||
src/cli/cmd_run.cpp
|
src/cli/cmd_run.cpp
|
||||||
|
|||||||
339
docs/version-resolution.md
Normal file
339
docs/version-resolution.md
Normal file
@@ -0,0 +1,339 @@
|
|||||||
|
# Version-resolution algorithm
|
||||||
|
|
||||||
|
Status: in progress (Phases 1–2 of 6 done). This doc fixes the contract
|
||||||
|
for **`(package, version) → nixpkgs commit_hash`** discovery and the
|
||||||
|
flake-codegen pipeline that consumes it. It overrides `SPEC.md` §10's
|
||||||
|
single-shared-rev model with a per-dep-rev model (user-directed; SPEC
|
||||||
|
amendment is Phase 6).
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
cargoxx add <pkg>@<ver>
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌──────────────────────┐
|
||||||
|
│ resolve_version(name,│
|
||||||
|
│ version) │
|
||||||
|
└──────────────────────┘
|
||||||
|
│ │
|
||||||
|
primary HTTP │ │ offline fallback
|
||||||
|
▼ ▼
|
||||||
|
┌──────────────────┐ ┌──────────────────────┐
|
||||||
|
│ devbox_resolve │ │ nixpkgs_git_resolve │
|
||||||
|
│ search.devbox.sh │ │ ~/.cache/cargoxx/ │
|
||||||
|
│ /v1/resolve │ │ nixpkgs/ (lazy) │
|
||||||
|
└──────────────────┘ └──────────────────────┘
|
||||||
|
│ │
|
||||||
|
└──┬───┘
|
||||||
|
▼
|
||||||
|
Result<std::string /*commit_hash*/>
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
cmd_add writes nixpkgs_rev into Cargoxx.lock
|
||||||
|
│
|
||||||
|
▼ (later)
|
||||||
|
cargoxx build
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
codegen::flake_nix reads lockfile
|
||||||
|
emits per-pinned-dep nixpkgs input
|
||||||
|
```
|
||||||
|
|
||||||
|
## When does resolution run?
|
||||||
|
|
||||||
|
| Trigger | What gets resolved |
|
||||||
|
| --- | --- |
|
||||||
|
| `cargoxx add <pkg>@<ver>` | `(pkg, ver)` is resolved exactly once. The resulting commit is written to `Cargoxx.lock` next to the dep entry. |
|
||||||
|
| `cargoxx add <pkg>` (no `@<ver>`) | **Not** resolved. Lockfile entry's `nixpkgs_rev` stays `nullopt`. The generated flake.nix uses only the shared `nixpkgs.url = github:NixOS/nixpkgs/nixos-unstable`. |
|
||||||
|
| `cargoxx build` (lockfile already has rev) | **Not re-resolved.** `cargoxx build` reads existing lockfile entries and preserves `nixpkgs_rev`. Re-resolution would require an explicit `cargoxx update` (deferred to v0.3). |
|
||||||
|
| `cargoxx build` (lockfile missing the rev for a dep) | Synthesized as null — same as the wildcard path. (Future: also call `resolve_version` here when manifest spec is concrete.) |
|
||||||
|
|
||||||
|
`cargoxx build` is **idempotent with respect to the lockfile** —
|
||||||
|
running it twice produces byte-identical `flake.nix` + `Cargoxx.lock`
|
||||||
|
provided the manifest hasn't changed. This is the property the
|
||||||
|
"lockfile merge" change in Phase 4 enforces.
|
||||||
|
|
||||||
|
## resolve_version
|
||||||
|
|
||||||
|
```
|
||||||
|
auto resolve_version(name: string, version: string) -> Result<string /*sha40*/>:
|
||||||
|
if r := devbox_resolve(name, version); r.has_value():
|
||||||
|
return r->commit_hash
|
||||||
|
if r := nixpkgs_git_resolve(name, version); r.has_value():
|
||||||
|
return *r
|
||||||
|
return std::unexpected(ResolutionVersionNotFound)
|
||||||
|
```
|
||||||
|
|
||||||
|
Implementation point: this orchestrator lives in
|
||||||
|
`src/resolver/resolver.cppm` (declaration) +
|
||||||
|
`src/resolver/version_resolve.cpp` (definition). Both probes are
|
||||||
|
already implemented — Phase 3 just wires them into the orchestrator
|
||||||
|
and into `cmd_add`.
|
||||||
|
|
||||||
|
### Probe A — devbox_resolve (primary, HTTP)
|
||||||
|
|
||||||
|
**File:** `src/resolver/search_devbox.cpp` (committed `df2c25b`)
|
||||||
|
|
||||||
|
**URL pattern:**
|
||||||
|
|
||||||
|
```
|
||||||
|
GET https://search.devbox.sh/v1/resolve?name=<urlencoded-name>&version=<urlencoded-version>
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the same endpoint devbox itself uses
|
||||||
|
(`devbox/internal/searcher/client.go` `Resolve()`). Behind the URL is
|
||||||
|
the same Jetify backend that powers nixhub.io.
|
||||||
|
|
||||||
|
**Response shape (real, abbreviated for `fmt 10.2.1`):**
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"commit_hash": "f4b140d5b253f5e2a1ff4e5506edbf8267724bde",
|
||||||
|
"version": "10.2.1",
|
||||||
|
"name": "fmt",
|
||||||
|
"attr_paths": ["fmt"],
|
||||||
|
"systems": {
|
||||||
|
"x86_64-linux": {
|
||||||
|
"commit_hash": "f4b140d5b253f5e2a1ff4e5506edbf8267724bde",
|
||||||
|
"attr_paths": ["fmt"], ...
|
||||||
|
}, ...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Parser contract** (`parse_devbox_resolve`):
|
||||||
|
- `commit_hash` is mandatory. If the top-level field is missing, fall
|
||||||
|
back to the first non-empty `systems.<plat>.commit_hash`.
|
||||||
|
- `name`, `version`, `attr_paths` are best-effort; absence leaves them
|
||||||
|
blank.
|
||||||
|
- 404 / curl exit 22 → `ResolutionUnknownPackage`.
|
||||||
|
- Empty `commit_hash` after fallback → `ResolutionVersionNotFound`.
|
||||||
|
- Other curl exits, JSON parse errors → `ResolutionNetworkError`.
|
||||||
|
|
||||||
|
**Timeout:** 10 s on `--max-time`, 15 s wrapping `ExecOptions.timeout`.
|
||||||
|
|
||||||
|
### Probe B — nixpkgs_git_resolve (offline fallback)
|
||||||
|
|
||||||
|
**File:** `src/resolver/nixpkgs_git.cpp` (committed in Phase 2 series)
|
||||||
|
|
||||||
|
**Setup:** lazy clone of
|
||||||
|
`https://github.com/NixOS/nixpkgs.git` into
|
||||||
|
`$XDG_CACHE_HOME/cargoxx/nixpkgs/` (or `$HOME/.cache/...`) on first
|
||||||
|
use. ~9 GB and slow (5–15 min); subsequent calls are fast and offline.
|
||||||
|
|
||||||
|
**Search:**
|
||||||
|
|
||||||
|
```
|
||||||
|
git -C <repo> log --all \
|
||||||
|
-S 'version = "<urlencoded-version>"' \
|
||||||
|
--pretty='%H %ct' \
|
||||||
|
-- pkgs/
|
||||||
|
```
|
||||||
|
|
||||||
|
`-S '<term>'` returns commits that *introduced or removed* the literal
|
||||||
|
string. `--pretty='%H %ct'` emits `<sha40> <committer-time>` per
|
||||||
|
line. We restrict to `pkgs/` to keep noise down (out-of-tree match
|
||||||
|
sites in `lib/`, `nixos/`, etc. don't matter).
|
||||||
|
|
||||||
|
**Pick:** youngest committer-time (`%ct` highest) wins. The pure
|
||||||
|
helper `pick_youngest_commit(text)` does this; it tolerates malformed
|
||||||
|
lines (skips them).
|
||||||
|
|
||||||
|
**Errors:**
|
||||||
|
- `pick_youngest_commit` returns `nullopt` → `ResolutionVersionNotFound`.
|
||||||
|
- Clone failure → `ResolutionNetworkError`.
|
||||||
|
- Subsequent `git log` failure → `ResolutionNetworkError`.
|
||||||
|
|
||||||
|
**Test fixture trick:** instead of cloning real nixpkgs in tests, the
|
||||||
|
unit test builds a tiny throwaway repo with
|
||||||
|
`pkgs/development/libraries/<pkg>/default.nix` files at two versions
|
||||||
|
and asserts introducing-commit detection works.
|
||||||
|
|
||||||
|
### Heuristic limits
|
||||||
|
|
||||||
|
`-S 'version = "<v>"'` is fuzzy — it matches **any** file in `pkgs/`
|
||||||
|
that has that literal. Two real-world failure modes:
|
||||||
|
|
||||||
|
1. **Unrelated package match.** `version = "1.0.0"` appears in many
|
||||||
|
nix derivations. The youngest-commit tiebreaker biases toward
|
||||||
|
"the most recent thing that touched this string", which usually
|
||||||
|
*is* the package's bump commit, but not guaranteed.
|
||||||
|
2. **Non-string-formed versions.** Some derivations build the version
|
||||||
|
via `lib.removeSuffix`, interpolation, or an inherited
|
||||||
|
`pname`/`finalAttrs.version`. `-S` won't see those. For those
|
||||||
|
packages, only the devbox HTTP path can answer.
|
||||||
|
|
||||||
|
Both are accepted as known limits — the HTTP path is primary and fast
|
||||||
|
when reachable; the git fallback exists only for offline determinism.
|
||||||
|
|
||||||
|
## Lockfile interaction
|
||||||
|
|
||||||
|
`Cargoxx.lock` already carries `LockfilePackage.nixpkgs_rev`
|
||||||
|
(`std::optional<std::string>`). No schema change.
|
||||||
|
|
||||||
|
### Add path
|
||||||
|
|
||||||
|
`cmd_add fmt@10.2.1`:
|
||||||
|
|
||||||
|
1. existing manifest validation, duplicate check, linkdb resolve /
|
||||||
|
discover (separate auto-resolution feature, already shipped).
|
||||||
|
2. **NEW:** call `resolve_version("fmt", "10.2.1")`. On success,
|
||||||
|
capture `commit_hash`.
|
||||||
|
3. existing manifest write of `[dependencies] fmt = "10.2.1"`.
|
||||||
|
4. **NEW:** load lockfile (or initialize empty), find/insert the
|
||||||
|
`LockfilePackage{ name="fmt", version="10.2.1" }` entry, set
|
||||||
|
`nixpkgs_rev = "<commit_hash>"`, write lockfile back.
|
||||||
|
|
||||||
|
`cmd_add fmt` (wildcard) skips step 2 and step 4's `nixpkgs_rev`
|
||||||
|
assignment.
|
||||||
|
|
||||||
|
### Build path (Phase 4 fix)
|
||||||
|
|
||||||
|
Today, `synthesize_lockfile` overwrites the lockfile every time. With
|
||||||
|
per-dep revs in scope this would erase pinned revs on every build.
|
||||||
|
|
||||||
|
The fix:
|
||||||
|
|
||||||
|
```
|
||||||
|
build_lockfile(manifest, recipes):
|
||||||
|
let prior = parse(project_root / "Cargoxx.lock") or empty
|
||||||
|
for each dep in manifest.dependencies:
|
||||||
|
let prior_entry = prior.find(dep.name, dep.version_spec)
|
||||||
|
new_entry = LockfilePackage{ name, version=dep.version_spec, ... }
|
||||||
|
if prior_entry: new_entry.nixpkgs_rev = prior_entry.nixpkgs_rev
|
||||||
|
emit new_entry
|
||||||
|
```
|
||||||
|
|
||||||
|
The lookup key is `(name, version)`. If the user changes the version,
|
||||||
|
the prior rev is dropped (correct — the rev was for the old version).
|
||||||
|
If the user neither edited nor `cargoxx update`d, the rev survives.
|
||||||
|
|
||||||
|
### Update path (deferred to v0.3)
|
||||||
|
|
||||||
|
`cargoxx update <pkg>` would call `resolve_version` again with the
|
||||||
|
existing manifest version_spec, possibly upgrading the rev to a
|
||||||
|
newer one, even when the user-visible version string is unchanged.
|
||||||
|
Out of scope for this milestone.
|
||||||
|
|
||||||
|
## Flake codegen — per-dep inputs
|
||||||
|
|
||||||
|
**Phase 5.** Today's `flake.nix` template has a single
|
||||||
|
`@@NIXPKGS_REV@@` placeholder. The new template emits:
|
||||||
|
|
||||||
|
### Inputs block
|
||||||
|
|
||||||
|
```nix
|
||||||
|
inputs = {
|
||||||
|
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
|
||||||
|
# one line per dep with non-null nixpkgs_rev:
|
||||||
|
nixpkgs-fmt-10_2_1.url = "github:NixOS/nixpkgs/f4b140d5b...";
|
||||||
|
nixpkgs-spdlog-1_13_0.url = "github:NixOS/nixpkgs/abcdef0...";
|
||||||
|
flake-utils.url = "github:numtide/flake-utils";
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Outputs lambda
|
||||||
|
|
||||||
|
```nix
|
||||||
|
outputs = { self, nixpkgs, nixpkgs-fmt-10_2_1, nixpkgs-spdlog-1_13_0,
|
||||||
|
flake-utils }: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Let bindings
|
||||||
|
|
||||||
|
```nix
|
||||||
|
let
|
||||||
|
pkgs = import nixpkgs { inherit system; };
|
||||||
|
pkgs_fmt_10_2_1 = import nixpkgs-fmt-10_2_1 { inherit system; };
|
||||||
|
pkgs_spdlog_1_13_0 = import nixpkgs-spdlog-1_13_0 { inherit system; };
|
||||||
|
llvmPkgs = pkgs.llvmPackages;
|
||||||
|
in {...}
|
||||||
|
```
|
||||||
|
|
||||||
|
### buildInputs
|
||||||
|
|
||||||
|
```nix
|
||||||
|
buildInputs = [
|
||||||
|
pkgs_fmt_10_2_1.fmt # pinned dep
|
||||||
|
pkgs_spdlog_1_13_0.spdlog # pinned dep
|
||||||
|
pkgs.zlib # unpinned: uses default nixpkgs
|
||||||
|
];
|
||||||
|
```
|
||||||
|
|
||||||
|
Unpinned deps (where `nixpkgs_rev` is null) reference the shared
|
||||||
|
`pkgs` set as today.
|
||||||
|
|
||||||
|
### Sanitization
|
||||||
|
|
||||||
|
Helper in `src/codegen/flake.cpp`:
|
||||||
|
|
||||||
|
```cpp
|
||||||
|
auto sanitize_input_attr(std::string_view name, std::string_view version)
|
||||||
|
-> std::string;
|
||||||
|
```
|
||||||
|
|
||||||
|
Steps:
|
||||||
|
|
||||||
|
1. Concatenate `nixpkgs-<name>-<version>`.
|
||||||
|
2. Replace every char outside `[a-zA-Z0-9_-]` with `_`. Mostly
|
||||||
|
converts dots in versions: `10.2.1` → `10_2_1`.
|
||||||
|
3. Use the sanitized form in **all three** places: `inputs.<attr>`,
|
||||||
|
the `outputs = { …, <attr>, … }` parameter list, and the
|
||||||
|
`pkgs_<attr-with-dashes-as-underscores>` `let` binding.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- `fmt` + `10.2.1` → input attr `nixpkgs-fmt-10_2_1`,
|
||||||
|
`let` binding `pkgs_fmt_10_2_1`
|
||||||
|
- `range-v3` + `0.12.0` → `nixpkgs-range-v3-0_12_0`,
|
||||||
|
`pkgs_range_v3_0_12_0`
|
||||||
|
- `boost_filesystem` + `1.84.0` → `nixpkgs-boost_filesystem-1_84_0`
|
||||||
|
|
||||||
|
The `let`-binding name needs **all** non-alpha-num replaced with `_`
|
||||||
|
(hyphens included) because nix variable names disallow hyphens. The
|
||||||
|
**input** attr keeps hyphens (allowed in input names). Two derived
|
||||||
|
forms.
|
||||||
|
|
||||||
|
### Collision detection
|
||||||
|
|
||||||
|
Two pinned deps with the same `(sanitized_name, sanitized_version)`
|
||||||
|
collide. With the version stored fully (e.g. `10.2.1`, never the
|
||||||
|
manifest spec `10.2`) and dep names being unique within a manifest,
|
||||||
|
collisions are pathologically rare. If a real one is ever reported,
|
||||||
|
mitigation is to append `-<short-sha>` to the input attr.
|
||||||
|
|
||||||
|
## Phase status
|
||||||
|
|
||||||
|
| Phase | Status | Commit |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 1. devbox_resolve + parser | ✅ | `df2c25b` |
|
||||||
|
| 2. nixpkgs_git_resolve fallback | ✅ | (this commit) |
|
||||||
|
| 3. resolve_version + cmd_add wire-up | pending | — |
|
||||||
|
| 4. cmd_build lockfile merge | pending | — |
|
||||||
|
| 5. flake codegen for per-dep inputs | pending | — |
|
||||||
|
| 6. SPEC §7/§10 amendment + smoke | pending | — |
|
||||||
|
|
||||||
|
## End-to-end verification (Phase 6)
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /tmp && rm -rf demo && mkdir demo && cd demo
|
||||||
|
cargoxx new app && cd app
|
||||||
|
cargoxx add fmt@10.2.1
|
||||||
|
grep "nixpkgs-fmt-10_2_1" flake.nix # input present
|
||||||
|
grep "f4b140d5" flake.nix # commit_hash substituted
|
||||||
|
cargoxx build && ./build/debug/app # binary builds + runs
|
||||||
|
cargoxx build # second run is no-op
|
||||||
|
diff <prev-flake.nix> flake.nix # byte-identical
|
||||||
|
```
|
||||||
|
|
||||||
|
A second `cargoxx build` regenerates byte-identical
|
||||||
|
`Cargoxx.lock` + `flake.nix` — proves the merge path preserves the
|
||||||
|
rev, not re-resolves it.
|
||||||
|
|
||||||
|
## ABI note
|
||||||
|
|
||||||
|
Mixing nixpkgs revisions across pinned deps trades the single-rev
|
||||||
|
ABI guarantee (SPEC §10) for flexibility. Two pinned deps may have
|
||||||
|
been compiled against different glibc / libc++ majors and fail to
|
||||||
|
link cleanly. v0.2 silently accepts the risk; surfacing a
|
||||||
|
compatibility warning is a future polish item.
|
||||||
138
src/resolver/nixpkgs_git.cpp
Normal file
138
src/resolver/nixpkgs_git.cpp
Normal file
@@ -0,0 +1,138 @@
|
|||||||
|
module cargoxx.resolver;
|
||||||
|
|
||||||
|
import std;
|
||||||
|
import cargoxx.util;
|
||||||
|
import cargoxx.exec;
|
||||||
|
|
||||||
|
namespace cargoxx::resolver {
|
||||||
|
|
||||||
|
namespace fs = std::filesystem;
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
constexpr std::string_view NIXPKGS_REPO_URL = "https://github.com/NixOS/nixpkgs.git";
|
||||||
|
|
||||||
|
auto error(util::ErrorCode code, std::string msg) -> util::Error {
|
||||||
|
return util::Error{code, std::move(msg), "", std::nullopt, std::nullopt};
|
||||||
|
}
|
||||||
|
|
||||||
|
auto default_clone_path() -> fs::path {
|
||||||
|
if (auto* xdg = std::getenv("XDG_CACHE_HOME"); xdg && *xdg) {
|
||||||
|
return fs::path{xdg} / "cargoxx" / "nixpkgs";
|
||||||
|
}
|
||||||
|
if (auto* home = std::getenv("HOME"); home && *home) {
|
||||||
|
return fs::path{home} / ".cache" / "cargoxx" / "nixpkgs";
|
||||||
|
}
|
||||||
|
return fs::current_path() / ".cargoxx-nixpkgs";
|
||||||
|
}
|
||||||
|
|
||||||
|
auto ensure_clone(const fs::path& repo_path) -> util::Result<void> {
|
||||||
|
std::error_code ec;
|
||||||
|
if (fs::exists(repo_path / ".git", ec)) {
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
fs::create_directories(repo_path.parent_path(), ec);
|
||||||
|
if (ec) {
|
||||||
|
return std::unexpected(error(
|
||||||
|
util::ErrorCode::ResolutionNetworkError,
|
||||||
|
std::format("cannot create '{}': {}", repo_path.parent_path().string(),
|
||||||
|
ec.message())));
|
||||||
|
}
|
||||||
|
auto r = exec::run("git",
|
||||||
|
{"clone", std::string{NIXPKGS_REPO_URL}, repo_path.string()},
|
||||||
|
exec::ExecOptions{
|
||||||
|
.cwd = {},
|
||||||
|
.env_overrides = {},
|
||||||
|
.timeout = std::chrono::seconds{1800}, // up to 30 min
|
||||||
|
.inherit_stdio = true, // user wants progress
|
||||||
|
});
|
||||||
|
if (!r) {
|
||||||
|
return std::unexpected(r.error());
|
||||||
|
}
|
||||||
|
if (r->exit_code != 0) {
|
||||||
|
return std::unexpected(error(
|
||||||
|
util::ErrorCode::ResolutionNetworkError,
|
||||||
|
std::format("git clone of nixpkgs failed (exit {})", r->exit_code)));
|
||||||
|
}
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
auto pick_youngest_commit(std::string_view git_log_output) -> std::optional<std::string> {
|
||||||
|
std::string best_sha;
|
||||||
|
std::int64_t best_time = -1;
|
||||||
|
|
||||||
|
std::size_t pos = 0;
|
||||||
|
while (pos < git_log_output.size()) {
|
||||||
|
auto eol = git_log_output.find('\n', pos);
|
||||||
|
auto line = git_log_output.substr(
|
||||||
|
pos, eol == std::string_view::npos ? git_log_output.size() - pos
|
||||||
|
: eol - pos);
|
||||||
|
auto space = line.find(' ');
|
||||||
|
if (space != std::string_view::npos) {
|
||||||
|
auto sha = line.substr(0, space);
|
||||||
|
auto ts = line.substr(space + 1);
|
||||||
|
std::int64_t t = 0;
|
||||||
|
auto [ptr, ec] = std::from_chars(ts.data(), ts.data() + ts.size(), t);
|
||||||
|
if (ec == std::errc{} && t > best_time) {
|
||||||
|
best_time = t;
|
||||||
|
best_sha = std::string{sha};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (eol == std::string_view::npos) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
pos = eol + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (best_sha.empty()) {
|
||||||
|
return std::nullopt;
|
||||||
|
}
|
||||||
|
return best_sha;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto nixpkgs_git_resolve(const std::string& name, const std::string& version,
|
||||||
|
std::optional<fs::path> repo_path) -> util::Result<std::string> {
|
||||||
|
if (name.empty() || version.empty()) {
|
||||||
|
return std::unexpected(error(util::ErrorCode::ResolutionUnknownPackage,
|
||||||
|
"nixpkgs_git_resolve: name and version are required"));
|
||||||
|
}
|
||||||
|
auto repo = repo_path.value_or(default_clone_path());
|
||||||
|
if (auto ok = ensure_clone(repo); !ok) {
|
||||||
|
return std::unexpected(ok.error());
|
||||||
|
}
|
||||||
|
|
||||||
|
// -S looks for changes that added or removed the literal string. We
|
||||||
|
// restrict the path filter to pkgs/ to keep noise down. The youngest
|
||||||
|
// commit wins (highest committer timestamp).
|
||||||
|
auto needle = std::format("version = \"{}\"", version);
|
||||||
|
auto r = exec::run(
|
||||||
|
"git",
|
||||||
|
{"-C", repo.string(), "log", "--all", "-S", needle,
|
||||||
|
"--pretty=%H %ct", "--", "pkgs/"},
|
||||||
|
exec::ExecOptions{
|
||||||
|
.cwd = {},
|
||||||
|
.env_overrides = {},
|
||||||
|
.timeout = std::chrono::seconds{60},
|
||||||
|
.inherit_stdio = false,
|
||||||
|
});
|
||||||
|
if (!r) {
|
||||||
|
return std::unexpected(r.error());
|
||||||
|
}
|
||||||
|
if (r->exit_code != 0) {
|
||||||
|
return std::unexpected(error(
|
||||||
|
util::ErrorCode::ResolutionNetworkError,
|
||||||
|
std::format("git log failed (exit {}): {}", r->exit_code, r->stderr_text)));
|
||||||
|
}
|
||||||
|
auto sha = pick_youngest_commit(r->stdout_text);
|
||||||
|
if (!sha) {
|
||||||
|
return std::unexpected(error(
|
||||||
|
util::ErrorCode::ResolutionVersionNotFound,
|
||||||
|
std::format("nixpkgs git: no commit introduced 'version = \"{}\"' for {}",
|
||||||
|
version, name)));
|
||||||
|
}
|
||||||
|
return *sha;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace cargoxx::resolver
|
||||||
@@ -158,4 +158,20 @@ auto parse_devbox_resolve(std::string_view json)
|
|||||||
auto devbox_resolve(const std::string& name, const std::string& version)
|
auto devbox_resolve(const std::string& name, const std::string& version)
|
||||||
-> util::Result<DevboxResolution>;
|
-> util::Result<DevboxResolution>;
|
||||||
|
|
||||||
|
// Pure: parse the output of
|
||||||
|
// git log --all -S 'version = "<v>"' --pretty='%H %ct' -- pkgs/
|
||||||
|
// (one commit per line: "<40-char-sha> <unix-epoch-seconds>") and
|
||||||
|
// return the youngest matching SHA, or nullopt on empty input.
|
||||||
|
auto pick_youngest_commit(std::string_view git_log_output)
|
||||||
|
-> std::optional<std::string>;
|
||||||
|
|
||||||
|
// Searches a local nixpkgs clone for the youngest commit that
|
||||||
|
// introduced `version = "<version>"` under pkgs/. `repo_path`
|
||||||
|
// defaults to ~/.cache/cargoxx/nixpkgs/ — when the directory doesn't
|
||||||
|
// exist the repo is cloned lazily (multi-GB, only on first miss).
|
||||||
|
// 404-equivalent: ResolutionVersionNotFound.
|
||||||
|
auto nixpkgs_git_resolve(const std::string& name, const std::string& version,
|
||||||
|
std::optional<std::filesystem::path> repo_path = std::nullopt)
|
||||||
|
-> util::Result<std::string>;
|
||||||
|
|
||||||
} // namespace cargoxx::resolver
|
} // namespace cargoxx::resolver
|
||||||
|
|||||||
@@ -35,3 +35,4 @@ cargoxx_add_test(vcpkg_probe_live)
|
|||||||
cargoxx_add_test(verify_link_unit)
|
cargoxx_add_test(verify_link_unit)
|
||||||
cargoxx_add_test(devbox_resolve_parse)
|
cargoxx_add_test(devbox_resolve_parse)
|
||||||
cargoxx_add_test(devbox_resolve_live)
|
cargoxx_add_test(devbox_resolve_live)
|
||||||
|
cargoxx_add_test(nixpkgs_git_resolve)
|
||||||
|
|||||||
115
tests/nixpkgs_git_resolve.cpp
Normal file
115
tests/nixpkgs_git_resolve.cpp
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
// Unit + fixture-based test for resolver::nixpkgs_git_resolve. Avoids
|
||||||
|
// the multi-GB nixos/nixpkgs clone by building a tiny throwaway repo
|
||||||
|
// that mirrors the relevant slice of the nixpkgs layout.
|
||||||
|
|
||||||
|
#include <catch2/catch_test_macros.hpp>
|
||||||
|
|
||||||
|
import cargoxx.resolver;
|
||||||
|
import cargoxx.exec;
|
||||||
|
import cargoxx.util;
|
||||||
|
import std;
|
||||||
|
|
||||||
|
using cargoxx::resolver::nixpkgs_git_resolve;
|
||||||
|
using cargoxx::resolver::pick_youngest_commit;
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
auto run_git(const std::filesystem::path& cwd, std::vector<std::string> args)
|
||||||
|
-> int {
|
||||||
|
auto r = cargoxx::exec::run(
|
||||||
|
"git", args,
|
||||||
|
cargoxx::exec::ExecOptions{
|
||||||
|
.cwd = cwd,
|
||||||
|
.env_overrides = {{"GIT_COMMITTER_NAME", "t"},
|
||||||
|
{"GIT_COMMITTER_EMAIL", "t@t"},
|
||||||
|
{"GIT_AUTHOR_NAME", "t"},
|
||||||
|
{"GIT_AUTHOR_EMAIL", "t@t"},
|
||||||
|
{"GIT_COMMITTER_DATE", "1700000000 +0000"},
|
||||||
|
{"GIT_AUTHOR_DATE", "1700000000 +0000"}},
|
||||||
|
.timeout = std::chrono::seconds{30},
|
||||||
|
.inherit_stdio = false,
|
||||||
|
});
|
||||||
|
REQUIRE(r.has_value());
|
||||||
|
return r->exit_code;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto write_text(const std::filesystem::path& p, std::string_view content) {
|
||||||
|
std::filesystem::create_directories(p.parent_path());
|
||||||
|
std::ofstream{p} << content;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto fixture_repo(std::string_view pkg, std::string_view v1,
|
||||||
|
std::string_view v2) -> std::filesystem::path {
|
||||||
|
auto root = std::filesystem::temp_directory_path() /
|
||||||
|
std::format("cargoxx-git-fixture-{}", std::random_device{}());
|
||||||
|
std::filesystem::create_directories(root);
|
||||||
|
REQUIRE(run_git(root, {"init", "-q", "-b", "main"}) == 0);
|
||||||
|
|
||||||
|
auto nix_path =
|
||||||
|
root / "pkgs" / "development" / "libraries" / std::string{pkg} / "default.nix";
|
||||||
|
write_text(nix_path, std::format("{{ stdenv }}: stdenv.mkDerivation {{ "
|
||||||
|
"version = \"{}\"; }}\n",
|
||||||
|
v1));
|
||||||
|
REQUIRE(run_git(root, {"add", "."}) == 0);
|
||||||
|
REQUIRE(run_git(root, {"commit", "-q", "-m", std::format("init {}", v1)}) == 0);
|
||||||
|
|
||||||
|
write_text(nix_path, std::format("{{ stdenv }}: stdenv.mkDerivation {{ "
|
||||||
|
"version = \"{}\"; }}\n",
|
||||||
|
v2));
|
||||||
|
REQUIRE(run_git(root, {"add", "."}) == 0);
|
||||||
|
REQUIRE(run_git(root, {"commit", "-q", "-m", std::format("bump {}", v2)}) == 0);
|
||||||
|
|
||||||
|
return root;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
TEST_CASE("pick_youngest_commit picks the highest-timestamp SHA",
|
||||||
|
"[resolver][git]") {
|
||||||
|
constexpr std::string_view input =
|
||||||
|
"abc111 1700000000\n"
|
||||||
|
"def222 1800000000\n"
|
||||||
|
"ghi333 1750000000\n";
|
||||||
|
auto r = pick_youngest_commit(input);
|
||||||
|
REQUIRE(r == std::string{"def222"});
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST_CASE("pick_youngest_commit returns nullopt on empty input",
|
||||||
|
"[resolver][git]") {
|
||||||
|
REQUIRE_FALSE(pick_youngest_commit("").has_value());
|
||||||
|
REQUIRE_FALSE(pick_youngest_commit("\n\n").has_value());
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST_CASE("pick_youngest_commit skips malformed lines", "[resolver][git]") {
|
||||||
|
constexpr std::string_view input =
|
||||||
|
"garbage\n"
|
||||||
|
"abc111 1700000000\n"
|
||||||
|
"no-timestamp\n";
|
||||||
|
auto r = pick_youngest_commit(input);
|
||||||
|
REQUIRE(r == std::string{"abc111"});
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST_CASE("nixpkgs_git_resolve finds the introducing commit",
|
||||||
|
"[resolver][git]") {
|
||||||
|
auto repo = fixture_repo("fmt", "10.2.0", "10.3.0");
|
||||||
|
|
||||||
|
auto r = nixpkgs_git_resolve("fmt", "10.2.0", repo);
|
||||||
|
REQUIRE(r.has_value());
|
||||||
|
REQUIRE(r->size() == 40); // git sha
|
||||||
|
|
||||||
|
std::error_code ec;
|
||||||
|
std::filesystem::remove_all(repo, ec);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST_CASE("nixpkgs_git_resolve returns ResolutionVersionNotFound for an absent version",
|
||||||
|
"[resolver][git]") {
|
||||||
|
auto repo = fixture_repo("fmt", "10.2.0", "10.3.0");
|
||||||
|
|
||||||
|
auto r = nixpkgs_git_resolve("fmt", "99.99.99", repo);
|
||||||
|
REQUIRE_FALSE(r.has_value());
|
||||||
|
REQUIRE(r.error().code ==
|
||||||
|
cargoxx::util::ErrorCode::ResolutionVersionNotFound);
|
||||||
|
|
||||||
|
std::error_code ec;
|
||||||
|
std::filesystem::remove_all(repo, ec);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user