# Library reuse + public registry Design plan for two interlocking features: 1. **Library reuse** — cargoxx-built libraries are consumable by other cargoxx projects, by plain CMake projects (`find_package`), and by plain clang invocations (`pkg-config`). 2. **Public registry + publish** — a Gitea-hosted package registry (`cargoxx-pkgs`), a `cargoxx publish` CLI, and self-hosted CI that validates PRs, builds, pushes derivations to a binary cache, and auto-merges. Phase 1 covers library reuse end-to-end with local path and git deps. Phase 2 layers the registry on top, with cache-backed substitution so consumers don't recompile. --- ## Part 1 — reusable libraries ### 1.1 Producer install layout ``` $out/ ├── bin/ # binaries (existing path) ├── lib/ │ ├── libmylib.a # archive of impl bits │ ├── pkgconfig/mylib.pc # pkg-config descriptor │ └── cmake/mylib/ │ ├── mylibConfig.cmake # entry point for find_package │ ├── mylibConfigVersion.cmake # semver gate │ └── mylibTargets.cmake # IMPORTED tgt + CXX_MODULES FILE_SET └── include/mylib/ # public headers + source-form modules ``` Covers three consumption paths: - **cargoxx → cargoxx**: consumer's CMakeLists emits `find_package(mylib CONFIG REQUIRED)` + link against `mylib::mylib`. `$out` is on `CMAKE_PREFIX_PATH` via `buildInputs`. - **plain CMake**: same `find_package` call, manual `HINTS` if not on the default prefix path. - **plain clang + pkg-config**: works for non-module libraries (header-only or `.a` with extern API). Module libraries require a CMake-aware consumer — industry-wide pkg-config has no story for module BMIs. ### 1.2 Codegen changes (`src/codegen/cmake.cpp`) When `layout.library` is set, append: ```cmake include(GNUInstallDirs) include(CMakePackageConfigHelpers) install(TARGETS mylib EXPORT mylibTargets FILE_SET CXX_MODULES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/mylib ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) install(EXPORT mylibTargets FILE mylibTargets.cmake NAMESPACE mylib:: DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/mylib) # Config.cmake.in written inline so we don't need an off-tree template file. file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/mylibConfig.cmake.in [[ @PACKAGE_INIT@ include(CMakeFindDependencyMacro) @FIND_DEPENDENCY_BLOCK@ include("${CMAKE_CURRENT_LIST_DIR}/mylibTargets.cmake") check_required_components(mylib) ]]) configure_package_config_file( ${CMAKE_CURRENT_BINARY_DIR}/mylibConfig.cmake.in ${CMAKE_CURRENT_BINARY_DIR}/mylibConfig.cmake INSTALL_DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/mylib) write_basic_package_version_file( ${CMAKE_CURRENT_BINARY_DIR}/mylibConfigVersion.cmake VERSION ${PROJECT_VERSION} COMPATIBILITY SameMajorVersion) install(FILES ${CMAKE_CURRENT_BINARY_DIR}/mylibConfig.cmake ${CMAKE_CURRENT_BINARY_DIR}/mylibConfigVersion.cmake DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/mylib) # pkg-config descriptor — also inline. file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/mylib.pc.in [[ prefix=@CMAKE_INSTALL_PREFIX@ exec_prefix=${prefix} libdir=${prefix}/lib includedir=${prefix}/include Name: @PROJECT_NAME@ Version: @PROJECT_VERSION@ Description: @PROJECT_DESCRIPTION@ Cflags: -I${includedir} Libs: -L${libdir} -lmylib ]]) configure_file(${CMAKE_CURRENT_BINARY_DIR}/mylib.pc.in ${CMAKE_CURRENT_BINARY_DIR}/mylib.pc @ONLY) install(FILES ${CMAKE_CURRENT_BINARY_DIR}/mylib.pc DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig) ``` `@FIND_DEPENDENCY_BLOCK@` is computed by codegen — one `find_dependency(X)` line per cargoxx-source dep, so the transitive graph reconstitutes when a consumer does `find_package(mylib …)`. Binaries also get an install rule so `cmake --install` handles everything in one shot: ```cmake install(TARGETS mylib_bin RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) install(TARGETS tool RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) ``` ### 1.3 `buildCppPackage.installPhase` switches to `cmake --install` ```nix installPhase = '' cmake --install build/release --prefix $out ''; ``` Replaces the `cp -a build/release/bin/. $out/bin/` hack — now bins, library archive, headers, cmake config, pc file all land in the right place from one install command. ### 1.4 Manifest schema — discriminated dep source Backward-compat (string form unchanged): ```toml [dependencies] sqlite = "*" # nixpkgs / linkdb (today) fmt = { version = ">=10" } # nixpkgs explicit (today) mylib = { path = "../mylib" } # cargoxx local path ← NEW remote = { git = "https://gitea/me/remote", rev = "abc123" } # cargoxx git ← NEW shared = { version = "1.0", registry = "cargoxx" } # cargoxx registry ← NEW ``` `manifest::Dependency` grows discriminated source fields. Parser rejects unknown keys in dep tables (today silently ignored). ### 1.5 Lockfile schema additions For cargoxx-source deps the lockfile carries source provenance and the recipe shape (`find_package`, `targets`) — codegen consumes the latter unchanged, so cargoxx deps look like any other linkdb recipe to the emitted CMakeLists. Path deps: record relative path. Git deps: record `(url, commit, sha256)`. Registry deps: record only `(registry_rev, registry_attr)`; the source URL + sha256 live in the registry repo, not the consumer's lockfile. ### 1.6 `buildCppPackage` recursion ```nix buildCargoxxDep = p: if p.source_kind == "path" then buildCppPackage { src = src + ("/" + p.path); } else if p.source_kind == "git" then let depSrc = pkgs.fetchgit { inherit (p) url; rev = p.commit; hash = p.sha256; }; in buildCppPackage { src = depSrc; } else if p.source_kind == "registry" then # Pull the prebuilt $out from the registry flake — no source fetch, # no recompile. The registry's `packages..` derivation # is content-addressed and substituted from the binary cache. let registry = builtins.getFlake "git+https://gitea/.../cargoxx-pkgs?rev=${lock.registry_rev}"; in registry.packages.${system}.${p.registry_attr} else throw "unknown source_kind: ${p.source_kind}"; cargoxxDepDrvs = map buildCargoxxDep (builtins.filter (p: p ? source_kind) lock.package); ``` `cargoxxDepDrvs` join `buildInputs`. nixpkgs's `cmakeConfigurePhase` automatically prepends every `buildInputs` `$out` to `CMAKE_PREFIX_PATH`, so consumer `find_package( CONFIG REQUIRED)` resolves without extra plumbing. ### 1.7 Resolver flow for cargoxx-source deps ``` manifest dep → cmd_build::resolve_dep() ├─ source kind == Nixpkgs/Auto: existing resolver chain ├─ source kind == CargoxxPath: │ read ${path}/Cargoxx.toml, take [package].name/version, │ synthesize Recipe { find_package = " CONFIG REQUIRED", │ targets = ["::"] } ├─ source kind == CargoxxGit: │ fetch the repo at rev (resolver cache), │ same as CargoxxPath against the checkout └─ source kind == CargoxxRegistry: fetch the registry's recipe TOML, take name + version, synthesize the same Recipe shape. ``` The recipe shape — `find_package( CONFIG REQUIRED)` plus `::` target — matches what the producer's installed `Config.cmake` defines. Codegen needs no cargoxx-dep special case. ### 1.8 Phasing | Phase | Deliverable | |---|---| | 1a | install rules + Config.cmake + .pc; `cmake --install` in buildCppPackage | | 1b | `{ path = "..." }` deps end-to-end (manifest parse + lock + recursive buildCppPackage); e2e fixture | | 1c | `{ git = "...", rev = "..." }` deps via `pkgs.fetchgit` | | 1d | `{ version = "1.0", registry = "cargoxx" }` deps (gated on Phase 2 being live) | --- ## Part 2 — Gitea-hosted registry ### 2.1 `cargoxx-pkgs` repo layout ``` cargoxx-pkgs/ ├── flake.nix ├── recipes/ │ └── / │ ├── maintainers.txt # gitea usernames, one per line │ ├── meta.toml # description, homepage, license, repo │ └── versions/ │ ├── 1.0.0.toml │ └── 1.0.1.toml └── .gitea/workflows/ ├── validate-pr.yml └── auto-merge.yml ``` ### 2.2 Recipe `versions/.toml` ```toml schema = 1 name = "mylib" version = "1.0.0" [source] type = "git" url = "https://gitea/me/mylib" commit = "abc123…" # 40-char sha256 = "sha256-…" # FOD pin [dependencies] fmt = "10.2" otherlib = { version = "0.3", registry = "cargoxx" } [lock] nixpkgs_rev = "…" flake_utils_rev = "…" cargoxx_rev = "…" [meta] description = "…" homepage = "https://…" license = "MIT" ``` ### 2.3 `cargoxx-pkgs` flake (the substitution surface) ```nix { description = "cargoxx package registry"; inputs.nixpkgs.url = "github:NixOS/nixpkgs/"; inputs.flake-utils.url = "github:numtide/flake-utils"; inputs.cargoxx.url = "git+https://gitea/.../cargoxx?rev="; outputs = { self, nixpkgs, flake-utils, cargoxx }: flake-utils.lib.eachDefaultSystem (system: let pkgs = import nixpkgs { inherit system; }; mkPackage = recipeFile: let r = builtins.fromTOML (builtins.readFile recipeFile); in cargoxx.lib.${system}.buildCppPackage { src = pkgs.fetchgit { inherit (r.source) url; rev = r.source.commit; hash = r.source.sha256; }; }; # Enumerate recipes//versions/.toml, build packagesAttrs. packagesAttrs = …; in { packages = packagesAttrs; }); } ``` CI evaluates this flake, builds the new attr, pushes `$out` to the binary cache. Consumers later get a cache hit for the SAME derivation hash via `builtins.getFlake` at the same `registry_rev`. ### 2.4 Publishing flow (`cargoxx publish`) ``` cargoxx publish [--dry-run] 1. Validate Cargoxx.toml has name, version, description, license, repository. 2. Validate working tree clean; current commit on a remote branch. 3. Validate Cargoxx.lock pins every cargoxx-source dep to a registry version (path = "..." deps disallowed in publish). 4. Compute recipe TOML: [source].url = git remote URL [source].commit = HEAD commit [source].sha256 = sha256 of `git archive HEAD` [dependencies] = mirror Cargoxx.toml [lock] = mirror Cargoxx.lock pins 5. Clone cargoxx-pkgs (cached under ~/.cache/cargoxx/registry-clone). 6. Create branch publish/-; new package → also create recipes//{maintainers.txt,meta.toml}. 7. Write recipes//versions/.toml; commit. 8. Push branch; `tea pr create` (tea login is already set up). ``` ### 2.5 Validation bot (`.gitea/workflows/validate-pr.yml`) Runs on every PR via self-hosted runners: 1. **Schema check**: PR touches only `recipes//**`; recipe TOML validates. 2. **Source fixity**: re-fetch `[source].url` at `[source].commit`, recompute sha256, compare. 3. **Build**: `nix build .#packages.x86_64-linux.`. 4. **Cache push**: ``` nix copy --to "file:///srv/cargoxx-cache?secret-key=$KEY" \ .#packages.x86_64-linux. ``` `/srv/cargoxx-cache` served by nginx as the binary cache. 5. **Dependency closure**: every `[dependencies]` entry exists at the same PR HEAD. 6. **Maintainer match**: - New package: PR author becomes maintainer (write `maintainers.txt`). - Existing package: PR author appears in `maintainers.txt`. Else label `needs-human-review`. 7. On all-green + maintainer-match: label `auto-merge`. `auto-merge.yml` triggers on the `auto-merge` label and merges via the Gitea API. Maintainer transfer is a separate PR editing `maintainers.txt`. Same validation: PR author must already be on the list. ### 2.6 Resolving registry deps (`cargoxx add foo`) ``` cargoxx add foo[@] 1. Fetch https://gitea/.../cargoxx-pkgs/raw/branch/master/index.json (the index, emitted by CI, maps name → list of versions). 2. Pick the highest matching version (or @). 3. Fetch recipes/foo/versions/.toml. 4. Write Cargoxx.toml: foo = { version = "", registry = "cargoxx" }. 5. Write Cargoxx.lock: source_kind = "registry", registry_attr = "foo_". 6. Also persist lock.registry_rev = . ``` The bundled cargoxx wrapper has `git`, `curl` on PATH, so this works on non-Nix hosts too. ### 2.7 Binary cache config Once the cache URL + signing key are decided, the cargoxx wrapper's `NIX_CONFIG` (in `flake.nix`'s `cargoxxNixConfig`) gains: ``` substituters = https://cache.cargoxx./... https://cache.nixos.org trusted-public-keys = cache.cargoxx.:= cache.nixos.org:6NCH…= ``` Set via `--set-default`, so a user with a different setup can override by exporting `NIX_CONFIG`. ### 2.8 Hosting | Concern | Choice | |---|---| | Repo host | Self-hosted Gitea (your instance). | | CI | Gitea Actions on self-hosted runners (your existing pool). | | Binary cache | Static-HTTPS to start (`nix copy --to file:///srv/cargoxx-cache`); migrate to `attic` (https://github.com/zhaofengli/attic) if scale demands. | | Publishing auth | `tea login` (already configured). | | Architectures | `x86_64-linux` only for v1. aarch64/darwin are follow-ups. | ### 2.9 Phasing | Phase | Deliverable | |---|---| | 2a | `cargoxx-pkgs` repo skeleton + `flake.nix` (zero recipes). | | 2b | `.gitea/workflows/validate-pr.yml`; manual recipe PRs. | | 2c | `cargoxx publish` CLI; `auto-merge.yml`. | | 2d | `cargoxx add` ↔ registry index; CI cache push; wrapper substituters config. | --- ## Module BMI strategy Following the libstdc++/libc++ model in nixpkgs: ship **source `.cppm`** via `install(TARGETS … FILE_SET CXX_MODULES …)`, ship the **compiled archive** (`libfoo.a`) of impl bits. Consumers regenerate the BMI in their own build (cmake's `FILE_SET CXX_MODULES` handles it when `find_package(foo)` is called). The compiled archive is what's cached — the BMI is cheap to regenerate against an already-cached archive. cargoxx already enables `CMAKE_CXX_MODULE_STD ON`, which CMake uses to rebuild the `std` BMI in each consumer project. The same machinery covers cargoxx libraries. --- ## Why `buildCppPackage` doesn't fetch source for registry deps The registry flake (`cargoxx-pkgs`) owns the `source-fetch → cargoxx-build → cmake-install` pipeline. Its `packages..` is an input-addressed derivation: same sources + same toolchain pins → same hash → cache hit. Consumers reference that derivation via `builtins.getFlake` at the locked `registry_rev`. Nix's binary cache substitutes the `$out` directly — no source clone, no compile. Source fetch + build live inside the registry's eval, run once per recipe at CI time. This is the same model as nixpkgs/`cache.nixos.org`: you don't fetch fmt's source to use it from nixpkgs. --- ## Open follow-ups 1. Gitea domain + cache URL (for wrapper config in Phase 2d). 2. Signing key generation (`nix-store --generate-binary-cache-key cache.cargoxx. `); private into runner secrets, public into wrapper config. 3. Multi-arch (`aarch64-linux`, `*-darwin`) — Phase 3+. 4. Conan-style wrap recipes for non-cargoxx libraries — design now, defer implementation.