From a4a3bc1da3213543fb15027beed52fa4360b58cc Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 09:23:00 +0000 Subject: [PATCH 1/6] Start vex vendored gem leftover fix (#2076) Assisted-by: Claude Code:claude-fable-5-2 From c5a018ccae9087081e40c93c736b8ca1906752dd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 09:48:33 +0000 Subject: [PATCH 2/6] Skip gem-home leftovers in vex's drift probe Once Gemfile.lock takes a vendored gem from its PATH source, Bundler loads the vendored directory and installs nothing for the gem. The registry copy an earlier bundle install left in the gem home stays, unpatched, until bundle clean. vex counted it as the live tree and warned vendored_tree_out_of_sync on every run, with a remedy (re-run the install) that could never clear it. The probe now skips a vendored gem's gem-home copies, as it does Go's module cache and Gradle's files-2.1 copies. A default gem's copy still counts, since RubyGems can load it before Bundler sets the load path. The verdict was never affected and is unchanged. Fixes #2076 Assisted-by: Claude Code:claude-fable-5-2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 6 +- .../tests/e2e_vendor_gem_build.rs | 44 +++++++++++ crates/socket-patch-core/src/vex/verify.rs | 76 ++++++++++++++++++- 3 files changed, 121 insertions(+), 5 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 1755cd6fb..f131fe302 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -460,7 +460,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm *The unwind direction.* After a hosted unwind — `rollback` / `remove` of a hosted pin, scan's withdrawn-patch restore, `vendor --revert` of a package vendored over hosted wiring before v5 — put a gem back on its upstream registry entry, its installed copies and archives hold the PATCHED gem, with the same consequences in reverse: the patched code keeps running, and on bundler ≥ 2.6 `bundle pristine` fails on the restored upstream checksum. The unwind applies the same refresh with the upstream gem: the sha256 is the restored lock's `CHECKSUMS` entry, else the rubygems.org compact index; the archive is taken from disk, else downloaded from rubygems.org when the lock resolves the gem from it (a gem from another remote is refreshed only from an archive already on disk). An installed copy is stale when its `/cache/.gem` is readable and is not the upstream archive (ruby-platform gems only: the lock's checksum names that archive). Notes are `upstream_gem_install_refreshed` / `upstream_gem_install_would_refresh`; an installed copy left on the patched build is `upstream_gem_stale_install` (one consolidated warning in a human run), and an archive left in bundler's cache dir keeps its `upstream_gem_stale_cache` warning. A gem whose installed copies were refreshed is not counted in rollback's "keep their patched bytes" note. -*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact. Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. +*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one. `vex` does not count those leftovers as the installed tree of a vendored gem (no `vendored_tree_out_of_sync` for them; a default gem's copy still counts). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. **Pipenv hosted redirect (`Pipfile.lock`, pipfile-spec 6)**: every category other than `_meta` (`default`, `develop`, and Pipenv 2022+ named categories) that pins the package at the patched version is rewritten to the hosted reference — `{"file" | "path": "#sha256=", "hashes": ["sha256:"]}` with `markers`/`extras`/`index` kept exactly as Pipenv wrote them (present or absent: whether Pipenv records `index` depends on its release, the Pipfile spelling and the locking environment, so only the entry itself knows) and `version` dropped; `_meta` (the Pipfile content hash) and the Pipfile itself are never touched, so `pipenv install --deploy`/`sync`/`verify` keep passing. The reference KEY depends on the installing Pipenv: releases 7–11 only install `path` references, 2018 and later `file` ones (0–6 write pipfile-spec < 6 and are refused). The release is probed once per command with `pipenv --version`, resolved on ABSOLUTE `PATH` entries only (a relative entry would run a `pipenv` planted in the scanned repository; `.bat`/`.cmd` shims are found through `PATHEXT` on Windows), only when a pypi patch actually targets an entry of the lock, and `SOCKET_PIPENV_MAJOR=` pins the answer without spawning anything. An unknown installer selects `file` and warns `redirect_pipenv_installer_unknown` only when the lock was rewritten. **Refusal scope**: a pin/source CONFLICT (another version pinned, a foreign `file`/`path` source, a VCS/editable dependency) refuses the whole dependency atomically across categories as `redirect_pipenv_refused` AND vetoes the sibling Python rewriters (requirements.txt / uv.lock / pyproject) for that patch — the project's Pipenv install could not pick the patch up, so a half-redirected checkout is refused; anything else (no entry for the package, an old pipfile-spec, an unparseable lock, a digest-less patch) is `redirect_pipenv_skipped` and leaves the siblings alone (a stale Pipfile.lock in a uv/Poetry/requirements project must not block them). The veto applies to a LIVE lock only: a `Pipfile.lock` with no `Pipfile` beside it is abandoned, so its conflict refuses that file but never the siblings. Hash enforcement at install time is split by era — the `#sha256=` URL fragment is what Pipenv 2023+ verifies, the `hashes` list what 2018–2022 verify, Pipenv 11 either — so both are load-bearing. **Pipenv stale-install guard**: Pipenv never reinstalls a release that is already present (`pipenv install`, `install --deploy` and `sync` all exit 0 and keep the installed bytes — measured on 11.10.4, 2018.11.26 and 2026.8.0, hosted and vendored), so after the rewrite the run probes the Python crawler's site-packages (VIRTUAL_ENV, `./.venv`, `./venv`, Pipenv's out-of-tree `WORKON_HOME` venv; `--global`/`--global-prefix` honoured) for each confirmed Pipfile.lock redirect with the Python stale-install guard's evidence (the pinned patched wheel, else the patch record; STALE needs positive evidence, read-only, skipped on `--dry-run`, stale purls excluded from the same-run `--vex` `assume_applied` set) and the Python stale-install guard (`redirect_pypi_stale_install`, see above) names the site-packages dir and the Pipenv-specific verified remedy: `pipenv run pip uninstall -y && pipenv sync` (or `pipenv --rm && pipenv sync`), with the `sync` arguments following the lock, since plain `pipenv sync` installs only `default`: the targeted form re-syncs the categories that pin the package (`--dev` for `develop`, `--categories ""` for a named category) and the `--rm` form re-syncs every non-empty category — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away. The vendored backend emits the twin `pypi_pipenv_stale_install` (`skipped` warning event). **Rollback** (v5.0, upstream restore): each hosted entry gets its registry shape back — `"version": "=="`, the entry's own `index` carried back unchanged (refused unless it — and the Pipfile's explicit `index`, if any — names a PyPI source in `_meta.sources`), and every release file's sha256 from PyPI's JSON API (`SOCKET_PYPI_JSON_API`), sorted by filename as Pipenv records them; an entry that pins another version beside the hosted reference is refused with the `git checkout` remedy (see "Hosted unwind coverage"). A Pipfile names no project, so a same-run `--vex` on a Pipenv project needs `--vex-product` (or a git remote) to detect a product purl. **Discovery**: `Pipfile.lock` is part of the lockfile inventory (every category's `==` pins, with the lock's digest set as `Sha256AnyOf` integrity so a lock-only checkout can be vendored by fetching the pure wheel through PyPI's JSON API — only when `_meta.sources` name the public index; a private-index lock stays discovery-only and never reaches pypi.org), and Socket's own hosted / vendored references stay discoverable as the package they replace, so a re-scan of an already-redirected or already-vendored lock-only checkout re-confirms it (`--vex` attests, vendored reports `already_vendored`) instead of finding nothing. @@ -652,7 +652,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x); the attestation itself is unchanged; a vendored gem's registry copy left in a gem home is not reported, since Bundler loads the vendored `path:` source and no install replaces that copy, except a default gem's, which RubyGems can load before Bundler runs). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) @@ -720,7 +720,7 @@ Recognition rules that hold for every ecosystem: | Wiring | Evidence (verify mode) | Marker | |---|---|---| -| Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync`. | `(vendored)` | +| Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync` (not for a copy the vendored build never loads: Go's module cache, a Gradle `files-2.1` copy, a vendored gem's gem-home leftover). | `(vendored)` | | Hosted: a discovered patch-host reference (or a live pre-v5 redirect-ledger record) | The installed copies the build **consumes** through the hosted wiring are hash-verified when any exist: the Go replacement module, never the pristine `M@v` in the module cache; the Socket-registry cargo source dir; maven's suffixed version. Installed evidence wins: `hash_mismatch` / `not_applied` are omitted. With **nothing installed**, a discovered reference whose lock pins the artifact (or whose format's rewriter never writes a pin) attests from that pin, which is the same evidence as in-run `scan --mode hosted --vex`. A pre-v5 ledger-only record, or a reference whose required pin is missing, stays `package_not_found`. So do purls that `--ecosystems` kept out of the crawl, because "not installed" has to mean the crawler looked. The same goes for npm purls when an installed pnpm tree records its virtual store outside the project (`enableGlobalVirtualStore`, or a `virtualStoreDir` that climbs out): transitive deps there are invisible to the crawler. A pnpm `modulesDir` inside the project is crawled. A Composer purl is omitted as `composer_install_unread` instead when `composer.json`'s `config.vendor-dir` puts the install outside the project (an absolute path, `~/…`, `$VAR/…`, or one that climbs out) and an install is there (its `composer/installed.json` exists, or the directory cannot be resolved): the crawler reads no tree outside the project that only a project file names, so that install was not checked and may predate the redirect (#686). While nothing is installed there the pin still attests, and with `COMPOSER_VENDOR_DIR` set to the directory the crawler reads it and the installed bytes decide. A yarn Plug'n'Play project (`.pnp.cjs` / `.pnp.js`) has no tree to crawl: an npm purl there attests from the lock's pin only when the PnP loader itself resolves it through the patch (berry names the hosted url, yarn 1 the resolved url's `#` in its cache folder). A loader written before the lock was rewired runs the registry copy, so the purl stays omitted until `yarn install` rewrites it (#519). | `(redirected)` | | Agent: a manifest record with no live hosted/vendored wiring | The installed tree, unchanged. **Every** installed copy the crawler finds for the purl (npm nests duplicates of one `name@version`; pnpm, vlt, Bun and Deno stores add peer-variant copies and copies bundled inside other packages) must hash to the patched bytes, as `apply` patches every copy (Maven: every copy a build consumes, Gradle hash dirs included — see [Gradle builds](#gradle-builds-v50)). One unpatched copy omits the purl with that copy's tag (`not_applied` / `hash_mismatch`). | none | diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index aed069313..c5a3a8044 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -654,6 +654,50 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { "vendored attestation must carry the (vendored) marker: {impact}" ); + // REGRESSION (#2076): the registry copy the first `bundle install` put + // in the gem home is a leftover now. Bundler loads the vendored path + // and installs nothing for rack, so the install the run prescribes + // keeps the leftover, unpatched. `vex` must not report it as an + // out-of-sync live tree, before that install or after it. + assert!( + !stderr.contains("vendored artifact"), + "a gem-home leftover is no out-of-sync tree:\n{stderr}" + ); + let reinstall = bundle(&proj, &["install"], false); + assert!( + reinstall.status.success(), + "`bundle install` over the vendored pair must succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&reinstall.stdout), + String::from_utf8_lossy(&reinstall.stderr), + ); + assert_eq!( + std::fs::read(&installed_rb).expect("the leftover lib/rack.rb"), + orig, + "bundler keeps the gem-home copy it no longer loads (test premise)" + ); + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vex", + "--json", + "--cwd", + proj.to_str().unwrap(), + "--output", + vex_path.to_str().unwrap(), + "--product", + "pkg:gem/app@1.0.0", + ], + ); + assert_eq!(code, 0, "vex failed.\nstdout:\n{stdout}\nstderr:\n{stderr}"); + let env = parse_json_envelope(&stdout); + assert_eq!(env["status"], "success", "envelope: {env}"); + assert!( + env["warnings"] + .as_array() + .is_none_or(|w| w.iter().all(|w| w["code"] != "vendored_tree_out_of_sync")), + "a gem-home leftover is no out-of-sync tree: {env}" + ); + // 5. FRESH-CHECKOUT PROOF: ONLY the committable files, frozen lock. The // vendored path source is the only provider of rack (the fresh dir // has no vendor/bundle), and the patched constant must be visible at diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 197e7be54..527f56be8 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -15,6 +15,7 @@ use std::collections::HashMap; use std::path::{Path, PathBuf}; +use crate::crawlers::ruby_crawler::is_default_gem_copy; use crate::crawlers::Ecosystem; use crate::manifest::schema::{PatchManifest, PatchRecord}; use crate::patch::apply::{verify_file_patch, VerifyStatus}; @@ -238,14 +239,26 @@ pub async fn applied_patches_with_copies( // build resolves the GAV from the committed repository // only (exclusiveContent), so the cache copy is a // pristine sibling the build never reads. - let go_cache_copy = Ecosystem::from_purl(purl) == Some(Ecosystem::Golang); + // + // So is a gem-home copy of a vendored gem (#2076): the + // lock takes the gem from its `PATH` source, so Bundler + // loads the committed directory and installs nothing + // for it. The registry copy an earlier `bundle install` + // left behind stays until `bundle clean`, and no + // install resyncs it. A default gem's copy still + // counts: RubyGems can have loaded it before Bundler + // sets the load path. + let ecosystem = Ecosystem::from_purl(purl); + let go_cache_copy = ecosystem == Some(Ecosystem::Golang); + let gem = ecosystem == Some(Ecosystem::Gem); let installed = package_copies .get(purl) .filter(|_| !go_cache_copy) .map(Vec::as_slice) .unwrap_or_default() .iter() - .filter(|p| !crate::crawlers::gradle_cache::is_gradle_version_dir(p)); + .filter(|p| !crate::crawlers::gradle_cache::is_gradle_version_dir(p)) + .filter(|p| !gem || is_default_gem_copy(p)); for pkg_path in installed { let in_sync = if is_vlt_dir_entry(entry) { vlt_installed_copy_matches(&ctx.project_root, pkg_path, entry, record) @@ -2014,6 +2027,65 @@ mod tests { ); } + /// Regression (#2076): once the lock takes a vendored gem from its + /// `PATH` source, Bundler loads the vendored directory and installs + /// nothing for the gem, so the registry copy an earlier `bundle install` + /// left in the gem home is a leftover no install resyncs. It is not + /// reported out of sync. A default gem's copy still is: RubyGems can + /// have loaded it before Bundler sets the load path. + #[tokio::test] + async fn gem_home_leftover_of_a_vendored_gem_is_not_out_of_sync() { + let root = tempfile::tempdir().unwrap(); + let purl = "pkg:gem/rack@2.2.8"; + let rel = format!(".socket/vendor/gem/{VUUID}/rack-2.2.8"); + let patched = b"module Rack; end # patched\n"; + let vdir = root.path().join(&rel); + tokio::fs::create_dir_all(&vdir).await.unwrap(); + tokio::fs::write(vdir.join("index.js"), patched) + .await + .unwrap(); + let home = root.path().join("vendor/bundle/ruby/3.3.0"); + let leftover = home.join("gems/rack-2.2.8"); + tokio::fs::create_dir_all(&leftover).await.unwrap(); + tokio::fs::write(leftover.join("index.js"), b"module Rack; end\n") + .await + .unwrap(); + + let mut rec = record_with_one_file(&compute_git_sha256_from_bytes(patched)); + rec.uuid = VUUID.to_string(); + let mut manifest = PatchManifest::new(); + manifest.patches.insert(purl.to_string(), rec); + let ctx = VendorContext { + project_root: root.path().to_path_buf(), + entries: HashMap::from([(purl.to_string(), vendor_entry(purl, &rel))]), + go_patches: HashMap::new(), + hosted: HashMap::new(), + }; + let paths = HashMap::from([(purl.to_string(), leftover)]); + + let out = applied_patches_with_vendor(&manifest, &paths, Some(&ctx)).await; + assert_eq!(out.applied, vec![purl.to_string()]); + assert_eq!(out.vendored, vec![purl.to_string()]); + assert!( + out.vendored_out_of_sync.is_empty(), + "Bundler does not load the gem-home leftover: {:?}", + out.vendored_out_of_sync + ); + + let default_specs = home.join("specifications/default"); + tokio::fs::create_dir_all(&default_specs).await.unwrap(); + tokio::fs::write(default_specs.join("rack-2.2.8.gemspec"), b"") + .await + .unwrap(); + let out = applied_patches_with_vendor(&manifest, &paths, Some(&ctx)).await; + assert_eq!(out.vendored, vec![purl.to_string()]); + assert_eq!( + out.vendored_out_of_sync, + vec![purl.to_string()], + "a default gem's copy can load ahead of Bundler" + ); + } + /// Disclosure probe, tampered direction: the vendor artifact is healthy /// (attest + vendored) while the installed tree is present with bytes /// matching NEITHER `beforeHash` nor `afterHash`. The verdict stands but From a63ceb6f5bee0e3aaca161a3945e9a1dfaf86bd7 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 09:57:46 +0000 Subject: [PATCH 3/6] Name a working remedy for a gem leftover in vex This replaces the previous commit's approach. Dropping the gem-home copy from the drift probe hid a copy that a load outside Bundler still runs (plain ruby, a stale --standalone setup script), so the probe is back to what main has. What #2076 reports is the wording. For a vendored gem the warning said the live tree carries different bytes and to re-run the install. Bundler loads the vendored path and bundle install never replaces the old registry copy, so the advice could not clear it. The gem detail now says the copy is a leftover Bundler no longer loads, and names bundle clean (a project bundle path) or gem uninstall NAME -v VERSION (a shared gem home). The warning code, the verdict and every other ecosystem's text are unchanged. Fixes #2076 Assisted-by: Claude Code:claude-fable-5-2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 6 +- crates/socket-patch-cli/src/commands/vex.rs | 71 ++++++++++++++-- .../tests/e2e_vendor_gem_build.rs | 84 +++++++++++-------- crates/socket-patch-core/src/vex/verify.rs | 76 +---------------- 4 files changed, 116 insertions(+), 121 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index f131fe302..5b4851b5e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -460,7 +460,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm *The unwind direction.* After a hosted unwind — `rollback` / `remove` of a hosted pin, scan's withdrawn-patch restore, `vendor --revert` of a package vendored over hosted wiring before v5 — put a gem back on its upstream registry entry, its installed copies and archives hold the PATCHED gem, with the same consequences in reverse: the patched code keeps running, and on bundler ≥ 2.6 `bundle pristine` fails on the restored upstream checksum. The unwind applies the same refresh with the upstream gem: the sha256 is the restored lock's `CHECKSUMS` entry, else the rubygems.org compact index; the archive is taken from disk, else downloaded from rubygems.org when the lock resolves the gem from it (a gem from another remote is refreshed only from an archive already on disk). An installed copy is stale when its `/cache/.gem` is readable and is not the upstream archive (ruby-platform gems only: the lock's checksum names that archive). Notes are `upstream_gem_install_refreshed` / `upstream_gem_install_would_refresh`; an installed copy left on the patched build is `upstream_gem_stale_install` (one consolidated warning in a human run), and an archive left in bundler's cache dir keeps its `upstream_gem_stale_cache` warning. A gem whose installed copies were refreshed is not counted in rollback's "keep their patched bytes" note. -*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one. `vex` does not count those leftovers as the installed tree of a vendored gem (no `vendored_tree_out_of_sync` for them; a default gem's copy still counts). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. +*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact (`vendored_tree_out_of_sync`, whose gem wording names `bundle clean`). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. **Pipenv hosted redirect (`Pipfile.lock`, pipfile-spec 6)**: every category other than `_meta` (`default`, `develop`, and Pipenv 2022+ named categories) that pins the package at the patched version is rewritten to the hosted reference — `{"file" | "path": "#sha256=", "hashes": ["sha256:"]}` with `markers`/`extras`/`index` kept exactly as Pipenv wrote them (present or absent: whether Pipenv records `index` depends on its release, the Pipfile spelling and the locking environment, so only the entry itself knows) and `version` dropped; `_meta` (the Pipfile content hash) and the Pipfile itself are never touched, so `pipenv install --deploy`/`sync`/`verify` keep passing. The reference KEY depends on the installing Pipenv: releases 7–11 only install `path` references, 2018 and later `file` ones (0–6 write pipfile-spec < 6 and are refused). The release is probed once per command with `pipenv --version`, resolved on ABSOLUTE `PATH` entries only (a relative entry would run a `pipenv` planted in the scanned repository; `.bat`/`.cmd` shims are found through `PATHEXT` on Windows), only when a pypi patch actually targets an entry of the lock, and `SOCKET_PIPENV_MAJOR=` pins the answer without spawning anything. An unknown installer selects `file` and warns `redirect_pipenv_installer_unknown` only when the lock was rewritten. **Refusal scope**: a pin/source CONFLICT (another version pinned, a foreign `file`/`path` source, a VCS/editable dependency) refuses the whole dependency atomically across categories as `redirect_pipenv_refused` AND vetoes the sibling Python rewriters (requirements.txt / uv.lock / pyproject) for that patch — the project's Pipenv install could not pick the patch up, so a half-redirected checkout is refused; anything else (no entry for the package, an old pipfile-spec, an unparseable lock, a digest-less patch) is `redirect_pipenv_skipped` and leaves the siblings alone (a stale Pipfile.lock in a uv/Poetry/requirements project must not block them). The veto applies to a LIVE lock only: a `Pipfile.lock` with no `Pipfile` beside it is abandoned, so its conflict refuses that file but never the siblings. Hash enforcement at install time is split by era — the `#sha256=` URL fragment is what Pipenv 2023+ verifies, the `hashes` list what 2018–2022 verify, Pipenv 11 either — so both are load-bearing. **Pipenv stale-install guard**: Pipenv never reinstalls a release that is already present (`pipenv install`, `install --deploy` and `sync` all exit 0 and keep the installed bytes — measured on 11.10.4, 2018.11.26 and 2026.8.0, hosted and vendored), so after the rewrite the run probes the Python crawler's site-packages (VIRTUAL_ENV, `./.venv`, `./venv`, Pipenv's out-of-tree `WORKON_HOME` venv; `--global`/`--global-prefix` honoured) for each confirmed Pipfile.lock redirect with the Python stale-install guard's evidence (the pinned patched wheel, else the patch record; STALE needs positive evidence, read-only, skipped on `--dry-run`, stale purls excluded from the same-run `--vex` `assume_applied` set) and the Python stale-install guard (`redirect_pypi_stale_install`, see above) names the site-packages dir and the Pipenv-specific verified remedy: `pipenv run pip uninstall -y && pipenv sync` (or `pipenv --rm && pipenv sync`), with the `sync` arguments following the lock, since plain `pipenv sync` installs only `default`: the targeted form re-syncs the categories that pin the package (`--dev` for `develop`, `--categories ""` for a named category) and the `--rm` form re-syncs every non-empty category — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away. The vendored backend emits the twin `pypi_pipenv_stale_install` (`skipped` warning event). **Rollback** (v5.0, upstream restore): each hosted entry gets its registry shape back — `"version": "=="`, the entry's own `index` carried back unchanged (refused unless it — and the Pipfile's explicit `index`, if any — names a PyPI source in `_meta.sources`), and every release file's sha256 from PyPI's JSON API (`SOCKET_PYPI_JSON_API`), sorted by filename as Pipenv records them; an entry that pins another version beside the hosted reference is refused with the `git checkout` remedy (see "Hosted unwind coverage"). A Pipfile names no project, so a same-run `--vex` on a Pipenv project needs `--vex-product` (or a git remote) to detect a product purl. **Discovery**: `Pipfile.lock` is part of the lockfile inventory (every category's `==` pins, with the lock's digest set as `Sha256AnyOf` integrity so a lock-only checkout can be vendored by fetching the pure wheel through PyPI's JSON API — only when `_meta.sources` name the public index; a private-index lock stays discovery-only and never reaches pypi.org), and Socket's own hosted / vendored references stay discoverable as the package they replace, so a re-scan of an already-redirected or already-vendored lock-only checkout re-confirms it (`--vex` attests, vendored reports `already_vendored`) instead of finding nothing. @@ -652,7 +652,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x); the attestation itself is unchanged; a vendored gem's registry copy left in a gem home is not reported, since Bundler loads the vendored `path:` source and no install replaces that copy, except a default gem's, which RubyGems can load before Bundler runs). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail says instead that the copy is a leftover of an earlier install which Bundler no longer loads and no `bundle install` replaces, now that `Gemfile.lock` takes the gem from the vendored path, and names `bundle clean` for a project bundle path or `gem uninstall -v ` for a gem home no other project needs it from); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) @@ -720,7 +720,7 @@ Recognition rules that hold for every ecosystem: | Wiring | Evidence (verify mode) | Marker | |---|---|---| -| Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync` (not for a copy the vendored build never loads: Go's module cache, a Gradle `files-2.1` copy, a vendored gem's gem-home leftover). | `(vendored)` | +| Vendored: a lockfile/config wires a `.socket/vendor` artifact, or a live vendor ledger entry | The **committed artifact** is hashed against the record's `afterHash`. The ledger entry is used when it names the wired artifact (it carries the dir-artifact inventory); otherwise an entry is synthesized from the reference. A present installed tree with different bytes only warns `vendored_tree_out_of_sync`. | `(vendored)` | | Hosted: a discovered patch-host reference (or a live pre-v5 redirect-ledger record) | The installed copies the build **consumes** through the hosted wiring are hash-verified when any exist: the Go replacement module, never the pristine `M@v` in the module cache; the Socket-registry cargo source dir; maven's suffixed version. Installed evidence wins: `hash_mismatch` / `not_applied` are omitted. With **nothing installed**, a discovered reference whose lock pins the artifact (or whose format's rewriter never writes a pin) attests from that pin, which is the same evidence as in-run `scan --mode hosted --vex`. A pre-v5 ledger-only record, or a reference whose required pin is missing, stays `package_not_found`. So do purls that `--ecosystems` kept out of the crawl, because "not installed" has to mean the crawler looked. The same goes for npm purls when an installed pnpm tree records its virtual store outside the project (`enableGlobalVirtualStore`, or a `virtualStoreDir` that climbs out): transitive deps there are invisible to the crawler. A pnpm `modulesDir` inside the project is crawled. A Composer purl is omitted as `composer_install_unread` instead when `composer.json`'s `config.vendor-dir` puts the install outside the project (an absolute path, `~/…`, `$VAR/…`, or one that climbs out) and an install is there (its `composer/installed.json` exists, or the directory cannot be resolved): the crawler reads no tree outside the project that only a project file names, so that install was not checked and may predate the redirect (#686). While nothing is installed there the pin still attests, and with `COMPOSER_VENDOR_DIR` set to the directory the crawler reads it and the installed bytes decide. A yarn Plug'n'Play project (`.pnp.cjs` / `.pnp.js`) has no tree to crawl: an npm purl there attests from the lock's pin only when the PnP loader itself resolves it through the patch (berry names the hosted url, yarn 1 the resolved url's `#` in its cache folder). A loader written before the lock was rewired runs the registry copy, so the purl stays omitted until `yarn install` rewrites it (#519). | `(redirected)` | | Agent: a manifest record with no live hosted/vendored wiring | The installed tree, unchanged. **Every** installed copy the crawler finds for the purl (npm nests duplicates of one `name@version`; pnpm, vlt, Bun and Deno stores add peer-variant copies and copies bundled inside other packages) must hash to the patched bytes, as `apply` patches every copy (Maven: every copy a build consumes, Gradle hash dirs included — see [Gradle builds](#gradle-builds-v50)). One unpatched copy omits the purl with that copy's tag (`not_applied` / `hash_mismatch`). | none | diff --git a/crates/socket-patch-cli/src/commands/vex.rs b/crates/socket-patch-cli/src/commands/vex.rs index 2231d8378..dc68eabbc 100644 --- a/crates/socket-patch-cli/src/commands/vex.rs +++ b/crates/socket-patch-cli/src/commands/vex.rs @@ -1325,13 +1325,7 @@ async fn generate_vex( warnings, common, "vendored_tree_out_of_sync", - format!( - "{purl}: the installed tree does not match its vendored artifact; the \ - attestation is based on the committed .socket/vendor artifact (the project's \ - lockfile or package-manager config wires it in), but the live tree carries \ - different bytes — re-run your package manager's install to resync it.\ - {hatch_note}{pdm_note}" - ), + vendored_out_of_sync_detail(purl, &format!("{hatch_note}{pdm_note}")), ); } @@ -1809,6 +1803,40 @@ async fn withhold_stale_go_tools( })); } +/// The `vendored_tree_out_of_sync` detail for `purl`; `notes` are the +/// manager-specific remedies (Hatch, PDM) appended to the generic one. +/// +/// A gem gets its own wording (#2076). Once `Gemfile.lock` takes the gem +/// from the vendored `path:` source, Bundler loads that directory and +/// installs nothing for the gem, so the registry copy an earlier install +/// left in the gem home stays as it is: "re-run your install" never clears +/// the warning, and "the live tree" misnames a copy Bundler no longer +/// loads. The copy is still disclosed, because a load that bypasses +/// Bundler (plain `ruby`, a stale `--standalone` setup script) runs it. +fn vendored_out_of_sync_detail(purl: &str, notes: &str) -> String { + use socket_patch_core::utils::purl::{purl_name_version, strip_purl_qualifiers}; + let base = strip_purl_qualifiers(purl); + let gem = purl_name_version(base).filter(|_| base.starts_with("pkg:gem/")); + match gem { + Some((name, version)) => format!( + "{purl}: a copy of this gem left in the gem home by an earlier install does not \ + match its vendored artifact. The attestation is based on the committed \ + .socket/vendor artifact: Gemfile.lock takes the gem from that path, so Bundler \ + loads the patched copy and `bundle install` neither replaces nor removes the \ + other one, which only code that loads the gem without Bundler still runs. \ + Remove it with `bundle clean` when the project has its own bundle path, or \ + with `gem uninstall {name} -v {version}` in a gem home no other project needs \ + it from.{notes}" + ), + None => format!( + "{purl}: the installed tree does not match its vendored artifact; the \ + attestation is based on the committed .socket/vendor artifact (the project's \ + lockfile or package-manager config wires it in), but the live tree carries \ + different bytes — re-run your package manager's install to resync it.{notes}" + ), + } +} + /// Record a run-level advisory the way `update`/`vendor` do: stderr /// (`Warning: `) in human mode, and into `warnings` so the `--json` /// envelope — which silences stderr — carries it in `warnings[]` instead. @@ -2470,6 +2498,35 @@ mod tests { use super::*; use clap::Parser; + /// #2076: `bundle install` never replaces a vendored gem's gem-home + /// leftover, so the gem warning names the commands that do remove it + /// and does not call the leftover the live tree. Other ecosystems keep + /// the install remedy and their manager-specific notes. + #[test] + fn out_of_sync_detail_names_a_remedy_bundler_honors() { + for purl in [ + "pkg:gem/rack@2.2.8", + "pkg:gem/ffi@1.17.0?platform=x86_64-linux", + ] { + let detail = vendored_out_of_sync_detail(purl, ""); + assert!(detail.starts_with(&format!("{purl}: ")), "{detail}"); + assert!(detail.contains("`bundle clean`"), "{detail}"); + assert!(!detail.contains("re-run your package manager"), "{detail}"); + assert!(!detail.contains("live tree"), "{detail}"); + } + let detail = vendored_out_of_sync_detail("pkg:gem/rack@2.2.8", ""); + assert!(detail.contains("`gem uninstall rack -v 2.2.8`"), "{detail}"); + let detail = vendored_out_of_sync_detail("pkg:gem/ffi@1.17.0?platform=x86_64-linux", ""); + assert!(detail.contains("`gem uninstall ffi -v 1.17.0`"), "{detail}"); + + let detail = vendored_out_of_sync_detail("pkg:pypi/idna@3.6", " Hatch note."); + assert!( + detail.ends_with("re-run your package manager's install to resync it. Hatch note."), + "{detail}" + ); + assert!(!detail.contains("bundle"), "{detail}"); + } + /// The go-patches synthesis guards its copy-dir keys with core's /// `are_safe_redirect_coords`; pin the accept/reject set from the CLI /// side — a regression here would let a tampered go.mod `replace` key diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index c5a3a8044..55d7bc5c7 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -656,46 +656,16 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { // REGRESSION (#2076): the registry copy the first `bundle install` put // in the gem home is a leftover now. Bundler loads the vendored path - // and installs nothing for rack, so the install the run prescribes - // keeps the leftover, unpatched. `vex` must not report it as an - // out-of-sync live tree, before that install or after it. + // and installs nothing for rack, so no install replaces that copy. The + // warning about it must name a remedy that removes it (`bundle clean`, + // run below), not a reinstall. assert!( - !stderr.contains("vendored artifact"), - "a gem-home leftover is no out-of-sync tree:\n{stderr}" + stderr.contains(&format!("Warning: {purl}: ")) && stderr.contains("`bundle clean`"), + "the leftover gem-home copy is disclosed with a remedy that clears it:\n{stderr}" ); - let reinstall = bundle(&proj, &["install"], false); assert!( - reinstall.status.success(), - "`bundle install` over the vendored pair must succeed.\nstdout:\n{}\nstderr:\n{}", - String::from_utf8_lossy(&reinstall.stdout), - String::from_utf8_lossy(&reinstall.stderr), - ); - assert_eq!( - std::fs::read(&installed_rb).expect("the leftover lib/rack.rb"), - orig, - "bundler keeps the gem-home copy it no longer loads (test premise)" - ); - let (code, stdout, stderr) = run_socket( - &proj, - &[ - "vex", - "--json", - "--cwd", - proj.to_str().unwrap(), - "--output", - vex_path.to_str().unwrap(), - "--product", - "pkg:gem/app@1.0.0", - ], - ); - assert_eq!(code, 0, "vex failed.\nstdout:\n{stdout}\nstderr:\n{stderr}"); - let env = parse_json_envelope(&stdout); - assert_eq!(env["status"], "success", "envelope: {env}"); - assert!( - env["warnings"] - .as_array() - .is_none_or(|w| w.iter().all(|w| w["code"] != "vendored_tree_out_of_sync")), - "a gem-home leftover is no out-of-sync tree: {env}" + !stderr.contains("re-run your package manager"), + "no install resyncs a vendored gem's leftover:\n{stderr}" ); // 5. FRESH-CHECKOUT PROOF: ONLY the committable files, frozen lock. The @@ -798,6 +768,46 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { "re-vendor must leave Gemfile.lock byte-identical" ); + // #2076, the remedy: `bundle clean` removes the leftover, touches + // neither file of the pair, and `vex` then has nothing to warn about. + let clean = bundle(&proj, &["clean"], false); + assert!( + clean.status.success(), + "`bundle clean` failed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&clean.stdout), + String::from_utf8_lossy(&clean.stderr), + ); + assert!( + !gem_dir.exists(), + "`bundle clean` must remove the leftover {}", + gem_dir.display() + ); + assert_eq!( + std::fs::read(&lock_path).unwrap(), + lock_wired, + "`bundle clean` must leave Gemfile.lock byte-identical" + ); + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vex", + "--json", + "--cwd", + proj.to_str().unwrap(), + "--output", + vex_path.to_str().unwrap(), + "--product", + "pkg:gem/app@1.0.0", + ], + ); + assert_eq!(code, 0, "vex failed.\nstdout:\n{stdout}\nstderr:\n{stderr}"); + let env = parse_json_envelope(&stdout); + assert_eq!(env["status"], "success", "envelope: {env}"); + assert!( + env.get("warnings").is_none(), + "nothing is left to warn about after `bundle clean`: {env}" + ); + // 7. REVERT PROOF: both halves of the pair edit byte-restored, artifacts // gone. let (code, stdout, stderr) = run_socket( diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 527f56be8..197e7be54 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -15,7 +15,6 @@ use std::collections::HashMap; use std::path::{Path, PathBuf}; -use crate::crawlers::ruby_crawler::is_default_gem_copy; use crate::crawlers::Ecosystem; use crate::manifest::schema::{PatchManifest, PatchRecord}; use crate::patch::apply::{verify_file_patch, VerifyStatus}; @@ -239,26 +238,14 @@ pub async fn applied_patches_with_copies( // build resolves the GAV from the committed repository // only (exclusiveContent), so the cache copy is a // pristine sibling the build never reads. - // - // So is a gem-home copy of a vendored gem (#2076): the - // lock takes the gem from its `PATH` source, so Bundler - // loads the committed directory and installs nothing - // for it. The registry copy an earlier `bundle install` - // left behind stays until `bundle clean`, and no - // install resyncs it. A default gem's copy still - // counts: RubyGems can have loaded it before Bundler - // sets the load path. - let ecosystem = Ecosystem::from_purl(purl); - let go_cache_copy = ecosystem == Some(Ecosystem::Golang); - let gem = ecosystem == Some(Ecosystem::Gem); + let go_cache_copy = Ecosystem::from_purl(purl) == Some(Ecosystem::Golang); let installed = package_copies .get(purl) .filter(|_| !go_cache_copy) .map(Vec::as_slice) .unwrap_or_default() .iter() - .filter(|p| !crate::crawlers::gradle_cache::is_gradle_version_dir(p)) - .filter(|p| !gem || is_default_gem_copy(p)); + .filter(|p| !crate::crawlers::gradle_cache::is_gradle_version_dir(p)); for pkg_path in installed { let in_sync = if is_vlt_dir_entry(entry) { vlt_installed_copy_matches(&ctx.project_root, pkg_path, entry, record) @@ -2027,65 +2014,6 @@ mod tests { ); } - /// Regression (#2076): once the lock takes a vendored gem from its - /// `PATH` source, Bundler loads the vendored directory and installs - /// nothing for the gem, so the registry copy an earlier `bundle install` - /// left in the gem home is a leftover no install resyncs. It is not - /// reported out of sync. A default gem's copy still is: RubyGems can - /// have loaded it before Bundler sets the load path. - #[tokio::test] - async fn gem_home_leftover_of_a_vendored_gem_is_not_out_of_sync() { - let root = tempfile::tempdir().unwrap(); - let purl = "pkg:gem/rack@2.2.8"; - let rel = format!(".socket/vendor/gem/{VUUID}/rack-2.2.8"); - let patched = b"module Rack; end # patched\n"; - let vdir = root.path().join(&rel); - tokio::fs::create_dir_all(&vdir).await.unwrap(); - tokio::fs::write(vdir.join("index.js"), patched) - .await - .unwrap(); - let home = root.path().join("vendor/bundle/ruby/3.3.0"); - let leftover = home.join("gems/rack-2.2.8"); - tokio::fs::create_dir_all(&leftover).await.unwrap(); - tokio::fs::write(leftover.join("index.js"), b"module Rack; end\n") - .await - .unwrap(); - - let mut rec = record_with_one_file(&compute_git_sha256_from_bytes(patched)); - rec.uuid = VUUID.to_string(); - let mut manifest = PatchManifest::new(); - manifest.patches.insert(purl.to_string(), rec); - let ctx = VendorContext { - project_root: root.path().to_path_buf(), - entries: HashMap::from([(purl.to_string(), vendor_entry(purl, &rel))]), - go_patches: HashMap::new(), - hosted: HashMap::new(), - }; - let paths = HashMap::from([(purl.to_string(), leftover)]); - - let out = applied_patches_with_vendor(&manifest, &paths, Some(&ctx)).await; - assert_eq!(out.applied, vec![purl.to_string()]); - assert_eq!(out.vendored, vec![purl.to_string()]); - assert!( - out.vendored_out_of_sync.is_empty(), - "Bundler does not load the gem-home leftover: {:?}", - out.vendored_out_of_sync - ); - - let default_specs = home.join("specifications/default"); - tokio::fs::create_dir_all(&default_specs).await.unwrap(); - tokio::fs::write(default_specs.join("rack-2.2.8.gemspec"), b"") - .await - .unwrap(); - let out = applied_patches_with_vendor(&manifest, &paths, Some(&ctx)).await; - assert_eq!(out.vendored, vec![purl.to_string()]); - assert_eq!( - out.vendored_out_of_sync, - vec![purl.to_string()], - "a default gem's copy can load ahead of Bundler" - ); - } - /// Disclosure probe, tampered direction: the vendor artifact is healthy /// (attest + vendored) while the installed tree is present with bytes /// matching NEITHER `beforeHash` nor `afterHash`. The verdict stands but From 5ff0fa02c4a06399c36b7fcac44bd32a557b866f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 10:09:49 +0000 Subject: [PATCH 4/6] Name the gem leftover and its one remedy Review of the previous commit found the gem wording wrong in three cases: a default gem (no command removes it, and RubyGems can load it before Bundler), a shared gem home (gem uninstall needs -i and the platform), and a Hatch note appended to a gem's warning. The drift probe now records which copy drifted, and the warning names that directory and gives the remedy for where it is: bundle clean inside the project, gem uninstall with -i and --platform outside it, none for a default gem. A gem takes no pypi notes. Refs #2076 Assisted-by: Claude Code:claude-fable-5-2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- crates/socket-patch-cli/src/commands/vex.rs | 160 ++++++++++++------ .../tests/e2e_vendor_gem_build.rs | 5 +- crates/socket-patch-core/src/vex/verify.rs | 10 ++ 4 files changed, 124 insertions(+), 53 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 5b4851b5e..00a131dd7 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -652,7 +652,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail says instead that the copy is a leftover of an earlier install which Bundler no longer loads and no `bundle install` replaces, now that `Gemfile.lock` takes the gem from the vendored path, and names `bundle clean` for a project bundle path or `gem uninstall -v ` for a gem home no other project needs it from); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail instead names the gem-home copy and says that a Bundler run of the lock loads the vendored path and no `bundle install` replaces or removes that copy, then gives the one remedy for where it is: `bundle clean` for a copy inside the project, `gem uninstall -v [--platform

] -i ` for a gem home outside it (if no other project needs the copy), and none for a default gem, which ships with Ruby and which RubyGems can load before Bundler runs); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) diff --git a/crates/socket-patch-cli/src/commands/vex.rs b/crates/socket-patch-cli/src/commands/vex.rs index dc68eabbc..4f5dd13a1 100644 --- a/crates/socket-patch-cli/src/commands/vex.rs +++ b/crates/socket-patch-cli/src/commands/vex.rs @@ -1325,7 +1325,15 @@ async fn generate_vex( warnings, common, "vendored_tree_out_of_sync", - vendored_out_of_sync_detail(purl, &format!("{hatch_note}{pdm_note}")), + vendored_out_of_sync_detail( + purl, + outcome + .vendored_out_of_sync_copies + .get(purl) + .map(PathBuf::as_path), + &common.cwd, + &format!("{hatch_note}{pdm_note}"), + ), ); } @@ -1803,38 +1811,66 @@ async fn withhold_stale_go_tools( })); } -/// The `vendored_tree_out_of_sync` detail for `purl`; `notes` are the -/// manager-specific remedies (Hatch, PDM) appended to the generic one. +/// The `vendored_tree_out_of_sync` detail for `purl`. `copy` is the +/// installed copy that drifted, `notes` the manager-specific remedies +/// (Hatch, PDM) appended to the generic one. /// /// A gem gets its own wording (#2076). Once `Gemfile.lock` takes the gem -/// from the vendored `path:` source, Bundler loads that directory and -/// installs nothing for the gem, so the registry copy an earlier install -/// left in the gem home stays as it is: "re-run your install" never clears -/// the warning, and "the live tree" misnames a copy Bundler no longer -/// loads. The copy is still disclosed, because a load that bypasses -/// Bundler (plain `ruby`, a stale `--standalone` setup script) runs it. -fn vendored_out_of_sync_detail(purl: &str, notes: &str) -> String { - use socket_patch_core::utils::purl::{purl_name_version, strip_purl_qualifiers}; - let base = strip_purl_qualifiers(purl); - let gem = purl_name_version(base).filter(|_| base.starts_with("pkg:gem/")); - match gem { - Some((name, version)) => format!( - "{purl}: a copy of this gem left in the gem home by an earlier install does not \ - match its vendored artifact. The attestation is based on the committed \ - .socket/vendor artifact: Gemfile.lock takes the gem from that path, so Bundler \ - loads the patched copy and `bundle install` neither replaces nor removes the \ - other one, which only code that loads the gem without Bundler still runs. \ - Remove it with `bundle clean` when the project has its own bundle path, or \ - with `gem uninstall {name} -v {version}` in a gem home no other project needs \ - it from.{notes}" - ), - None => format!( +/// from the vendored path, Bundler loads that directory and installs +/// nothing for the gem, so the registry copy an earlier install left in the +/// gem home stays as it is: "re-run your install" never clears the warning. +/// The copy is still disclosed, because a load that bypasses Bundler (plain +/// `ruby`, a stale `--standalone` setup script) runs it. The remedy depends +/// on where the copy is: `bundle clean` only prunes a project's own bundle +/// path, a shared gem home needs `gem uninstall`, and nothing removes a +/// default gem. +fn vendored_out_of_sync_detail( + purl: &str, + copy: Option<&Path>, + project_root: &Path, + notes: &str, +) -> String { + use socket_patch_core::crawlers::ruby_crawler::is_default_gem_copy; + use socket_patch_core::utils::purl::{purl_parts, purl_qualifier}; + let gem = purl_parts(purl).filter(|(ty, _, _)| ty == "gem").zip(copy); + let Some(((_, name, version), copy)) = gem else { + return format!( "{purl}: the installed tree does not match its vendored artifact; the \ attestation is based on the committed .socket/vendor artifact (the project's \ lockfile or package-manager config wires it in), but the live tree carries \ different bytes — re-run your package manager's install to resync it.{notes}" - ), - } + ); + }; + let remedy = if is_default_gem_copy(copy) { + "It is a default gem that ships with Ruby: RubyGems can load it before Bundler sets \ + the load path, and neither `bundle clean` nor `gem uninstall` removes it." + .to_string() + } else if copy.starts_with(project_root) { + "Only code that loads the gem without Bundler still runs it. Remove it with \ + `bundle clean`." + .to_string() + } else { + let platform = purl_qualifier(purl, "platform") + .map(|platform| format!(" --platform {platform}")) + .unwrap_or_default(); + let home = copy + .parent() + .and_then(Path::parent) + .map(|home| format!(" -i {}", home.display())) + .unwrap_or_default(); + format!( + "Only code that loads the gem without Bundler still runs it. Remove it with \ + `gem uninstall {name} -v {version}{platform}{home}` if no other project needs \ + it from that gem home." + ) + }; + format!( + "{purl}: {} does not match the vendored artifact. Gemfile.lock takes the gem from \ + the committed .socket/vendor path, which the attestation is based on, so a Bundler \ + run of this lock loads the patched copy, and `bundle install` neither replaces nor \ + removes this one. {remedy}", + copy.display() + ) } /// Record a run-level advisory the way `update`/`vendor` do: stderr @@ -2499,32 +2535,56 @@ mod tests { use clap::Parser; /// #2076: `bundle install` never replaces a vendored gem's gem-home - /// leftover, so the gem warning names the commands that do remove it - /// and does not call the leftover the live tree. Other ecosystems keep - /// the install remedy and their manager-specific notes. + /// leftover, so the gem warning names the copy and the one command + /// that removes it from where it is. Other ecosystems keep the install + /// remedy and their manager-specific notes; a gem never gets those. #[test] fn out_of_sync_detail_names_a_remedy_bundler_honors() { - for purl in [ - "pkg:gem/rack@2.2.8", - "pkg:gem/ffi@1.17.0?platform=x86_64-linux", - ] { - let detail = vendored_out_of_sync_detail(purl, ""); - assert!(detail.starts_with(&format!("{purl}: ")), "{detail}"); - assert!(detail.contains("`bundle clean`"), "{detail}"); - assert!(!detail.contains("re-run your package manager"), "{detail}"); - assert!(!detail.contains("live tree"), "{detail}"); - } - let detail = vendored_out_of_sync_detail("pkg:gem/rack@2.2.8", ""); - assert!(detail.contains("`gem uninstall rack -v 2.2.8`"), "{detail}"); - let detail = vendored_out_of_sync_detail("pkg:gem/ffi@1.17.0?platform=x86_64-linux", ""); - assert!(detail.contains("`gem uninstall ffi -v 1.17.0`"), "{detail}"); - - let detail = vendored_out_of_sync_detail("pkg:pypi/idna@3.6", " Hatch note."); - assert!( - detail.ends_with("re-run your package manager's install to resync it. Hatch note."), - "{detail}" + let tmp = tempfile::tempdir().unwrap(); + let project = tmp.path().join("app"); + let detail = |purl: &str, copy: &Path| { + vendored_out_of_sync_detail(purl, Some(copy), &project, " Hatch note.") + }; + + let local = project.join("vendor/bundle/ruby/3.3.0/gems/rack-2.2.8"); + let text = detail("pkg:gem/rack@2.2.8", &local); + let named = format!("pkg:gem/rack@2.2.8: {} does not match", local.display()); + assert!(text.starts_with(&named), "{text}"); + assert!(text.ends_with("Remove it with `bundle clean`."), "{text}"); + assert!(!text.contains("gem uninstall"), "{text}"); + assert!(!text.contains("re-run your package manager"), "{text}"); + + let home = tmp.path().join("shared/ruby/3.3.0"); + let shared = home.join("gems/ffi-1.17.0-x86_64-linux"); + let text = detail("pkg:gem/ffi@1.17.0?platform=x86_64-linux", &shared); + let uninstall = format!( + "`gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i {}`", + home.display() ); - assert!(!detail.contains("bundle"), "{detail}"); + assert!(text.contains(&uninstall), "{text}"); + assert!(!text.contains("`bundle clean`"), "{text}"); + assert!(!text.contains("Hatch"), "{text}"); + + // A default gem ships with Ruby: no command removes it, and + // RubyGems can load it ahead of Bundler. + let specs = home.join("specifications/default"); + std::fs::create_dir_all(&specs).unwrap(); + std::fs::write(specs.join("json-2.7.1.gemspec"), "").unwrap(); + let text = detail("pkg:gem/json@2.7.1", &home.join("gems/json-2.7.1")); + assert!(text.contains("default gem"), "{text}"); + assert!(!text.contains("Remove it with"), "{text}"); + assert!(!text.contains("Only code that loads"), "{text}"); + + for (purl, copy) in [ + ("pkg:pypi/idna@3.6", Some(local.as_path())), + ("pkg:gem/rack@2.2.8", None), + ] { + let text = vendored_out_of_sync_detail(purl, copy, &project, " Hatch note."); + assert!( + text.ends_with("re-run your package manager's install to resync it. Hatch note."), + "{text}" + ); + } } /// The go-patches synthesis guards its copy-dir keys with core's diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index 55d7bc5c7..35c5293cf 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -659,9 +659,10 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { // and installs nothing for rack, so no install replaces that copy. The // warning about it must name a remedy that removes it (`bundle clean`, // run below), not a reinstall. + let leftover = format!("Warning: {purl}: {} does not match", gem_dir.display()); assert!( - stderr.contains(&format!("Warning: {purl}: ")) && stderr.contains("`bundle clean`"), - "the leftover gem-home copy is disclosed with a remedy that clears it:\n{stderr}" + stderr.contains(&leftover) && stderr.contains("Remove it with `bundle clean`."), + "the leftover gem-home copy is named, with the remedy that clears it:\n{stderr}" ); assert!( !stderr.contains("re-run your package manager"), diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 197e7be54..0c89298f7 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -53,6 +53,9 @@ pub struct VerifyOutcome { /// manager re-installs. An ABSENT installed tree is the expected /// post-vendor state and is never flagged. pub vendored_out_of_sync: Vec, + /// The first installed copy that put each purl in + /// `vendored_out_of_sync`, so a caller can name the directory. + pub vendored_out_of_sync_copies: HashMap, /// Maven / Gradle: `(purl, copy)` for every installed copy of a failed /// purl that does not verify — the copy a build may still load /// unpatched. Callers name them (`vex_gradle_unpatched_copy`). @@ -257,6 +260,8 @@ pub async fn applied_patches_with_copies( }; if !in_sync { out.vendored_out_of_sync.push(purl.clone()); + out.vendored_out_of_sync_copies + .insert(purl.clone(), pkg_path.clone()); break; } } @@ -1966,6 +1971,11 @@ mod tests { assert_eq!(out.applied, vec![purl.to_string()]); assert_eq!(out.vendored, vec![purl.to_string()]); assert_eq!(out.vendored_out_of_sync, vec![purl.to_string()]); + // The copy that drifted is the one named, not the first crawled. + assert_eq!( + out.vendored_out_of_sync_copies, + HashMap::from([(purl.to_string(), root.path().join("drifted"))]) + ); } /// Go vendored: the crawler's module-cache `M@v` is pristine by From f213d05899fcee1fc3a4abfa21779fbdeb77332a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 10:22:55 +0000 Subject: [PATCH 5/6] List every gem leftover with gem uninstall -i Review of the previous commit: choosing bundle clean by whether the copy sits under --cwd broke on a relative --cwd, on a GEM_HOME inside the project (bundle clean refuses there) and on a bundle path outside it, and only the first drifted copy was named. The gem warning now lists every drifted copy and gives one gem uninstall NAME -v VERSION -i HOME per gem home, which removes the copy from any home, a Bundler path included, and mentions bundle clean for the project's own bundle path. The home is shell-quoted and the platform percent-decoded. The text no longer names Gemfile.lock, since the lock may be gems.locked. Refs #2076 Assisted-by: Claude Code:claude-fable-5-2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 4 +- crates/socket-patch-cli/src/commands/vex.rs | 195 +++++++++++------- .../tests/e2e_vendor_gem_build.rs | 14 +- crates/socket-patch-core/src/vex/verify.rs | 16 +- 4 files changed, 140 insertions(+), 89 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 00a131dd7..e05dbfa2e 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -460,7 +460,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm *The unwind direction.* After a hosted unwind — `rollback` / `remove` of a hosted pin, scan's withdrawn-patch restore, `vendor --revert` of a package vendored over hosted wiring before v5 — put a gem back on its upstream registry entry, its installed copies and archives hold the PATCHED gem, with the same consequences in reverse: the patched code keeps running, and on bundler ≥ 2.6 `bundle pristine` fails on the restored upstream checksum. The unwind applies the same refresh with the upstream gem: the sha256 is the restored lock's `CHECKSUMS` entry, else the rubygems.org compact index; the archive is taken from disk, else downloaded from rubygems.org when the lock resolves the gem from it (a gem from another remote is refreshed only from an archive already on disk). An installed copy is stale when its `/cache/.gem` is readable and is not the upstream archive (ruby-platform gems only: the lock's checksum names that archive). Notes are `upstream_gem_install_refreshed` / `upstream_gem_install_would_refresh`; an installed copy left on the patched build is `upstream_gem_stale_install` (one consolidated warning in a human run), and an archive left in bundler's cache dir keeps its `upstream_gem_stale_cache` warning. A gem whose installed copies were refreshed is not counted in rollback's "keep their patched bytes" note. -*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact (`vendored_tree_out_of_sync`, whose gem wording names `bundle clean`). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. +*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact (`vendored_tree_out_of_sync`, whose gem wording names the copy and the command that removes it). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. **Pipenv hosted redirect (`Pipfile.lock`, pipfile-spec 6)**: every category other than `_meta` (`default`, `develop`, and Pipenv 2022+ named categories) that pins the package at the patched version is rewritten to the hosted reference — `{"file" | "path": "#sha256=", "hashes": ["sha256:"]}` with `markers`/`extras`/`index` kept exactly as Pipenv wrote them (present or absent: whether Pipenv records `index` depends on its release, the Pipfile spelling and the locking environment, so only the entry itself knows) and `version` dropped; `_meta` (the Pipfile content hash) and the Pipfile itself are never touched, so `pipenv install --deploy`/`sync`/`verify` keep passing. The reference KEY depends on the installing Pipenv: releases 7–11 only install `path` references, 2018 and later `file` ones (0–6 write pipfile-spec < 6 and are refused). The release is probed once per command with `pipenv --version`, resolved on ABSOLUTE `PATH` entries only (a relative entry would run a `pipenv` planted in the scanned repository; `.bat`/`.cmd` shims are found through `PATHEXT` on Windows), only when a pypi patch actually targets an entry of the lock, and `SOCKET_PIPENV_MAJOR=` pins the answer without spawning anything. An unknown installer selects `file` and warns `redirect_pipenv_installer_unknown` only when the lock was rewritten. **Refusal scope**: a pin/source CONFLICT (another version pinned, a foreign `file`/`path` source, a VCS/editable dependency) refuses the whole dependency atomically across categories as `redirect_pipenv_refused` AND vetoes the sibling Python rewriters (requirements.txt / uv.lock / pyproject) for that patch — the project's Pipenv install could not pick the patch up, so a half-redirected checkout is refused; anything else (no entry for the package, an old pipfile-spec, an unparseable lock, a digest-less patch) is `redirect_pipenv_skipped` and leaves the siblings alone (a stale Pipfile.lock in a uv/Poetry/requirements project must not block them). The veto applies to a LIVE lock only: a `Pipfile.lock` with no `Pipfile` beside it is abandoned, so its conflict refuses that file but never the siblings. Hash enforcement at install time is split by era — the `#sha256=` URL fragment is what Pipenv 2023+ verifies, the `hashes` list what 2018–2022 verify, Pipenv 11 either — so both are load-bearing. **Pipenv stale-install guard**: Pipenv never reinstalls a release that is already present (`pipenv install`, `install --deploy` and `sync` all exit 0 and keep the installed bytes — measured on 11.10.4, 2018.11.26 and 2026.8.0, hosted and vendored), so after the rewrite the run probes the Python crawler's site-packages (VIRTUAL_ENV, `./.venv`, `./venv`, Pipenv's out-of-tree `WORKON_HOME` venv; `--global`/`--global-prefix` honoured) for each confirmed Pipfile.lock redirect with the Python stale-install guard's evidence (the pinned patched wheel, else the patch record; STALE needs positive evidence, read-only, skipped on `--dry-run`, stale purls excluded from the same-run `--vex` `assume_applied` set) and the Python stale-install guard (`redirect_pypi_stale_install`, see above) names the site-packages dir and the Pipenv-specific verified remedy: `pipenv run pip uninstall -y && pipenv sync` (or `pipenv --rm && pipenv sync`), with the `sync` arguments following the lock, since plain `pipenv sync` installs only `default`: the targeted form re-syncs the categories that pin the package (`--dev` for `develop`, `--categories ""` for a named category) and the `--rm` form re-syncs every non-empty category — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away. The vendored backend emits the twin `pypi_pipenv_stale_install` (`skipped` warning event). **Rollback** (v5.0, upstream restore): each hosted entry gets its registry shape back — `"version": "=="`, the entry's own `index` carried back unchanged (refused unless it — and the Pipfile's explicit `index`, if any — names a PyPI source in `_meta.sources`), and every release file's sha256 from PyPI's JSON API (`SOCKET_PYPI_JSON_API`), sorted by filename as Pipenv records them; an entry that pins another version beside the hosted reference is refused with the `git checkout` remedy (see "Hosted unwind coverage"). A Pipfile names no project, so a same-run `--vex` on a Pipenv project needs `--vex-product` (or a git remote) to detect a product purl. **Discovery**: `Pipfile.lock` is part of the lockfile inventory (every category's `==` pins, with the lock's digest set as `Sha256AnyOf` integrity so a lock-only checkout can be vendored by fetching the pure wheel through PyPI's JSON API — only when `_meta.sources` name the public index; a private-index lock stays discovery-only and never reaches pypi.org), and Socket's own hosted / vendored references stay discoverable as the package they replace, so a re-scan of an already-redirected or already-vendored lock-only checkout re-confirms it (`--vex` attests, vendored reports `already_vendored`) instead of finding nothing. @@ -652,7 +652,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail instead names the gem-home copy and says that a Bundler run of the lock loads the vendored path and no `bundle install` replaces or removes that copy, then gives the one remedy for where it is: `bundle clean` for a copy inside the project, `gem uninstall -v [--platform

] -i ` for a gem home outside it (if no other project needs the copy), and none for a default gem, which ships with Ruby and which RubyGems can load before Bundler runs); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail instead names each gem-home copy and says that a Bundler run of the lock loads the vendored path and no `bundle install` replaces or removes what an earlier install left there, then gives the command that does, `gem uninstall -v [--platform

] -i ` per copy (unless another project uses that gem home; `bundle clean` also prunes the project's own bundle path), and for a default gem's copy says that it ships with Ruby, that RubyGems can load it before Bundler runs and that no command removes it); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) diff --git a/crates/socket-patch-cli/src/commands/vex.rs b/crates/socket-patch-cli/src/commands/vex.rs index 4f5dd13a1..b3c9244ed 100644 --- a/crates/socket-patch-cli/src/commands/vex.rs +++ b/crates/socket-patch-cli/src/commands/vex.rs @@ -1327,11 +1327,12 @@ async fn generate_vex( "vendored_tree_out_of_sync", vendored_out_of_sync_detail( purl, - outcome + &outcome .vendored_out_of_sync_copies - .get(purl) - .map(PathBuf::as_path), - &common.cwd, + .iter() + .filter(|(drifted, _)| drifted == purl) + .map(|(_, copy)| copy.as_path()) + .collect::>(), &format!("{hatch_note}{pdm_note}"), ), ); @@ -1811,29 +1812,25 @@ async fn withhold_stale_go_tools( })); } -/// The `vendored_tree_out_of_sync` detail for `purl`. `copy` is the -/// installed copy that drifted, `notes` the manager-specific remedies +/// The `vendored_tree_out_of_sync` detail for `purl`. `copies` are the +/// installed copies that drifted, `notes` the manager-specific remedies /// (Hatch, PDM) appended to the generic one. /// -/// A gem gets its own wording (#2076). Once `Gemfile.lock` takes the gem +/// A gem gets its own wording (#2076). Once Bundler's lock takes the gem /// from the vendored path, Bundler loads that directory and installs -/// nothing for the gem, so the registry copy an earlier install left in the +/// nothing for the gem, so the registry copy an earlier install left in a /// gem home stays as it is: "re-run your install" never clears the warning. /// The copy is still disclosed, because a load that bypasses Bundler (plain -/// `ruby`, a stale `--standalone` setup script) runs it. The remedy depends -/// on where the copy is: `bundle clean` only prunes a project's own bundle -/// path, a shared gem home needs `gem uninstall`, and nothing removes a +/// `ruby`, a stale `--standalone` setup script) runs it. `gem uninstall -i` +/// removes it from any gem home, a Bundler path included; nothing removes a /// default gem. -fn vendored_out_of_sync_detail( - purl: &str, - copy: Option<&Path>, - project_root: &Path, - notes: &str, -) -> String { +fn vendored_out_of_sync_detail(purl: &str, copies: &[&Path], notes: &str) -> String { use socket_patch_core::crawlers::ruby_crawler::is_default_gem_copy; - use socket_patch_core::utils::purl::{purl_parts, purl_qualifier}; - let gem = purl_parts(purl).filter(|(ty, _, _)| ty == "gem").zip(copy); - let Some(((_, name, version), copy)) = gem else { + use socket_patch_core::utils::purl::{ + percent_decode_purl_component, purl_parts, purl_qualifier, + }; + let gem = purl_parts(purl).filter(|(ty, _, _)| ty == "gem" && !copies.is_empty()); + let Some((_, name, version)) = gem else { return format!( "{purl}: the installed tree does not match its vendored artifact; the \ attestation is based on the committed .socket/vendor artifact (the project's \ @@ -1841,36 +1838,57 @@ fn vendored_out_of_sync_detail( different bytes — re-run your package manager's install to resync it.{notes}" ); }; - let remedy = if is_default_gem_copy(copy) { - "It is a default gem that ships with Ruby: RubyGems can load it before Bundler sets \ - the load path, and neither `bundle clean` nor `gem uninstall` removes it." - .to_string() - } else if copy.starts_with(project_root) { - "Only code that loads the gem without Bundler still runs it. Remove it with \ - `bundle clean`." - .to_string() - } else { - let platform = purl_qualifier(purl, "platform") - .map(|platform| format!(" --platform {platform}")) - .unwrap_or_default(); - let home = copy - .parent() - .and_then(Path::parent) - .map(|home| format!(" -i {}", home.display())) - .unwrap_or_default(); - format!( - "Only code that loads the gem without Bundler still runs it. Remove it with \ - `gem uninstall {name} -v {version}{platform}{home}` if no other project needs \ - it from that gem home." - ) + let platform = purl_qualifier(purl, "platform") + .map(|platform| format!(" --platform {}", percent_decode_purl_component(platform))) + .unwrap_or_default(); + let (default, removable): (Vec<&Path>, Vec<&Path>) = + copies.iter().partition(|copy| is_default_gem_copy(copy)); + let listed = |paths: &[&Path]| { + let paths: Vec = paths.iter().map(|p| p.display().to_string()).collect(); + paths.join(", ") }; - format!( - "{purl}: {} does not match the vendored artifact. Gemfile.lock takes the gem from \ - the committed .socket/vendor path, which the attestation is based on, so a Bundler \ - run of this lock loads the patched copy, and `bundle install` neither replaces nor \ - removes this one. {remedy}", - copy.display() - ) + let mut detail = format!( + "{purl}: an installed copy of this gem does not match the vendored artifact ({}). \ + Bundler's lock takes the gem from the committed .socket/vendor path, which the \ + attestation is based on, so a Bundler run of that lock loads the patched copy, and \ + `bundle install` neither replaces nor removes what an earlier install left in a \ + gem home.", + listed(copies) + ); + if !removable.is_empty() { + let commands: Vec = removable + .iter() + .filter_map(|copy| copy.parent().and_then(Path::parent)) + .map(|home| { + let home = shell_quoted(&home.display().to_string()); + format!("`gem uninstall {name} -v {version}{platform} -i {home}`") + }) + .collect(); + detail.push_str(&format!( + " Only code that loads the gem without Bundler still runs such a copy. Remove it \ + with {} unless another project uses that gem home; `bundle clean` also prunes it \ + from the project's own bundle path.", + commands.join(" and ") + )); + } + if !default.is_empty() { + detail.push_str(&format!( + " A default gem ships with Ruby ({}): RubyGems can load it before Bundler sets \ + the load path, and no command removes it.", + listed(&default) + )); + } + detail +} + +/// `arg` as one word of a POSIX shell command line a reader may paste. +fn shell_quoted(arg: &str) -> String { + let plain = |c: char| c.is_ascii_alphanumeric() || "/._-+:@%=,\\".contains(c); + if !arg.is_empty() && arg.chars().all(plain) { + arg.to_string() + } else { + format!("'{}'", arg.replace('\'', "'\\''")) + } } /// Record a run-level advisory the way `update`/`vendor` do: stderr @@ -2535,51 +2553,72 @@ mod tests { use clap::Parser; /// #2076: `bundle install` never replaces a vendored gem's gem-home - /// leftover, so the gem warning names the copy and the one command - /// that removes it from where it is. Other ecosystems keep the install + /// leftover, so the gem warning names each copy and the command that + /// removes it from its gem home. Other ecosystems keep the install /// remedy and their manager-specific notes; a gem never gets those. #[test] fn out_of_sync_detail_names_a_remedy_bundler_honors() { let tmp = tempfile::tempdir().unwrap(); - let project = tmp.path().join("app"); - let detail = |purl: &str, copy: &Path| { - vendored_out_of_sync_detail(purl, Some(copy), &project, " Hatch note.") + let detail = |purl: &str, copies: &[&Path]| { + vendored_out_of_sync_detail(purl, copies, " Hatch note.") }; - let local = project.join("vendor/bundle/ruby/3.3.0/gems/rack-2.2.8"); - let text = detail("pkg:gem/rack@2.2.8", &local); - let named = format!("pkg:gem/rack@2.2.8: {} does not match", local.display()); - assert!(text.starts_with(&named), "{text}"); - assert!(text.ends_with("Remove it with `bundle clean`."), "{text}"); - assert!(!text.contains("gem uninstall"), "{text}"); + let local = Path::new("vendor/bundle/ruby/3.3.0/gems/rack-2.2.8"); + let text = detail("pkg:gem/rack@2.2.8", &[local]); + assert!( + text.starts_with("pkg:gem/rack@2.2.8: an installed copy of this gem"), + "{text}" + ); + assert!(text.contains(&format!("({})", local.display())), "{text}"); + assert!( + text.contains("`gem uninstall rack -v 2.2.8 -i vendor/bundle/ruby/3.3.0`"), + "{text}" + ); + assert!(text.contains("`bundle clean`"), "{text}"); assert!(!text.contains("re-run your package manager"), "{text}"); + assert!(!text.contains("Hatch"), "{text}"); + assert!(!text.contains("default gem"), "{text}"); - let home = tmp.path().join("shared/ruby/3.3.0"); + // Every drifted copy gets its command; the platform is passed on + // decoded, and a gem home with a space is one shell word. + let home = tmp.path().join("Jane Doe/ruby/3.3.0"); let shared = home.join("gems/ffi-1.17.0-x86_64-linux"); - let text = detail("pkg:gem/ffi@1.17.0?platform=x86_64-linux", &shared); - let uninstall = format!( - "`gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i {}`", + let text = detail( + "pkg:gem/ffi@1.17.0?platform=x86_64%2Dlinux", + &[Path::new("/opt/gems/gems/ffi-1.17.0-x86_64-linux"), &shared], + ); + let quoted = format!( + "`gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i /opt/gems` and \ + `gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i '{}'`", home.display() ); - assert!(text.contains(&uninstall), "{text}"); - assert!(!text.contains("`bundle clean`"), "{text}"); - assert!(!text.contains("Hatch"), "{text}"); + assert!(text.contains("ed), "{text}"); // A default gem ships with Ruby: no command removes it, and // RubyGems can load it ahead of Bundler. let specs = home.join("specifications/default"); std::fs::create_dir_all(&specs).unwrap(); std::fs::write(specs.join("json-2.7.1.gemspec"), "").unwrap(); - let text = detail("pkg:gem/json@2.7.1", &home.join("gems/json-2.7.1")); - assert!(text.contains("default gem"), "{text}"); - assert!(!text.contains("Remove it with"), "{text}"); + let default = home.join("gems/json-2.7.1"); + let text = detail("pkg:gem/json@2.7.1", &[&default]); + assert!(text.contains("no command removes it"), "{text}"); + assert!(!text.contains("gem uninstall"), "{text}"); assert!(!text.contains("Only code that loads"), "{text}"); + let both = detail( + "pkg:gem/json@2.7.1", + &[&default, Path::new("/opt/gems/gems/json-2.7.1")], + ); + assert!(both.contains("no command removes it"), "{both}"); + assert!( + both.contains("`gem uninstall json -v 2.7.1 -i /opt/gems`"), + "{both}" + ); - for (purl, copy) in [ - ("pkg:pypi/idna@3.6", Some(local.as_path())), - ("pkg:gem/rack@2.2.8", None), + for (purl, copies) in [ + ("pkg:pypi/idna@3.6", vec![local]), + ("pkg:gem/rack@2.2.8", vec![]), ] { - let text = vendored_out_of_sync_detail(purl, copy, &project, " Hatch note."); + let text = detail(purl, &copies); assert!( text.ends_with("re-run your package manager's install to resync it. Hatch note."), "{text}" @@ -2587,6 +2626,14 @@ mod tests { } } + #[test] + fn shell_quoted_keeps_plain_words_and_quotes_the_rest() { + assert_eq!(shell_quoted("/opt/gems-3.3.0/a_b"), "/opt/gems-3.3.0/a_b"); + assert_eq!(shell_quoted("/a b"), "'/a b'"); + assert_eq!(shell_quoted("it's"), "'it'\\''s'"); + assert_eq!(shell_quoted(""), "''"); + } + /// The go-patches synthesis guards its copy-dir keys with core's /// `are_safe_redirect_coords`; pin the accept/reject set from the CLI /// side — a regression here would let a tampered go.mod `replace` key diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index 35c5293cf..5925461cd 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -657,12 +657,14 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { // REGRESSION (#2076): the registry copy the first `bundle install` put // in the gem home is a leftover now. Bundler loads the vendored path // and installs nothing for rack, so no install replaces that copy. The - // warning about it must name a remedy that removes it (`bundle clean`, - // run below), not a reinstall. - let leftover = format!("Warning: {purl}: {} does not match", gem_dir.display()); - assert!( - stderr.contains(&leftover) && stderr.contains("Remove it with `bundle clean`."), - "the leftover gem-home copy is named, with the remedy that clears it:\n{stderr}" + // warning about it must name the copy and a remedy that removes it + // (`bundle clean`, run below), not a reinstall. + assert!( + stderr.contains(&format!("Warning: {purl}: ")) + && stderr.contains(&format!("({})", gem_dir.display())) + && stderr.contains(&format!("`gem uninstall {DEP} -v {version} -i ")) + && stderr.contains("`bundle clean`"), + "the leftover gem-home copy is named, with remedies that clear it:\n{stderr}" ); assert!( !stderr.contains("re-run your package manager"), diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 0c89298f7..27ed282b5 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -53,9 +53,10 @@ pub struct VerifyOutcome { /// manager re-installs. An ABSENT installed tree is the expected /// post-vendor state and is never flagged. pub vendored_out_of_sync: Vec, - /// The first installed copy that put each purl in - /// `vendored_out_of_sync`, so a caller can name the directory. - pub vendored_out_of_sync_copies: HashMap, + /// `(purl, copy)` for every installed copy that put a purl in + /// `vendored_out_of_sync`, in crawl order, so a caller can name the + /// directories. + pub vendored_out_of_sync_copies: Vec<(String, PathBuf)>, /// Maven / Gradle: `(purl, copy)` for every installed copy of a failed /// purl that does not verify — the copy a build may still load /// unpatched. Callers name them (`vex_gradle_unpatched_copy`). @@ -259,12 +260,13 @@ pub async fn applied_patches_with_copies( .is_ok() }; if !in_sync { - out.vendored_out_of_sync.push(purl.clone()); out.vendored_out_of_sync_copies - .insert(purl.clone(), pkg_path.clone()); - break; + .push((purl.clone(), pkg_path.clone())); } } + if matches!(out.vendored_out_of_sync_copies.last(), Some((p, _)) if p == purl) { + out.vendored_out_of_sync.push(purl.clone()); + } } } Err(reason) => out.failed.push(FailedPatch { @@ -1974,7 +1976,7 @@ mod tests { // The copy that drifted is the one named, not the first crawled. assert_eq!( out.vendored_out_of_sync_copies, - HashMap::from([(purl.to_string(), root.path().join("drifted"))]) + vec![(purl.to_string(), root.path().join("drifted"))] ); } From a27356df605002a99e03a8e7643ed2bf8d3b5a7b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 10:47:56 +0000 Subject: [PATCH 6/6] Describe the gem remedies instead of a command Review of the previous commit: the printed gem uninstall command stopped at the dependents prompt, failed through a symlinked gem home, and carried purl text from committed files into a line meant to be pasted into a shell. The gem warning now lists the drifted copies and says in words what removes one: bundle clean for the project's own bundle path, gem uninstall in any other gem home no other project uses, and nothing for a default gem. No part of the purl reaches the remedy. Only a gem's probe collects every drifted copy; the others stop at the first, as before. The e2e test runs bundle clean on a copy of the project, so the revert leg keeps its starting tree. Refs #2076 Assisted-by: Claude Code:claude-fable-5-2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 4 +- crates/socket-patch-cli/src/commands/vex.rs | 168 +++++------------- .../tests/e2e_vendor_gem_build.rs | 30 ++-- crates/socket-patch-core/src/vex/verify.rs | 63 ++++++- 4 files changed, 127 insertions(+), 138 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index e05dbfa2e..241bc3172 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -460,7 +460,7 @@ The rewriter reads a fixed set of candidate files from the project root: the npm *The unwind direction.* After a hosted unwind — `rollback` / `remove` of a hosted pin, scan's withdrawn-patch restore, `vendor --revert` of a package vendored over hosted wiring before v5 — put a gem back on its upstream registry entry, its installed copies and archives hold the PATCHED gem, with the same consequences in reverse: the patched code keeps running, and on bundler ≥ 2.6 `bundle pristine` fails on the restored upstream checksum. The unwind applies the same refresh with the upstream gem: the sha256 is the restored lock's `CHECKSUMS` entry, else the rubygems.org compact index; the archive is taken from disk, else downloaded from rubygems.org when the lock resolves the gem from it (a gem from another remote is refreshed only from an archive already on disk). An installed copy is stale when its `/cache/.gem` is readable and is not the upstream archive (ruby-platform gems only: the lock's checksum names that archive). Notes are `upstream_gem_install_refreshed` / `upstream_gem_install_would_refresh`; an installed copy left on the patched build is `upstream_gem_stale_install` (one consolidated warning in a human run), and an archive left in bundler's cache dir keeps its `upstream_gem_stale_cache` warning. A gem whose installed copies were refreshed is not counted in rollback's "keep their patched bytes" note. -*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact (`vendored_tree_out_of_sync`, whose gem wording names the copy and the command that removes it). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. +*What never refreshes.* A hosted → vendored takeover (`vendor`, `scan` / `get --mode vendored` over a hosted pin) puts the gem on its upstream entry only on its way to the vendored `path:` source, which is what Bundler loads from then on. The copies Bundler already holds are unused leftovers there, so the takeover changes none of them and downloads no gem for them: resetting the installed copy to the upstream build would cost a registry download in the mode that needs none, and would turn a patched leftover into an unpatched one that `vex` reports against the vendored artifact (`vendored_tree_out_of_sync`, whose gem wording names the copy and how to remove it). Its `upstream_gem_stale_cache` advisory about a patched cache-dir archive is as before. A vendored unwind (`remove`, `rollback`, `vendor --revert`, `scan --prune`, `vendor`'s reconcile of a vendored gem) likewise only names that archive. **Pipenv hosted redirect (`Pipfile.lock`, pipfile-spec 6)**: every category other than `_meta` (`default`, `develop`, and Pipenv 2022+ named categories) that pins the package at the patched version is rewritten to the hosted reference — `{"file" | "path": "#sha256=", "hashes": ["sha256:"]}` with `markers`/`extras`/`index` kept exactly as Pipenv wrote them (present or absent: whether Pipenv records `index` depends on its release, the Pipfile spelling and the locking environment, so only the entry itself knows) and `version` dropped; `_meta` (the Pipfile content hash) and the Pipfile itself are never touched, so `pipenv install --deploy`/`sync`/`verify` keep passing. The reference KEY depends on the installing Pipenv: releases 7–11 only install `path` references, 2018 and later `file` ones (0–6 write pipfile-spec < 6 and are refused). The release is probed once per command with `pipenv --version`, resolved on ABSOLUTE `PATH` entries only (a relative entry would run a `pipenv` planted in the scanned repository; `.bat`/`.cmd` shims are found through `PATHEXT` on Windows), only when a pypi patch actually targets an entry of the lock, and `SOCKET_PIPENV_MAJOR=` pins the answer without spawning anything. An unknown installer selects `file` and warns `redirect_pipenv_installer_unknown` only when the lock was rewritten. **Refusal scope**: a pin/source CONFLICT (another version pinned, a foreign `file`/`path` source, a VCS/editable dependency) refuses the whole dependency atomically across categories as `redirect_pipenv_refused` AND vetoes the sibling Python rewriters (requirements.txt / uv.lock / pyproject) for that patch — the project's Pipenv install could not pick the patch up, so a half-redirected checkout is refused; anything else (no entry for the package, an old pipfile-spec, an unparseable lock, a digest-less patch) is `redirect_pipenv_skipped` and leaves the siblings alone (a stale Pipfile.lock in a uv/Poetry/requirements project must not block them). The veto applies to a LIVE lock only: a `Pipfile.lock` with no `Pipfile` beside it is abandoned, so its conflict refuses that file but never the siblings. Hash enforcement at install time is split by era — the `#sha256=` URL fragment is what Pipenv 2023+ verifies, the `hashes` list what 2018–2022 verify, Pipenv 11 either — so both are load-bearing. **Pipenv stale-install guard**: Pipenv never reinstalls a release that is already present (`pipenv install`, `install --deploy` and `sync` all exit 0 and keep the installed bytes — measured on 11.10.4, 2018.11.26 and 2026.8.0, hosted and vendored), so after the rewrite the run probes the Python crawler's site-packages (VIRTUAL_ENV, `./.venv`, `./venv`, Pipenv's out-of-tree `WORKON_HOME` venv; `--global`/`--global-prefix` honoured) for each confirmed Pipfile.lock redirect with the Python stale-install guard's evidence (the pinned patched wheel, else the patch record; STALE needs positive evidence, read-only, skipped on `--dry-run`, stale purls excluded from the same-run `--vex` `assume_applied` set) and the Python stale-install guard (`redirect_pypi_stale_install`, see above) names the site-packages dir and the Pipenv-specific verified remedy: `pipenv run pip uninstall -y && pipenv sync` (or `pipenv --rm && pipenv sync`), with the `sync` arguments following the lock, since plain `pipenv sync` installs only `default`: the targeted form re-syncs the categories that pin the package (`--dev` for `develop`, `--categories ""` for a named category) and the `--rm` form re-syncs every non-empty category — NOT `pipenv uninstall`, which rewrites the Pipfile and re-locks the patch away. The vendored backend emits the twin `pypi_pipenv_stale_install` (`skipped` warning event). **Rollback** (v5.0, upstream restore): each hosted entry gets its registry shape back — `"version": "=="`, the entry's own `index` carried back unchanged (refused unless it — and the Pipfile's explicit `index`, if any — names a PyPI source in `_meta.sources`), and every release file's sha256 from PyPI's JSON API (`SOCKET_PYPI_JSON_API`), sorted by filename as Pipenv records them; an entry that pins another version beside the hosted reference is refused with the `git checkout` remedy (see "Hosted unwind coverage"). A Pipfile names no project, so a same-run `--vex` on a Pipenv project needs `--vex-product` (or a git remote) to detect a product purl. **Discovery**: `Pipfile.lock` is part of the lockfile inventory (every category's `==` pins, with the lock's digest set as `Sha256AnyOf` integrity so a lock-only checkout can be vendored by fetching the pure wheel through PyPI's JSON API — only when `_meta.sources` name the public index; a private-index lock stays discovery-only and never reaches pypi.org), and Socket's own hosted / vendored references stay discoverable as the package they replace, so a re-scan of an already-redirected or already-vendored lock-only checkout re-confirms it (`--vex` attests, vendored reports `already_vendored`) instead of finding nothing. @@ -652,7 +652,7 @@ Contract details: * **JSON success surface**: `apply` adds a top-level `vex` object to its envelope; `scan` adds a top-level `vex` key to its result. Both carry `{ path, statements, format: "openvex-0.2.0" }`. * `apply`'s no-manifest early exit (the `noManifest` success no-op; v5.0: its human line is `No patch manifest found; nothing to apply.` — it names the missing `.socket/manifest.json`, not the folder, since `.socket/` may legitimately hold vendored state) and `vendor`'s (`No manifest found, nothing to vendor.` — a project with hosted pins ejects instead, v5.0) still generate the document from the lockfiles and the vendor ledger (manifest-less VEX: hosted / vendored checkouts carry no manifest). Nothing referenced anywhere keeps the calm exit 0 (a stale document at the path is removed; `--json` carries any discovery diagnostics in `warnings[]`); any other VEX failure fails the command with exit 1 — including a run whose only candidates are omitted `record_unavailable` (an `--offline` run over a lockfile-wired checkout with no local records), so an ambient `SOCKET_VEX` there fails the install. `--dry-run` skips generation on both, and so does `apply --check` — it stays read-only and offline-safe, leaving the output path untouched. `scan` has no such early exit: with no manifest and nothing wired anywhere its `--vex` fails with `manifest_not_found`. * **Stale-doc removal (v3.5)**: a run that ends in a VEX error removes a recognizably-OpenVEX file (JSON whose `@context` names openvex.dev) already sitting at the output path — a pipeline reusing one path can never ship yesterday's attestation for a now-unpatched tree. Unrelated files at the path are never touched; a mid-write partial that no longer parses as JSON is left for downstream parsers to reject loudly. -* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail instead names each gem-home copy and says that a Bundler run of the lock loads the vendored path and no `bundle install` replaces or removes what an earlier install left there, then gives the command that does, `gem uninstall -v [--platform

] -i ` per copy (unless another project uses that gem home; `bundle clean` also prunes the project's own bundle path), and for a default gem's copy says that it ships with Ruby, that RubyGems can load it before Bundler runs and that no command removes it); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). +* **Additive warnings (v3.5)**: `product_not_iri` (the `--product`/`--vex-product` override is neither a `pkg:` purl nor an absolute IRI; honored verbatim, warned), its document-id twin `doc_id_not_iri` (the `--doc-id`/`--vex-doc-id` override has no IRI scheme; honored, warned), `vex_not_verified` (the run used `--no-verify` / `--vex-no-verify` / `SOCKET_VEX_NO_VERIFY`: the detail starts ` of VEX statements were emitted without on-disk verification`, see the provenance markers below) and `vendored_tree_out_of_sync` (a healthy vendored attestation stands on the committed artifact + lock wiring while the PRESENT installed tree hash-mismatches the patched bytes — run the package manager's install (for a project with Hatch environments the detail also names `hatch env remove `, since Hatch keeps an installed release; for a pypi package of a project whose governing pdm.lock was written by PDM < 2.11 it names `pdm sync --reinstall`, or removing the installed copy first on PDM 1.x; for a gem the detail instead names each gem-home copy and says that a Bundler run of the lock loads the vendored path, that no `bundle install` replaces or removes what an earlier install left there, and that only a load outside that Bundler setup still runs it; the remedy is `bundle clean` for the project's own bundle path, or `gem uninstall` in any other gem home no other project uses); the attestation itself is unchanged). Both ride stderr in human mode and `warnings[]` in the standalone `vex --json` envelope. Same channel for `product_multiple_manifests` (auto-detect found several project manifests and names the one it used), `vex_stale_doc_removed` (the stale-doc removal above happened), the manifest-less plan's advisories — `vex_wiring_conflict` (the lockfiles wire a package to different patches: which files, which uuids, how to fix it), `vex_record_superseded` (a recorded patch replaced by the lockfile-wired one), `vex_claim_unwired` (a ledger claim whose patch the lockfiles still mention, but not as wiring), `vex_record_offline` / `vex_record_not_found` / `vex_record_fetch_failed` (why a lockfile-wired patch has no record — the detail behind a `record_unavailable` skip) and `api_auth_fallback` (the public proxy served free patches only: the authenticated API refused the credentials, `get` / `scan`'s warning text; or, v5.0, a token was set with no `--org` / `SOCKET_ORG_SLUG` / socket-cli `defaultOrg` and the org could not be resolved, so the whole run used the proxy — standalone `vex --json` carries it when it fetched records, and `scan --json` / `get --json` report this case in their top-level `warnings[]`) — and, standalone only, `org_looks_like_path` (`-o`/`--org` given a file-shaped value — `-O` is `--output`). The standalone error envelope carries `warnings[]` too. An embedded `--vex` that fails also folds each omitted patch into the host command's `warnings[]` as `vex_omitted` (`: ()` — standalone `vex` lists them as `skipped` events), and `--silent` lists them as `omitted: ()` lines under the error. A corrupt `.socket/vendor/state.json` is no longer degraded with a warning: every form of vex fails with `vendor_ledger_corrupt` (see the error-code table). A malformed pre-v5 `redirect-state.json` is, as of v5.0, only the `redirect_ledger_corrupt` warning (hosted mode keeps no ledger; the file is an optional migration record source). ### VEX provenance markers (contract) diff --git a/crates/socket-patch-cli/src/commands/vex.rs b/crates/socket-patch-cli/src/commands/vex.rs index b3c9244ed..3bc2fa301 100644 --- a/crates/socket-patch-cli/src/commands/vex.rs +++ b/crates/socket-patch-cli/src/commands/vex.rs @@ -1820,75 +1820,33 @@ async fn withhold_stale_go_tools( /// from the vendored path, Bundler loads that directory and installs /// nothing for the gem, so the registry copy an earlier install left in a /// gem home stays as it is: "re-run your install" never clears the warning. -/// The copy is still disclosed, because a load that bypasses Bundler (plain -/// `ruby`, a stale `--standalone` setup script) runs it. `gem uninstall -i` -/// removes it from any gem home, a Bundler path included; nothing removes a -/// default gem. +/// The copy is still disclosed, because a load outside that Bundler setup +/// runs it. The remedies are described, not spelled as a command: the purl +/// comes from committed files, and which `gem uninstall` flags work depends +/// on the gem home. fn vendored_out_of_sync_detail(purl: &str, copies: &[&Path], notes: &str) -> String { - use socket_patch_core::crawlers::ruby_crawler::is_default_gem_copy; - use socket_patch_core::utils::purl::{ - percent_decode_purl_component, purl_parts, purl_qualifier, - }; - let gem = purl_parts(purl).filter(|(ty, _, _)| ty == "gem" && !copies.is_empty()); - let Some((_, name, version)) = gem else { + use socket_patch_core::utils::purl::strip_purl_qualifiers; + if !strip_purl_qualifiers(purl).starts_with("pkg:gem/") || copies.is_empty() { return format!( "{purl}: the installed tree does not match its vendored artifact; the \ attestation is based on the committed .socket/vendor artifact (the project's \ lockfile or package-manager config wires it in), but the live tree carries \ different bytes — re-run your package manager's install to resync it.{notes}" ); - }; - let platform = purl_qualifier(purl, "platform") - .map(|platform| format!(" --platform {}", percent_decode_purl_component(platform))) - .unwrap_or_default(); - let (default, removable): (Vec<&Path>, Vec<&Path>) = - copies.iter().partition(|copy| is_default_gem_copy(copy)); - let listed = |paths: &[&Path]| { - let paths: Vec = paths.iter().map(|p| p.display().to_string()).collect(); - paths.join(", ") - }; - let mut detail = format!( + } + let copies: Vec = copies.iter().map(|p| p.display().to_string()).collect(); + format!( "{purl}: an installed copy of this gem does not match the vendored artifact ({}). \ Bundler's lock takes the gem from the committed .socket/vendor path, which the \ attestation is based on, so a Bundler run of that lock loads the patched copy, and \ `bundle install` neither replaces nor removes what an earlier install left in a \ - gem home.", - listed(copies) - ); - if !removable.is_empty() { - let commands: Vec = removable - .iter() - .filter_map(|copy| copy.parent().and_then(Path::parent)) - .map(|home| { - let home = shell_quoted(&home.display().to_string()); - format!("`gem uninstall {name} -v {version}{platform} -i {home}`") - }) - .collect(); - detail.push_str(&format!( - " Only code that loads the gem without Bundler still runs such a copy. Remove it \ - with {} unless another project uses that gem home; `bundle clean` also prunes it \ - from the project's own bundle path.", - commands.join(" and ") - )); - } - if !default.is_empty() { - detail.push_str(&format!( - " A default gem ships with Ruby ({}): RubyGems can load it before Bundler sets \ - the load path, and no command removes it.", - listed(&default) - )); - } - detail -} - -/// `arg` as one word of a POSIX shell command line a reader may paste. -fn shell_quoted(arg: &str) -> String { - let plain = |c: char| c.is_ascii_alphanumeric() || "/._-+:@%=,\\".contains(c); - if !arg.is_empty() && arg.chars().all(plain) { - arg.to_string() - } else { - format!("'{}'", arg.replace('\'', "'\\''")) - } + gem home. Code that loads the gem outside that Bundler setup still runs such a \ + copy (plain `ruby`, or RubyGems activating a default gem before Bundler runs). \ + `bundle clean` prunes it from the project's own bundle path; in any other gem \ + home, `gem uninstall` it there if no other project uses it (a default gem that \ + ships with Ruby cannot be uninstalled).", + copies.join(", ") + ) } /// Record a run-level advisory the way `update`/`vendor` do: stderr @@ -2553,66 +2511,42 @@ mod tests { use clap::Parser; /// #2076: `bundle install` never replaces a vendored gem's gem-home - /// leftover, so the gem warning names each copy and the command that - /// removes it from its gem home. Other ecosystems keep the install - /// remedy and their manager-specific notes; a gem never gets those. + /// leftover, so the gem warning names each copy and the commands that + /// remove one, and never a reinstall. Nothing from the purl is spelled + /// into a command. Other ecosystems keep the install remedy and their + /// manager-specific notes; a gem never gets those. #[test] fn out_of_sync_detail_names_a_remedy_bundler_honors() { - let tmp = tempfile::tempdir().unwrap(); let detail = |purl: &str, copies: &[&Path]| { vendored_out_of_sync_detail(purl, copies, " Hatch note.") }; - - let local = Path::new("vendor/bundle/ruby/3.3.0/gems/rack-2.2.8"); - let text = detail("pkg:gem/rack@2.2.8", &[local]); - assert!( - text.starts_with("pkg:gem/rack@2.2.8: an installed copy of this gem"), - "{text}" - ); - assert!(text.contains(&format!("({})", local.display())), "{text}"); - assert!( - text.contains("`gem uninstall rack -v 2.2.8 -i vendor/bundle/ruby/3.3.0`"), - "{text}" - ); - assert!(text.contains("`bundle clean`"), "{text}"); - assert!(!text.contains("re-run your package manager"), "{text}"); - assert!(!text.contains("Hatch"), "{text}"); - assert!(!text.contains("default gem"), "{text}"); - - // Every drifted copy gets its command; the platform is passed on - // decoded, and a gem home with a space is one shell word. - let home = tmp.path().join("Jane Doe/ruby/3.3.0"); - let shared = home.join("gems/ffi-1.17.0-x86_64-linux"); - let text = detail( - "pkg:gem/ffi@1.17.0?platform=x86_64%2Dlinux", - &[Path::new("/opt/gems/gems/ffi-1.17.0-x86_64-linux"), &shared], - ); - let quoted = format!( - "`gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i /opt/gems` and \ - `gem uninstall ffi -v 1.17.0 --platform x86_64-linux -i '{}'`", - home.display() - ); - assert!(text.contains("ed), "{text}"); - - // A default gem ships with Ruby: no command removes it, and - // RubyGems can load it ahead of Bundler. - let specs = home.join("specifications/default"); - std::fs::create_dir_all(&specs).unwrap(); - std::fs::write(specs.join("json-2.7.1.gemspec"), "").unwrap(); - let default = home.join("gems/json-2.7.1"); - let text = detail("pkg:gem/json@2.7.1", &[&default]); - assert!(text.contains("no command removes it"), "{text}"); - assert!(!text.contains("gem uninstall"), "{text}"); - assert!(!text.contains("Only code that loads"), "{text}"); - let both = detail( - "pkg:gem/json@2.7.1", - &[&default, Path::new("/opt/gems/gems/json-2.7.1")], - ); - assert!(both.contains("no command removes it"), "{both}"); - assert!( - both.contains("`gem uninstall json -v 2.7.1 -i /opt/gems`"), - "{both}" - ); + let local = Path::new("vendor/bundle/ruby/3.3.0/gems/ffi-1.17.0"); + let shared = Path::new("/opt/gems/gems/ffi-1.17.0-x86_64-linux"); + let remedy = |text: &str| text.split_once("). ").map(|(_, rest)| rest.to_string()); + + for purl in [ + "pkg:gem/ffi@1.17.0", + "pkg:gem/ffi@1.17.0?platform=x86_64-linux", + "pkg:gem/ffi@1.17.0?platform=%24%28id%29", + ] { + let text = detail(purl, &[local, shared]); + let head = format!( + "{purl}: an installed copy of this gem does not match the vendored artifact \ + ({}, {}). ", + local.display(), + shared.display() + ); + let rest = text.strip_prefix(&head).unwrap_or_else(|| panic!("{text}")); + assert!(rest.contains("`bundle clean` prunes it"), "{text}"); + assert!(rest.contains("`gem uninstall` it there"), "{text}"); + assert!(!rest.contains("re-run your package manager"), "{text}"); + assert!(!rest.contains("Hatch"), "{text}"); + // The remedy text is fixed: no purl part reaches it. + assert_eq!( + Some(rest.to_string()), + remedy(&detail("pkg:gem/a@1", &[local, shared])) + ); + } for (purl, copies) in [ ("pkg:pypi/idna@3.6", vec![local]), @@ -2626,14 +2560,6 @@ mod tests { } } - #[test] - fn shell_quoted_keeps_plain_words_and_quotes_the_rest() { - assert_eq!(shell_quoted("/opt/gems-3.3.0/a_b"), "/opt/gems-3.3.0/a_b"); - assert_eq!(shell_quoted("/a b"), "'/a b'"); - assert_eq!(shell_quoted("it's"), "'it'\\''s'"); - assert_eq!(shell_quoted(""), "''"); - } - /// The go-patches synthesis guards its copy-dir keys with core's /// `are_safe_redirect_coords`; pin the accept/reject set from the CLI /// side — a regression here would let a tampered go.mod `replace` key diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index 5925461cd..c17fc7d09 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -662,8 +662,7 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { assert!( stderr.contains(&format!("Warning: {purl}: ")) && stderr.contains(&format!("({})", gem_dir.display())) - && stderr.contains(&format!("`gem uninstall {DEP} -v {version} -i ")) - && stderr.contains("`bundle clean`"), + && stderr.contains("`bundle clean` prunes it"), "the leftover gem-home copy is named, with remedies that clear it:\n{stderr}" ); assert!( @@ -772,33 +771,38 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { ); // #2076, the remedy: `bundle clean` removes the leftover, touches - // neither file of the pair, and `vex` then has nothing to warn about. - let clean = bundle(&proj, &["clean"], false); + // neither file of the pair, and `vex` then has nothing to report. Run + // on a copy, so the revert below still starts from the tree a user + // reverts from. + let cleaned = tmp.path().join("cleaned"); + common::fs_tree::copy_tree(&proj, &cleaned); + let clean = bundle(&cleaned, &["clean"], false); assert!( clean.status.success(), "`bundle clean` failed.\nstdout:\n{}\nstderr:\n{}", String::from_utf8_lossy(&clean.stdout), String::from_utf8_lossy(&clean.stderr), ); + let cleaned_gem_dir = cleaned.join(gem_dir.strip_prefix(&proj).unwrap()); assert!( - !gem_dir.exists(), + !cleaned_gem_dir.exists(), "`bundle clean` must remove the leftover {}", - gem_dir.display() + cleaned_gem_dir.display() ); assert_eq!( - std::fs::read(&lock_path).unwrap(), + std::fs::read(cleaned.join("Gemfile.lock")).unwrap(), lock_wired, "`bundle clean` must leave Gemfile.lock byte-identical" ); let (code, stdout, stderr) = run_socket( - &proj, + &cleaned, &[ "vex", "--json", "--cwd", - proj.to_str().unwrap(), + cleaned.to_str().unwrap(), "--output", - vex_path.to_str().unwrap(), + cleaned.join("out.vex.json").to_str().unwrap(), "--product", "pkg:gem/app@1.0.0", ], @@ -807,8 +811,10 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { let env = parse_json_envelope(&stdout); assert_eq!(env["status"], "success", "envelope: {env}"); assert!( - env.get("warnings").is_none(), - "nothing is left to warn about after `bundle clean`: {env}" + env["warnings"] + .as_array() + .is_none_or(|w| w.iter().all(|w| w["code"] != "vendored_tree_out_of_sync")), + "nothing is out of sync after `bundle clean`: {env}" ); // 7. REVERT PROOF: both halves of the pair edit byte-restored, artifacts diff --git a/crates/socket-patch-core/src/vex/verify.rs b/crates/socket-patch-core/src/vex/verify.rs index 27ed282b5..08b1c6da7 100644 --- a/crates/socket-patch-core/src/vex/verify.rs +++ b/crates/socket-patch-core/src/vex/verify.rs @@ -53,9 +53,9 @@ pub struct VerifyOutcome { /// manager re-installs. An ABSENT installed tree is the expected /// post-vendor state and is never flagged. pub vendored_out_of_sync: Vec, - /// `(purl, copy)` for every installed copy that put a purl in + /// `(purl, copy)` for the installed copies that put a purl in /// `vendored_out_of_sync`, in crawl order, so a caller can name the - /// directories. + /// directories: every drifted copy of a gem, the first one otherwise. pub vendored_out_of_sync_copies: Vec<(String, PathBuf)>, /// Maven / Gradle: `(purl, copy)` for every installed copy of a failed /// purl that does not verify — the copy a build may still load @@ -250,6 +250,8 @@ pub async fn applied_patches_with_copies( .unwrap_or_default() .iter() .filter(|p| !crate::crawlers::gradle_cache::is_gradle_version_dir(p)); + let every_copy = Ecosystem::from_purl(purl) == Some(Ecosystem::Gem); + let mut drifted = false; for pkg_path in installed { let in_sync = if is_vlt_dir_entry(entry) { vlt_installed_copy_matches(&ctx.project_root, pkg_path, entry, record) @@ -262,9 +264,15 @@ pub async fn applied_patches_with_copies( if !in_sync { out.vendored_out_of_sync_copies .push((purl.clone(), pkg_path.clone())); + drifted = true; + // Only a gem's warning lists each copy (its + // remedy is per gem home). + if !every_copy { + break; + } } } - if matches!(out.vendored_out_of_sync_copies.last(), Some((p, _)) if p == purl) { + if drifted { out.vendored_out_of_sync.push(purl.clone()); } } @@ -1980,6 +1988,55 @@ mod tests { ); } + /// #2076: a gem's warning names every gem-home copy that drifted (each + /// gem home needs its own cleanup), where other ecosystems stop at the + /// first. The purl is still flagged once. + #[tokio::test] + async fn vendored_drift_probe_lists_every_drifted_gem_copy() { + for (eco, listed) in [("gem", 2), ("cargo", 1)] { + let purl = &format!("pkg:{eco}/rack@2.2.8"); + let root = tempfile::tempdir().unwrap(); + let rel = format!(".socket/vendor/{eco}/{VUUID}/rack-2.2.8"); + let patched = b"patched-content"; + let vdir = root.path().join(&rel); + tokio::fs::create_dir_all(&vdir).await.unwrap(); + tokio::fs::write(vdir.join("index.js"), patched) + .await + .unwrap(); + let mut installed = Vec::new(); + for home in ["home-a", "home-b"] { + let dir = root.path().join(home).join("gems/rack-2.2.8"); + tokio::fs::create_dir_all(&dir).await.unwrap(); + tokio::fs::write(dir.join("index.js"), b"pristine") + .await + .unwrap(); + installed.push(dir); + } + + let mut rec = record_with_one_file(&compute_git_sha256_from_bytes(patched)); + rec.uuid = VUUID.to_string(); + let mut manifest = PatchManifest::new(); + manifest.patches.insert(purl.to_string(), rec); + let ctx = VendorContext { + project_root: root.path().to_path_buf(), + entries: HashMap::from([(purl.to_string(), vendor_entry(purl, &rel))]), + go_patches: HashMap::new(), + hosted: HashMap::new(), + }; + let copies = HashMap::from([(purl.to_string(), installed.clone())]); + + let out = applied_patches_with_copies(&manifest, &copies, Some(&ctx)).await; + assert_eq!(out.vendored, vec![purl.to_string()]); + assert_eq!(out.vendored_out_of_sync, vec![purl.to_string()]); + let expected: Vec<(String, PathBuf)> = installed + .into_iter() + .take(listed) + .map(|copy| (purl.to_string(), copy)) + .collect(); + assert_eq!(out.vendored_out_of_sync_copies, expected, "{purl}"); + } + } + /// Go vendored: the crawler's module-cache `M@v` is pristine by /// construction (immutable, go.sum-verified; the directory `replace` /// builds the committed copy), so it is never reported out of sync —