Repository navigation
Add custom-server support for variant analysis through companion extension support - #4636
nicolaswill wants to merge 18 commits into
Conversation
| ? ` Re-authenticate using the token provider '${server.tokenProviderExtensionId}' and retry.` | ||
| : ""; | ||
| throw new RequestError( | ||
| `Variant analysis server request failed: ${method} ${path}: ${response.status} ${response.statusText}${describeErrorBody(data)}${authenticationMessage}`, |
| Authorization: `Bearer ${token}`, | ||
| ...(body === undefined ? {} : { "Content-Type": "application/json" }), | ||
| }, | ||
| body: body === undefined ? undefined : JSON.stringify(body), |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical issues can misroute query-pack uploads, allow writes or deletion outside analysis storage, and mix or expose results through cross-server ID collisions.
8 open issues
Reject non-string server URL settings instead of falling back to GitHub · New Validate response IDs and repository paths before storage access · New Namespace analysis identity and storage by server URL and ID · New Allow custom-server mode through the repositories tree gate · New Reject URLs with empty query or fragment delimiters · New Avoid blocking history deletion on an unattended notification · New Treat invalid server config as unavailable in rendering paths · New Enable stopping custom-server runs without a workflow ID · New
What changed in this PR
Adds internal custom-server integration for multi-repository variant analysis, aimed at GitHub Enterprise Server deployments while reusing CodeQL’s existing history and results UI.
Changes:
- Routes analysis operations to GitHub or a custom HTTPS endpoint using companion-provided authentication.
- Persists server origins and repository URLs, and adapts logs, cancellation, source links, and exports for custom runs.
- Documents the internal contracts and adds configuration, lifecycle, persistence, and rendering coverage.
| File | Description |
|---|---|
extensions/ql-vscode/test/vscode-tests/test-config.ts |
Supports configuration inspection in tests. |
extensions/ql-vscode/test/vscode-tests/no-workspace/variant-analysis/variant-analysis-api.test.ts |
Covers routing, authentication, logs, configuration isolation, and errors. |
extensions/ql-vscode/test/vscode-tests/no-workspace/variant-analysis/gh-api-client.test.ts |
Exercises GitHub requests through the routing API. |
extensions/ql-vscode/test/vscode-tests/no-workspace/query-history/store/query-history-store.test.ts |
Tests persistence of server origins and repository URLs. |
extensions/ql-vscode/test/vscode-tests/no-workspace/query-history/query-history-manager.test.ts |
Covers delegated logs and cancellation during removal. |
extensions/ql-vscode/test/vscode-tests/no-workspace/query-history/history-tree-data-provider.test.ts |
Tests capability-based history actions. |
extensions/ql-vscode/test/vscode-tests/cli-integration/variant-analysis/variant-analysis-manager.test.ts |
Updates submission mocks for the routing API. |
extensions/ql-vscode/test/vscode-tests/activated-extension/variant-analysis/variant-analysis-results-manager.test.ts |
Covers custom repository links and plain-text fallbacks. |
extensions/ql-vscode/test/vscode-tests/activated-extension/variant-analysis/variant-analysis-monitor.test.ts |
Updates polling mocks and cancellation-status coverage. |
extensions/ql-vscode/test/vscode-tests/activated-extension/variant-analysis/variant-analysis-manager.test.ts |
Adds custom-server lifecycle, logs, and cancellation coverage. |
extensions/ql-vscode/test/unit-tests/variant-analysis/variant-analysis-mapper.test.ts |
Tests origin and source-URL preservation. |
extensions/ql-vscode/test/unit-tests/variant-analysis/markdown-generation.test.ts |
Tests Markdown locations without repository links. |
extensions/ql-vscode/test/unit-tests/common/bqrs-utils.test.ts |
Covers raw-result links with optional prefixes. |
extensions/ql-vscode/src/view/variant-analysis/VariantAnalysisStatusStats.tsx |
Uses server-neutral log-link wording. |
extensions/ql-vscode/src/view/variant-analysis/VariantAnalysisOutcomePanels.tsx |
Passes server and log capabilities to failure alerts. |
extensions/ql-vscode/src/view/variant-analysis/VariantAnalysis.tsx |
Renders custom runs without workflow IDs. |
extensions/ql-vscode/src/view/variant-analysis/RawResultsTable.tsx |
Accepts missing file-link prefixes. |
extensions/ql-vscode/src/view/variant-analysis/RawResultRow.tsx |
Propagates optional file-link prefixes. |
extensions/ql-vscode/src/view/variant-analysis/RawResultCell.tsx |
Supports cells without repository links. |
extensions/ql-vscode/src/view/variant-analysis/FailureReasonAlert.tsx |
Adapts failure messages and log actions for custom runs. |
extensions/ql-vscode/src/view/variant-analysis/__tests__/VariantAnalysisStatusStats.spec.tsx |
Updates log-label expectations. |
extensions/ql-vscode/src/view/variant-analysis/__tests__/VariantAnalysisStats.spec.tsx |
Updates log-action interaction coverage. |
extensions/ql-vscode/src/view/variant-analysis/__tests__/VariantAnalysisOutcomePanels.spec.tsx |
Covers custom-server failure alerts and logs. |
extensions/ql-vscode/src/view/variant-analysis/__tests__/VariantAnalysis.spec.tsx |
Tests rendering without workflow IDs. |
extensions/ql-vscode/src/view/variant-analysis/__tests__/AnalyzedRepoItemContent.spec.tsx |
Tests plain-text locations without links. |
extensions/ql-vscode/src/view/model-alerts/ModelAlerts.tsx |
Uses shared log-capability checks. |
extensions/ql-vscode/src/view/common/FileCodeSnippet/FileCodeSnippet.tsx |
Renders unlinked file titles as text. |
extensions/ql-vscode/src/view/common/FileCodeSnippet/CodeSnippetMessage.tsx |
Renders unlinked message locations as text. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-view.ts |
Refreshes views after asynchronous log discovery. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-results-manager.ts |
Builds source links from repository URLs with revision fallbacks. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-monitor.ts |
Polls using the run’s originating server. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-mapper.ts |
Maps submissions and preserves server metadata. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-manager.ts |
Integrates custom routing, result loading, cancellation, and logs. |
extensions/ql-vscode/src/variant-analysis/variant-analysis-api.ts |
Centralizes routing, origin checks, logs, and gist restrictions. |
extensions/ql-vscode/src/variant-analysis/shared/variant-analysis.ts |
Adds server-origin and runtime log-capability fields. |
extensions/ql-vscode/src/variant-analysis/shared/repository.ts |
Adds optional repository URLs. |
extensions/ql-vscode/src/variant-analysis/shared/analysis-result.ts |
Makes file-link prefixes optional. |
extensions/ql-vscode/src/variant-analysis/sarif-processing.ts |
Propagates optional links through interpreted results. |
extensions/ql-vscode/src/variant-analysis/run-remote-query.ts |
Bypasses controller-repository lookup for custom servers. |
extensions/ql-vscode/src/variant-analysis/repo-tasks-store/repo-tasks-dto.ts |
Adds repository URLs to stored task data. |
extensions/ql-vscode/src/variant-analysis/repo-tasks-store/repo-tasks-dto-mapper.ts |
Saves repository URLs with tasks. |
extensions/ql-vscode/src/variant-analysis/repo-tasks-store/repo-tasks-domain-mapper.ts |
Restores repository URLs from stored tasks. |
extensions/ql-vscode/src/variant-analysis/markdown-generation.ts |
Emits plain-text locations when links are unavailable. |
extensions/ql-vscode/src/variant-analysis/ghec-dr.ts |
Enables variant analysis with a configured custom server. |
extensions/ql-vscode/src/variant-analysis/gh-api/variant-analysis-server-client.ts |
Adds companion activation, token timeouts, and HTTPS requests. |
extensions/ql-vscode/src/variant-analysis/gh-api/repository.ts |
Accepts repository URLs in API responses. |
extensions/ql-vscode/src/variant-analysis/gh-api/gh-api-client.ts |
Moves analysis operations into the routing API. |
extensions/ql-vscode/src/variant-analysis/export-results.ts |
Restricts custom runs to local Markdown export. |
extensions/ql-vscode/src/variant-analysis/bqrs-processing.ts |
Accepts raw results without file-link prefixes. |
extensions/ql-vscode/src/query-history/store/query-history-variant-analysis-dto.ts |
Adds persisted origins and repository URLs. |
extensions/ql-vscode/src/query-history/store/query-history-variant-analysis-dto-mapper.ts |
Restores custom-server metadata from history. |
extensions/ql-vscode/src/query-history/store/query-history-variant-analysis-domain-mapper.ts |
Saves custom-server metadata to history. |
extensions/ql-vscode/src/query-history/query-history-manager.ts |
Delegates logs and offers server cancellation during removal. |
extensions/ql-vscode/src/query-history/history-tree-data-provider.ts |
Selects history actions using log capabilities. |
extensions/ql-vscode/src/model-editor/model-alerts/model-alerts-view.ts |
Refreshes model-alert log capabilities asynchronously. |
extensions/ql-vscode/src/extension.ts |
Registers log-capability configuration watching. |
extensions/ql-vscode/src/databases/ui/db-panel.ts |
Guards GitHub-opening actions in custom-server mode. |
extensions/ql-vscode/src/config.ts |
Reads hidden global settings and validates server URLs. |
extensions/ql-vscode/src/common/location-link-utils.ts |
Omits remote references when link prefixes are unavailable. |
extensions/ql-vscode/src/common/bqrs-utils.ts |
Omits raw-result links without a repository prefix. |
extensions/ql-vscode/package.json |
Updates history actions and custom-server UI conditions. |
docs/variant-analysis-custom-server.md |
Documents settings, companion hooks, behavior, and server contracts. |
Review effort: Max (auto)
| throw new Error("Variant analysis server returned an empty response."); | ||
| } | ||
|
|
||
| return data as T; |
| this.getVariantAnalysisStorageLocation(variantAnalysisId), | ||
| repositoryFullName, | ||
| options, | ||
| { ...options, serverUrl: variantAnalysis.serverUrl }, |
| "view": "codeQLVariantAnalysisRepositories", | ||
| "contents": "Set up a controller repository to start using variant analysis. [Learn more](https://codeql.github.com/docs/codeql-for-visual-studio-code/running-codeql-queries-at-scale-with-mrva#controller-repository) about controller repositories. \n[Set up controller repository](command:codeQLVariantAnalysisRepositories.setupControllerRepository)", | ||
| "when": "!config.codeQL.variantAnalysis.controllerRepo" | ||
| "when": "!config.codeQL.variantAnalysis.controllerRepo && !config.codeQL.variantAnalysis.serverUrl" |
| if ( | ||
| variantAnalysis === undefined || | ||
| (variantAnalysis.status === VariantAnalysisStatus.InProgress && | ||
| !variantAnalysis.serverUrl && |
cklin
left a comment
There was a problem hiding this comment.
This is a very large PR, and it is in everyone's interest that the PR is as easy to review as possible. Given that this is an unsupported feature, a key focus of the review is that existing behavior remains unchanged.
To facilitate timely review, please restructure the PR into a series of self-contained commits, with each commit clearly labeled as being in one of the following four categories:
- Refactoring: The commit completely preserves existing behavior. For example, the commit extracts new functions, or moves functions from one file to another. Commits in this category should make it apparent that all behavior remain unchanged.
- Switching to custom-server support: The commit adds new behavior based on the hidden user settings (or based on consequences of those hidden user settings). There should be very few such commits, and they should be very small. Commits in this category should make it apparent that existing supported behavior (i.e., when the hidden settings are absent) remains unchanged.
- Adding custom-server support: The commit changes behavior, but the behavior change can only affect custom-server support. Commits in this category should make it apparent that code changes affect only custom-server support, for example, because it changes code paths that are used only by custom-server support.
- Non-production-code changes: The commit does not change production code. For example, it only adds documentation or tests.
Please reach out when the restructuring is done. Thanks!
Move the code that builds the variant analysis submission request body from gh-api-client into variant-analysis-mapper, as mapVariantAnalysisSubmission, so that the custom-server client can reuse it. Category: Refactoring. submitVariantAnalysis sends the same request body, built by the same code in another module. The existing gh-api-client tests pass unchanged.
Add variant-analysis-api.ts with submitVariantAnalysis, getVariantAnalysis, getVariantAnalysisRepo and cancelVariantAnalysis. Each one calls the existing GitHub function with the same arguments. The manager and the monitor import them instead of the GitHub clients, so that a later commit can choose the server in one place. Category: Refactoring. The new functions only forward to the unchanged GitHub clients, and the call sites change only their imports. The existing manager, monitor and CLI tests, which spy on the GitHub clients, pass unchanged.
Results from a custom server may have no repository URL to link to. Allow fileLinkPrefix to be undefined, and in that case show file locations as plain text instead of links, both in the results view and in exported Markdown. Category: Refactoring. The changes widen the type of fileLinkPrefix and add branches for an undefined prefix. The results manager always builds a prefix for GitHub results, from the GitHub host and the repository name, so those branches never run, and GitHub results render and export as before. New tests cover linked and plain-text output.
Add an optional serverUrl to variant analyses and submissions. The mappers copy it from the submission to the run and keep it across status updates, and query history saves and restores it. Later commits use it to keep each run bound to the server that ran it. Category: Refactoring. Nothing sets serverUrl yet, so it stays undefined: requests, runs and views are unchanged. JSON omits undefined fields, so saved query history is unchanged as well.
Repositories analyzed by a custom server are not hosted on the GitHub instance, so their results cannot be linked through the GitHub host and the repository name. Map html_url from the API into repositories and repository tasks, store it in query history and in the repository task store, and pass the run's serverUrl to the results manager. For server runs, file links are built from html_url and point to HEAD when the task has no commit SHA. Without a usable html_url, locations are shown as plain text. Category: Adding custom-server support. Link creation changes only inside `if (serverUrl)`, which only server runs reach. GitHub runs keep the missing-SHA error and their existing link prefix. GitHub's responses also include html_url, so GitHub runs now store it too, but nothing reads it for them.
Results from a custom server must not be uploaded to GitHub, so the export picker offers only local Markdown for server runs. Category: Adding custom-server support. The options change only for runs with a serverUrl; GitHub runs get the same two options as before.
Server runs have no GitHub Actions workflow run. For them, allow cancellation without a workflow run ID, enable Stop query, show the results instead of the loading screen, and, when a running analysis is removed from query history, ask in a modal dialog whether to stop it on the server instead of linking to GitHub Actions. Category: Adding custom-server support. Each change is an early branch or an added `!serverUrl &&` condition, so runs without a serverUrl take the original code paths.
Add the hidden codeQL.variantAnalysis.serverUrl and serverTokenProvider settings. They are read only from user settings, so a workspace cannot redirect queries or tokens, and an invalid server URL throws instead of falling back to GitHub. The test configuration mock gains inspect(), which reading user settings needs. Category: Adding custom-server support. Only new functions are added, and no production code calls them until the switching commits.
Add requestVariantAnalysisServer, which sends variant analysis requests to the configured server. It gets a bearer token from the configured token provider extension, waits at most 30 seconds for the provider, refuses redirects, and reports HTTP errors as RequestError, so callers can handle statuses as they do for GitHub. Category: Adding custom-server support. The module is new, and no production code calls it yet.
Server runs have no GitHub Actions logs. A token provider can now implement an optional openVariantAnalysisLogs(serverUrl, analysisId) hook. For server runs, View logs in the results view and in query history call that hook, after checking that the run belongs to the configured server. Category: Adding custom-server support. The manager and query history only add early branches for runs with a serverUrl. GitHub runs still open their workflow run URL.
Whether a server run has logs depends on the provider implementing the log hook. Discover that in the background, cache the result per server and provider, clear the cache when either setting changes, and refresh query history and the webviews when discovery finishes, so drawing never waits for the provider. Failed server runs show a failure message without the GitHub Actions wording, with a View logs link if supported. Category: Adding custom-server support. History context values, webview messages and failure messages change only for runs with a serverUrl. The new history context values are only used for server runs, and the menus list them next to the existing ones. The view-logs check moves into hasVariantAnalysisLogs, which gives the same result as before for runs without a serverUrl. The configuration watcher only reacts to the two hidden settings.
When codeQL.variantAnalysis.serverUrl is set, submit, poll, fetch repository tasks and cancel through the server client; stamp new runs with the server URL; use controller repository 0; and enable variant analysis regardless of github-enterprise.uri. Each run stays bound to the server that created it: its requests fail while another server, or none, is configured. Category: Switching to custom-server support. Without the setting, getVariantAnalysisServerConfig() returns undefined, so new runs get no serverUrl, runs without a serverUrl are routed to GitHub, and the other added conditions are false. Requests and error handling are then the same as before.
With a server configured, the repositories panel does not ask for a controller repository, and GitHub's built-in repository lists and the Open on GitHub action are hidden. Category: Switching to custom-server support. Each change adds a condition on the server setting. Without the setting, the when-clauses and isVariantAnalysisServerConfigured() see no server, and the panel behaves as before.
Add an activated-extension test that submits a variant analysis to a mocked server, monitors it, downloads its results and checks their file links. It also checks that the server token is only sent to the server and that GitHub authentication is never used. Category: Non-production-code changes. Only a test is added; production code is unchanged.
Document the hidden settings, the token provider contract, the server's REST contract, and how CodeQL behaves when a server is configured. Category: Non-production-code changes. Only documentation is added; production code is unchanged.
7d42d34 to
0588021
Compare
| : ""; | ||
| throw new RequestError( | ||
| `Variant analysis server request failed: ${method} ${path}: ${response.status} ${response.statusText}${describeErrorBody(data)}${authenticationMessage}`, | ||
| response.status, |
| throw new RequestError( | ||
| `Variant analysis server request failed: ${method} ${path}: ${response.status} ${response.statusText}${describeErrorBody(data)}${authenticationMessage}`, | ||
| response.status, | ||
| { request: { method, url, headers: {} } }, |
The server's IDs and repository names become request URLs and file paths. A server that returned a repository name such as "../x" could make CodeQL read or write files outside its storage directory. Reject server responses whose IDs are not integers, or whose repository names have an empty or dots-only part, a backslash or a colon. Other names, including ones with more than two parts, are accepted. Category: Adding custom-server support. Only the server branches of the variant analysis API check responses, and only runs with a serverUrl reach them. GitHub responses are not checked, so GitHub runs are unchanged.
Changes made. |
There was a problem hiding this comment.
🟡 Changes recommended
Run-ID collisions across endpoints can overwrite local history and mix artifacts, while non-unique Markdown export filenames can silently overwrite repository results.
7 open issues
Prevent Markdown export filename collisions · New Qualify local run identities by endpoint · New Namespace analysis identity and storage by server URL and ID Validate response IDs and repository paths before storage access Tailor server URL guidance to the run type · New Enable stopping custom-server runs without a workflow ID Allow custom-server mode through the repositories tree gate
4 resolved since last review
Review effort: Max
| Number.isSafeInteger(repo.id) && | ||
| typeof repo.full_name === "string" && | ||
| /^[^\\:]+$/.test(repo.full_name) && | ||
| repo.full_name.split("/").every((part) => !/^\.*$/.test(part)), |
| id: response.id, | ||
| serverUrl: currentVariantAnalysis.serverUrl, |
| if (server?.url !== serverUrl) { | ||
| throw new Error( | ||
| "This variant analysis belongs to a different server. Restore the server setting used to submit it before retrying.", | ||
| ); | ||
| } |
Treat only missing values and blank strings as unset. An explicit null must fail validation rather than falling back to GitHub.
Reserve numeric local IDs for custom runs without changing existing GitHub storage or local interfaces. Persist the server ID for API calls, number custom-server Markdown export files, and give endpoint-specific guidance for saved runs.


This PR adds a configurable server for multi-repository variant analysis (MRVA). Submission, polling, task retrieval, and cancellation go through
variant-analysis-api.ts, which selects GitHub or the configured endpoint.Purpose:
To unblock GitHub Expert-Services-guided customer use of multi-repo variant analysis on GitHub Enterprise Server using existing tooling, in a stable, extensible way without breaking any existing logic.
Implementation:
getVariantAnalysisServerToken(serverUrl)and can implementopenVariantAnalysisLogs(serverUrl, analysisId). This lets the companion handle sign-in, token refresh, and log presentation while CodeQL keeps the existing analysis UI. The server must implement the existing variant-analysis REST contract.codeQL.variantAnalysis.serverUrlandcodeQL.variantAnalysis.serverTokenProvider. Workspace values are ignored. Custom endpoints must use HTTPS; redirects are rejected. Custom submissions use controller repository ID0, bypassing controller-repository setup.HEADwhen the SHA is missing; unavailable repository URLs produce plain-text locations.docs/variant-analysis-custom-server.md. This PR does not include either implementation./blob/{revision}, and canary autofix still retrieves source through GitHub.