Skip to content

Python: experiment with shared CFG and SSA activation - #161

Draft
yoff wants to merge 2198 commits into
mainfrom
yoff-python-shared-ssa-port
Draft

yoff wants to merge 2198 commits into
mainfrom
yoff-python-shared-ssa-port

Conversation

@yoff

@yoff yoff commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Experimental current-main port

This draft ports the Python shared CFG/SSA adapter and dataflow activation from github#21923 and github#21925 onto the pinned current-main baseline for isolated performance measurement.

  • Baseline: 9ffea9596ef3a45ad8524ad2700836db2454472a
  • Candidate: f6e989a86971765b667992661714ff9ab0257a99
  • Validation: 89/89 focused and control-flow tests passed; QL formatting and git diff --check passed.
  • Exact semantic evidence: definitions, phis, phi inputs, first-use, adjacent-use, canonical local flow, and production outputs are preserved in the local study artifacts with every added/removed tuple enumerated.

Local deterministic measurements show large lifecycle regressions (+62.48% joined tuples on NiceGUI, +47.73% on Langroid, +45.96% on DeepDiff). Central SSA caching reduces some duplicate liveness and Local-only shared-SSA plans, but does not remove the dominant CFG/local-flow cost.

This activation is not recommended for landing based on the local results. This draft exists only to obtain an isolated same-repository DCA fleet picture. It makes no landing claim.

nickrolfe and others added 30 commits November 10, 2026 12:23
…sions-java

Java: convert data extensions to JSON
…sions-shared

Shared: convert data extensions to JSON
Moving the C++ library target only aligns solib ownership in standalone builds. When CodeQL is a dependency, the solib artifacts still live in the root repository's runfiles but are owned by the CodeQL repository, so rules_pkg 1.3 cannot resolve them.

Overlay the extractor's runfiles with the original prebuilt shared libraries at the same installed destinations. This preserves package contents and permissions while using source paths that resolve correctly in both build layouts.
Also changes the `InterpolationContext` counter to use `u64` instead of
`usize`.
…ects

Python: treat await expressions as side-effecting
…ested-types

Unified: Type inference for nested types
No semantic
paldepind and others added 19 commits November 16, 2026 14:02
With Rust 1.100 Cargo now includes a lint for unused dependencies:
https://blog.rust-lang.org/2026/11/12/Rust-1.100.0/#reporting-unused-dependencies-in-cargo

Fix all results from the lint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7f912f98-5361-47d2-b1f3-46af0843e5e0
yeast: Add support for associating schemas with `tree!`/`trees!` calls.
…m-escape-consistency-check

Type inference: Consistency check for escaping type parameters
Update CODEOWNERS to specify ownership for README.md and CODE_OF_CONDUCT.md.

changes to general markdown files in the root should get caught by the overall CODEOWNERS rule
Bumps the extractor-dependencies group in /go/extractor with 1 update: [golang.org/x/mod](https://github.com/golang/mod).


Updates `golang.org/x/mod` from 0.42.0 to 0.43.0
- [Commits](golang/mod@v0.42.0...v0.43.0)

---
updated-dependencies:
- dependency-name: golang.org/x/mod
  dependency-version: 0.43.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: extractor-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
…extractor/extractor-dependencies-9f099c314d

Bump golang.org/x/mod from 0.42.0 to 0.43.0 in /go/extractor in the extractor-dependencies group
…return

Rust: Model early `Err` returns in `?` expressions
…-default-json

modelgenerator: default to generating JSON
…sions-python

Python: convert data extensions to JSON
Port the additive Python SSA adapter and focused parity tests onto current main while preserving module-exit phis, captures, bindings, and legacy-shaped compatibility APIs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Switch the Python dataflow implementation and compatibility call sites to the shared CFG/SSA adapter. Generated expected-file conflicts remain on current-main outputs pending repository-standard regeneration.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Port the follow-up binding, guard-specialization, capture, and explicit-successor fixes while combining them with current main's optional conditional branches.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Regenerate the regression expectation through codeql test run --learn for the shared CFG/SSA activation on current main.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Regenerate the query expectation after shared dataflow activation exposes the additional interprocedural integer flow.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

QHelp previews:

cpp/ql/src/Likely Bugs/Likely Typos/AmbiguousAssignmentOfComparison.qhelp

Ambiguous assignment of comparison result used as truth value

Assignment operators have lower precedence than comparison operators. For example, status = read_status() < 0 assigns the comparison result (zero or one) to status. This can be unintended when the programmer meant to assign the return value first and then compare it with zero.

Recommendation

Use parentheses to make the intended order of operations explicit. To assign first and compare the assigned value, parenthesize the assignment. To intentionally assign the comparison result, parenthesize the comparison. An explicit cast around the comparison also makes that order clear.

Example

In the first condition, status receives either zero or one instead of the value returned by read_status(). The second condition explicitly performs the assignment before the comparison.

int read_status();

int check_status() {
  int status;
  if (status = read_status() < 0) // BAD: assigns the comparison result.
    return status;

  if ((status = read_status()) < 0) // GOOD: assigns first, then compares.
    return status;

  return 0;
}

References

cpp/ql/src/Security/CWE/CWE-078/ExecTainted.qhelp

Uncontrolled data used in OS command

The code passes user input as part of a call to system or popen without escaping special elements. It generates a command line using sprintf, with the user-supplied data directly passed as a formatting argument. This leaves the code vulnerable to attack by command injection.

Recommendation

Use a library routine to escape characters in the user-supplied string before passing it to a command shell.

Example

The following example runs an external command in two ways. The first way uses sprintf to build a command directly out of a user-supplied argument. As such, it is vulnerable to command injection. The second way quotes the user-provided value before embedding it in the command; assuming the encodeShellString utility is correct, this code should be safe against command injection.

int main(int argc, char** argv) {
  char *userName = argv[2];
  
  {
    // BAD: a string from the user is injected directly into
    // a command line.
    char command1[1000] = {0};
    sprintf(command1, "userinfo -v \"%s\"", userName);
    system(command1);
  }

  {
    // GOOD: the user string is encoded by a library routine.
    char userNameQuoted[1000] = {0};
    encodeShellString(userNameQuoted, 1000, userName); 
    char command2[1000] = {0};
    sprintf(command2, "userinfo -v %s", userNameQuoted);
    system(command2);
  }
}

References

cpp/ql/src/experimental/Security/CWE/CWE-078/WordexpTainted.qhelp

Uncontrolled data used in wordexp command

The code passes user input to wordexp. This leaves the code vulnerable to attack by command injection, because wordexp performs command substitution. Command substitution is a feature that replaces $(command) or `command` with the output of the given command, allowing the user to run arbitrary code on the system.

Recommendation

When calling wordexp, pass the WRDE_NOCMD flag to prevent command substitution.

Example

The following example passes a user-supplied file path to wordexp in two ways. The first way uses wordexp with no specified flags. As such, it is vulnerable to command injection. The second way uses wordexp with the WRDE_NOCMD flag. As such, no command substitution is performed, making this safe from command injection.

int main(int argc, char** argv) {
  char *filePath = argv[2];

  {
    // BAD: the user-controlled string is injected
    // directly into `wordexp` which performs command substitution

    wordexp_t we;
    wordexp(filePath, &we, 0);
  }

  {
    // GOOD: command substitution is disabled

    wordexp_t we;
    wordexp(filePath, &we, WRDE_NOCMD);
  }
}

References

cpp/ql/src/experimental/Security/CWE/CWE-120/MmioUnsanitizedMemcpy.qhelp

MMIO/DMA unsanitized memory copy

Firmware and embedded drivers often copy data into buffers using lengths read from allowlisted MMIO register macros such as READ_REG or GET_MMIO. When those lengths are not validated against the destination buffer size, an attacker who can influence hardware registers or DMA metadata can trigger buffer overflows.

Recommendation

Always validate MMIO/DMA-derived lengths before passing them to memcpy, memmove, or strncpy. Compare against a compile-time maximum and reject or clamp out-of-range values before copying.

Example

Bad: length from an MMIO register used directly as the copy size.

#define READ_REG(addr) (*(volatile unsigned int *)(addr))
#define MAX_DMA_LEN 64

void *memcpy(void *dest, const void *src, unsigned long n);

void bad_mmio_memcpy(char *dst, char *src) {
  unsigned int len = READ_REG(0x40001000);
  memcpy(dst, src, len);
}

Good: defensive bounds check before the copy.

#define READ_REG(addr) (*(volatile unsigned int *)(addr))
#define MAX_DMA_LEN 64

void *memcpy(void *dest, const void *src, unsigned long n);

void good_mmio_memcpy(char *dst, char *src) {
  unsigned int len = READ_REG(0x40001000);
  if (len <= MAX_DMA_LEN)
    memcpy(dst, src, len);
}

References

  • Common Weakness Enumeration: CWE-120.
  • Common Weakness Enumeration: CWE-787.
csharp/ql/src/Security Features/CWE-078/CommandInjection.qhelp

Uncontrolled command line

Code that passes user input directly to System.Diagnostic.Process.Start, or some other library routine that executes a command, allows the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command to run or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to System.Diagnostic.Process.Start without examining it first.

using System;
using System.Web;
using System.Diagnostics;

public class CommandInjectionHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext ctx)
    {
        string param = ctx.Request.QueryString["param"];
        Process.Start("process.exe", "/c " + param);
    }
}

References

csharp/ql/src/Security Features/CWE-451/MissingXFrameOptions.qhelp

Missing clickjacking protection

Web sites that do not restrict framing using the X-Frame-Options HTTP header or the frame-ancestors Content Security Policy directive may be vulnerable to UI redress attacks ("clickjacking"). In these attacks, the vulnerable site is loaded in a frame on an attacker-controlled site which uses opaque or transparent layers to trick the user into unintentionally clicking a button or link on the vulnerable site.

Recommendation

Set the X-Frame-Options HTTP header to DENY, to instruct web browsers to block attempts to load the site in a frame. Alternatively, if framing is needed in certain circumstances, specify SAMEORIGIN to permit framing by the same origin. The frame-ancestors directive in an enforced Content-Security-Policy header provides a more flexible alternative. For example, use frame-ancestors 'none' to prevent all framing, or use its source list to specify which origins may embed the application.

For ASP.NET Framework applications, the header may be specified either in the Web.config file, using the <customHeaders> tag, or within the source code of the application using the HttpResponse.AddHeader method. In general, prefer specifying the header in the Web.config file to ensure it is added to all requests. If adding it to the source code, ensure that it is added unconditionally to all requests. For example, add the header in the Application_BeginRequest method in the global.asax file.

For ASP.NET Core applications, set the header on HttpResponse.Headers. This can be done using the header dictionary's indexer or its Append, Add, or TryAdd methods.

Example

The following example shows how to specify the X-Frame-Options header within the Web.config file for ASP.NET:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.web>
  </system.web>
  <system.webServer>
    <httpProtocol>
      <customHeaders>
        <add name="X-Frame-Options" value="SAMEORIGIN" />
      </customHeaders>
    </httpProtocol>
  </system.webServer>
</configuration>

This next example shows how to specify the X-Frame-Options header within the global.asax file for an ASP.NET application:

protected void Application_BeginRequest(object sender, EventArgs e)
{
    HttpContext.Current.Response.AddHeader("X-Frame-Options", "DENY");
}

The following ASP.NET Core example uses an enforced Content Security Policy to disallow framing:

void Configure(IApplicationBuilder app)
{
    app.Use(async (context, next) =>
    {
        context.Response.Headers["Content-Security-Policy"] = "frame-ancestors 'none'";
        await next();
    });
}

References

csharp/ql/src/Security Features/CWE-730/ReDoS.qhelp

Denial of Service from comparison of user input against expensive regex

Matching user input against a regular expression which takes exponential time in the worst case can allow a malicious user to perform a Denial of Service ("DoS") attack by crafting input that takes a long time to execute.

Most regular expression engines, including the C# standard library implementation, are designed to work with an extended regular expression syntax. Although this provides flexibility for the user, it can prevent the engine from constructing an efficient implementation of the matcher in all circumstances. In particular, the "worst case time complexity" (see the references) of certain regular expressions may be "exponential". This would allow a malicious user to provide some input which causes the regular expression to take a very long time to execute.

Typically, a regular expression is vulnerable to this attack if it applies repetition to a sub-expression which itself is repeated, or contains overlapping options. For example, (a+)+ is vulnerable to a string such as aaaaaaaaaaaaaaaaaaaaaaaaaaab. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to avoid the exponential worst case time. If this is not possible, then a timeout should be used to avoid a denial of service. For C# applications, a timeout can be provided to the Regex constructor. Alternatively, apply a global timeout by setting the REGEX_DEFAULT_MATCH_TIMEOUT application domain property, using the AppDomain.SetData method.

Example

The following example shows a HTTP request parameter that is matched against a regular expression which has exponential worst case performance. In the first case, it is matched without a timeout, which can lead to a denial of service. In the second case, a timeout is used to cancel the evaluation of the regular expression after 1 second.

using System;
using System.Web;
using System.Text.RegularExpressions;

public class ReDoSHandler : IHttpHandler
{

    public void ProcessRequest(HttpContext ctx)
    {
        string userInput = ctx.Request.QueryString["userInput"];

        // BAD: User input is matched against a regex with exponential worst case behavior
        new Regex("^([a-z]*)*$").Match(userInput);

        // GOOD: Regex is given a timeout to avoid DoS
        new Regex("^([a-z]*)*$",
                  RegexOptions.IgnoreCase,
                  TimeSpan.FromSeconds(1)).Match(userInput);
    }
}

References

csharp/ql/src/Security Features/CWE-730/RegexInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous as a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

For user input that is intended to be referenced as a string literal in a regular expression, use the Regex.Escape method to escape any special characters. If the regular expression is intended to be configurable by the user, then a timeout should be used to avoid Denial of Service attacks. For C# applications, a timeout can be provided to the Regex constructor. Alternatively, apply a global timeout by setting the REGEX_DEFAULT_MATCH_TIMEOUT application domain property, using the AppDomain.SetData method.

Example

The following example shows a HTTP request parameter that is used as a regular expression, and matched against another request parameter.

In the first case, the regular expression is used without a timeout, and the user-provided regex is not escaped. If a malicious user provides a regex that has exponential worst case performance, then this could lead to a Denial of Service.

In the second case, the user input is escaped using Regex.Escape before being included in the regular expression. This ensures that the user cannot insert characters which have a special meaning in regular expressions.

using System;
using System.Web;
using System.Text.RegularExpressions;

public class RegexInjectionHandler : IHttpHandler
{

    public void ProcessRequest(HttpContext ctx)
    {
        string name = ctx.Request.QueryString["name"];
        string userInput = ctx.Request.QueryString["userInput"];

        // BAD: Unsanitized user input is used to construct a regular expression
        new Regex("^" + name + "=.*$").Match(userInput);

        // GOOD: User input is sanitized before constructing the regex
        string safeName = Regex.Escape(name);
        new Regex("^" + safeName + "=.*$").Match(userInput);
    }
}

References

go/ql/src/Security/CWE-078/CommandInjection.qhelp

Command built from user-controlled sources

If a system command invocation is built from user-provided data without sufficient sanitization, a malicious user may be able to run commands to exfiltrate data or compromise the system.

Recommendation

Whenever possible, use hard-coded string literals for commands and avoid shell string interpreters like sh -c.

If given arguments as a single string, avoid simply splitting the string on whitespace. Arguments may contain quoted whitespace, causing them to split into multiple arguments.

If this is not possible, sanitize user input to avoid characters like spaces and various kinds of quotes that can alter the meaning of the command.

Example

In the following example, assume the function handler is an HTTP request handler in a web application, whose parameter req contains the request object:

package main

import (
	"net/http"
	"os/exec"
)

func handler(req *http.Request) {
	imageName := req.URL.Query()["imageName"][0]
	outputPath := "/tmp/output.svg"
	cmd := exec.Command("sh", "-c", fmt.Sprintf("imagetool %s > %s", imageName, outputPath))
	cmd.Run()
	// ...
}

The handler extracts the image file name from the request and uses the image name to construct a shell command that is executed using `sh -c`, which can lead to command injection.

It's better to avoid shell commands by using the exec.Command function directly, as shown in the following example:

package main

import (
	"log"
	"net/http"
	"os"
	"os/exec"
)

func handler(req *http.Request) {
	imageName := req.URL.Query()["imageName"][0]
	outputPath := "/tmp/output.svg"

	// Create the output file
	outfile, err := os.Create(outputPath)
	if err != nil {
		log.Fatal(err)
	}
	defer outfile.Close()

	// Prepare the command
	cmd := exec.Command("imagetool", imageName)

	// Set the output to our file
	cmd.Stdout = outfile

	cmd.Run()
}

Alternatively, a regular expression can be used to ensure that the image name is safe to use in a shell command:

package main

import (
	"log"
	"net/http"
	"os/exec"
	"regexp"
)

func handler(req *http.Request) {
	imageName := req.URL.Query()["imageName"][0]
	outputPath := "/tmp/output.svg"

	// Validate the imageName with a regular expression
	validImageName := regexp.MustCompile(`^[a-zA-Z0-9_\-\.]+$`)
	if !validImageName.MatchString(imageName) {
		log.Fatal("Invalid image name")
		return
	}

	cmd := exec.Command("sh", "-c", fmt.Sprintf("imagetool %s > %s", imageName, outputPath))
	cmd.Run()
}

Some commands, like git, can indirectly execute commands if an attacker specifies the flags given to the command.

To mitigate this risk, either add a -- argument to ensure subsequent arguments are not interpreted as flags, or verify that the argument does not start with "--".

package main

import (
	"log"
	"net/http"
	"os/exec"
	"strings"
)

func handler(req *http.Request) {
	repoURL := req.URL.Query()["repoURL"][0]
	outputPath := "/tmp/repo"

	// Sanitize the repoURL to ensure it does not start with "--"
	if strings.HasPrefix(repoURL, "--") {
		log.Fatal("Invalid repository URL")
	} else {
		cmd := exec.Command("git", "clone", repoURL, outputPath)
		err := cmd.Run()
		if err != nil {
			log.Fatal(err)
		}
	}

	// Or: add "--" to ensure that the repoURL is not interpreted as a flag
	cmd := exec.Command("git", "clone", "--", repoURL, outputPath)
	err := cmd.Run()
	if err != nil {
		log.Fatal(err)
	}
}

References

go/ql/src/Security/CWE-078/StoredCommand.qhelp

Command built from stored data

If a system command invocation is built from stored data without sufficient sanitization, and that data is stored from a user input, a malicious user may be able to run commands to exfiltrate data or compromise the system.

Recommendation

If possible, use hard-coded string literals to specify the command to run. Instead of interpreting stored input directly as command names, examine the input and then choose among hard-coded string literals.

If this is not possible, then add sanitization code to verify that the user input is safe before using it.

Example

In the following example, the function run runs a command directly from the result of a query:

package main

import (
	"database/sql"
	"os/exec"
)

var db *sql.DB

func run(query string) {
	rows, _ := db.Query(query)
	var cmdName string
	rows.Scan(&cmdName)
	cmd := exec.Command(cmdName)
	cmd.Run()
}

The function extracts the name of a system command from the database query, and then runs it without any further checks, which can cause a command-injection vulnerability. A possible solution is to ensure that commands are checked against a whitelist:

package main

import (
	"database/sql"
	"os/exec"
)

var db *sql.DB

func run(query string) {
	rows, _ := db.Query(query)
	var cmdName string
	rows.Scan(&cmdName)
	if cmdName == "mybinary1" || cmdName == "mybinary2" {
		cmd := exec.Command(cmdName)
	}
	cmd.Run()
}

References

go/ql/src/Security/CWE-089/StringBreak.qhelp

Potentially unsafe quoting

Code that constructs a quoted string literal containing user-provided data needs to ensure that this data does not itself contain a quote. Otherwise the embedded data could (accidentally or intentionally) terminate the string literal early and thereby change the structure of the overall string, with potentially severe consequences. If, for example, the string is later used as part of an operating-system command or database query, an attacker may be able to craft input data that injects a malicious command.

Recommendation

Sanitize the embedded data appropriately to ensure quotes are escaped, or use an API that does not rely on manually constructing quoted substrings. Make sure to use the appropriate escaping mechanism, for example, double quoting for SQL strings or backslash escaping for shell commands. When using backslash escaping, the backslash character itself must also be escaped.

Example

In the following example, assume that version is an object from an untrusted source. The code snippet first uses json.Marshal to serialize this object into a string, and then embeds it into a SQL query built using the Squirrel library.

package main

import (
	"encoding/json"
	"fmt"
	sq "github.com/Masterminds/squirrel"
)

func save(id string, version interface{}) {
	versionJSON, _ := json.Marshal(version)
	sq.StatementBuilder.
		Insert("resources").
		Columns("resource_id", "version_md5").
		Values(id, sq.Expr(fmt.Sprintf("md5('%s')", versionJSON))).
		Exec()
}

Note that JSON encoding does not escape single quotes in any way, so this code is vulnerable: any single-quote character in version will prematurely close the surrounding string literal, changing the structure of the SQL expression being constructed. This could be exploited to mount a SQL injection attack.

To fix this vulnerability, use the placeholder syntax from Squirrel's structured API for building queries, which avoids the need to explicitly construct a quoted string.

package main

import (
	"encoding/json"
	sq "github.com/Masterminds/squirrel"
)

func saveGood(id string, version interface{}) {
	versionJSON, _ := json.Marshal(version)
	sq.StatementBuilder.
		Insert("resources").
		Columns("resource_id", "version_md5").
		Values(id, sq.Expr("md5(?)", versionJSON)).
		Exec()
}

In situations where a structured API is not available, make sure that you escape quotes before embedding user-provided data into a quoted string. For example, this is how you can backslash-escape single quotes using strings.ReplaceAll:

  quoted := strings.ReplaceAll(raw, `\`, `\\`)
  quoted = strings.ReplaceAll(quoted, "'", "\\'")

Note that any existing backslash characters in the string must be escaped first, so that they do not interfere with the escaping of single quotes.

In some cases, strconv.Quote is a convenient option for backslash escaping, but note that it has two limitations:

  1. It only supports double quotes, not single quotes (as in the example).
  2. It puts quotes around the entire string, so it can only be used to construct complete string literals, not parts of larger string literals.

References

go/ql/src/Security/CWE-312/CleartextLogging.qhelp

Clear-text logging of sensitive information

Sensitive information that is logged unencrypted is accessible to an attacker who gains access to the logs.

Recommendation

Ensure that sensitive information is always encrypted or obfuscated before being logged.

In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext.

Be aware that external processes often store the standard out and standard error streams of the application, causing logged sensitive information to be stored.

Example

The following example code logs user credentials (in this case, their password) in plain text:

package main

import (
	"log"
	"net/http"
)

func serve() {
	http.HandleFunc("/register", func(w http.ResponseWriter, r *http.Request) {
		r.ParseForm()
		user := r.Form.Get("user")
		pw := r.Form.Get("password")

		log.Printf("Registering new user %s with password %s.\n", user, pw)
	})
	http.ListenAndServe(":80", nil)
}

Instead, the credentials should be encrypted, obfuscated, or omitted entirely:

package main

import (
	"log"
	"net/http"
)

func serve1() {
	http.HandleFunc("/register", func(w http.ResponseWriter, r *http.Request) {
		r.ParseForm()
		user := r.Form.Get("user")
		pw := r.Form.Get("password")

		log.Printf("Registering new user %s.\n", user)

		// ...
		use(pw)
	})
	http.ListenAndServe(":80", nil)
}

References

  • M. Dowd, J. McDonald and J. Schuhm, The Art of Software Security Assessment, 1st Edition, Chapter 2 - 'Common Vulnerabilities of Encryption', p. 43. Addison Wesley, 2006.
  • M. Howard and D. LeBlanc, Writing Secure Code, 2nd Edition, Chapter 9 - 'Protecting Secret Data', p. 299. Microsoft, 2002.
  • OWASP: Logging Cheat Sheet.
  • Common Weakness Enumeration: CWE-312.
  • Common Weakness Enumeration: CWE-315.
  • Common Weakness Enumeration: CWE-359.
java/ql/src/Security/CWE/CWE-078/ExecTainted.qhelp

Uncontrolled command line

Code that passes user input directly to Runtime.exec, or some other library routine that executes a command, allows the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command to run or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to Runtime.exec without examining it first.

class Test {
    public static void main(String[] args) {
        String script = System.getenv("SCRIPTNAME");
        if (script != null) {
            // BAD: The script to be executed is controlled by the user.
            Runtime.getRuntime().exec(script);
        }
    }
}

References

java/ql/src/Security/CWE/CWE-078/ExecUnescaped.qhelp

Building a command line with string concatenation

Code that builds a command line by concatenating strings that have been entered by a user allows the user to execute malicious code.

Recommendation

Execute external commands using an array of strings rather than a single string. By using an array, many possible vulnerabilities in the formatting of the string are avoided.

Example

In the following example, latlonCoords contains a string that has been entered by a user but not validated by the program. This allows the user to, for example, append an ampersand (&) followed by the command for a malicious program to the end of the string. The ampersand instructs Windows to execute another program. In the block marked 'BAD', latlonCoords is passed to exec as part of a concatenated string, which allows more than one command to be executed. However, in the block marked 'GOOD', latlonCoords is passed as part of an array, which means that exec treats it only as an argument.

class Test {
    public static void main(String[] args) {
        // BAD: user input might include special characters such as ampersands
        {
            String latlonCoords = args[1];
            Runtime rt = Runtime.getRuntime();
            Process exec = rt.exec("cmd.exe /C latlon2utm.exe " + latlonCoords);
        }

        // GOOD: use an array of arguments instead of executing a string
        {
            String latlonCoords = args[1];
            Runtime rt = Runtime.getRuntime();
            Process exec = rt.exec(new String[] {
                    "c:\\path\to\latlon2utm.exe",
                    latlonCoords });
        }
    }
}

References

java/ql/src/Security/CWE/CWE-730/PolynomialReDoS.qhelp

Polynomial regular expression used on uncontrolled data

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine provided by Java uses a backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Note that Java versions 9 and above have some mitigations against ReDoS; however they aren't perfect and more complex regular expressions can still be affected by this problem.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter. Alternatively, an alternate regex library that guarantees linear time execution, such as Google's RE2J, may be used.

Example

Consider this use of a regular expression, which removes all leading and trailing whitespace in a string:

Pattern.compile("^\\s+|\\s+$").matcher(text).replaceAll("") // BAD

The sub-expression "\\s+$" will match the whitespace characters in text from left to right, but it can start matching anywhere within a whitespace sequence. This is problematic for strings that do not end with a whitespace character. Such a string will force the regular expression engine to process each whitespace sequence once per whitespace character in the sequence.

This ultimately means that the time cost of trimming a string is quadratic in the length of the string. So a string like "a b" will take milliseconds to process, but a similar string with a million spaces instead of just one will take several minutes.

Avoid this problem by rewriting the regular expression to not contain the ambiguity about when to start matching whitespace sequences. For instance, by using a negative look-behind ("^\\s+|(?<!\\s)\\s+$"), or just by using the built-in trim method (text.trim()).

Note that the sub-expression "^\\s+" is not problematic as the ^ anchor restricts when that sub-expression can start matching, and as the regular expression engine matches from left to right.

Example

As a similar, but slightly subtler problem, consider the regular expression that matches lines with numbers, possibly written using scientific notation:

"^0\\.\\d+E?\\d+$"" 

The problem with this regular expression is in the sub-expression \d+E?\d+ because the second \d+ can start matching digits anywhere after the first match of the first \d+ if there is no E in the input string.

This is problematic for strings that do not end with a digit. Such a string will force the regular expression engine to process each digit sequence once per digit in the sequence, again leading to a quadratic time complexity.

To make the processing faster, the regular expression should be rewritten such that the two \d+ sub-expressions do not have overlapping matches: "^0\\.\\d+(E\\d+)?$".

Example

Sometimes it is unclear how a regular expression can be rewritten to avoid the problem. In such cases, it often suffices to limit the length of the input string. For instance, the following regular expression is used to match numbers, and on some non-number inputs it can have quadratic time complexity:

Pattern.matches("^(\\+|-)?(\\d+|(\\d*\\.\\d*))?(E|e)?([-+])?(\\d+)?$", str); 

It is not immediately obvious how to rewrite this regular expression to avoid the problem. However, you can mitigate performance issues by limiting the length to 1000 characters, which will always finish in a reasonable amount of time.

if (str.length() > 1000) {
    throw new IllegalArgumentException("Input too long");
}

Pattern.matches("^(\\+|-)?(\\d+|(\\d*\\.\\d*))?(E|e)?([-+])?(\\d+)?$", str); 

References

java/ql/src/Security/CWE/CWE-730/ReDoS.qhelp

Inefficient regular expression

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine provided by Java uses a backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Note that Java versions 9 and above have some mitigations against ReDoS; however they aren't perfect and more complex regular expressions can still be affected by this problem.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter. Alternatively, an alternate regex library that guarantees linear time execution, such as Google's RE2J, may be used.

Example

Consider this regular expression:

^_(__|.)+_$

Its sub-expression "(__|.)+?" can match the string "__" either by the first alternative "__" to the left of the "|" operator, or by two repetitions of the second alternative "." to the right. Thus, a string consisting of an odd number of underscores followed by some other character will cause the regular expression engine to run for an exponential amount of time before rejecting the input.

This problem can be avoided by rewriting the regular expression to remove the ambiguity between the two branches of the alternative inside the repetition:

^_(__|[^_])+_$

References

java/ql/src/Security/CWE/CWE-730/RegexInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous as a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as Pattern.quote to escape meta-characters that have special meaning.

Example

The following example shows an HTTP request parameter that is used to construct a regular expression.

In the first case the user-provided regex is not escaped. If a malicious user provides a regex whose worst-case performance is exponential, then this could lead to a Denial of Service.

In the second case, the user input is escaped using Pattern.quote before being included in the regular expression. This ensures that the user cannot insert characters which have a special meaning in regular expressions.

import java.util.regex.Pattern;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;

public class RegexInjectionDemo extends HttpServlet {

  public boolean badExample(javax.servlet.http.HttpServletRequest request) {
    String regex = request.getParameter("regex");
    String input = request.getParameter("input");

    // BAD: Unsanitized user input is used to construct a regular expression
    return input.matches(regex);
  }

  public boolean goodExample(javax.servlet.http.HttpServletRequest request) {
    String regex = request.getParameter("regex");
    String input = request.getParameter("input");

    // GOOD: User input is sanitized before constructing the regex
    return input.matches(Pattern.quote(regex));
  }
}

References

java/ql/src/experimental/Security/CWE/CWE-078/CommandInjectionRuntimeExec.qhelp

Command Injection into Runtime.exec() with dangerous command

Code that passes remote user input to an arugment of a call of Runtime.exec that executes a scripting executable will allow the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command or script to run, or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to the array going into Runtime.exec without examining it first.

class Test {
    public static void main(String[] args) {
        String script = System.getenv("SCRIPTNAME");
        if (script != null) {
            // BAD: The script to be executed by /bin/sh is controlled by the user.
            Runtime.getRuntime().exec(new String[]{"/bin/sh", script});
        }
    }
}

References

java/ql/src/experimental/Security/CWE/CWE-078/CommandInjectionRuntimeExecLocal.qhelp

Command Injection into Runtime.exec() with dangerous command

Code that passes local user input to an arugment of a call of Runtime.exec that executes a scripting executable will allow the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command or script to run, or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to the array going into Runtime.exec without examining it first.

class Test {
    public static void main(String[] args) {
        String script = System.getenv("SCRIPTNAME");
        if (script != null) {
            // BAD: The script to be executed by /bin/sh is controlled by the user.
            Runtime.getRuntime().exec(new String[]{"/bin/sh", script});
        }
    }
}

References

java/ql/src/experimental/Security/CWE/CWE-078/ExecTainted.qhelp

Uncontrolled command line (experimental sinks)

Code that passes user input directly to Runtime.exec, or some other library routine that executes a command, allows the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command to run or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to Runtime.exec without examining it first.

class Test {
    public static void main(String[] args) {
        String script = System.getenv("SCRIPTNAME");
        if (script != null) {
            // BAD: The script to be executed is controlled by the user.
            Runtime.getRuntime().exec(script);
        }
    }
}

References

javascript/ql/src/Performance/PolynomialReDoS.qhelp

Polynomial regular expression used on uncontrolled data

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engines provided by many popular JavaScript platforms use backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this use of a regular expression, which removes all leading and trailing whitespace in a string:

text.replace(/^\s+|\s+$/g, ''); // BAD

The sub-expression "\s+$" will match the whitespace characters in text from left to right, but it can start matching anywhere within a whitespace sequence. This is problematic for strings that do not end with a whitespace character. Such a string will force the regular expression engine to process each whitespace sequence once per whitespace character in the sequence.

This ultimately means that the time cost of trimming a string is quadratic in the length of the string. So a string like "a b" will take milliseconds to process, but a similar string with a million spaces instead of just one will take several minutes.

Avoid this problem by rewriting the regular expression to not contain the ambiguity about when to start matching whitespace sequences. For instance, by using a negative look-behind (/^\s+|(?<!\s)\s+$/g), or just by using the built-in trim method (text.trim()).

Note that the sub-expression "^\s+" is not problematic as the ^ anchor restricts when that sub-expression can start matching, and as the regular expression engine matches from left to right.

Example

As a similar, but slightly subtler problem, consider the regular expression that matches lines with numbers, possibly written using scientific notation:

/^0\.\d+E?\d+$/.test(str) // BAD

The problem with this regular expression is in the sub-expression \d+E?\d+ because the second \d+ can start matching digits anywhere after the first match of the first \d+ if there is no E in the input string.

This is problematic for strings that do not end with a digit. Such a string will force the regular expression engine to process each digit sequence once per digit in the sequence, again leading to a quadratic time complexity.

To make the processing faster, the regular expression should be rewritten such that the two \d+ sub-expressions do not have overlapping matches: ^0\.\d+(E\d+)?$.

Example

Sometimes it is unclear how a regular expression can be rewritten to avoid the problem. In such cases, it often suffices to limit the length of the input string. For instance, the following regular expression is used to match numbers, and on some non-number inputs it can have quadratic time complexity:

/^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$/.test(str) // BAD

It is not immediately obvious how to rewrite this regular expression to avoid the problem. However, you can mitigate performance issues by limiting the length to 1000 characters, which will always finish in a reasonable amount of time.

if (str.length > 1000) {
    throw new Error("Input too long");
}

/^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$/.test(str)

References

javascript/ql/src/Performance/ReDoS.qhelp

Inefficient regular expression

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engines provided by many popular JavaScript platforms use backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this regular expression:

/^_(__|.)+_$/

Its sub-expression "(__|.)+?" can match the string "__" either by the first alternative "__" to the left of the "|" operator, or by two repetitions of the second alternative "." to the right. Thus, a string consisting of an odd number of underscores followed by some other character will cause the regular expression engine to run for an exponential amount of time before rejecting the input.

This problem can be avoided by rewriting the regular expression to remove the ambiguity between the two branches of the alternative inside the repetition:

/^_(__|[^_])+_$/

References

javascript/ql/src/Security/CWE-078/CommandInjection.qhelp

Uncontrolled command line

Code that passes untrusted user input directly to child_process.exec or similar APIs that execute shell commands allows the user to execute malicious code.

Recommendation

If possible, use APIs that don't run shell commands and that accept command arguments as an array of strings rather than a single concatenated string. This is both safer and more portable.

If given arguments as a single string, avoid simply splitting the string on whitespace. Arguments may contain quoted whitespace, causing them to split into multiple arguments. Use a library like shell-quote to parse the string into an array of arguments instead.

If this approach is not viable, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that extracts a filename from an HTTP query parameter that may contain untrusted data, and then embeds it into a shell command to count its lines without examining it first:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    cp.execSync(`wc -l ${file}`); // BAD
});

A malicious user can take advantage of this code by executing arbitrary shell commands. For example, by providing a filename like foo.txt; rm -rf ., the user can first count the lines in foo.txt and subsequently delete all files in the current directory.

To avoid this catastrophic behavior, use an API such as child_process.execFileSync that does not spawn a shell by default:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    cp.execFileSync('wc', ['-l', file]); // GOOD
});

If you want to allow the user to specify other options to wc, you can use a library like shell-quote to parse the user input into an array of arguments without risking command injection:

var cp = require("child_process"),
    http = require('http'),
    url = require('url'),
    shellQuote = require('shell-quote');

var server = http.createServer(function(req, res) {
    let options = url.parse(req.url, true).query.options;

    cp.execFileSync('wc', shellQuote.parse(options)); // GOOD
});

Alternatively, the original example can be made safe by checking the filename against an allowlist of safe characters before using it:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    // only allow safe characters in file name
    if (file.match(/^[\w\.\-\/]+$/)) {
        cp.execSync(`wc -l ${file}`); // GOOD
    }
});

References

javascript/ql/src/Security/CWE-078/IndirectCommandInjection.qhelp

Indirect uncontrolled command line

Forwarding command-line arguments to child_process.exec or some other library routine that executes a system command within a shell can change the meaning of the command unexpectedly due to unescaped special characters.

When the forwarded command-line arguments come from a parent process that has not escaped the special characters in the arguments, then the parent process may indirectly be vulnerable to command-line injection since the special characters are evaluated unexpectedly.

Recommendation

If possible, use APIs that don't run shell commands and accept command arguments as an array of strings rather than a single concatenated string. This is both safer and more portable.

If given arguments as a single string, avoid simply splitting the string on whitespace. Arguments may contain quoted whitespace, causing them to split into multiple arguments. Use a library like shell-quote to parse the string into an array of arguments instead.

If this approach is not viable, then add code to verify that each forwarded command-line argument is properly escaped before using it.

Example

The following wrapper script example executes another JavaScript file in a child process and forwards some command-line arguments. This is problematic because the special characters in the command-line arguments may change the meaning of the child process invocation unexpectedly. For instance, if one of the command-line arguments is "dollar$separated$name", then the child process will substitute the two environment variables $separated and $name before invoking node.

var cp = require("child_process");

const args = process.argv.slice(2);
const script = path.join(__dirname, 'bin', 'main.js');
cp.execSync(`node ${script} ${args.join(' ')}`); // BAD

If another program uses child_process.execFile to invoke the above wrapper script with input from a remote user, then there may be a command-line injection vulnerability. This may be surprising, since a command-line invocation with child_process.execFile is generally considered safe. But in this case, the remote user input is simply forwarded to the problematic process.exec call in the wrapper script.

To guard against this, use an API that does not perform environment variable substitution, such as child_process.execFile:

var cp = require("child_process");

const args = process.argv.slice(2);
const script = path.join(__dirname, 'bin', 'main.js');
cp.execFileSync('node', [script].concat(args)); // GOOD

If you want to allow the user to specify other options to node, you can use a library like shell-quote to parse the user input into an array of arguments without risking command injection:

var cp = require("child_process"),
    shellQuote = require("shell-quote");

const args = process.argv.slice(2);
let nodeOpts = '';
if (args[0] === '--node-opts') {
    nodeOpts = args[1];
    args.splice(0, 2);
}
const script = path.join(__dirname, 'bin', 'main.js');
cp.execFileSync('node', shellQuote.parse(nodeOpts).concat(script).concat(args)); // GOOD

References

javascript/ql/src/Security/CWE-078/SecondOrderCommandInjection.qhelp

Second order command injection

Some shell commands, like git ls-remote, can execute arbitrary commands if a user provides a malicious URL that starts with --upload-pack. This can be used to execute arbitrary code on the server.

Recommendation

Sanitize user input before passing it to the shell command. For example, ensure that URLs are valid and do not contain malicious commands.

Example

The following example shows code that executes git ls-remote on a URL that can be controlled by a malicious user.

const express = require("express");
const app = express();

const cp = require("child_process");

app.get("/ls-remote", (req, res) => {
  const remote = req.query.remote;
  cp.execFile("git", ["ls-remote", remote]); // NOT OK
});

The problem has been fixed in the snippet below, where the URL is validated before being passed to the shell command.

const express = require("express");
const app = express();

const cp = require("child_process");

app.get("/ls-remote", (req, res) => {
  const remote = req.query.remote;
  if (!(remote.startsWith("git@") || remote.startsWith("https://"))) {
    throw new Error("Invalid remote: " + remote);
  }
  cp.execFile("git", ["ls-remote", remote]); // OK
});

References

javascript/ql/src/Security/CWE-078/ShellCommandInjectionFromEnvironment.qhelp

Shell command built from environment values

Dynamically constructing a shell command with values from the local environment, such as file paths, may inadvertently change the meaning of the shell command. Such changes can occur when an environment value contains characters that the shell interprets in a special way, for instance quotes and spaces. This can result in the shell command misbehaving, or even allowing a malicious user to execute arbitrary commands on the system.

Recommendation

If possible, use hard-coded string literals to specify the shell command to run, and provide the dynamic arguments to the shell command separately to avoid interpretation by the shell.

Alternatively, if the shell command must be constructed dynamically, then add code to ensure that special characters in environment values do not alter the shell command unexpectedly.

Example

The following example shows a dynamically constructed shell command that recursively removes a temporary directory that is located next to the currently executing JavaScript file. Such utilities are often found in custom build scripts.

var cp = require("child_process"),
  path = require("path");
function cleanupTemp() {
  let cmd = "rm -rf " + path.join(__dirname, "temp");
  cp.execSync(cmd); // BAD
}

The shell command will, however, fail to work as intended if the absolute path of the script's directory contains spaces. In that case, the shell command will interpret the absolute path as multiple paths, instead of a single path.

For instance, if the absolute path of the temporary directory is /home/username/important project/temp, then the shell command will recursively delete /home/username/important and project/temp, where the latter path gets resolved relative to the working directory of the JavaScript process.

Even worse, although less likely, a malicious user could provide the path /home/username/; cat /etc/passwd #/important project/temp in order to execute the command cat /etc/passwd.

To avoid such potentially catastrophic behaviors, provide the directory as an argument that does not get interpreted by a shell:

var cp = require("child_process"),
  path = require("path");
function cleanupTemp() {
  let cmd = "rm",
    args = ["-rf", path.join(__dirname, "temp")];
  cp.execFileSync(cmd, args); // GOOD
}

References

javascript/ql/src/Security/CWE-078/UnsafeShellCommandConstruction.qhelp

Unsafe shell command constructed from library input

Dynamically constructing a shell command with inputs from exported functions may inadvertently change the meaning of the shell command. Clients using the exported function may use inputs containing characters that the shell interprets in a special way, for instance quotes and spaces. This can result in the shell command misbehaving, or even allowing a malicious user to execute arbitrary commands on the system.

Recommendation

If possible, provide the dynamic arguments to the shell as an array using a safe API such as child_process.execFile to avoid interpretation by the shell.

If given arguments as a single string, avoid simply splitting the string on whitespace. Arguments may contain quoted whitespace, causing them to split into multiple arguments. Use a library like shell-quote to parse the string into an array of arguments instead.

Alternatively, if the command must be interpreted by a shell (for example because it includes I/O redirections), you can use shell-quote to escape any special characters in the input before embedding it in the command.

Example

The following example shows a dynamically constructed shell command that downloads a file from a remote URL.

var cp = require("child_process");

module.exports = function download(path, callback) {
  cp.exec("wget " + path, callback);
}

The shell command will, however, fail to work as intended if the input contains spaces or other special characters interpreted in a special way by the shell.

Even worse, a client might pass in user-controlled data, not knowing that the input is interpreted as a shell command. This could allow a malicious user to provide the input http://example.org; cat /etc/passwd in order to execute the command cat /etc/passwd.

To avoid such potentially catastrophic behaviors, provide the inputs from exported functions as an argument that does not get interpreted by a shell:

var cp = require("child_process");

module.exports = function download(path, callback) {
  cp.execFile("wget", [path], callback);
}

As another example, consider the following code which is similar to the preceding example, but pipes the output of wget into wc -l to count the number of lines in the downloaded file.

var cp = require("child_process");

module.exports = function download(path, callback) {
  cp.exec("wget " + path + " | wc -l", callback);
};

In this case, using child_process.execFile is not an option because the shell is needed to interpret the pipe operator. Instead, you can use shell-quote to escape the input before embedding it in the command:

var cp = require("child_process");

module.exports = function download(path, callback) {
  cp.exec("wget " + shellQuote.quote([path]) + " | wc -l", callback);
};

References

javascript/ql/src/Security/CWE-078/UselessUseOfCat.qhelp

Unnecessary use of cat process

Using the unix command cat only to read a file is an unnecessarily complex way to achieve something that can be done in a simpler and safer manner using the Node.js fs.readFile API.

The use of cat for simple file reads leads to code that is unportable, inefficient, complex, and can lead to subtle bugs or even security vulnerabilities.

Recommendation

Use fs.readFile or fs.readFileSync to read files from the file system.

Example

The following example shows code that reads a file using cat:

var child_process = require('child_process');

module.exports = function (name) {
    return child_process.execSync("cat " + name).toString();
};

The code in the example will break if the input name contains special characters (including space). Additionally, it does not work on Windows and if the input is user-controlled, a command injection attack can happen.

The fs.readFile API should be used to avoid these potential issues:

var fs = require('fs');

module.exports = function (name) {
    return fs.readFileSync(name).toString();
};

References

javascript/ql/src/Security/CWE-730/RegExpInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous as a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as lodash's _.escapeRegExp to escape meta-characters that have special meaning.

Example

The following example shows a HTTP request parameter that is used to construct a regular expression without sanitizing it first:

var express = require('express');
var app = express();

app.get('/findKey', function(req, res) {
  var key = req.param("key"), input = req.param("input");

  // BAD: Unsanitized user input is used to construct a regular expression
  var re = new RegExp("\\b" + key + "=(.*)\n");
});

Instead, the request parameter should be sanitized first, for example using the function _.escapeRegExp from the lodash package. This ensures that the user cannot insert characters which have a special meaning in regular expressions.

var express = require('express');
var _ = require('lodash');
var app = express();

app.get('/findKey', function(req, res) {
  var key = req.param("key"), input = req.param("input");

  // GOOD: User input is sanitized before constructing the regex
  var safeKey = _.escapeRegExp(key);
  var re = new RegExp("\\b" + safeKey + "=(.*)\n");
});

References

javascript/ql/src/experimental/heuristics/ql/src/Security/CWE-078/CommandInjection.qhelp

Uncontrolled command line with additional heuristic sources

Code that passes untrusted user input directly to child_process.exec or similar APIs that execute shell commands allows the user to execute malicious code.

Recommendation

If possible, use APIs that don't run shell commands and that accept command arguments as an array of strings rather than a single concatenated string. This is both safer and more portable.

If given arguments as a single string, avoid simply splitting the string on whitespace. Arguments may contain quoted whitespace, causing them to split into multiple arguments. Use a library like shell-quote to parse the string into an array of arguments instead.

If this approach is not viable, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that extracts a filename from an HTTP query parameter that may contain untrusted data, and then embeds it into a shell command to count its lines without examining it first:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    cp.execSync(`wc -l ${file}`); // BAD
});

A malicious user can take advantage of this code by executing arbitrary shell commands. For example, by providing a filename like foo.txt; rm -rf ., the user can first count the lines in foo.txt and subsequently delete all files in the current directory.

To avoid this catastrophic behavior, use an API such as child_process.execFileSync that does not spawn a shell by default:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    cp.execFileSync('wc', ['-l', file]); // GOOD
});

If you want to allow the user to specify other options to wc, you can use a library like shell-quote to parse the user input into an array of arguments without risking command injection:

var cp = require("child_process"),
    http = require('http'),
    url = require('url'),
    shellQuote = require('shell-quote');

var server = http.createServer(function(req, res) {
    let options = url.parse(req.url, true).query.options;

    cp.execFileSync('wc', shellQuote.parse(options)); // GOOD
});

Alternatively, the original example can be made safe by checking the filename against an allowlist of safe characters before using it:

var cp = require("child_process"),
    http = require('http'),
    url = require('url');

var server = http.createServer(function(req, res) {
    let file = url.parse(req.url, true).query.path;

    // only allow safe characters in file name
    if (file.match(/^[\w\.\-\/]+$/)) {
        cp.execSync(`wc -l ${file}`); // GOOD
    }
});

References

javascript/ql/src/experimental/heuristics/ql/src/Security/CWE-730/RegExpInjection.qhelp

Regular expression injection with additional heuristic sources

Constructing a regular expression with unsanitized user input is dangerous as a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as lodash's _.escapeRegExp to escape meta-characters that have special meaning.

Example

The following example shows a HTTP request parameter that is used to construct a regular expression without sanitizing it first:

var express = require('express');
var app = express();

app.get('/findKey', function(req, res) {
  var key = req.param("key"), input = req.param("input");

  // BAD: Unsanitized user input is used to construct a regular expression
  var re = new RegExp("\\b" + key + "=(.*)\n");
});

Instead, the request parameter should be sanitized first, for example using the function _.escapeRegExp from the lodash package. This ensures that the user cannot insert characters which have a special meaning in regular expressions.

var express = require('express');
var _ = require('lodash');
var app = express();

app.get('/findKey', function(req, res) {
  var key = req.param("key"), input = req.param("input");

  // GOOD: User input is sanitized before constructing the regex
  var safeKey = _.escapeRegExp(key);
  var re = new RegExp("\\b" + safeKey + "=(.*)\n");
});

References

python/ql/src/Security/CWE-078/CommandInjection.qhelp

Uncontrolled command line

Code that passes user input directly to exec, eval, or some other library routine that executes a command, allows the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command to run or the library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows two functions. The first is unsafe as it takes a shell script that can be changed by a user, and passes it straight to subprocess.call() without examining it first. The second is safe as it selects the command from a predefined allowlist.

urlpatterns = [
    # Route to command_execution
    url(r'^command-ex1$', command_execution_unsafe, name='command-execution-unsafe'),
    url(r'^command-ex2$', command_execution_safe, name='command-execution-safe')
]

COMMANDS = {
    "list" :"ls",
    "stat" : "stat"
}

def command_execution_unsafe(request):
    if request.method == 'POST':
        action = request.POST.get('action', '')
        #BAD -- No sanitizing of input
        subprocess.call(["application", action])

def command_execution_safe(request):
    if request.method == 'POST':
        action = request.POST.get('action', '')
        #GOOD -- Use an allowlist
        subprocess.call(["application", COMMANDS[action]])

References

python/ql/src/Security/CWE-078/UnsafeShellCommandConstruction.qhelp

Unsafe shell command constructed from library input

Dynamically constructing a shell command with inputs from library functions may inadvertently change the meaning of the shell command. Clients using the exported function may use inputs containing characters that the shell interprets in a special way, for instance quotes and spaces. This can result in the shell command misbehaving, or even allowing a malicious user to execute arbitrary commands on the system.

Recommendation

If possible, provide the dynamic arguments to the shell as an array to APIs such as subprocess.run to avoid interpretation by the shell.

Alternatively, if the shell command must be constructed dynamically, then add code to ensure that special characters do not alter the shell command unexpectedly.

Example

The following example shows a dynamically constructed shell command that downloads a file from a remote URL.

import os

def download(path): 
    os.system("wget " + path) # NOT OK

The shell command will, however, fail to work as intended if the input contains spaces or other special characters interpreted in a special way by the shell.

Even worse, a client might pass in user-controlled data, not knowing that the input is interpreted as a shell command. This could allow a malicious user to provide the input http://example.org; cat /etc/passwd in order to execute the command cat /etc/passwd.

To avoid such potentially catastrophic behaviors, provide the input from library functions as an argument that does not get interpreted by a shell:

import subprocess

def download(path): 
    subprocess.run(["wget", path]) # OK

References

python/ql/src/Security/CWE-730/PolynomialReDoS.qhelp

Polynomial regular expression used on uncontrolled data

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine provided by Python uses a backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this use of a regular expression, which removes all leading and trailing whitespace in a string:

re.sub(r"^\s+|\s+$", "", text) # BAD

The sub-expression "\s+$" will match the whitespace characters in text from left to right, but it can start matching anywhere within a whitespace sequence. This is problematic for strings that do not end with a whitespace character. Such a string will force the regular expression engine to process each whitespace sequence once per whitespace character in the sequence.

This ultimately means that the time cost of trimming a string is quadratic in the length of the string. So a string like "a b" will take milliseconds to process, but a similar string with a million spaces instead of just one will take several minutes.

Avoid this problem by rewriting the regular expression to not contain the ambiguity about when to start matching whitespace sequences. For instance, by using a negative look-behind (^\s+|(?<!\s)\s+$), or just by using the built-in strip method (text.strip()).

Note that the sub-expression "^\s+" is not problematic as the ^ anchor restricts when that sub-expression can start matching, and as the regular expression engine matches from left to right.

Example

As a similar, but slightly subtler problem, consider the regular expression that matches lines with numbers, possibly written using scientific notation:

^0\.\d+E?\d+$ # BAD

The problem with this regular expression is in the sub-expression \d+E?\d+ because the second \d+ can start matching digits anywhere after the first match of the first \d+ if there is no E in the input string.

This is problematic for strings that do not end with a digit. Such a string will force the regular expression engine to process each digit sequence once per digit in the sequence, again leading to a quadratic time complexity.

To make the processing faster, the regular expression should be rewritten such that the two \d+ sub-expressions do not have overlapping matches: ^0\.\d+(E\d+)?$.

Example

Sometimes it is unclear how a regular expression can be rewritten to avoid the problem. In such cases, it often suffices to limit the length of the input string. For instance, the following regular expression is used to match numbers, and on some non-number inputs it can have quadratic time complexity:

match = re.search(r'^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$', str) 

It is not immediately obvious how to rewrite this regular expression to avoid the problem. However, you can mitigate performance issues by limiting the length to 1000 characters, which will always finish in a reasonable amount of time.

if len(str) > 1000:
    raise ValueError("Input too long")

match = re.search(r'^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$', str) 

References

python/ql/src/Security/CWE-730/ReDoS.qhelp

Inefficient regular expression

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine provided by Python uses a backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this regular expression:

^_(__|.)+_$

Its sub-expression "(__|.)+?" can match the string "__" either by the first alternative "__" to the left of the "|" operator, or by two repetitions of the second alternative "." to the right. Thus, a string consisting of an odd number of underscores followed by some other character will cause the regular expression engine to run for an exponential amount of time before rejecting the input.

This problem can be avoided by rewriting the regular expression to remove the ambiguity between the two branches of the alternative inside the repetition:

^_(__|[^_])+_$

References

python/ql/src/Security/CWE-730/RegexInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous as a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as re.escape to escape meta-characters that have a special meaning regarding regular expressions' syntax.

Example

The following examples are based on a simple Flask web server environment.

The following example shows a HTTP request parameter that is used to construct a regular expression without sanitizing it first:

from flask import request, Flask
import re


@app.route("/direct")
def direct():
    unsafe_pattern = request.args["pattern"]
    re.search(unsafe_pattern, "")


@app.route("/compile")
def compile():
    unsafe_pattern = request.args["pattern"]
    compiled_pattern = re.compile(unsafe_pattern)
    compiled_pattern.search("")

Instead, the request parameter should be sanitized first, for example using the function re.escape. This ensures that the user cannot insert characters which have a special meaning in regular expressions.

from flask import request, Flask
import re


@app.route("/direct")
def direct():
    unsafe_pattern = request.args['pattern']
    safe_pattern = re.escape(unsafe_pattern)
    re.search(safe_pattern, "")


@app.route("/compile")
def compile():
    unsafe_pattern = request.args['pattern']
    safe_pattern = re.escape(unsafe_pattern)
    compiled_pattern = re.compile(safe_pattern)
    compiled_pattern.search("")

References

ruby/ql/src/queries/security/cwe-078/CommandInjection.qhelp

Uncontrolled command line

Code that passes user input directly to Kernel.system, Kernel.exec, or some other library routine that executes a command, allows the user to execute malicious code.

Recommendation

If possible, use hard-coded string literals to specify the command to run or library to load. Instead of passing the user input directly to the process or library function, examine the user input and then choose among hard-coded string literals.

If the applicable libraries or commands cannot be determined at compile time, then add code to verify that the user input string is safe before using it.

Example

The following example shows code that takes a shell script that can be changed maliciously by a user, and passes it straight to Kernel.system without examining it first.

class UsersController < ActionController::Base
  def create
    command = params[:command]
    system(command) # BAD
  end
end

References

ruby/ql/src/queries/security/cwe-078/KernelOpen.qhelp

Use of Kernel.open, IO.read or similar sinks with user-controlled input

If Kernel.open is given a file name that starts with a | character, it will execute the remaining string as a shell command. If a malicious user can control the file name, they can execute arbitrary code. The same vulnerability applies to IO.read, IO.write, IO.binread, IO.binwrite, IO.foreach, IO.readlines and URI.open.

Recommendation

Use File.open instead of Kernel.open, as the former does not have this vulnerability. Similarly, use the methods from the File class instead of the IO class e.g. File.read instead of IO.read.

Instead of URI.open use URI(..).open or an HTTP Client.

Example

The following example shows code that calls Kernel.open on a user-supplied file path.

require "open-uri"

class UsersController < ActionController::Base
  def create
    filename = params[:filename]
    open(filename) # BAD

    web_page = params[:web_page]
    URI.open(web_page) # BAD - calls `Kernel.open` internally
  end
end

Instead, File.open should be used, as in the following example.

class UsersController < ActionController::Base
  def create
    filename = params[:filename]
    File.open(filename)

    web_page = params[:web_page]
    Net::HTTP.get(URI.parse(web_page))
  end
end

References

ruby/ql/src/queries/security/cwe-078/NonConstantKernelOpen.qhelp

Use of Kernel.open or IO.read or similar sinks with a non-constant value

If Kernel.open is given a file name that starts with a | character, it will execute the remaining string as a shell command. If a malicious user can control the file name, they can execute arbitrary code. The same vulnerability applies to IO.read, IO.write, IO.binread, IO.binwrite, IO.foreach, IO.readlines and URI.open.

Recommendation

Use File.open instead of Kernel.open, as the former does not have this vulnerability. Similarly, use the methods from the File class instead of the IO class e.g. File.read instead of IO.read.

Instead of URI.open use URI(..).open or an HTTP Client.

Example

The following example shows code that calls Kernel.open on a user-supplied file path.

require "open-uri"

class UsersController < ActionController::Base
  def create
    filename = params[:filename]
    open(filename) # BAD

    web_page = params[:web_page]
    URI.open(web_page) # BAD - calls `Kernel.open` internally
  end
end

Instead, File.open should be used, as in the following example.

class UsersController < ActionController::Base
  def create
    filename = params[:filename]
    File.open(filename)

    web_page = params[:web_page]
    Net::HTTP.get(URI.parse(web_page))
  end
end

References

ruby/ql/src/queries/security/cwe-078/UnsafeShellCommandConstruction.qhelp

Unsafe shell command constructed from library input

Dynamically constructing a shell command with inputs from exported functions may inadvertently change the meaning of the shell command. Clients using the exported function may use inputs containing characters that the shell interprets in a special way, for instance quotes and spaces. This can result in the shell command misbehaving, or even allowing a malicious user to execute arbitrary commands on the system.

Recommendation

If possible, avoid concatenating shell strings to APIs such as system(..) to avoid interpretation by the shell.

Instead, provide the arguments to the shell command as separate arguments to the API, such as system("echo", arg1, arg2).

Alternatively, if the shell command must be constructed dynamically, then add code to ensure that special characters do not alter the shell command unexpectedly.

Example

The following example shows a dynamically constructed shell command that downloads a file from a remote URL.

module Utils 
    def download(path)
        system("wget #{path}") # NOT OK
    end
end

The shell command will, however, fail to work as intended if the input contains spaces or other special characters interpreted in a special way by the shell.

Even worse, a client might pass in user-controlled data, not knowing that the input is interpreted as a shell command. This could allow a malicious user to provide the input http://example.org; cat /etc/passwd in order to execute the command cat /etc/passwd.

To avoid such potentially catastrophic behaviors, provide the input from exported functions as an argument that does not get interpreted by a shell:

module Utils 
    def download(path)
        # using an API that doesn't interpret the path as a shell command
        system("wget", path) # OK
    end
end

References

ruby/ql/src/queries/security/cwe-1333/PolynomialReDoS.qhelp

Polynomial regular expression used on uncontrolled data

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine used by the Ruby interpreter (MRI) uses backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Note that Ruby 3.2 and later have implemented a caching mechanism that completely eliminates the worst-case time complexity for the regular expressions flagged by this query. The regular expressions flagged by this query are therefore only problematic for Ruby versions prior to 3.2.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this use of a regular expression, which removes all leading and trailing whitespace in a string:

text.gsub!(/^\s+|\s+$/, '') # BAD

The sub-expression "\s+$" will match the whitespace characters in text from left to right, but it can start matching anywhere within a whitespace sequence. This is problematic for strings that do not end with a whitespace character. Such a string will force the regular expression engine to process each whitespace sequence once per whitespace character in the sequence.

This ultimately means that the time cost of trimming a string is quadratic in the length of the string. So a string like "a b" will take milliseconds to process, but a similar string with a million spaces instead of just one will take several minutes.

Avoid this problem by rewriting the regular expression to not contain the ambiguity about when to start matching whitespace sequences. For instance, by using a negative look-behind (/^\s+|(?<!\s)\s+$/), or just by using the built-in strip method (text.strip!).

Note that the sub-expression "^\s+" is not problematic as the ^ anchor restricts when that sub-expression can start matching, and as the regular expression engine matches from left to right.

Example

As a similar, but slightly subtler problem, consider the regular expression that matches lines with numbers, possibly written using scientific notation:

/^0\.\d+E?\d+$/ # BAD

The problem with this regular expression is in the sub-expression \d+E?\d+ because the second \d+ can start matching digits anywhere after the first match of the first \d+ if there is no E in the input string.

This is problematic for strings that do not end with a digit. Such a string will force the regular expression engine to process each digit sequence once per digit in the sequence, again leading to a quadratic time complexity.

To make the processing faster, the regular expression should be rewritten such that the two \d+ sub-expressions do not have overlapping matches: /^0\.\d+(E\d+)?$/.

Example

Sometimes it is unclear how a regular expression can be rewritten to avoid the problem. In such cases, it often suffices to limit the length of the input string. For instance, the following regular expression is used to match numbers, and on some non-number inputs it can have quadratic time complexity:

is_matching = /^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$/.match?(str)

It is not immediately obvious how to rewrite this regular expression to avoid the problem. However, you can mitigate performance issues by limiting the length to 1000 characters, which will always finish in a reasonable amount of time.

if str.length > 1000
    raise ArgumentError, "Input too long"
end

is_matching = /^(\+|-)?(\d+|(\d*\.\d*))?(E|e)?([-+])?(\d+)?$/.match?(str)

References

ruby/ql/src/queries/security/cwe-1333/ReDoS.qhelp

Inefficient regular expression

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, or even allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine used by the Ruby interpreter (MRI) uses backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or even exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Note that Ruby 3.2 and later have implemented a caching mechanism that completely eliminates the worst-case time complexity for the regular expressions flagged by this query. The regular expressions flagged by this query are therefore only problematic for Ruby versions prior to 3.2.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time-complexity does not matter.

Example

Consider this regular expression:

/^_(__|.)+_$/

Its sub-expression "(__|.)+?" can match the string "__" either by the first alternative "__" to the left of the "|" operator, or by two repetitions of the second alternative "." to the right. Thus, a string consisting of an odd number of underscores followed by some other character will cause the regular expression engine to run for an exponential amount of time before rejecting the input.

This problem can be avoided by rewriting the regular expression to remove the ambiguity between the two branches of the alternative inside the repetition:

/^_(__|[^_])+_$/

References

ruby/ql/src/queries/security/cwe-1333/RegExpInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous, since a malicious user may be able to modify the meaning of the expression. In particular, such a user may be able to provide a regular expression fragment that takes exponential time in the worst case, and use that to perform a Denial of Service attack.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as Regexp.escape to escape meta-characters that have special meaning.

Example

The following examples construct regular expressions from an HTTP request parameter without sanitizing it first:

class UsersController < ActionController::Base
  def first_example
    # BAD: Unsanitized user input is used to construct a regular expression
    regex = /#{ params[:key] }/
  end

  def second_example
    # BAD: Unsanitized user input is used to construct a regular expression
    regex = Regexp.new(params[:key])
  end
end

Instead, the request parameter should be sanitized first. This ensures that the user cannot insert characters that have special meanings in regular expressions.

class UsersController < ActionController::Base
  def example
    # GOOD: User input is sanitized before constructing the regular expression
    regex = Regexp.new(Regex.escape(params[:key]))
  end
end

References

rust/ql/src/queries/security/CWE-311/CleartextTransmission.qhelp

Cleartext transmission of sensitive information

Sensitive information that is transmitted without encryption may be accessible to an attacker.

Recommendation

Ensure that sensitive information is always encrypted before being transmitted over the network. In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext. Avoid transmitting sensitive information when it is not necessary to.

Example

The following example shows three cases of transmitting information. In the 'BAD' case, the transmitted data is sensitive (a credit card number) and is included as cleartext in the URL. URLs are often logged or otherwise visible in cleartext, and should not contain sensitive information.

In the 'GOOD' cases, the data is either not sensitive, or is protected with encryption. When encryption is used, ensure that you select a secure modern encryption algorithm, and put suitable key management practices into place.

func getData() {
	// ...

	// GOOD: not sensitive information
	let body = reqwest::get("https://example.com/song/{faveSong}").await?.text().await?;

	// BAD: sensitive information sent in cleartext in the URL
	let body = reqwest::get(format!("https://example.com/card/{creditCardNo}")).await?.text().await?;

	// GOOD: encrypted sensitive information sent in the URL
	let encryptedPassword = encrypt(password, encryptionKey);
	let body = reqwest::get(format!("https://example.com/card/{creditCardNo}")).await?.text().await?;

	// ...
}

References

rust/ql/src/queries/security/CWE-312/CleartextStorageDatabase.qhelp

Cleartext storage of sensitive information in a database

Sensitive information that is stored unencrypted in a database is accessible to an attacker who gains access to that database. For example, the information could be accessed by any process or user in a rooted device, or exposed through another vulnerability.

Recommendation

Either encrypt the entire database, or ensure that each piece of sensitive information is encrypted before being stored. In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext. Avoid storing sensitive information at all if you do not need to keep it.

Example

The following example stores sensitive information into a database without encryption, using the SQLx library:

let query = "INSERT INTO PAYMENTDETAILS(ID, CARDNUM) VALUES(?, ?)";
let result = sqlx::query(query)
	.bind(id)
	.bind(credit_card_number) // BAD: Cleartext storage of sensitive data in the database
	.execute(pool)
	.await?;

This is insecure because the sensitive data is stored in cleartext, making it accessible to anyone with access to the database.

To fix this, we can either encrypt the entire database or encrypt just the sensitive data before it is stored. Take care to select a secure modern encryption algorithm and put suitable key management practices into place. In the following example, we have encrypted the sensitive data using 256-bit AES before storing it in the database:

fn encrypt(text: String, encryption_key: &aes_gcm::Key<Aes256Gcm>) -> String {
    // encrypt text -> ciphertext
    let cipher = Aes256Gcm::new(&encryption_key);
    let nonce = Aes256Gcm::generate_nonce(&mut OsRng);
    let ciphertext = cipher.encrypt(&nonce, text.as_ref()).unwrap();

    // append (nonce, ciphertext)
    let mut combined = nonce.to_vec();
    combined.extend(ciphertext);

    // encode to base64 string
    BASE64_STANDARD.encode(combined)
}

fn decrypt(data: String, encryption_key: &aes_gcm::Key<Aes256Gcm>) -> String {
    let cipher = Aes256Gcm::new(&encryption_key);

    // decode base64 string
    let decoded = BASE64_STANDARD.decode(data).unwrap();

    // split into (nonce, ciphertext)
    let nonce_size = <Aes256Gcm as AeadCore>::NonceSize::to_usize();
    let (nonce, ciphertext) = decoded.split_at(nonce_size);

    // decrypt ciphertext -> plaintext
    let plaintext = cipher.decrypt(nonce.into(), ciphertext).unwrap();
    String::from_utf8(plaintext).unwrap()
}

...

let encryption_key = Aes256Gcm::generate_key(OsRng);

...

let query = "INSERT INTO PAYMENTDETAILS(ID, CARDNUM) VALUES(?, ?)";
let result = sqlx::query(query)
	.bind(id)
	.bind(encrypt(credit_card_number, &encryption_key)) // GOOD: Encrypted storage of sensitive data in the database
	.execute(pool)
	.await?;

References

rust/ql/src/queries/security/CWE-502/UnsafeDeserialization.qhelp

Deserialization of user-controlled data

Deserializing untrusted data without validation can allow an attacker to cause denial of service, consume excessive resources, or in some cases execute arbitrary code. Rust's memory safety mitigates some risks, but deserializing untrusted data with libraries likeserde, bincode, or rmp-serde can still lead to panics, excessive memory allocation, or logic bugs. This is especially true when trait objects or polymorphic types are involved.

Recommendation

Avoid deserializing untrusted data with formats that allow unbounded allocation or polymorphic dispatch. Where possible, prefer formats with schema validation, such as Protocol Buffers, when processing untrusted input. If you do use serde, consider:

  • Validating input size before deserialization.
  • Using #[serde(deny_unknown_fields)] to reject unexpected data.
  • Avoiding #[typetag] or trait object deserialization with untrusted input.
  • Using bounded containers (e.g., limiting Vec length via custom deserializers).

Example

In the following example, data from an HTTP request is directly deserialized without any validation. An attacker could send a crafted payload that causes excessive memory allocation or other unintended behavior.

use serde::Deserialize;

#[derive(Deserialize)]
struct UserData {
    name: String,
    items: Vec<String>,
}

fn handle_request(body: &[u8]) -> UserData {
    // BAD: deserializing user-controlled data without size validation
    serde_json::from_slice(body).unwrap()
}

A safer approach validates the input size and uses strict deserialization settings:

use serde::Deserialize;

const MAX_BODY_SIZE: usize = 1024 * 1024; // 1 MB limit

#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct UserData {
    name: String,
    #[serde(deserialize_with = "bounded_vec")]
    items: Vec<String>,
}

fn bounded_vec<'de, D>(deserializer: D) -> Result<Vec<String>, D::Error>
where
    D: serde::Deserializer<'de>,
{
    let v = Vec::<String>::deserialize(deserializer)?;
    if v.len() > 100 {
        return Err(serde::de::Error::custom("too many items"));
    }
    Ok(v)
}

fn handle_request(body: &[u8]) -> Result<UserData, String> {
    // GOOD: validate input size before deserialization, use bounded containers
    if body.len() > MAX_BODY_SIZE {
        return Err("payload too large".to_string());
    }
    serde_json::from_slice(body).map_err(|e| e.to_string())
}

References

swift/ql/src/queries/Security/CWE-078/CommandInjection.qhelp

System command built from user-controlled sources

Constructing a system command with unsanitized user input is dangerous, since a malicious user may be able to craft input that executes arbitrary code.

Recommendation

If possible, use hard-coded string literals to specify the command to run. Instead of interpreting user input directly as command names, examine the input and then choose among hard-coded string literals.

If this is not possible, then add sanitization code to verify that the user input is safe before using it.

Example

The following example executes code from user input without sanitizing it first:

var task = Process()
task.launchPath = "/bin/bash"
task.arguments = ["-c", userControlledString] // BAD

task.launch()

If user input is used to construct a command it should be checked first. This ensures that the user cannot insert characters that have special meanings:

func validateCommand(_ command: String) -> String? {
    let allowedCommands = ["ls -l", "pwd", "echo"]
    if allowedCommands.contains(command) {
        return command
    }
    return nil
}

if let validatedString = validateCommand(userControlledString) {
    var task = Process()
    task.launchPath = "/bin/bash"
    task.arguments = ["-c", validatedString] // GOOD

    task.launch()
}

References

swift/ql/src/queries/Security/CWE-1333/ReDoS.qhelp

Inefficient regular expression

Some regular expressions take a long time to match certain input strings to the point where the time it takes to match a string of length n is proportional to nk or even 2n. Such regular expressions can negatively affect performance, and potentially allow a malicious user to perform a Denial of Service ("DoS") attack by crafting an expensive input string for the regular expression to match.

The regular expression engine used by Swift uses backtracking non-deterministic finite automata to implement regular expression matching. While this approach is space-efficient and allows supporting advanced features like capture groups, it is not time-efficient in general. The worst-case time complexity of such an automaton can be polynomial or exponential, meaning that for strings of a certain shape, increasing the input length by ten characters may make the automaton about 1000 times slower.

Typically, a regular expression is affected by this problem if it contains a repetition of the form r* or r+ where the sub-expression r is ambiguous in the sense that it can match some string in multiple ways. More information about the precise circumstances can be found in the references.

Recommendation

Modify the regular expression to remove the ambiguity, or ensure that the strings matched with the regular expression are short enough that the time complexity does not matter.

Example

Consider the following regular expression:


/^_(__|.)+_$/

Its sub-expression "(__|.)+" can match the string "__" either by the first alternative "__" to the left of the "|" operator, or by two repetitions of the second alternative "." to the right. Therefore, a string consisting of an odd number of underscores followed by some other character will cause the regular expression engine to run for an exponential amount of time before rejecting the input.

This problem can be avoided by rewriting the regular expression to remove the ambiguity between the two branches of the alternative inside the repetition:


/^_(__|[^_])+_$/

References

swift/ql/src/queries/Security/CWE-311/CleartextStorageDatabase.qhelp

Cleartext storage of sensitive information in a local database

Sensitive information that is stored unencrypted in a database is accessible to an attacker who gains access to that database. For example, the information could be accessed by any process or user in a rooted device, or exposed through another vulnerability.

Recommendation

Either encrypt the entire database, or ensure that each piece of sensitive information is encrypted before being stored. In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext. Avoid storing sensitive information at all if you do not need to keep it.

Example

The following example shows three cases of storing information using the Core Data library. In the 'BAD' case, the data that is stored is sensitive (a credit card number) and is not encrypted. In the 'GOOD' cases, the data is either not sensitive, or is protected with encryption. When encryption is used, take care to select a secure modern encryption algorithm, and put suitable key management practices into place.

import CryptoKit

private func encrypt(_ text: String, _ encryptionKey: SymmetricKey) -> String {
	let sealedBox = try! AES.GCM.seal(Data(text.utf8), using: encryptionKey)
	return sealedBox.combined!.base64EncodedString()
}

func storeMyData(databaseObject : NSManagedObject, faveSong : String, creditCardNo : String, encryptionKey: SymmetricKey) {
	// ...

	// GOOD: not sensitive information
	databaseObject.setValue(faveSong, forKey: "myFaveSong")

	// BAD: sensitive information saved in cleartext
	databaseObject.setValue(creditCardNo, forKey: "myCreditCardNo")

	// GOOD: encrypted sensitive information saved
	databaseObject.setValue(encrypt(creditCardNo, encryptionKey), forKey: "myCreditCardNo")

	// ...
}

References

swift/ql/src/queries/Security/CWE-311/CleartextTransmission.qhelp

Cleartext transmission of sensitive information

Sensitive information that is transmitted without encryption may be accessible to an attacker.

Recommendation

Ensure that sensitive information is always encrypted before being transmitted over the network. In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext. Avoid transmitting sensitive information when it is not necessary to.

Example

The following example shows three cases of transmitting information. In the 'BAD' case, the data transmitted is sensitive (a credit card number) and is not encrypted. In the 'GOOD' cases, the data is either not sensitive, or is protected with encryption. When encryption is used, take care to select a secure modern encryption algorithm, and put suitable key management practices into place.

import CryptoKit

private func encrypt(_ text: String, _ encryptionKey: SymmetricKey) -> String {
	let sealedBox = try! AES.GCM.seal(Data(text.utf8), using: encryptionKey)
	return sealedBox.combined!.base64EncodedString()
}

func transmitMyData(connection : NWConnection, faveSong : String, creditCardNo : String, encryptionKey: SymmetricKey) {
	// ...

	// GOOD: not sensitive information
	connection.send(content: faveSong, completion: .idempotent)

	// BAD: sensitive information saved in cleartext
	connection.send(content: creditCardNo, completion: .idempotent)

	// GOOD: encrypted sensitive information saved
	connection.send(content: encrypt(creditCardNo, encryptionKey), completion: .idempotent)

	// ...
}

References

swift/ql/src/queries/Security/CWE-312/CleartextLogging.qhelp

Cleartext logging of sensitive information

Attackers could gain access to sensitive information that is logged unencrypted.

Recommendation

Always make sure to encrypt or obfuscate sensitive information before you log it.

Generally, you should decrypt sensitive information only at the point where it is necessary for it to be used in cleartext.

Be aware that external processes often store the standard output and standard error streams of the application. This will include logged sensitive information.

Example

The following example code logs user credentials (in this case, their password) in plaintext:

let password = "P@ssw0rd"
NSLog("User password changed to \(password)")

Instead, you should encrypt or obfuscate the credentials, or omit them entirely:

let password = "P@ssw0rd"
NSLog("User password changed")

References

  • M. Dowd, J. McDonald and J. Schuhm, The Art of Software Security Assessment, 1st Edition, Chapter 2 - 'Common Vulnerabilities of Encryption', p. 43. Addison Wesley, 2006.
  • M. Howard and D. LeBlanc, Writing Secure Code, 2nd Edition, Chapter 9 - 'Protecting Secret Data', p. 299. Microsoft, 2002.
  • OWASP: Logging Cheat Sheet.
  • Common Weakness Enumeration: CWE-312.
  • Common Weakness Enumeration: CWE-359.
  • Common Weakness Enumeration: CWE-532.
swift/ql/src/queries/Security/CWE-312/CleartextStoragePreferences.qhelp

Cleartext storage of sensitive information in an application preference store

Sensitive information that is stored unencrypted in an application preference store, such as the user defaults database or the iCloud-backed ubiquitous key-value store, is accessible to an attacker who gains access to that data store. For example, the information could be accessed by any process or user in a rooted device, by compromised app extensions, or could be exposed through another vulnerability.

Recommendation

Either store the data in an encrypted database, or ensure that each piece of sensitive information is encrypted before being stored. In general, decrypt sensitive information only at the point where it is necessary for it to be used in cleartext. Avoid storing sensitive information at all if you do not need to keep it.

Example

The following example shows three cases of storing information using UserDefaults. In the 'BAD' case, the data that is stored is sensitive (a credit card number) and is not encrypted. In the 'GOOD' cases, the data is either not sensitive, or is protected with encryption. When encryption is used, take care to select a secure modern encryption algorithm, and put suitable key management practices into place.

import CryptoKit

private func encrypt(_ text: String, _ encryptionKey: SymmetricKey) -> String {
	let sealedBox = try! AES.GCM.seal(Data(text.utf8), using: encryptionKey)
	return sealedBox.combined!.base64EncodedString()
}

func storeMyData(faveSong : String, creditCardNo : String, encryptionKey: SymmetricKey) {
	// ...

	// GOOD: not sensitive information
	UserDefaults.standard.set(faveSong, forKey: "myFaveSong")

	// BAD: sensitive information saved in cleartext
	UserDefaults.standard.set(creditCardNo, forKey: "myCreditCardNo")

	// GOOD: encrypted sensitive information saved
	UserDefaults.standard.set(encrypt(creditCardNo, encryptionKey), forKey: "myCreditCardNo")

	// ...
}

References

swift/ql/src/queries/Security/CWE-730/RegexInjection.qhelp

Regular expression injection

Constructing a regular expression with unsanitized user input is dangerous, since a malicious user may be able to modify the meaning of the expression. They may be able to cause unexpected program behaviour, or perform a denial-of-service attack. For example, they may provide a regular expression fragment that takes exponential time to evaluate in the worst case.

Recommendation

Before embedding user input into a regular expression, use a sanitization function such as NSRegularExpression::escapedPattern(for:) to escape meta-characters that have special meaning.

Example

The following examples construct regular expressions from user input without sanitizing it first:

func processRemoteInput(remoteInput: String) {
  ...

  // BAD: Unsanitized user input is used to construct a regular expression
  let regex1 = try Regex(remoteInput)

  // BAD: Unsanitized user input is used to construct a regular expression
  let regexStr = "abc|\(remoteInput)"
  let regex2 = try NSRegularExpression(pattern: regexStr)

  ...
}

If user input is used to construct a regular expression it should be sanitized first. This ensures that the user cannot insert characters that have special meanings in regular expressions.

func processRemoteInput(remoteInput: String) {
  ...

  // GOOD: Regular expression is not derived from user input
  let regex1 = try Regex(myRegex)

  // GOOD: User input is sanitized before being used to construct a regular expression
  let escapedInput = NSRegularExpression.escapedPattern(for: remoteInput)
  let regexStr = "abc|\(escapedInput)"
  let regex2 = try NSRegularExpression(pattern: regexStr)

  ...
}

References

unified/ql/src/queries/security/CWE-022/PathInjection.qhelp

Uncontrolled data used in path expression

Accessing paths controlled by users can expose resources to attackers.

Paths that are naively constructed from data controlled by a user may contain unexpected special characters, such as ... Such a path could point to any directory on the file system.

Recommendation

Validate user input before using it to construct a file path. Ideally, follow these rules:

  • Do not allow more than a single . character.
  • Do not allow directory separators such as / or \ (depending on the file system).
  • Do not rely on simply replacing problematic sequences such as ../. For example, after applying this filter to .../...// the resulting string would still be ../.
  • Use a whitelist of known good patterns.

Example

The following code shows two bad examples.

let fm = FileManager.default
let path = try String(contentsOf: URL(string: "http://example.com/")!)

// BAD
return fm.contents(atPath: path)

// BAD
if (path.hasPrefix(NSHomeDirectory() + "/Library/Caches")) {
    return fm.contents(atPath: path)
}

In the first, a file name is read from an HTTP request and then used to access a file. In this case, a malicious response could include a file name that is an absolute path, such as "/Applications/(current_application)/Documents/sensitive.data".

In the second bad example, it appears that the user is restricted to opening a file within the "/Library/Caches" home directory. In this case, a malicious response could contain a file name containing special characters. For example, the string "../../Documents/sensitive.data" will result in the code reading the file located at "/Applications/(current_application)/Library/Caches/../../Documents/sensitive.data", which contains users' sensitive data. This file may then be made accessible to an attacker, giving them access to all this data.

In the following (good) example, the path used to access the file system is normalized before being checked against a known prefix. This ensures that regardless of the user input, the resulting path is safe.

let fm = FileManager.default
let path = try String(contentsOf: URL(string: "http://example.com/")!)

// GOOD
let filePath = FilePath(stringLiteral: path)
if (filePath.lexicallyNormalized().starts(with: FilePath(stringLiteral: NSHomeDirectory() + "/Library/Caches"))) {
    return fm.contents(atPath: path)
}

References

unified/ql/src/queries/security/CWE-078/CommandInjection.qhelp

System command built from user-controlled sources

Constructing a system command with unsanitized user input is dangerous, since a malicious user may be able to craft input that executes arbitrary code.

Recommendation

If possible, use hard-coded string literals to specify the command to run. Instead of interpreting user input directly as command names, examine the input and then choose among hard-coded string literals.

If this is not possible, then add sanitization code to verify that the user input is safe before using it.

Example

The following example executes code from user input without sanitizing it first:

var task = Process()
task.launchPath = "/bin/bash"
task.arguments = ["-c", userControlledString] // BAD

task.launch()

If user input is used to construct a command it should be checked first. This ensures that the user cannot insert characters that have special meanings:

func validateCommand(_ command: String) -> String? {
    let allowedCommands = ["ls -l", "pwd", "echo"]
    if allowedCommands.contains(command) {
        return command
    }
    return nil
}

if let validatedString = validateCommand(userControlledString) {
    var task = Process()
    task.launchPath = "/bin/bash"
    task.arguments = ["-c", validatedString] // GOOD

    task.launch()
}

References

unified/ql/src/queries/security/CWE-094/UnsafeJsEval.qhelp

JavaScript Injection

Evaluating JavaScript that contains a substring from a remote origin may lead to remote code execution. Code written by an attacker can execute unauthorized actions, including exfiltration of local data through a third party web service.

Recommendation

When loading JavaScript into a web view, evaluate only known, locally-defined source code. If part of the input comes from a remote source, do not inject it into the JavaScript code to be evaluated. Instead, send it to the web view as data using an API such as WKWebView.callAsyncJavaScript with the arguments dictionary to pass remote data objects.

Example

In the following (bad) example, a call to WKWebView.evaluateJavaScript evaluates JavaScript source code that is tainted with remote data, potentially introducing a code injection vulnerability.

let webview: WKWebView
let remoteData = try String(contentsOf: URL(string: "http://example.com/evil.json")!)

...

_ = try await webview.evaluateJavaScript("console.log(" + remoteData + ")") // BAD

In the following (good) example, we sanitize the remote data by passing it using the arguments dictionary of WKWebView.callAsyncJavaScript. This ensures that untrusted data cannot be evaluated as JavaScript source code.

let webview: WKWebView
let remoteData = try String(contentsOf: URL(string: "http://example.com/evil.json")!)

...

_ = try await webview.callAsyncJavaScript(
    "console.log(data)",
    arguments: ["data": remoteData], // GOOD
    contentWorld: .page
)

References

unified/ql/src/queries/security/CWE-134/UncontrolledFormatString.qhelp

Uncontrolled format string

Passing untrusted format strings to functions that use printf style formatting can lead to buffer overflows and data representation problems. An attacker may be able to exploit this weakness to crash the program or obtain sensitive information from its internal state.

Recommendation

Use a constant string literal for the format string to prevent the possibility of data flow from an untrusted source. This also helps to prevent errors where the format arguments do not match the format string.

If the format string cannot be constant, ensure that it comes from a secure data source or is compiled into the source code. If you need to include a string value from the user, use an appropriate specifier (such as %@) in the format string and include the user provided value as a format argument.

Example

In this example, the format string includes a user-controlled inputString:

print(String(format: "User input: " + inputString)) // vulnerable

To fix it, make inputString a format argument rather than part of the format string, as in the following code:

print(String(format: "User input: %@", inputString)) // fixed

References

unified/ql/src/queries/security/CWE-312/CleartextLogging.qhelp

Cleartext logging of sensitive information

Attackers could gain access to sensitive information that is logged unencrypted.

Recommendation

Always make sure to encrypt or obfuscate sensitive information before you log it.

Generally, you should decrypt sensitive information only at the point where it is necessary for it to be used in cleartext.

Be aware that external processes often store the standard output and standard error streams of the application. This will include logged sensitive information.

Example

The following example code logs user credentials (in this case, their password) in plaintext:

let password = "P@ssw0rd"
NSLog("User password changed to \(password)")

Instead, you should encrypt or obfuscate the credentials, or omit them entirely:

let password = "P@ssw0rd"
NSLog("User password changed")

References

  • M. Dowd, J. McDonald and J. Schuhm, The Art of Software Security Assessment, 1st Edition, Chapter 2 - 'Common Vulnerabilities of Encryption', p. 43. Addison Wesley, 2006.
  • M. Howard and D. LeBlanc, Writing Secure Code, 2nd Edition, Chapter 9 - 'Protecting Secret Data', p. 299. Microsoft, 2002.
  • OWASP: Logging Cheat Sheet.
  • Common Weakness Enumeration: CWE-312.
  • Common Weakness Enumeration: CWE-359.
  • Common Weakness Enumeration: CWE-532.

@yoff

yoff commented Nov 17, 2026

Copy link
Copy Markdown
Owner Author

DCA launched: github/codeql-dca-main#40713

  • Baseline: 9ffea9596ef3a45ad8524ad2700836db2454472a
  • Candidate: f6e989a86971765b667992661714ff9ab0257a99
  • Toolchain / extractor / compiler / evaluator: 7eb0047c90fffaeb822564a0ac56d585dc871f30 (CodeQL CLI 2.27.4)
  • DCA revision: 5421656584fbfb892a3a68ca0064dca8d514f833
  • Sources: exact canonical 48-source Python set and source SHAs from github/codeql-dca-main#38848, reused from the successful github/codeql-dca-main#39772 inputs
  • Suite/source configuration: nightly.qls / historical canonical nightly.yml
  • Database cache: disabled
  • Grouping/action/source/query-suite inputs: structurally identical to #39772 after normalizing experiment branches/descriptions, variant SHAs/labels, toolchain/DCA revisions, and generated bundle IDs
  • Generated-input SHA-256: e32012369730f24a2d664b483f4ddc0e6eba99666c08a1dff60b296164a0475b
  • Controller: Tasks run 44132025853
  • Reports: data/PR-161-python-shared-ssa-current-main/reports

Two supported preflight exceptions were required: generation skipped source-repository access because the historical ONSdigital/eq-questionnaire-runner repository now returns 404, and dispatch skipped worker-repository access because the local token is not SSO-authorized for codeql-dca-runners. Neither exception changes worker inputs. The current DCA launcher also received the same narrow fork-recognition adaptation used by the earlier successful experiment, so unmerged yoff/codeql commits are recognized as CodeQL query commits.

No follow-up DCA will be launched and the candidate will not be modified while this experiment is unresolved.

@yoff

yoff commented Nov 17, 2026

Copy link
Copy Markdown
Owner Author

DCA #40713 is complete. The isolated current-main comparison remains unfavorable, so activation is not recommended.

Exact comparison:

  • baseline 9ffea9596ef3a45ad8524ad2700836db2454472a
  • candidate f6e989a86971765b667992661714ff9ab0257a99
  • CLI/toolchain 7eb0047c90fffaeb822564a0ac56d585dc871f30 (2.27.4)
  • 47/48 sources completed per variant; the sole shared failure is the unavailable historical ONSdigital/eq-questionnaire-runner source

Headline results:

  • all successful query runs: 7,988s -> 8,596s, +7.61%
  • complete-source sum: 5,526s -> 5,838s, +5.65%; paired-source median +5.32%
  • Airflow +16.4%, Nova +8.37%, Salt +2.65%, FreeCAD -0.17%
  • DIL +29.3%, retained cache after normal cleanup +27.0%, tuple pool +30.5%, database including cache +8.49%
  • 275 changed source/query tuple rows: 514 tuples added, 2,958 removed; 50 candidate-only and 16 baseline-only alert locations

Compared with the historical +9.824%/+7.70% findings, current main partially alleviates the regression (about 22-27% relative reduction), but does not remove it. The broad output changes and space costs reinforce the local deterministic conclusion: keep/narrow the adapter if useful, reject activation as-is, and investigate demand architecture rather than another broad cache.

Detailed reports: https://github.com/github/codeql-dca-main/tree/data/PR-161-python-shared-ssa-current-main/reports

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.