ZIV LABS
Engineering notes

A deleted secret is still in Git history

Deleting a secret in a new commit does not remove it from the repository: the old commit still holds it, and so does every copy of that commit. This is a general, common cause of exposed credentials rather than one incident, and we do not cite a specific one here. The guidance comes from GitHub's documentation on removing sensitive data and from Git's own documentation. We reproduce the parts that can be tested locally in a throw-away repository with made-up tokens, and we say which parts we only report. Everything ran with Git 2.54.0 and Node.js v24.7.0 on macOS arm64.

Rotate first

GitHub's documentation says that if the sensitive data is a secret such as a password, token, or credential, the first step is to revoke or rotate it. Once it can no longer be used for access, that may be enough, and the extra work of rewriting history may not be warranted. Everything below is about the leftover copy, which matters less once the secret is dead and matters a lot while it is alive.

1. A later commit does not remove an earlier one

The demo makes a repository, commits a config file with a made-up token (fk_live_ followed by 16 characters, not a real format), then commits a change that reads the token from the environment instead. It then looks for the token in the current files and in history, and copies the repository, a stand-in for a clone made after the leak:

1. A secret is committed, then removed
  the file now: export const apiToken = process.env.API_TOKEN;
  working tree scan: nothing found
  history scan: config.js in commit "add config"
  git log -S finds it: true
  a copy made after the leak still has it: true

A scan of the current files finds nothing, which is why the problem looks solved. The history scan finds the token in the earlier commit, git log -S finds it, and the copy still has it, which is true by construction: a copy made after the leak contains the leak. The point is that fixing your own repository does not reach copies that already exist. That scan is the code in scan.mjs below: it walks every commit reachable from any ref and tests every file. It starts a Git process per file and reads blobs as text, so it suits a tiny repository like this one, not a real project.

2. A pattern scanner only finds what its pattern describes

The scanner above is deliberately simple, one regular expression for one token format. Real scanners have many rules and also check how random a string looks; we did not install or compare any of them. What the simple one shows is the principle: a rule for a format finds that format.

2. What a simple pattern scanner misses
  the token as written
    -> detected
  base64 of the token
    -> not detected
  split in two strings
    -> not detected
  same value with another prefix
    -> not detected

The token as written is detected. Encoded, split across two strings, or with another prefix, the same value is not. A scan that reports nothing means nothing matched its rules, not that no secret is present. The history scan is the part to keep: scanning only the current files, or only new changes, would have missed the leak in section 1.

3. Removing a commit locally is two steps, not one

A common reaction is to reset the branch. We commit a second made-up token, reset the branch back one commit, and check whether the object is still in the repository, then expire the reflog and run garbage collection:

3. Reset a local commit, then clean up
  after git reset --hard HEAD~1
    found by git log --all -S: false
    object still in the repository: true
  after reflog expire and gc --prune=now
    object still in the repository: false

After the reset the commit no longer shows up in git log --all -S, but the object is still stored. The reflog is one thing that keeps it reachable; we did not test other places such as ORIG_HEAD, stashes, or worktrees. In this run it disappeared only after the reflog entries were expired and unreachable objects were pruned. This is local only; it changes nothing in any copy that already exists elsewhere.

What GitHub's documentation says about the rest

We did not run any of this. For removing a secret from the history of a repository hosted on GitHub, the documentation describes rewriting the history with git-filter-repo (a version with the --sensitive-data-removal flag, at least 2.47), which changes commit hashes and affects open pull requests, then force-pushing all refs. It warns that the commits may still be reachable through clones and forks, through cached views on GitHub by their SHA-1 hashes, and through pull requests that reference them. You cannot remove the data from other people's clones. After the other references are cleaned up, GitHub Support can remove cached views and pull request references and run garbage collection on the server, but only when rotating the credential cannot mitigate the risk. Read the current page before acting; the steps change with the tools.

Checklist

  • Treat a committed secret as exposed from the moment it was pushed or copied. Revoke or rotate it first.
  • Scan the whole history, all branches and tags, not only the current files or the latest diff.
  • Do not read a clean scan as proof of no secrets: patterns miss encoded, split, or unfamiliar formats.
  • If you rewrite history, plan for clones, forks, open pull requests, and cached views. Rewriting your own repository does not reach other copies.
  • Keep secrets out of the repository in the first place: read them from the environment or a secret store, and see OAuth grants and leaked tokens for what a leaked token is worth.

Reproduce it

Save the two files in one folder and run node demo.mjs with Git and Node.js installed. The demo creates a repository under your system's temporary directory, uses made-up tokens, never touches the network or any other repository, and removes its directory at the end. The output is deterministic.

scan.mjs
import { execFileSync } from "node:child_process";

// A deliberately simple detector for a made-up token format: fk_live_ followed by 16 letters or digits.
// Real scanners use many rules and entropy checks; this one only shows what scanning history means.
const TOKEN = /fk_live_[A-Za-z0-9]{16}/;

const git = (cwd, ...args) => execFileSync("git", args, { cwd, encoding: "utf8", maxBuffer: 1 << 26 });

export function scanWorkingTree(repo) {
  // Files as they are now.
  return git(repo, "ls-files").split("\n").filter(Boolean)
    .filter((file) => TOKEN.test(git(repo, "show", `:${file}`)))
    .map((file) => ({ where: "working tree", file }));
}

export function scanHistory(repo) {
  // Every commit reachable from any ref, every file in it.
  const found = [];
  for (const sha of git(repo, "rev-list", "--all").split("\n").filter(Boolean)) {
    const subject = git(repo, "log", "-1", "--format=%s", sha).trim();
    for (const file of git(repo, "ls-tree", "-r", "--name-only", sha).split("\n").filter(Boolean)) {
      if (TOKEN.test(git(repo, "show", `${sha}:${file}`))) found.push({ where: `commit "${subject}"`, file });
    }
  }
  return found;
}

export const looksLikeToken = (text) => TOKEN.test(text);
demo.mjs
import { execFileSync } from "node:child_process";
import { mkdtempSync, writeFileSync, cpSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { scanWorkingTree, scanHistory, looksLikeToken } from "./scan.mjs";

// Everything happens in a throw-away repository under the OS temp directory. The "secrets" are made up.
const root = mkdtempSync(join(tmpdir(), "leak-demo-"));
const repo = join(root, "repo");
const env = { ...process.env, GIT_CONFIG_GLOBAL: "/dev/null", GIT_CONFIG_SYSTEM: "/dev/null", GIT_AUTHOR_NAME: "demo", GIT_AUTHOR_EMAIL: "demo@example.invalid", GIT_COMMITTER_NAME: "demo", GIT_COMMITTER_EMAIL: "demo@example.invalid" };
const git = (cwd, ...args) => execFileSync("git", args, { cwd, env, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }).trim();
const commit = (message, files) => {
  for (const [name, text] of Object.entries(files)) writeFileSync(join(repo, name), text);
  git(repo, "add", "-A");
  git(repo, "commit", "-q", "-m", message);
};
const TOKEN1 = "fk_live_A1b2C3d4E5f6G7h8";
const TOKEN2 = "fk_live_Z9y8X7w6V5u4T3s2";

execFileSync("git", ["init", "-q", "-b", "main", repo], { env });

console.log("1. A secret is committed, then removed");
commit("add config", { "config.js": `export const apiToken = "${TOKEN1}";\n` });
commit("read the token from the environment", { "config.js": "export const apiToken = process.env.API_TOKEN;\n" });
console.log("  the file now: " + git(repo, "show", "HEAD:config.js"));
console.log("  working tree scan: " + (scanWorkingTree(repo).length ? "found" : "nothing found"));
const hits = scanHistory(repo);
console.log("  history scan: " + hits.map((h) => `${h.file} in ${h.where}`).join(", "));
console.log("  git log -S finds it: " + (git(repo, "log", "--all", "--format=%s", "-S", TOKEN1) !== ""));
cpSync(repo, join(root, "clone"), { recursive: true });
console.log("  a copy made after the leak still has it: " + (scanHistory(join(root, "clone")).length > 0));

console.log("\n2. What a simple pattern scanner misses");
const variants = {
  "the token as written": TOKEN1,
  "base64 of the token": Buffer.from(TOKEN1).toString("base64"),
  "split in two strings": `"fk_live_" + "A1b2C3d4E5f6G7h8"`,
  "same value with another prefix": "fk_test_A1b2C3d4E5f6G7h8",
};
for (const [name, text] of Object.entries(variants)) console.log(`  ${name}\n    -> ${looksLikeToken(text) ? "detected" : "not detected"}`);

console.log("\n3. Reset a local commit, then clean up");
commit("add a second token", { "extra.js": `export const other = "${TOKEN2}";\n` });
const sha = git(repo, "rev-parse", "HEAD");
const exists = () => { try { git(repo, "cat-file", "-e", sha); return true; } catch { return false; } };
git(repo, "reset", "-q", "--hard", "HEAD~1");
console.log("  after git reset --hard HEAD~1");
console.log("    found by git log --all -S: " + (git(repo, "log", "--all", "--format=%s", "-S", TOKEN2) !== ""));
console.log("    object still in the repository: " + exists());
git(repo, "reflog", "expire", "--expire=now", "--all");
git(repo, "gc", "-q", "--prune=now");
console.log("  after reflog expire and gc --prune=now");
console.log("    object still in the repository: " + exists());

rmSync(root, { recursive: true, force: true });

What we did not verify

  • git-filter-repo, force pushes, and every GitHub-side behavior described above. We report the documentation and did not test it.
  • Any real scanner, including its rules and entropy checks, or how well one would have done on the examples.
  • Behavior on other Git versions, and with options such as shallow clones, submodules, or LFS.
  • Any specific incident or how common this cause is. We do not cite one.

References