How to Remove Sensitive Data from Your Git History (For Real This Time)

Source: Dev.to

Nope. That API key, that .env file, that internal config with your database credentials — it’s all still there, sitting comfortably in your git history, waiting for anyone with git log and five minutes of curiosity.

I learned this the hard way about four years ago when a colleague pinged me to let me know our staging database password was visible in a public repo. I’d removed the file three months earlier. Didn’t matter. Git remembers everything.

Why Deleting a File Doesn’t Actually Delete It

Git is a content‑addressable filesystem. Every commit is a snapshot of your entire project at that point in time. When you git rm secrets.env and commit, you’re creating a new snapshot without that file — but every previous snapshot still has it.

Anyone can see it:

# Find all commits that touched a specific file, even deleted ones
git log --all --full-history -- path/to/secrets.env

# Show the contents of that file at a specific commit
git show a1b2c3d:path/to/secrets.env

This is by design. Git’s whole purpose is to never lose data. That’s great for source code. It’s terrible for secrets.

The Wrong Fix: git revert

I see people try this constantly. They run git revert thinking it undoes the damage. It doesn’t. A revert creates a new commit that reverses the changes — the original commit with your secrets is still right there in the history.

The same goes for git commit --amend on an already‑pushed commit. The old commit object still exists in the reflog and potentially on the remote.

The Right Fix: git filter-repo

The old advice was to use git filter-branch, but it’s painfully slow and easy to mess up. The Git project itself now recommends git-filter-repo instead.

Purge a file from your entire history

# Install git-filter-repo (requires Python 3.5+)
pip install git-filter-repo

# Clone a fresh copy — filter-repo requires a fresh clone
git clone --mirror https://github.com/you/your-repo.git
cd your-repo.git

# Remove a specific file from all history
git filter-repo --path secrets.env --invert-paths

# Remove a directory from all history
git filter-repo --path config/internal/ --invert-paths

The --invert-paths flag means “keep everything EXCEPT this path.” Without it, you’d keep only the specified path and delete everything else. Ask me how I know.

Scrub a specific string (e.g., a hard‑coded API key)

# Replace a specific string across all history
git filter-repo --replace-text REDACTED

Push the rewritten history

git push origin --force --all
git push origin --force --tags

Important: Tell Your Team

Force‑pushing rewrites history. Every collaborator needs to re‑clone or carefully rebase their local branches. If they push their old local copy, all your purged data comes right back. Send a message before you force‑push and coordinate the timing.

What About BFG Repo Cleaner?

BFG Repo Cleaner is another solid option, especially if you prefer a Java‑based tool. It’s faster than filter-branch and has a simpler interface for common operations.

# Remove files by name from all history
java -jar bfg.jar --delete-files secrets.env your-repo.git

# Replace specific text patterns
java -jar bfg.jar --replace-text passwords.txt your-repo.git

# Then clean up and push
cd your-repo.git
git reflog expire --expire=now --all
git gc --prune=now --aggressive
git push origin --force --all

BFG intentionally doesn’t modify your latest commit, only history. This is a safety feature — it assumes your current HEAD is already clean.

Preventing This in the First Place

Rewriting history is painful. Here’s how to avoid needing to do it.

1. Use a .gitignore That Actually Works

# Environment and secrets
.env
.env.*
*.pem
*.key
*.p12

# Cloud provider configs
.aws/credentials
.gcp-credentials.json

# IDE and OS junk that sometimes contains paths/tokens
.idea/
.vscode/settings.json
.DS_Store

2. Set Up Pre‑commit Hooks

pre‑commit with secret‑detection plugins will catch most accidental commits before they happen:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.5.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']

Run pre-commit install once, and it will scan every commit for things that look like secrets — high‑entropy strings, known API‑key patterns, private keys.

3. Use Environment Variables or a Secret Manager

This sounds obvious, but I still see PRs where someone hard‑coded a connection string. Use environment variables in development and a proper secret manager (Vault, cloud‑native options from your provider, or even pass for personal projects) in production.

Rule of thumb: if a value would cause damage in the wrong hands, it does not go in version control. Ever.

After the Purge: Rotate Everything

This is the step people skip, and it’s arguably the most important one. Rewriting Git history removes the secret from your repository. It does not remove it from:

• GitHub’s cached copies and any forks
• CI/CD logs or artifact stores
• Local clones that already fetched the secret
• Any third‑party services that may have cached the data

Therefore, after you’ve scrubbed the repo:

1. Rotate all compromised credentials (API keys, passwords, certificates, etc.).
2. Invalidate any tokens that were exposed.
3. Audit logs for any suspicious activity that might have used the leaked secrets.
4. Notify any stakeholders or customers if the breach could affect them.

TL;DR

• Deleting a file in Git doesn’t erase it from history.
• git revert and git commit --amend don’t solve the problem.
• Use git filter-repo (or BFG) to actually purge secrets.
• Force‑push the rewritten history and coordinate with your team.
• Prevent future leaks with a solid .gitignore, pre‑commit secret scans, and proper secret management.
• After cleaning, rotate every exposed credential.

# Secrets in Git – What Happens When You Accidentally Push a Credential?

## Where the secret can leak

- Forks of your repo  
- Anyone's local clone  
- Search‑engine caches  
- Services like the Wayback Machine  

> **If a secret was ever pushed to a public repo, even briefly, assume it’s compromised.**  
> Rotate the credential immediately. Regenerate the API key. Change the password. Update the certificate.

GitHub’s docs are blunt about this: they explicitly say that *force‑pushing does not remove data from cached views or cloned copies*. If you pushed a token to a public repo, treat it as burned.