Lessons from 500+ GitHub Migrations

Source: Dev.to

Introduction

After migrating 500+ repositories from Azure DevOps, BitBucket, SVN, and other systems to GitHub, we ran into pretty much every migration challenge imaginable. We built an automated framework to solve these problems and open‑sourced it so others facing enterprise GitHub migrations can benefit from our learnings.

This post shares the challenges we encountered, how we addressed them, and how GitHub Copilot helped us make the framework maintainable through skills‑based configuration.

Large Files Without LFS

Problem: Repositories contain large files (> 100 MB) that aren’t tracked by LFS. GitHub rejects pushes, blocking the entire migration.

Solution:

1. Detect large files before pushing.
2. Automatically configure Git LFS tracking for them.
3. Convert existing files to LFS storage.

Branch Naming Chaos

Problem: Different systems use main, master, or trunk. Pipelines break, protected branches fail.

Solution: Detect the source repository’s default branch and preserve it during migration instead of assuming main or master.

Lost History & Metadata

Problem: Commit authors become “migration‑bot”, timestamps vanish, branches flatten.

Solution: Use git clone (not migration APIs) to preserve full history with original authors, timestamps, all branches, tags, and merge topology.

Secret Scanning Roadblocks

Problem: Historical secrets block entire migrations, requiring hours of manual remediation.

Solution:

• Temporarily disable secret‑scanning push protection during migration.
• Re‑enable it immediately after.

This allows teams to remediate secrets post‑migration without blocking the process.

Room for improvement: Our current approach relies on teams revoking any exposed secrets after migration, which worked at our scale. A better solution could use pre‑commit scanning to detect secrets, mask them, and export findings to a secure location for the issue initiator to address—but we found post‑migration revocation more practical for our needs.

SVN to Git Translation

Problem: SVN’s trunk/branches/tags structure doesn’t map cleanly to Git refs.

Solution: Use git‑svn with automatic layout detection to convert the trunk/branches/tags structure to Git conventions.

Permission Chaos

Problem: Teams lose access, wrong permissions assigned, visibility mis‑configured.

Solution:

1. Create hierarchical teams (parent/owner/admin) automatically.
2. Validate the requestor’s team membership before allowing migration.
3. Set repository visibility based on criticality level.

Monorepo Decomposition

Problem: Extracting one folder from a monorepo loses history or requires manual git‑filter‑repo surgery.

Solution: Add an only‑folder parameter that uses git filtering to migrate specific paths while preserving relevant commit history.

Integration Breakage

Problem: CI/CD pipelines, Azure Boards, webhooks—everything points to the old repo.

Solution (Azure DevOps migrations):

• Automatically rewire pipelines to point to GitHub.
• Configure board integration.
• Lock the source repository to read‑only.

Framework Overview

We built the framework on GitHub Actions and PowerShell with a simple workflow:

1. Create a GitHub issue with migration parameters (org, team, repo name, source URL).
2. Workflow validates team membership and naming conventions.
3. Creates repository and migrates code with full history.
4. Sets up team hierarchy and permissions.
5. Rewires pipelines and integrations (for ADO sources).
6. Comments on the issue with success or failure details.

What We Learned

One challenge with enterprise frameworks is keeping them maintainable as requirements change. We use GitHub Copilot to help with this.

Skills Directory

We created .github/skills/ with instructions for common customization tasks. Example:

Use the update-app-name skill to update all references with:
App Name: repo-migrate
App ID: 123456789

Copilot can scan the entire repo and update references across multiple files.

Available Skills

How Copilot Helped

• Added new source‑system support (BitBucket, SVN) much faster.
• Refactored complex PowerShell functions with proper error handling.
• Generated troubleshooting documentation from workflow files.
• Onboarded new contributors by answering questions about the codebase.

This approach makes the framework easier for others to customize for their specific needs.

Known Limitations & Future Work

We’re being honest—this framework isn’t perfect. Here are areas we know could be better:

1. Secret Handling – Currently we temporarily disable secret scanning and let teams handle revocation after migration. A more sophisticated approach would involve proactive secret detection, masking secrets in history, and providing a secure report for remediation before migration completes.
2. Pre‑Migration Validation – Add repository health checks (detect LFS files, large binaries, malformed branch names) and report them upfront.
3. Post‑Migration Verification – Automated comparison of source vs. migrated repositories (commit counts, file checksums, branch structures) to catch edge cases we might miss.

If you implement improvements in these areas, we’d love to see your approach!

Get Started

The framework is open source and still evolving. If you’re facing enterprise GitHub migrations, give it a try:

• Repository: (link to repo)
• Use the repository template to create your own instance.
• Configure GitHub App authentication (see README).
• Set up secrets for your source systems (ADO, BitBucket, etc.).
• Create an issue using the migration request template.

Challenges & Tips for Large‑Scale Migrations

Every organization’s needs are different, but the following guidance can help you navigate common pitfalls.

Contribute & Get Help

• Found a bug? – Open an issue
• Have a feature request? – Let us know by opening an issue.
• Added support for a new source system? – Submit a PR.
• Customized it for your org? – Share what worked.

The Copilot skills in .github/skills/ make customization easier—you can add new source systems, custom properties, or validation rules by describing what you need.

Migration Best Practices

• Test thoroughly with a pilot repo – Large‑scale migrations hide complexity; a small pilot helps surface hidden issues early.
• Automate wherever possible – Manual migrations don’t scale; investing in automation pays off in the long run.
• Plan permissions carefully – Mistakes in team structure are hard to fix later.
• Document edge cases – They’ll inevitably reappear in future migrations.
• Share solutions – Open source benefits everyone; contribute your fixes and learn from others.

Got Questions or War Stories?

Open an issue in the framework repository. We’d love to hear how it works (or doesn’t work) for your migrations!