All articles · Site Operations

Site Migration Without Losing Traffic: Mapping 3,900 Old URLs

Cover with the article title next to arrows rerouting from old paths to new ones, in amber on a dark background

When we rebuilt PerfoMetrix, the old site was an off-the-shelf PHP application that had been running on the domain for years. We threw away all of its code. What we couldn’t throw away were its URLs: thousands of them were indexed, linked from other sites and bookmarked by people.

Most migration advice boils down to “set up 301 redirects”. That’s true and not very useful. The hard part is knowing which URLs exist, deciding what each of them should become, and proving before launch day that every single one does what you decided. This is how we approached it, and what we’d tell anyone facing the same job.

1. Build the inventory before you build the map

You can’t redirect URLs you don’t know about. Sitemaps are the obvious starting point and almost never complete. We combined several sources:

  • The old application’s database. The legacy site generated a report page for every domain anyone had ever checked, at /domain/<hostname>. The database had a table of those hostnames: 3,884 report URLs right there, far more than any sitemap listed.
  • The old site’s routes and CMS pages. Reading the legacy routing table turned up sections we’d forgotten: site comparison pages, PDF exports, profile pages, a “recent scans” list.
  • Search Console. The Pages report and the Performance report list URLs Google knows about and URLs that actually get clicks.
  • Server logs and analytics, for URLs real people and bots still request.
  • Backlink data, if you have it, for URLs other sites link to. These carry the most value.

De-duplicate, normalize (scheme, host, trailing slash), and you have your inventory. Ours came to 3,908 URLs. We kept it in the repository as a plain text file, one URL per line. Only URLs, with no personal data from the old system.

2. Decide per pattern, not per URL

Nobody maps 3,900 URLs by hand. Group them by pattern and decide once per pattern:

Old pattern Decision Why
/domain/<host> (report pages) 301 → /?url=<host> Same intent: “check this site”. The form is prefilled; a GET never starts a scan
PDF and alternate report variants 301 → /?url=<host> Same intent as above
/page/about-us, privacy, terms 301 → /about, /privacy, /terms Direct equivalents exist
Login and account pages 301 → /login The new site has accounts, under a different path
Admin, AJAX, theme and asset paths 410 Implementation details of the old app with no equivalent
“Recent scans”, comparison tools, old extras 410 Features we deliberately didn’t rebuild

The questions behind each row:

  • Is there a page on the new site that serves the same intent? Then 301 to it. “Same intent” is the bar, not “same topic”. Redirecting an old blog post about X to a new page about X is fine. Redirecting it to the home page isn’t.
  • Is the content gone on purpose? Then 410 Gone. It tells crawlers the removal is intentional, and in practice they tend to drop such URLs a little faster than after a 404.
  • Never mass-redirect to the home page. Search engines treat a redirect to an unrelated page as a soft 404 anyway, and users who followed a link to something specific end up confused. A clear 410 is more honest.

We wrote the decisions as a small table of path rules in the new app’s middleware, evaluated before normal routing. That keeps them in one place, reviewable in code, and out of a CDN dashboard where they’d be easy to lose track of. Our redirect chains guide explains why each old URL should reach its destination in a single hop.

3. Test every URL, not a sample

This is the step most migrations skip, and it’s the one that matters most. We wrote a script that reads the inventory file, requests every URL against a given base URL, and checks the result against the rules:

  • expected 301s must point at exactly the expected target,
  • expected 410s must return 410,
  • and, importantly, any URL that matches no rule and returns 404 fails the run.

That last check is what found the gaps. The early runs against the new site flagged URLs from legacy features nobody had put on the list: PDF export paths, an alternate report format, a couple of CMS pages. Each failure was either a missing rule or a deliberate decision we hadn’t written down. We kept iterating until the run was clean: 3,908 of 3,908.

The script takes the base URL as an argument, so the same check runs locally, against a staging host, and against production after cutover. If it isn’t automated, it won’t be re-run after the next change to the routing.

4. Rehearse on a real host

We deployed the new site to production infrastructure on a subdomain first, behind the same CDN and with the same configuration, and ran the URL check there. Two rules made that safe:

  • The rehearsal host is never indexable. Every response on a host other than the canonical one carries X-Robots-Tag: noindex, and its robots.txt disallows everything. You don’t want Google finding a full copy of your site on beta. and choosing it as canonical.
  • The real domain keeps serving the old site until the switch, so the rehearsal can take as long as it needs.

Rehearsing on the real stack caught problems a local test can’t: CDN rules interacting with redirects, the www to apex redirect (which we handle at the CDN, so the app only ever sees one host; see www vs non-www), and HTTPS behaviour.

5. Make the cutover boring and the rollback quick

With the rehearsal clean, the cutover itself was a DNS change: point the apex and www at the new servers. Before doing it we wrote down, in advance, what would make us roll back (for example the URL check failing in production, or error rates above a threshold) so that the decision wouldn’t be made under pressure.

Rollback meant pointing DNS back at the old host, which we left running and untouched for 14 days after the switch. Old hosting costs a few euros; a migration with no way back costs much more.

Right after the switch:

  1. Run the URL check against the production domain.
  2. Submit the new sitemap in Search Console (it lists only new, canonical URLs, never redirected ones).
  3. Use URL Inspection on a few important old URLs to confirm Google sees the redirect.
  4. Watch the 404 and 410 counts in your logs for a couple of weeks. Unexpected 404s on URLs with traffic are rules you missed.

6. Keep the redirects for a long time

Google’s guidance is to keep redirects in place for at least a year, and longer is better while old URLs still receive links or traffic. Redirect rules in application code are cheap to keep. The inventory file and the test script stay in the repository too, so that any future change to routing is checked against the old URLs automatically.

What we’d do the same way again

  • Inventory from the database and the code, not just the sitemap. That’s where most of our URLs were.
  • Rules per pattern, written in code, with a short reason for each decision.
  • An automated check over every URL that treats “no rule and 404” as a failure.
  • A rehearsal on the real stack, kept out of the index.
  • A rollback that’s a single DNS change, with the triggers decided beforehand.

None of it is clever. It’s a checklist executed carefully, and the payoff is a launch day where nothing interesting happens. If you’re planning a migration of your own, our HTTP status codes guide is a good refresher on what each response tells crawlers, and a scan of the new site after cutover will catch redirect chains, broken canonicals and missing sitemaps.

Tags:site migrationredirects301410SEO