Pages.dev Duplicate Content Checker

Check whether your Cloudflare pages.dev URL may cause duplicate content and generate redirects, robots.txt, noindex headers, and canonical recommendations.

Output:A ready-to-use configuration file for Pages.dev Duplicate Content Checker with best practices applied.

What framework are you using?

Current Setup

Custom Domain

Pages.dev Domain

I have a custom domain activeMy site is available on example.com

pages.dev is already indexed in GoogleI searched 'site:my-project.pages.dev' and found results

My site uses absolute canonical tags<link rel="canonical" href="https://example.com" />

Mitigations you already have

pages.dev Redirects to Custom DomainTraffic hitting pages.dev is instantly forwarded

pages.dev has X-Robots-Tag: noindexI use middleware to block indexing on the staging host

Assessment Results

Quick Summary

Use the Pages.dev Duplicate Content Checker to analyze your Cloudflare Pages setup and determine if your default .pages.dev URL is leaking SEO juice, and get immediate fixes to plug the leak.

What is this tool?

Duplicate content occurs when identical content appears on the internet in more than one place (URL). When deploying to Cloudflare Pages, you often end up with your content living at both example.com and example.pages.dev.

This checker takes your current configuration, evaluates the risk level, and generates the exact code snippets you need to protect your search engine rankings.

How to Use This Tool

Enter your production custom domain and your pages.dev domain.
Answer the questions about your current setup (do you have canonical tags? is it already indexed?).
Review your Duplicate Content Risk Score.
Implement the highest-priority recommendation provided in the results.

What This Tool Generates

A Duplicate Content Risk Assessment (Low, Medium, High, Critical).
_redirects rules or Middleware snippets tailored to your risk profile.

Best Practices

Always set up a Bulk Redirect in the Cloudflare dashboard to forward pages.dev traffic to your custom domain immediately after you launch.
Ensure your framework (Next.js, Astro) automatically generates absolute URL canonical tags pointing to your production domain.
Periodically check 'site:yourproject.pages.dev' in Google to ensure nothing slipped through.

Common Mistakes

Assuming that buying a domain and adding it to Cloudflare Pages automatically deletes the pages.dev URL.
Adding 'Disallow: /' to a single robots.txt file, which ends up accidentally blocking the main production domain as well.
Using JavaScript-based redirects instead of HTTP 301 redirects, which are slower and less reliable for SEO crawlers.

Security Notes

If your pages.dev URL is used for staging unreleased code, protect it with Cloudflare Access (Zero Trust) instead of relying solely on SEO noindex tags.

Frequently Asked Questions

Can pages.dev cause duplicate content?

Yes. Every Cloudflare Pages deployment gets a default [project].pages.dev URL. If Google discovers it, it will index it alongside your custom domain, causing duplicate content issues.

How do I stop pages.dev from indexing?

The easiest way is to set up a Bulk Redirect in Cloudflare to forward all pages.dev traffic to your custom domain. Alternatively, you can use middleware to inject an X-Robots-Tag: noindex header when the request host is pages.dev.

Should I redirect pages.dev to my domain?

Yes, a 301 redirect is the SEO industry standard for handling staging domains that don't need to be publicly accessible.

Should I block pages.dev with robots.txt?

You can, but it requires serving a different robots.txt depending on the hostname (which requires a Worker or Middleware). Furthermore, blocking in robots.txt only prevents crawling; it does not reliably prevent indexing if the site has already been discovered.

How do I check if pages.dev is indexed?

Go to Google and search for: site:your-project.pages.dev. If any results appear, your staging domain is indexed.

How We Keep Your Configs Safe & Valid

Built-in Error Checking

Every file is checked against official rules. We catch missing fields and bad syntax. YAML indentation errors are flagged right away. Kubernetes, Terraform, and Docker specs are all covered. API versions and labels are verified too. You get valid output every time you generate.

100% Private & Local

All tools run in your browser only. Your API keys never leave your machine. We do not use any tracking scripts. No data is sent to any server. Passwords and secrets stay on your device. Crypto operations use the Web Crypto API. Your privacy is fully protected at all times.

Secure Settings by Default

Configs use safe defaults out of the box. Containers run as non-root users. Root filesystems are set to read-only. Dangerous Linux capabilities are dropped. Network policies limit pod-to-pod traffic. TLS 1.3 is enabled for web servers. Security headers are added where needed.

Ready for CI/CD & Git

Output files are ready for your Git repo. Use them with ArgoCD, Flux, or GitHub Actions. Files use clear formatting and comments. Code review is easy for your team. Indentation and key order are consistent. Test in staging before going to production. Every file is clean and well-structured.

Infrastructure as Code

Store configs in Git alongside your code. Terraform modules include typed variables. Backend configs support remote state locking. Outputs work across multiple modules. Ansible playbooks use clear task steps. Chef and Puppet configs are also supported. Every file works with version control tools.

Monitoring & Tracing

Set up Prometheus with auto-discovery rules. Create Grafana dashboards with template variables. Add alerting rules with severity labels. Use OpenTelemetry for trace collection. Forward logs to Loki or Elasticsearch. Connect to Jaeger or Tempo for tracing. Monitor metrics, logs, and traces together.

Container & Docker Safety

Dockerfiles use multi-stage builds for small images. Base images are pinned to exact versions. Dev files are excluded from final images. Health checks are added for orchestrator use. Containers switch to non-root users. Docker Compose uses named volumes and networks. Resource limits are set in deploy configs.

Multiple Output Formats

Export as YAML, JSON, HCL, or TOML. Kubernetes uses YAML with proper separators. Terraform uses HCL with correct escaping. JSON output has consistent indentation. Copy to clipboard with one click. Preview output with syntax highlighting. Line numbers help you review quickly.