Cloudflare Staging Domain Blocker

Generate robots.txt, _headers, redirects, and noindex strategies to stop Cloudflare staging, preview, and pages.dev URLs from causing duplicate content.

Output:A ready-to-use configuration file for Cloudflare Staging Domain Blocker with best practices applied.

What framework are you using?

Domain Settings

Custom Production Domain

Cloudflare pages.dev Domain

Select Strategy

File Location

Where to place your _redirects

Source Repository

public/_redirects

After Build

out/_redirects

•Next.js server-side redirects in next.config.ts do not work for static export on Cloudflare Pages.
•Use output: 'export' in your Next.js config.
•Place _redirects in the public directory before building.
•Cloudflare Pages should deploy the 'out' directory.

https://my-project.pages.dev/* https://www.example.com/:splat 301

Alternative: Cloudflare Bulk RedirectsYou can also configure this outside of your codebase by going to the main Cloudflare Dashboard -> Rules -> Bulk Redirects and creating a redirect from my-project.pages.dev to www.example.com.

Quick Summary

Use the Cloudflare Staging Domain Blocker to generate the exact _headers, _redirects, or robots.txt files needed to stop Google from indexing your .pages.dev preview domains and causing SEO duplicate content.

What is this tool?

By default, Cloudflare Pages assigns a [project].pages.dev URL to your site. Even if you connect a custom domain (like example.com), the pages.dev URL remains active and accessible to Google.

If Google indexes both your custom domain and your pages.dev domain, they will compete against each other for rankings (duplicate content). This tool helps you choose a strategy to block the staging domain.

How to Use This Tool

Select your framework to get the correct file output paths.
Choose a blocking strategy (Redirect, Noindex, or Canonical).
Enter your production and pages.dev domains.
Copy the generated configuration and place it in your project.

What This Tool Generates

_redirects rules (to forward staging traffic to production).
_headers rules (to add X-Robots-Tag).
robots.txt templates.

Best Practices

The absolute best method is to configure a Bulk Redirect in your main Cloudflare Dashboard (outside of Pages) to 301 redirect your pages.dev domain to your custom domain.
If you use a framework that supports middleware (Next.js, Astro), you can write logic to conditionally apply X-Robots-Tag only if the host contains 'pages.dev'.
Always include self-referencing canonical tags in your HTML.

Common Mistakes

Adding 'Disallow: /' to your main repository's robots.txt. If you do this, you will block your production domain from Google too!
Assuming that buying a custom domain automatically turns off the pages.dev domain.
Thinking that a 'noindex' tag takes effect immediately. Google must crawl the page again to see the 'noindex' tag.

Security Notes

If your staging environment contains unreleased features or sensitive testing data, do not rely on SEO tags. Secure the environment using Cloudflare Access to require a login.

Frequently Asked Questions

How do I block pages.dev from Google?

The most reliable method is to use Cloudflare Bulk Redirects to 301 redirect your pages.dev domain to your custom production domain. If you need the pages.dev domain for testing, you must deploy a separate robots.txt or an X-Robots-Tag: noindex header specifically for the pages.dev URL.

Should pages.dev redirect to my custom domain?

Yes, unless you actively use pages.dev for QA testing. If you redirect it, you eliminate any chance of duplicate content penalizing your SEO rankings.

Should I use noindex on staging?

Yes. Staging environments should always return 'X-Robots-Tag: noindex, nofollow' in their HTTP headers.

Does robots.txt prevent duplicate content?

Not necessarily. robots.txt prevents crawling, but if a URL is already known to Google, it can still index it without crawling it. 'noindex' headers or 301 redirects are much safer for preventing duplicate content.

How do I add X-Robots-Tag on Cloudflare Pages?

Create a _headers file and add a rule matching '/*' with the header 'X-Robots-Tag: noindex, nofollow'. Note that this will apply to whichever domain serves the file.

How We Keep Your Configs Safe & Valid

Built-in Error Checking

Every file is checked against official rules. We catch missing fields and bad syntax. YAML indentation errors are flagged right away. Kubernetes, Terraform, and Docker specs are all covered. API versions and labels are verified too. You get valid output every time you generate.

100% Private & Local

All tools run in your browser only. Your API keys never leave your machine. We do not use any tracking scripts. No data is sent to any server. Passwords and secrets stay on your device. Crypto operations use the Web Crypto API. Your privacy is fully protected at all times.

Secure Settings by Default

Configs use safe defaults out of the box. Containers run as non-root users. Root filesystems are set to read-only. Dangerous Linux capabilities are dropped. Network policies limit pod-to-pod traffic. TLS 1.3 is enabled for web servers. Security headers are added where needed.

Ready for CI/CD & Git

Output files are ready for your Git repo. Use them with ArgoCD, Flux, or GitHub Actions. Files use clear formatting and comments. Code review is easy for your team. Indentation and key order are consistent. Test in staging before going to production. Every file is clean and well-structured.

Infrastructure as Code

Store configs in Git alongside your code. Terraform modules include typed variables. Backend configs support remote state locking. Outputs work across multiple modules. Ansible playbooks use clear task steps. Chef and Puppet configs are also supported. Every file works with version control tools.

Monitoring & Tracing

Set up Prometheus with auto-discovery rules. Create Grafana dashboards with template variables. Add alerting rules with severity labels. Use OpenTelemetry for trace collection. Forward logs to Loki or Elasticsearch. Connect to Jaeger or Tempo for tracing. Monitor metrics, logs, and traces together.

Container & Docker Safety

Dockerfiles use multi-stage builds for small images. Base images are pinned to exact versions. Dev files are excluded from final images. Health checks are added for orchestrator use. Containers switch to non-root users. Docker Compose uses named volumes and networks. Resource limits are set in deploy configs.

Multiple Output Formats

Export as YAML, JSON, HCL, or TOML. Kubernetes uses YAML with proper separators. Terraform uses HCL with correct escaping. JSON output has consistent indentation. Copy to clipboard with one click. Preview output with syntax highlighting. Line numbers help you review quickly.