The latest episode of Google’s Search Off The Record podcast discusses the challenges of launching websites and preventing staging sites from being indexed by search engines.
Staging sites used for testing and development should ideally not be indexed, and Google highlights password protection as a method to prevent accidental indexing.
Presenters John Mueller, Gary Illyes and Lizzy Sassman delve into the importance of clicking the launch button on different platforms and how website launches and migrations are connected.
The discussion emphasizes the importance of balancing launching a large number of pages, maintaining content quality, and choosing an appropriate domain name for branding and visibility purposes.
Here are the highlights from Google’s recent podcast on staging and launching websites.
Accidental Leaks and Importance of DNS Setup
Mueller and Illyes discuss how to hide staging sites from search engines and share anecdotes about accidental leaks from Google’s past.
They talk about the launch of Google’s search status dashboard and strategies to hide it before its official release.
Illyes says they did “literally nothing” to keep the dashboard a secret before sharing it with everyone.
It was live for months and no one noticed because there were no links pointing to it.
Illyes explains:
“If you don’t link to something on the internet, it’s very hard to notice or you don’t mention it anywhere, and that’s what we’ve done. Don’t mention it anywhere, don’t link to it, don’t have it in random JavaScript files. By the way, one more reason not to use JavaScript. And it just worked, it was risky I would say.”
Sassman adds that there was no sitemap for the new Google dashboard, which made searching even more difficult.
The conversation also touches on the crucial role of DNS setup and the distinction between using a subdomain and a new domain.
Despite the risks associated with their starting strategy, the hosts agree the potential fallout was minimal.
Bring away: A viable but risky way to prevent a staging site from disclosing them so as not to mention them or link to them from somewhere.
Other ways to block staging sites
Illyes explains that robots.txt is usually enough to prevent search engines from crawling staging sites.
When asked about alternative methods of keeping a website hidden from launch, Illyes suggests using a noindex tag on every page.
You can also use password protection or IP allow lists for staging sites.
This led to a story by Mueller about using static IP addresses to access the server backend, resulting in a loss of access when his IP address changed.
Launching websites with a large number of pages and choosing domain names
Discussion continues on launching many pages at once and the importance of choosing the right domain name.
Illyes believes that if a server has sufficient resources, launching 7,000 items at a time shouldn’t be a problem, although quality assurance could be a challenge.
Quality over quantity is emphasized when it comes to the prospect of publishing a million pages at once.
The hosts also discuss using free hostnames provided by hosting providers versus custom domain names.
According to Illyes, the choice depends on the company’s needs and brand goals, with custom domain names being better suited for a long-term online presence.
Listen to the full podcast for more insights from Google’s Search Relations team.
Featured image: sacitarios/Shutterstock
source: Search unofficially
