Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Domain Masking SEO Impact
-
I hope I am explaining this correctly. If I need to provide any clarity please feel free to ask. We currently use a domain mask on an external platform that points back to our site. We are a non-profit and the external site allows users to create peer-to peer fundraisers that benefit our ministry. Currently we get many meta issues related to this site as well as broken links when fundraisers expire etc. We do not have a need to rank for the information from this site. Is there a way to index these pages so that they are not a part of the search engine site crawls as it relates to our site?
-
Glad to be of service!
-
Thank you for this response. This helps me a ton as I discuss with our web team on the best way to set the coding for the site so that we are not registering errors but also not hurting the actual site in any way.
-
This is a good idea, but Robots.txt stops pages being crawled - it doesn't stop pages being indexed. For that you need to fire the Meta No-Index directive on the affected URLs. If you can't edit their code you can fire the same directive through the HTTP header via X-Robots. On that linked post, you'll need to scroll down a little. If possible you could also alter those URLs to serve status code 410 (gone) so that Google knows, those URLs aren't really on your site
Note that you'll need to make the changes on the 'affected' site, not the site which is the 'source' of the masked pages / data. If you make the changes there, that site will have all the Google traffic killed as well (and they'll probably want to punch you!)
I recommend that you lead with hard signals and directives which stop Google indexing the pages on the 'affected' site (which is receiving the masked URLs / content and doesn't want them to rank). Once the pages fall out of Google's index, then you swoop in behind and put the robots.txt stuff in to stop them ever coming back
-
Depends on the CMS you use. Many CMS's can click a quick button to not index in Google search.
If that isnt an option, do it through a robots.txt file in webmaster tools.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does having a sub-domain on a different server affect SEO?
I'm working with a company that has a hard-coded website on the root domain, and then a WordPress blog on a subdomain on a separate server. We're planning on implementing a hub and spoke model for their content, hosting the main hubs on the root domain and the linked articles on the blog. Is having the blog on a different server going to hinder our SEO efforts?
Technical SEO | | KaraParlin0 -
Old domain to new domain
Hi, A website on server A is no longer required. The owner has redirected some URLS of this website (via plugin) to his new website on server B -but not all URLS. So when I use COMMAND site:website A , I see a mixture of redirected URLS and not redirected URLS.Therefore two websites are still being indexed in some form and causing duplication. However, weirdly when I crawl with Screaming Frog I only see one URL which is 301 redirected to the new website. I would have thought I'd see lots of URLs which hadn't been redirected. How come it is different to using the site:command? Anyway, how do I move to the new website completely without the old one being indexed anymore. I thought I knew this but have read so many blogs I've confused myself! Should I: Redirect all URLS via the HTACESS file on old website on server A? There are lots of pages indexed so a lot of URLs. What if I miss some? or Point the old domain via DNS to server B and do the redirects in website B HTaccess file? This seems more sensible but does this method still retain the website rankings? Thanks for any help
Technical SEO | | AL123al0 -
Do Letters With Accents Affect SEO?
Hi Guys, My company has a franchise of a foreign company that uses an accent/foreign letter in its brand name. We have to refer to this franchise with this symbol on our website to meet their standards. I've done some research on this but its not conclusive, so i was wondering whether anyone here can confirm this for me; Will using the letter with this symbol impair our rankings for this franchise name? Obviously as a UK business people search for this franchise with a regular letter and not the accented one. I would have thought that Google is clever enough to recognise the meaning of the accented letter by now and therefore it wouldn't affect rankings (much). Furthermore, do you think that it would make any difference to use the HTML element to represent the accent rather than copy and pasting the symbol onto our website? I would've thought this would help Google pick it up, but it might not make a difference anyway! Any help is appreciated. Thanks Sam
Technical SEO | | Sandicliffe1 -
Tool to Generate All the URLs on a Domain
Hi all, I've been using xml-sitemaps.com for a while to generate a list of all the URLs that exist on a domain. However, this tool only works for websites with under 500 URLs on a domain. The paid tool doesn't offer what we are looking for either. I'm hoping someone can help with a recommendation. We're looking for a tool that can: Crawl, and list, all the indexed URLs on a domain, including .pdf and .doc files (ideally in a .xls or .txt file) Crawl multiple domains with unlimited URLs (we have 5 websites with 500+ URLs on them) Seems pretty simple, but we haven't been able to find something that isn't tailored toward management of a single domain or that can crawl a huge volume of content.
Technical SEO | | timfrick0 -
Do Abbreviations Hurt SEO Results?
We have certain products that we've abbreviated since it's a bit too long. For example, the word Fair Trade Organic is one of our categories and we abbreviate it to FTO. If I put FTO on our meta tag titles and links instead of the actual word, would that provide a weaker result?
Technical SEO | | ckroaster0 -
How much impact does bad html coding really have on SEO?
My client has a site that we are trying to optimise. However the code is really pretty bad. There are 205 errors showing when W3C validating. The >title>, , <keywords> tags are appearing twice. There is truly excessive javascript. And everything has been put in tables.</keywords> How much do you think this is really impacting the opportunity to rank? There has been quite a bit of discussion recently along the lines of is on-page SEO impacting anymore. I just want to be sure before I recommend a whole heap of code changes that could cost her a lot - especially if the impact/return could be miniscule. Should it all be cleaned up? Many thanks
Technical SEO | | Chammy0 -
What's the SEO impact of url suffixes?
Is there an advantage/disadvantage to adding an .html suffix to urls in a CMS like WordPress. Plugins exist to do it, but it seems better for the user to leave it off. What do search engines prefer?
Technical SEO | | Cornucopia0 -
What is the best method to block a sub-domain, e.g. staging.domain.com/ from getting indexed?
Now that Google considers subdomains as part of the TLD I'm a little leery of testing robots.txt with something like: staging.domain.com
Technical SEO | | fthead9
User-agent: *
Disallow: / in fear it might get the www.domain.com blocked as well. Has anyone had any success using robots.txt to block sub-domains? I know I could add a meta robots tag to the staging.domain.com pages but that would require a lot more work.0