style variation resulting in massive google crawls

[dethfire]

Looks like the style-variation feature has dynamic parameters. Google doesn't respect nofollow, so I've got hundreds of thousands of pages google is trying to index. I had to do a robots block and that didn't really help as now I have hundreds of thousands of these pages indexed and blocked by robots according to google search console. I think I'll have to hide this all to the public. Is there any easy way to do this or put some conditionals in templates?

 

[Russ]

So in PAGE_CONTAINER we have two instances where we add the variation:

If you click into the editor, hit ctrl-f and search for:

<xf:if is="$xf.visitor.canChangeStyleVariation($xf.style) AND property('xbStyleVariationHeaderLink')">

Replace with:

<xf:if is="$xf.visitor.canChangeStyleVariation($xf.style) AND property('xbStyleVariationHeaderLink') AND $xf.visitor.user_id">

That will make it so only logged in users see it. Guests may have the page cached so it might take a bit for the link to disappear.

If you want to hide XenForo's switcher:

<xf:if is="$xf.visitor.canChangeStyleVariation($xf.style)">

Replace with:

<xf:if is="$xf.visitor.canChangeStyleVariation($xf.style)" AND $xf.visitor.user_id">

But not sure if hiding them ALL is a good choice. Maybe google is just picking up ours since it's in the main nav... I can look into adjusting this for the next release.

 

[dethfire]

Thank you! I'll give that a go.

 

[Russ]

Thank you! I'll give that a go.

Just following up on this, it appears 2.4 is going to formally address this issue:

Future fix - Google tries to crawl links from style variation chooser menu

We nofollow the links and noindex the pages. Google doesn't listen or care. There's not a whole lot more that we can do out of the box, but blocking the pages via robots.txt should help. This does not really help: If crawling is blocked via robots.txt Google still indexes those pages without...

xenforo.com