Doing a Google search with the string given in #1 I get a lots of New Posts links to:
https://www.diyaudio.com/community/whats-new/posts/******/
where the '******' are old update ID's, ie. outdated links that doesn't bring forth the indexed post.
Looking at robots.txt file there's no particular restrictions added:
https://www.diyaudio.com/robots.txt
Another link that probably shouldn't be updated is:
https://www.diyaudio.com/community/
I guess it's, either update the robots.txt ,
https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt?csw=1
...or add individual noindex <meta> tags in the HTML header to those two constantly updating New Posts pages.
https://developers.google.com/search/docs/crawling-indexing/block-indexing
by adding the:
but because Google and probably other crawlers have already indexed these two pages, one can also make them de-index with the following line:
See also:
https://www.ilfusion.com/how-to-prevent-google-from-indexing-certain-web-pages
https://www.diyaudio.com/community/whats-new/posts/******/
where the '******' are old update ID's, ie. outdated links that doesn't bring forth the indexed post.
Looking at robots.txt file there's no particular restrictions added:
https://www.diyaudio.com/robots.txt
Code:
User-agent: *
Allow: /
Sitemap: https://www.diyaudio.com/community/sitemap.xml
Another link that probably shouldn't be updated is:
https://www.diyaudio.com/community/
I guess it's, either update the robots.txt ,
https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt?csw=1
...or add individual noindex <meta> tags in the HTML header to those two constantly updating New Posts pages.
https://developers.google.com/search/docs/crawling-indexing/block-indexing
by adding the:
<meta name=”robots” content=”noindex”>
but because Google and probably other crawlers have already indexed these two pages, one can also make them de-index with the following line:
<meta name=”robots” content=”noindex,nofollow”>
See also:
https://www.ilfusion.com/how-to-prevent-google-from-indexing-certain-web-pages
Last edited: