asudox@lemmy.worldtoPrivacy@lemmy.ml•How to block AI Crawler Bots using robots.txt file
0·
3 months agoBlock? Nope, robots.txt does not block the bots. It’s just a text file that says: “Hey robot X, please do not crawl my website. Thanks :>”
forgejo: https://codeberg.org/asudox
matrix: https://matrix.to/#/@asudox:matrix.org
aspe:keyoxide.org:D63IYCGSU4XXB5JSCBBHXXFEHQ
Block? Nope, robots.txt does not block the bots. It’s just a text file that says: “Hey robot X, please do not crawl my website. Thanks :>”
Switched to Linux a year ago. Haven’t looked back since. I didn’t even know Windows was getting those creepy ass advertisements all over the operating system if it weren’t for the news here. Now I don’t want to go back more.
Surprised that happened. Very rare to see that these days.