How to block AI Crawler Bots using robots.txt filewww.cyberciti.bizexternal-link Cynicus Rex ( @CynicusRex@lemmy.ml ) Privacy@lemmy.mlEnglish • 1 month ago message-square18fedilinkarrow-up166
arrow-up166external-linkHow to block AI Crawler Bots using robots.txt filewww.cyberciti.biz Cynicus Rex ( @CynicusRex@lemmy.ml ) Privacy@lemmy.mlEnglish • 1 month ago message-square18fedilink
minus-square thanks_shakey_snake ( @thanks_shakey_snake@lemmy.ca ) linkfedilink9•1 month agoLie. Or at best, dangerously wrong. Like saying “Crosswalks make cars incapable of harming pedestrians who stay within them.”
minus-square JackbyDev ( @JackbyDev@programming.dev ) linkfedilinkEnglish1•1 month agoIt’s better than saying something like “there’s no point in robots.txt because bots can disobey is” though.
minus-square thanks_shakey_snake ( @thanks_shakey_snake@lemmy.ca ) linkfedilink3•1 month agoMaybe? But it’s not like that’s the only alternative thing to say, lol
minus-square ReversalHatchery ( @ReversalHatchery@beehaw.org ) linkfedilinkEnglish2•edit-21 month agoIs it, though? I mean, robots.txt is the Do Not Track of the opposite side of the connection.
Lie. Or at best, dangerously wrong. Like saying “Crosswalks make cars incapable of harming pedestrians who stay within them.”
It’s better than saying something like “there’s no point in robots.txt because bots can disobey is” though.
Maybe? But it’s not like that’s the only alternative thing to say, lol
Is it, though?
I mean, robots.txt is the Do Not Track of the opposite side of the connection.