Hi, I’m building a personal website and I don’t want it to be used to train AI. In my robots.txt file I blocked:
- ChatGPT-User
- GPTBot
- Google-Extended
- FacebookBot
What bots should I also add? Are there any other ways to block AI bots?
IMPORTANT: I don’t want to block search engine crawlers, only bots that are used to train AI.
deleted by creator
Specifically what about AI don’t you agree with?
deleted by creator
How about a bubble like the internet? 90% of dotcoms failed in the 90s but the internet is alive and strong today. AI is just a tool, and from my experience an extremely useful one.
deleted by creator
Agreed that privacy can be a concern. Ideally it will be possible to run LLMs locally in the near future, but we’ll see.
I was about to ask the same question. It’s one thing to think of the potential impacts of AI technology, but to be “against AI” in the most general sense is, to me, a weird concept, especially considering AI is so many things.
Nice, thats what I am looking for!
deleted by creator
I did, most of them are used for AI or business search engines. I copied everything except Yandex.