zydronium/neocities
1
0
Fork 0
mirror of https://github.com/neocities/neocities.git synced 2025-07-03 09:33:25 +02:00
Code Issues Projects Releases Packages Wiki Activity

remove Applebot from list (Applebot-Extended determins LLM usage)

Browse source
This commit is contained in:
Kyle Drake 2024-12-30 17:40:19 -06:00
parent 2cfc848ab2
commit bb30252000
1 changed files with 1 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

1
views/templates/robots.txt
View file

@ -9,7 +9,6 @@ Allow: /
#User-agent: Ai2Bot-Dolma #User-agent: Ai2Bot-Dolma
#User-agent: Amazonbot #User-agent: Amazonbot
#User-agent: anthropic-ai #User-agent: anthropic-ai
#User-agent: Applebot
#User-agent: Applebot-Extended #User-agent: Applebot-Extended
#User-agent: Bytespider #User-agent: Bytespider
#User-agent: CCBot #User-agent: CCBot