Linux 软件免费装
Banner图

Latchbot - AI Crawler Control

开发者 maksymkotenko
更新时间 2026年11月14日 19:50
PHP版本: 7.4 及以上
WordPress版本: 7.1
版权: GPLv2 or later
版权网址: 版权信息

标签

robots.txt ai crawlers block ai ai bots tdmrep

下载

0.1.0

详情介绍:

AI companies send crawlers for three different reasons: to collect training data, to build an AI search index, and to fetch a page because a user asked a chatbot about it. Latchbot lets you decide per crawler, from one screen under Settings > AI Crawlers. Features On activation Latchbot blocks the AI training crawlers and allows AI search and user fetches. Nothing else changes until you turn it on. Privacy The visit log stores the crawler name, the day, the kind of page and two counters. It never stores IP addresses, page addresses, cookies or any other visitor data, and it keeps 90 days. Latchbot makes no requests to other sites and loads nothing from them. Good to know For developers

安装:

  1. Install and activate Latchbot from Plugins > Add New.
  2. Open Settings > AI Crawlers, pick a preset or set each crawler, and save.
  3. Open your robots.txt from the link on the settings screen to check the result.

屏幕截图:

  • Signals for all crawlers: noai, TDMRep and Content Signals.
  • The robots.txt additions and what AI crawlers read.

常见问题:

Does blocking GPTBot or Google-Extended hurt my Google ranking?

No. Google-Extended only controls whether Google may use your pages for its AI models; Google Search keeps crawling with Googlebot, which Latchbot never touches. The same goes for Applebot-Extended and Applebot.

What is the difference between training, search and user fetches?

Training crawlers collect pages to train AI models. AI search crawlers build the index that AI answers cite and link to. User fetches happen when someone asks an assistant about your page. Many sites block training and allow the other two, which is the first preset.

Does it work with a caching plugin?

The robots.txt rules, the TDMRep file and the meta tags are part of the page, so cached copies keep them. The X-Robots-Tag and tdm-reservation headers, the visit counts and the 403 answers need WordPress to run, so they cover only requests your cache passes through; some caches store headers too.

My site is in a subdirectory.

Crawlers read robots.txt and /.well-known/tdmrep.json only at the root of the domain. Latchbot shows a warning; copy the rules from the preview into the files at the root.

更新日志:

0.1.0