Linux 软件免费装

在这里免费安装WordPress以及其他软件

Banner图

CrawlWatch – AI Bot Insights

开发者 kstmonowar
更新时间 2026年9月23日 00:31
PHP版本: 7.4 及以上
WordPress版本: 7.1
版权: GPLv2 or later
版权网址: 版权信息

标签

gptbot ai crawler llms txt ai bots bot traffic

下载

1.1.1 1.0.0 1.1.0 1.1.2

详情介绍:

CrawlWatch shows you the invisible traffic: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and 40+ AI crawlers visiting your WordPress site. Read the llms.txt guide — what llms.txt is, the exact format that works, and the mistakes that make one useless · AI bot traffic in WordPress — how to see it, measure it, and decide what to do about it · Plugin page · Development on GitHub — report issues or contribute. No API key. No external service in V1. Lightweight: normal visitors cause zero extra queries.

安装:

  1. Upload the plugin folder to /wp-content/plugins/ or install from Plugins > Add New.
  2. Activate through the Plugins screen.
  3. Go to CrawlWatch in the left menu and follow the 3-step setup.
  4. Open yoursite.com/llms.txt to verify (plain permalinks or subdirectory installs: use yoursite.com/?crawlwatch_file=llms.txt).

屏幕截图:

  • Bots table with filter, search and pagination.
  • llms.txt editor + robots.txt manager.
  • Settings with retention, privacy notice and danger zone.
  • AI-ready llms.txt served to crawlers.
  • llms-full.txt with full post content for deep AI context.
  • AI Readiness page with missing excerpts and alt text plus edit links.

升级注意事项:

1.1.2 Privacy + reliability + performance fixes. No settings change needed; rescan schema once to fill the batched report. 1.1.1 Composer distribution via Packagist plus internal QA tooling. No functional changes for existing installs.

常见问题:

Does this send data outside my site?

No. Everything is stored locally in one table (wp_crawlwatch_logs). IPs are SHA-256 hashed (IPv4 last octet and IPv6 /64 anonymized first). No third-party calls — the only HTTP requests CrawlWatch ever makes are loopback fetches of your own posts for the on-demand schema scan.

Will it slow my site?

No. Normal human visits do zero extra queries. Only detected AI hits do one dedupe check plus one insert. No CSS/JS on the frontend.

How long are logs kept?

Default 30 days (7/14/30/90/180 selectable). Daily WP-Cron auto-deletes older rows.

Does it work without Yoast/RankMath?

Yes. It only detects whether an SEO plugin is active (for the schema part of the score). No integration needed.

Will blocking a bot hurt my SEO?

Blocking AI training bots (GPTBot, ClaudeBot) does not affect Google search. Never block regular Googlebot — CrawlWatch only manages AI bots, and Google-Extended asks for confirmation first.

Why do I see few or zero hits with a cache plugin?

Tracking runs in PHP. Full-page caches (WP Rocket, LiteSpeed, W3 Total Cache, Cloudflare APO) serve cached pages without running PHP, so those visits are never logged. CrawlWatch shows a notice on its own pages when a known cache is active.

How do I block GPTBot in WordPress?

Add a Disallow rule for GPTBot in robots.txt — CrawlWatch writes it for you with preview. That keeps your content out of OpenAI training data. It does not remove anything already trained on.

Does blocking GPTBot stop ChatGPT from citing my site?

No — that is a different bot. GPTBot governs training; OAI-SearchBot governs ChatGPT Search citability. Blocking GPTBot hides you from training while leaving you citable. The same split exists at Anthropic (ClaudeBot vs Claude-SearchBot). Google-Extended only affects Gemini training, never AI Overviews.

Where does llms.txt go on a WordPress site?

At the domain root (yoursite.com/llms.txt). CrawlWatch generates and serves it with no file writes. Anything deeper and most consumers never look.

Is llms.txt actually read by AI assistants?

Some read it, some do not — adoption is limited and Google does not use it for search. It costs nothing to have, and it makes the assistants that do read it describe you accurately.

更新日志:

1.1.2 1.1.1 1.1.0 1.0.0