Crawler Record
| 开发者 | dizzysoft |
|---|---|
| 更新时间 | 2025年9月30日 02:15 |
| PHP版本: | 7.4 及以上 |
| WordPress版本: | 6.8 |
| 版权: | GPLv2 or later |
| 版权网址: | 版权信息 |
详情介绍:
Crawler Record logs the last time specific user agents (for search and AI chat/LLMs) accessed your content including:
- Bing
- ChatGPT (OpenAI)
- Claude (Anthropic)
- Perplexity
- DuckDuckGo
- Admin page: a grouped list of all crawlers, with last seen time, last page, and robots status for the crawler.
- Admin Bar (front-end): quick-glance status for the current singular or URL context (no dropdowns in wp-admin).
- Per Page/Post: From the edit screen on any page or post, you can see that last time each crawler visited that page.
- Supports common user-agent variants for Google, Bing, OpenAI (ChatGPT), Anthropic (Claude), Perplexity, and DuckDuckGo.
- Robots status computed from local robots.txt (physical or virtual) without outbound requests.
- Clear UI with grouped sections, microsecond timestamps, and a small diagnostics toggle showing the matched robots group and rule.
- robots.txt wildcards (
*) and end-of-line marker ($) are not interpreted; matching is prefix-based only. Future versions may add full spec support.
安装:
- Upload the plugin folder to
/wp-content/plugins/, or install via the admin Plugins screen. - Activate the plugin.
- Visit Crawler Record under Admin → Crawler Record to review crawler activity.
常见问题:
Why are there no crawler visits recorded?
This plugin can only begin to track crawler/bot visits from the time you have installed the plugun- not before. It may take several days or weeks (depending upon the popularity of your website) before any come for a visit.
Why do I see “Blocked by WordPress setting”?
If Settings → Reading → Discourage search engines is enabled, so none of these systems are allowed to read the pages on your site.
更新日志:
0.7.0
- The robots.txt checker wasn't actually working. It does now.
- When you look at the report in the admin section, the robots checker is looking for a site-wide rule; when you look at the back-end of a page, the robots checker is looking at that particular page.
- If a page is blocked by the robots.txt file, a link appears sending you to the robots.txt file.
- Added a video to explain how to use this plugin.
- Clarified distinction between Googlebots.
- Better distinguishes Bingbots.
- Cosmetic changes to page in admin section.
- More clear documentation.
- First public release