Linux 软件免费装
Banner图

AJ Agent Crawl Optimizer – AI Readiness Scanner & llms.txt for WordPress

开发者 ajmaurya
更新时间 2026年11月12日 15:54
PHP版本: 7.4 及以上
WordPress版本: 7.1
版权: GPL-2.0-or-later
版权网址: 版权信息

标签

ai chatgpt ai crawlers llms-txt mcp

下载

1.0.0 2.0.0 2.1.0

详情介绍:

AJ Agent Crawl Optimizer is a free WordPress plugin that scans your site against 33 AI-agent-readiness checks — Cloudflare's 21 isitagentready.com checks plus 12 modelled on what hosted scanners like ora.ai and is-agentic.com grade for — scores it 0–100 with a letter grade and a Level 0–5 readiness ladder, fixes failing checks in one click, and lets you hand-curate the llms.txt that AI agents like ChatGPT, Claude, and Perplexity read. It makes your site legible to AI agents — and proves it. The built-in readiness scanner runs 33 checks against your live site, scores you 0–100 with a letter grade (A+ to F) and on the Level 0–5 agent-readiness ladder (Not Ready → Basic Web Presence → Bot-Aware → Agent-Readable → Agent-Integrated → Agent-Native), and shows the full evidence trail — every request and response — behind every verdict. Failing checks get a Fix now button that enables the right feature and re-scans that single check to prove it went green; two more (Organization details, a real MCP server) get a guided button to the right screen; anything the plugin can't fix in WordPress (DNS records, server config) gets a copy-paste prompt for your coding agent. This is the technical, agent-readiness layer of answer engine optimization (AEO) and generative engine optimization (GEO): it improves your AI visibility by making your content legible to AI agents through llms.txt, JSON-LD, Markdown negotiation, and AI crawler rules. It is not a content SEO plugin — it doesn't write titles or meta descriptions, and it doesn't track brand mentions or AI citations. It prepares your WordPress site so AI search engines and agents can read, trust, and route on it, then verifies that over real HTTP. The scan → fix → verify loop Each publishing capability is a separate toggle under Agent Ready → Settings and ships opt-in (everything starts off). On first activation, a Quick Setup wizard suggests defaults based on your environment (it skips JSON-LD when an SEO plugin is detected) — re-runnable any time. Curated llms.txt — you decide what agents read Most plugins generate llms.txt from fixed rules and hand you the result. This one hands you the controls. Discovery — help agents find what your site offers Presentation — format content for agents Declarations How it compares to other AI / llms.txt plugins Most AI and llms.txt plugins only generate a file from fixed rules. AJ Agent Crawl Optimizer also scans your live site over real HTTP, scores it 0–100 with a grade and the Level 0–5 ladder, shows the request/response evidence behind every verdict, and fixes failures in one click — then re-scans to prove the check turned green. It writes per-bot robots.txt rules for 20 AI crawlers and lets you hand-curate what AI agents read instead of auto-dumping every post. Scan → score → fix → verify, not generate-and-hope. Why use it From the WordPress plugin directory
  1. In your WP admin, go to Plugins → Add New and search for "AJ Agent Crawl Optimizer".
  2. Click Install Now, then Activate.
  3. The Quick Setup wizard runs automatically on first activation — review the recommended toggles and click Apply.
  4. Open Agent Ready → Dashboard, run your first scan, and use Fix now on failing checks. Toggles live under Agent Ready → Settings.
Manual install
  1. Download the plugin zip.
  2. Upload the aj-agent-crawl-optimizer folder to /wp-content/plugins/.
  3. Activate AJ Agent Crawl Optimizer from the Plugins screen.
  4. Visit Agent Ready → Dashboard to scan, or Agent Ready → Settings to run the wizard / configure manually.
IndexNow setup (optional) If you want Bing and Yandex to re-crawl your content within minutes of publish:
  1. Generate a key at https://www.bing.com/webmasters/indexnow.
  2. Paste it into the IndexNow API Key field on the settings page.
  3. Enable the IndexNow toggle. The plugin hosts the key file at /{key}.txt for ownership verification automatically.

屏幕截图:

  • Evidence timeline — the fetch/parse/conclude audit trail behind a failing check, with the real request and response.
  • Bulk fix sheet — "Fix all safe items", plus copy-paste agent prompts for the fixes that need DNS or server access.
  • llms.txt curation — intro, per-content-type sections, custom Markdown block, and a live preview with entry/byte/token counts.
  • Settings — per-feature toggles, each with a live link to the endpoint it publishes.
  • AI bot rules — per-crawler allow/block policy for 20 AI crawlers, plus Content-Signal preferences.
  • Built-in feature guide — what every feature actually does, in the contextual Help tab.
  • Quick Setup wizard — environment-aware recommendations on first activation.
  • Per-post curation — the "Agent Ready (llms.txt)" panel in the editor: keep an entry out of the agent indexes, or write the one-line summary an AI agent sees instead of the excerpt.

升级注意事项:

2.1.0 Adds a 0–100 score and grade, 12 new checks with one-click or guided fixes, MCP Adapter integration, Organization details for JSON-LD, a revamped Dashboard and a score badge. Fixes Markdown negotiation serving HTML. Features stay opt-in; settings and bot policies are preserved. 2.0.0 Major update: adds an agent-readiness scanner (21 checks, Level 0–5, evidence trails), one-click fixes, a curated llms.txt editor, REST API and WP-CLI. All features stay opt-in and your existing settings are preserved. The menu moves to a new top-level Agent Ready. 1.0.1 Spec-compliance fixes: Agent Skills index and MCP Server Card now validate against external agent-readiness scanners; markdown negotiation is cache-safe (Vary: Accept); OpenAPI at /openapi.json. 1.0.0 Initial release.

常见问题:

Is my WordPress site ready for AI agents?

Run the built-in readiness scanner. AJ Agent Crawl Optimizer checks your live site against 33 AI-agent-readiness checks — Cloudflare's 21 isitagentready.com checks plus 12 modelled on what ora.ai / is-agentic.com grade for — scores you 0–100 with a grade and a Level 0–5 ladder from Not Ready to Agent-Native. Every verdict shows the real request and response behind it, and failing checks get a one-click Fix now button that re-scans to prove it passed.

What is llms.txt and does my WordPress site need one?

llms.txt is a plain-Markdown file at /llms.txt that gives AI agents a curated index of your most important content, proposed at llmstxt.org. AJ Agent Crawl Optimizer lets you hand-curate it: write the intro, pick which post types (and WooCommerce products) appear, and set a per-post "Summary for AI agents" that overrides the excerpt. It is a surface AI agents can route on — not a guaranteed path to AI citations, but a machine-readable map of what your site is for.

How do I allow or block GPTBot and other AI crawlers in WordPress?

The plugin writes explicit robots.txt rules for 20 AI crawlers — GPTBot, ChatGPT-User and OAI-SearchBot (OpenAI), ClaudeBot, Claude-User and Claude-Web (Anthropic), PerplexityBot and Perplexity-User, Google-Extended, CCBot and Bytespider among them — each allowed or blocked per a per-bot policy. Presets like "Allow search & user requests, block training" make it a single choice. Remember: to be cited by an AI engine you must allow its crawler.

How do I get my WordPress content found by ChatGPT and Perplexity?

There is no guaranteed path, but three things help and this plugin does all three: allow the AI search crawlers (OAI-SearchBot, PerplexityBot, Google-Extended), publish the machine-readable surfaces agents look for (llms.txt, JSON-LD, Markdown negotiation), and write a clear "Summary for AI agents" on key pages. The readiness scanner then verifies each of these is actually live over real HTTP — so you fix what's broken instead of guessing.

What is the difference between llms.txt and llms-full.txt?

/llms.txt is a curated index — a Markdown list of your key pages with links and short summaries, so an agent can decide what to fetch. /llms-full.txt serves the full Markdown content of those same curated entries in one file, for agents that want everything inline. The plugin serves both, and password-protected content is always excluded from each.

How is this different from an llms.txt generator or an AI SEO plugin?

Most plugins only generate an llms.txt file from fixed rules. AJ Agent Crawl Optimizer is an agent-readiness scanner and fixer: it audits your live site over real HTTP against 33 checks, scores it 0–100 with a grade and a Level 0–5 ladder, shows the request/response evidence behind every verdict, fixes failures in one click, and re-verifies. It covers the answer engine optimization (AEO) and generative engine optimization (GEO) groundwork — structured data, AI crawler rules, and MCP discovery — that plain generators and content-writing SEO plugins skip.

Does it work with WooCommerce?

Yes. WooCommerce products are detected automatically and can appear as their own section in your curated llms.txt, so AI agents can see your catalogue. You control whether products appear, how many, and in what order, and you can exclude individual products or give any product a per-product "Summary for AI agents."

What's the difference between the Dashboard and Settings screens?

Dashboard is the verifier: it scans your live site over real HTTP, shows your score, grade and Level 0–5 with evidence for every check, and fixes failures with one click. Settings is the switchboard: the per-feature toggles, wizard, and testing tools. A toggle can be ON in Settings while the Dashboard still shows a fail — that gap usually means a page cache or server rule is intercepting the endpoint, and the evidence view shows exactly what happened.

How are the score and the readiness Level calculated?

The scanner runs Cloudflare's 21 isitagentready.com checks plus 12 checks modelled on the ora.ai / is-agentic.com catalog and the AgentReady.org standard. The 0–100 score weights every check; checks that don't apply or can't be reached drop out of the denominator, bonus checks (Web Bot Auth, DNS-AID, A2A card, commerce protocols) add up to 5 points and never subtract, and grades use the ora.ai bands (A+ 95, A 86, B 70, C 48, D 28). The Level 0–5 ladder stays identical to Cloudflare's: Level 1 needs 2 of robots.txt/sitemap/Link headers; Level 2 adds AI bot rules and Content Signals; Level 3 adds Markdown negotiation; Level 4 adds one of MCP card/Agent Skills/API catalog/A2A card; Level 5 adds two of Web Bot Auth, all four integrations, and auth metadata. Nothing counts unless the scan verifies it — enabling a toggle earns nothing until the endpoint actually responds. Commerce checks are informational and never scored.

My host blocks /.well-known/ paths or doesn't allow creating files — will the plugin still work?

The plugin never creates files: every endpoint (/.well-known/, /llms.txt, /openapi.json, /auth.md, /agent-ready-badge.svg) is served virtually by WordPress, so hosts that forbid file creation are fully supported. What can break things is the web server intercepting requests before WordPress runs — typically an nginx dot-path deny rule returning 403 on /.well-known/, or a static-file rule returning 404 for /llms.txt-style paths. The readiness scan detects exactly this (feature enabled + server blocking) and the Dashboard shows a hosting notice with copy-paste nginx and Apache fixes you can apply yourself or forward to your hosting support.

Can I control what goes into llms.txt?

Yes — it is curated, not auto-generated from fixed rules. Under Agent Ready → llms.txt you write the intro, pick which post types appear (custom post types and WooCommerce products show up automatically), and set each section's heading, item count and order, plus your own Markdown block. A live preview shows the file as you edit, before you save. Individual entries are steered from the editor: the "Agent Ready (llms.txt)" panel has an Include in llms.txt toggle (excluded pages stay out of both /llms.txt and /llms-full.txt) and a Summary for AI agents field that overrides the excerpt for that entry — write it for a model deciding whether to fetch the page. The defaults keep the same sections and ordering as before, so upgrading needs no attention until you decide to curate.

Can I run scans from the command line or scripts?

Yes: wp agent-ready scan (add --format=json for machines or --format=agent for a markdown fix report), wp agent-ready fix <check> or --all-safe, and wp agent-ready status. Add --min-score=<n> or --min-level=<n> to scan or status for a CI gate that exits non-zero. The same operations are available over REST at /wp-json/ajaco/v1/scan, /wp-json/ajaco/v1/fix and /wp-json/ajaco/v1/external (admin capability required; /wp-json/ajaco/v1/health is public).

Will this conflict with my SEO plugin?

No. JSON-LD and the identity meta tags auto-suppress when Yoast SEO, Rank Math, All in One SEO, SEOPress, The SEO Framework, Slim SEO, Squirrly SEO, Schema Pro, or SASWP is active. The settings page shows a clear notice when this happens. Our robots_txt filter runs at PHP_INT_MAX priority, so any rules your SEO plugin adds (sitemap URLs, custom Disallow, etc.) are preserved — our Content-Signal line is appended last.

Does this change anything for my regular site visitors?

No. Browsers send Accept: text/html and get the normal HTML response. The .well-known/..., /llms.txt, and /openapi.json endpoints are paths a normal user never visits. What the enabled features add to a page is invisible to visitors: a JSON-LD <script> block and Open Graph meta tags (when no SEO plugin is active), a few <link rel> tags and Link headers pointing at the agent-facing files, and a small WebMCP script tag that no-ops in browsers without the API.

My site is multisite. Does it work?

Yes. Every endpoint resolves at both the root and per-subsite paths automatically — /llms.txt, /blog/llms.txt, /.well-known/api-catalog, /blog/.well-known/api-catalog, etc. Each subsite has its own settings.

How do I delegate plugin access to a non-admin role?

Use the ajaco_required_capability filter. The plugin defaults to manage_options (admin-only); change it to any capability of your choice, e.g. edit_pages for editors: add_filter( 'ajaco_required_capability', function () { return 'edit_pages'; } );

I enabled IndexNow but I'm not seeing pings to Bing or Yandex.

Check three things: (1) the IndexNow API Key field is filled in, (2) you're publishing a post of a public post type — revisions and autosaves are skipped, (3) you're on production. Pings are non-blocking, so failures are silent. Tail your server log for outbound requests to api.indexnow.org.

Can I add custom skills, schemas, or sections to llms.txt? Which filter hooks exist?

Yes. The plugin exposes nineteen filter hooks. Examples: add_filter( 'ajaco_required_capability', function () { return 'edit_posts'; } ); Delegate plugin access to a non-admin role. add_filter( 'ajaco_skill_definitions', function ( $skills ) { return $skills + [ 'products' => [ 'type' => 'information-retrieval', 'description' => 'WooCommerce products', 'endpoint' => rest_url( 'wc/v3/products' ) ] ]; } ); Register custom skills that ship in the Agent Skills Index as SKILL.md artifacts with verifiable sha256 digests. add_filter( 'ajaco_content_signal', function () { return 'ai-train=yes, search=yes, ai-input=yes'; } ); Customize the Content-Signal directive (e.g. permit AI training). Other hooks: ajaco_json_ld_graph (add Product, Recipe, Event… schema), ajaco_llms_txt_content and ajaco_llms_full_txt_content (append to or replace the files), ajaco_llms_post_types, ajaco_llms_exclude_post, ajaco_openapi_spec, ajaco_api_catalog_linkset, ajaco_ai_catalog, ajaco_mcp_server_card, ajaco_mcp_endpoint (declare an MCP server the plugin cannot detect), ajaco_auth_md_content, ajaco_ai_bot_list, ajaco_ai_bot_policy, ajaco_commerce_signals, ajaco_scan_sslverify, ajaco_active_seo_plugin. The Help tab's "For Developers" section on the settings page lists every hook with a description. Per-post curation is stored in two post meta fields: _ajaco_llms_exclude (boolean) and _ajaco_llms_summary (string, 300 chars). Both are registered with show_in_rest, so they round-trip through the REST API and WP-CLI (wp post meta update 42 _ajaco_llms_summary "...").

How do I undo everything and start over?

Use the Reset to Defaults button under the Save Changes button on the settings page. It clears every plugin option, wipes cached endpoint outputs, and shows a confirmation dialog before wiping anything.

Performance impact?

Minimal. The OpenAPI document is cached for a day in a transient and only regenerates on plugin activation/deactivation or theme switch. The llms.txt body is cached for an hour and invalidates on post edits, site name/description changes, and setting toggles. Endpoint handlers run on init only when the request path matches; on every other request they early-return after one path check. Markdown conversion happens in an output-buffer callback for requests that ask for Markdown, so normal page loads are untouched. The hosted-scanner lookup runs only when you click it and is cached for an hour.

How do I see if AI agents are actually using my site?

The plugin doesn't include a built-in access log. To verify activity, check your server access log for User-Agents like GPTBot, ClaudeBot, OAI-SearchBot, Google-Extended, PerplexityBot, CCBot, Bytespider, etc. hitting any of the plugin endpoints.

Can I disable a feature without deactivating the plugin?

Yes. Every feature is an independent toggle. Uncheck what you don't want and Save Changes. The corresponding behavior reverts immediately on the next request — no rewrite flush, no cache flush needed (the plugin handles cache invalidation itself).

Which external services does the plugin connect to, and what is sent?

Only in three situations you trigger yourself; otherwise the plugin makes no outbound requests at all. Publishing features (manifests, JSON-LD, robots.txt) only answer incoming requests.

What data does the plugin store, and what happens on uninstall?

Everything lives on your own server — no telemetry, analytics or third-party logging. Option rows store toggle states, the per-bot crawler policy, the Organization details you enter, the llms.txt curation config, the most recent scan result (with evidence snapshots of your own site's responses), a compact score history (at most 30 entries) and the IndexNow key (plain text — keep your database secure). Transients cache the OpenAPI document, the llms.txt bodies and the last hosted-scanner lookup. No request data, IP addresses, User-Agents or visitor information is recorded, and the plugin sets no cookies. Deleting the plugin (not just deactivating) removes every option, the per-post curation meta and all transients; multisite networks have every site cleaned in turn.

更新日志:

2.1.0 2.0.0 1.0.1 1.0.0