| 开发者 | charthouse |
|---|---|
| 更新时间 | 2026年10月26日 20:28 |
| PHP版本: | 7.4 及以上 |
| WordPress版本: | 7.1 |
| 版权: | Apache-2.0 OR GPL-2.0-or-later |
| 版权网址: | 版权信息 |
License:, Permits:, Prohibits:, Compensation:), is applied to the request path.Crawler-Price, Crawler-Price-Rail and Link headers plus a structured JSON payment offer when policy says so and your site can be paid; 403 "not licensed" when it cannot be paid yet or policy disallows without compensation; otherwise the request passes through with X-CrawlerToll-Action, X-CrawlerToll-Operator and X-CrawlerToll-Bot-Name headers for downstream logging./robots.txt (your RSL-style policy, added through WordPress's standard robots_txt filter) and /.well-known/context-license.json (machine-readable licence metadata built from your settings).
What this plugin does NOT do
unlock.succeeded events to your own systems.crawlertoll folder to /wp-content/plugins/, or install via the Plugins → Add New menu.Compensation: 5000 micros, i.e. $0.005). The bundled catalogue recognises 184 declared user-agents, including PerplexityBot; only those seven are priced, and the rest are allowed until you add a rule. Matching is the User-Agent string only — not every AI visit, and not a ceiling. Crawlers named in the policy are shown that price once your site can be paid (Stripe keys, a USDC address or a payment URL); until then they are told the content is not licensed. Adjust price, currency, rail and policy to fit your site.curl -sI -H 'user-agent: GPTBot/1.2' https://your-site.example/
Once your site can be paid you should see a 402 Payment Required response with the Crawler-Price header; before that, a 403 "not licensed" answer.No. Card payments run on your own Stripe account and USDC is paid to your own wallet. CrawlerToll holds no Stripe secret, no platform account, no facilitator key and no wallet key. Our only revenue is the optional Pro subscription.
Yes. Encryption controls who can read a post, not what a paying reader does afterwards, and every buyer of a post receives the same key. This is true of any paywall. What sealing removes is the free leaks: the article sitting in the page, the RSS feed, the REST API or a cache.
No payment can start while the unlock service is down: the wall shows that unlocking is unavailable. If a card payment goes through and the article doesn't open, the reader's browser remembers the payment and finishes it on the next try in the same browser, without a second charge. The rest of your site keeps working normally. There is no SLA at launch. See https://crawlertoll.com/docs/unlock-service for what the service stores.
Yes. CrawlerToll Pro adds access tiers and bundles, read packs, summaries sold separately, metered free articles, per-path pricing, per-crawler rail routing, a revenue dashboard, filterable bot-request logs with CSV/JSON export, email alerts, content provenance, unlock webhooks and access revocation. The free plugin's crawler recognition, 402 responses and sealed paywall with both payment rails remain fully functional without it. See crawlertoll.com/pro.
Yes. Readers paying with a USDC wallet can pay from $0.001 per article, and the money goes straight to your wallet. Card payments have a $0.50 minimum, so for card readers sell a read pack (Pro), for example 10 articles for $5, paid once on your own Stripe account.
Yes. Set "Article price for AI agents" in Settings. Declared AI crawlers such as ClaudeBot and GPTBot are quoted and charged that price, in USDC. It can be higher than the reader price, not lower. A bot that hides who it is pays the reader price, and the article stays encrypted either way.
No — payment settles before access. When a crawler or reader pays, the money lands directly in your own Stripe account or wallet, and only then is the content unlocked. There is no credit, no monthly billing cycle, and nothing to chase. Every paid unlock leaves a receipt you can read back in Settings → CrawlerToll (recent unlocks; the Pro revenue dashboard totals them by rail). On the Stripe rail the payment is in your own Stripe account; readers can ask for Stripe's emailed receipt on the card step. Stripe does not add tax or issue invoices for these payments unless you set that up: you are the seller, so VAT or sales tax on what you sell is yours to handle (ask your accountant). On the x402 rail, the on-chain transaction hash is the receipt, linked in the unlock record to the exact article and time.
Yes, normally 14 days, but for content that unlocks immediately it ends once the reader has agreed to immediate access and acknowledged losing it. With Settings → CrawlerToll → "EU/UK right of withdrawal" on (automatic for sites that look European), the paid options stay locked until the reader ticks that box. The consent time is stored on your Stripe payment and on the unlock receipt, and the reader can save a receipt that states it. You are the seller; this helps you meet the rule and is not legal advice.
No. The default policy targets specifically AI crawlers — GPTBot, ClaudeBot, Google-Extended (Google's training-data crawler, not Googlebot), Applebot-Extended (Apple's training crawler, not Applebot), etc. Search-engine crawlers continue to index your site normally unless you explicitly add them to the policy. PerplexityBot is recognised and allowed until you add a rule.
Only if your rules say so. The plugin ships with seven default rules and a catalogue of 184 declared user-agents. PerplexityBot and the other declared agents are recognised and allowed until you add a rule for them. Recognition is the User-Agent string only — it is not every AI visit, and community lists are larger and still growing.
Yes. They run at different layers and there is no integration between them. Pay Per Crawl runs at Cloudflare's edge; CrawlerToll runs inside WordPress on any host and adds the sealed paywall, the card rail and the human-reader unlock. You can use both, or just one.
If you already use a hosted paywall or licensing service, choose the custom payment URL rail and the 402 response will point crawlers at it via a Link: <…>; rel="payment" header. Sealing and the built-in rails are not required.
Activate it on each site. Every site has its own settings. Network activation has not been tested.
Edit the policy textarea in Settings → CrawlerToll. The policy uses standard robots.txt syntax extended with RSL-style directives: License:, Permits:, Prohibits: and Compensation:. See rslstandard.org for the vocabulary they are modelled on.
The crawler check is a user-agent match and a policy lookup on each front-end request, with no external calls. A sealed post is encrypted the first time it is viewed after it changes; later views reuse the stored ciphertext.
Plugins → Deactivate → Delete. Your /robots.txt returns to its previous form. Settings are kept unless you tick "Remove all data on uninstall" in Settings → CrawlerToll first; then deleting the plugin also removes every option (including your Stripe keys), its tables, per-post data and scheduled tasks.
Post in the plugin's support forum on WordPress.org, or email hello@crawlertoll.com.
Yes. CrawlerToll limits payment attempts per reader, and reads the reader's address from the proxy's headers when the request comes from Cloudflare or from your own server or private network. If your proxy uses other public addresses, add them with the crawlertoll_trusted_proxies filter (a list of IPs or CIDR ranges). Headers from any other source are ignored, so a visitor cannot fake their address.
match() and all() merge the weekly crawlertoll_bot_catalogue option over the bundle, so a successful catalogue fetch changes detection. The bundle is used when that option is missing or unusable. Bundled rows win on a name or token collision.