Log Hero
Do you want to know which bots visit your site, and what they find there? Log Hero records every request your server answers and turns it into a clear picture of how search engines, AI crawlers and all the other bots move through your site.
Because the plugin works on the server, it sees what a tracking script never will. Crawlers do not run JavaScript, so they are invisible to ordinary analytics. Log Hero catches every one of them, including the requests to robots.txt and your sitemaps, the 404s they run into and the redirects they follow. No downloading log files, no grep.
Search bots, AI crawlers and everything else
Next to the classic search engine bots, Log Hero recognises the AI crawlers that now make up a large share of automated traffic, and it tells you which is which:
- Search engines such as Googlebot, bingbot, YandexBot, Amazonbot and Sogou.
- AI crawlers and assistants such as GPTBot, ClaudeBot and ChatGPT-User.
- SEO tools such as AhrefsBot and SemrushBot.
- Social networks and link previews such as facebookexternalhit.
Bots that identify themselves with an unknown user agent are grouped so you can still see how much traffic they cause.
Where your data lives
The data is available in two places, and you can use both.
In the Keyword Hero app. A bot activity dashboard: hits, error rate, how many bots are active and how fast your site answers them, daily traffic split per bot, the paths that get crawled most and the errors bots run into.
In Google Analytics 4. Log Hero pushes the data into your GA4 property, so you can build your own explorations across bot name, landing page and status code, and combine bot traffic with the rest of your reporting.
What you can analyse
Everything below is included, there is no paid tier holding features back.
- The path each bot requested, so you can see what gets crawled and how often.
- The bot name, checked against the IP address rather than taken at face value, so a crawler claiming to be Googlebot has to prove it.
- The user agent it identified itself with.
- IP addresses, used for that verification (see "External services" below for exactly what this plugin transmits).
- HTTP status codes, to find broken links, redirect chains and server errors.
- Response times, so you can see how long your site kept a bot waiting.
- The request method.
- Referrer and channel, to see how bots reached a page.
- Sessions and users, so you can follow a bot through your site the way you would a human visitor.
- Spam bots, marked as such, so you can tell junk traffic apart from the crawlers you care about.
Analysis in Google Analytics 4
Because the data lands in your own GA4 property, it stays yours and remains available even if you stop using Log Hero.
- Free form explorations across bot name, landing page and status code.
- Filtering and segmentation, regular expressions included.
- Export to Sheets, Excel or CSV, and access through the GA4 API.
- Dashboards and alerts alongside your existing reporting.
Monitoring critical technical issues
Log Hero helps you to identify many issues concerning technical optimization of your site and can derive quick action items to resolve these.
- Which bots crawl my site and how often?
- Which orphan pages has my site that are never crawled?
- Which status codes does my site return to search engines?
- Are my robots.txt and sitemap.xml crawled by search engine bots?
- How long does the bot need to download the resources of my site?
- Which AI crawlers read my content, and how much of my traffic do they account for?
- How much traffic never reaches my analytics because it does not run JavaScript?
- and much more. Visit log-hero.com! for more information
From within WordPress
- Visit 'Plugins > Add New'.
- Search for 'Log Hero'.
- Activate Log Hero in your plugins page.
- Follow the steps on "after activation" below.
Manually
- Upload the
loghero folder to the /wp-content/plugins/ directory.
- Activate the Log Hero plugin through the 'Plugins' menu in WordPress.
- Follow the steps on "after activation" below.
After activation
- Click on Settings > LogHero.
- Now you have to enter the API key you received in the sign-up flow on log-hero.com. If you don't have an API key, get one for free on https://log-hero.com!
- You're done. Bot traffic starts showing up in the Log Hero dashboard in the Keyword Hero app, and in the Google Analytics 4 property named "Log Hero - your-domain.com".
If you have any problems, write our support team.