Log file analysis without log files.
Screaming Frog Log Analyser is an excellent desktop tool for sites that can access their own server logs. Shopify, managed WordPress, Webflow, Squarespace, and Magento Commerce do not give you those. Botsanalyser captures the same crawl data directly from the CMS, no log export required.
Desktop log-file analyser. Needs raw server logs.
- ✓Deep analysis of every request in a raw log file.Response codes, crawl frequency, discovered vs known URLs.
- ✓Custom reports, filtered exports, crawl comparison.Desktop software, full control.
- ✗Requires a raw access log from your web server.Apache, Nginx, IIS, or a CDN export.
- ✗Not possible on Shopify, managed WordPress.com, Webflow, Squarespace, Magento Commerce.They do not expose logs to merchants.
- ✗Manual. Import, analyse, export, rinse, repeat.No real-time feed, no alerts, no API.
Plugin-based. Captures the same data live, no server access.
- ✓Captures every crawler request from inside the CMS plugin or universal script tag.Same data Screaming Frog would parse from a log file.
- ✓Real-time dashboard, weekly email, and alerts on new bots and redirect loops.Not a batch job.
- ✓Works on Shopify, managed WordPress, Webflow, Squarespace, Magento Commerce.The platforms log analysers cannot reach.
- ✓REST API, outbound webhooks, CSV export.Feed Looker, HubSpot, BigQuery. No desktop app required.
- ✓Identifies every major AI bot out of the box.GPTBot, ClaudeBot, PerplexityBot, and 20+ others, plus new ones within 24 hours.
| Capability | Screaming Frog | Botsanalyser |
|---|---|---|
| Needs server log file access | Yes (required) | No |
| Works on Shopify, managed WordPress, Webflow | No | Yes |
| Real-time feed and dashboard | Batch desktop | Yes |
| Alerts on new bots, redirect loops, crawl spikes | No | Yes |
| REST API and outbound webhooks | No | Yes |
| Auto-detects new AI crawlers | Manual | Within 24h |
| Runs on a single laptop, no infra | Yes | Hosted on Cloudflare |
| One-off deep ad-hoc analysis | Excellent | CSV export covers it |
When to use Screaming Frog
If you run a server you control, you can export raw access logs, and you want a one-off deep crawl audit. It is the best desktop tool for that job, and it is not expensive. Nothing on this page is arguing against it.
When to use Botsanalyser
If you work at a marketing team that runs on Shopify, managed WordPress, Webflow, Squarespace, or Magento Commerce, and you cannot get anyone to email you a log file, Botsanalyser is the tool that fits the stack you have.
Or if you run on infrastructure where logs are technically available but you need an always-on, real-time, per-URL, per-bot feed that pipes into GA4 and your BI tool instead of a desktop app.
Bot data without a log file.
Reserve your spot on the waitlist.
- One email at launch
- Unsubscribe any time
- GDPR compliant