More of your customers will arrive through software acting for them. Wolfix checks that it can find your app, get in, and land on the right page, and tells you plainly where it can't.
The report answers that question in one brief, in three steps — find, get in, land — built from rows in two departments. Growth owns finding and routing; Technology owns getting in. The rows themselves stay in their departments.
Who the agents are
Agents are grouped by what they do, not by brand. The group decides what a block means.
Training crawlers
Collect pages to train models. Blocking them opts you out of training. It does not remove you from AI search answers.
Search and answer-index crawlers
Build the indexes AI search answers from. Blocking these is the one robots.txt choice that can leave you out of those answers.
User-initiated fetchers
Fetch one page because a person asked. Several vendors say robots.txt may not apply to them, and many fetch without running JavaScript.
Browsing and acting agents
Drive a full browser for a person. A challenge page or CAPTCHA stops them.
Two more groups matter for some products. Tool-calling agents — coding assistants and MCP clients — often start from an llms.txt file or an API description. Transactional agents buy on a person's behalf through commerce protocols and merchant feeds.
What Wolfix checks
Find — Growth.
- Robots posture by purpose. Wolfix reads your robots.txt group by group. Only a block on search and answer-index crawlers is a visibility finding. A block that names the crawler in its own
User-agentgroup counts as your choice, with the trade-off stated. - llms.txt links. When you publish an
llms.txt, Wolfix follows the first 20 links on your own domain and names any that do not load. - Discovery surfaces. Whether you publish a commerce profile at
/.well-known/ucp, and, from your code, whether you ship an MCP server or an OpenAPI description. These are suggestions, never problems.
Get in — Technology.
- Edge challenge. Whether your site challenges or blocks an automated visitor that identifies itself honestly. Agents that are not on an allow-list get the same treatment.
- Edge bot policy. Whether a bot-management edge sits in front of your site. Wolfix cannot read that edge's settings, so this row always says unknown and tells you where to look.
- Content without JavaScript. Whether your main content, headings and links are in the page before JavaScript runs. Fetchers that skip JavaScript see an empty shell otherwise.
Land — Growth. Whether your pricing page (when you charge) and your docs (when your audience is developers) are one link from your homepage, your llms.txt or your structured data. Support, terms and privacy links are checked by their own rows and listed alongside.
How Wolfix measures
Wolfix always identifies itself as Wolfix and never sends another vendor's user agent. Imitating a vendor bot does not reproduce how a site treats the real one, which vendors verify by IP range or signature. So a passed edge check means one thing: an honest automated visitor got through.
How serious these rows are
None of them is ever a blocker. While your app is private, link-only or invite-only, they show as not yet applicable — visible, and phrased for later. Once strangers can find your app, a row becomes an issue where what you intended and what happens disagree: an llms.txt whose links break, a public page whose content needs JavaScript, an honest visitor turned away at the edge. A missing optional surface, like a commerce profile, stays a suggestion. If your app handles sensitive data, keeping unverified automated visitors out is treated as a reasonable posture.
What no outside scan can see
- The bot setting inside your edge provider's dashboard.
- Whether verified vendor bots get through.
- Merchant-feed or agentic-checkout enrolment.
- Anything behind your login.
- Agent traffic in your own logs.
Partners
This topic lists no partners, on purpose. The fixes are settings on your own host and edits to your own files — a robots.txt block, an llms.txt, a server-rendered page, an edge rule — not services you buy. When there is nothing worth buying, a referral would only be noise.
Wolfix's own setup
Wolfix holds itself to the same checks. Its robots.txt names every crawler Wolfix reads, in three groups — search, user-initiated and training — each allowed on purpose and each kept off /api/ and /r/. Its llms.txt links the pages an agent most needs: home, pricing, the help center, the FAQ, this page and the legal pages.
Related articles