SkoupBot, the Skoup crawler.

Who we are

Skoup measures where a brand's products stand in the answers of AI assistants: ChatGPT, Claude, Gemini, Perplexity and the others. The service is published by Lunify SAS, in Paris.

SkoupBot reads sites to tell their owners what AI assistants can find there: readable pages, product data, robots.txt, llms.txt.

When it comes

A brand that uses Skoup, or its agency, added the site to its workspace. SkoupBot then audits the site every Monday, and when the customer runs the audit again. A store with no Shopify or WooCommerce connection can also have its catalog read, page by page.

A brand signs up or runs the free audit. SkoupBot reads about a dozen pages, once, to describe the brand and its products.

A visitor uses one of our free tools: robots.txt checker, llms.txt checker, product page test. SkoupBot reads only the address entered, when the visitor asks for it, as their browser would.

Its pace

One page after the other, waiting 250 ms before each page. The audits of all our customers go through a single process: one page at a time, for the whole web.

At most 500 pages per audit, 6,000 for reading a catalog.

Short timeouts: 5 seconds to connect, 10 seconds to answer. A page that does not answer is left aside, never asked again in a loop.

Read only: SkoupBot fills in no form and signs in to no account. It follows redirects one by one, 5 at most.

It obeys your robots.txt

SkoupBot reads your robots.txt under RFC 9309, like Google: the group that names SkoupBot applies, otherwise the User-agent: * group. The longest rule wins, and Allow wins a tie.

A disallowed path is never audited. A site entirely closed to SkoupBot is not read: the customer sees that their robots.txt blocks us.

Blocking it

Add the group above to your robots.txt to close the whole site, or replace Disallow: / with the paths to leave out, for example Disallow: /account/.

Blocking SkoupBot changes nothing for AI assistants: they have their own crawlers. To see which ones your robots.txt lets in, use our robots.txt checker.

Write to us

A problem with SkoupBot: too many requests, a page it should not read? Write to support@skoup.ai with your site's address and the time of the requests. We will answer and fix it.