EveryvoreBot

If you found this URL in your access logs, this page tells you exactly what we fetched, how often, what we do with it, and how to make us stop. Blocking us takes one line and no email.

Everyvore is a dietary-constraint restaurant finder for coastal Carolina, built by NeuralConfig. It answers "where can this specific group of people eat?" for people with dietary restrictions — including coeliac disease, where a wrong answer makes someone ill. To do that it reads published menus and quotes them back, with the date they were read.

The exact User-Agent

We never send a User-Agent that impersonates a browser, and we never crawl under a second, undisclosed identity. We run on Cloudflare Workers, so we cannot offer a stable IP range to allowlist — the User-Agent is the only honest way to identify us, and we would rather say so than publish a range that changes.

What it fetches

• Your home page, and pages linked from it that appear to be menus — /menu, /dinner, a PDF menu, and so on.
• GET and HEAD only. We never submit a form or send a POST.
• Nothing behind a login, a paywall, a booking flow or a checkout. We do not bypass access controls of any kind.
• Never social platforms or menu aggregators. Facebook, Instagram and the large aggregators prohibit this and we stay off them, even though it means a real share of independent restaurants are simply not covered. Our coverage reporting says so rather than hiding it.

How often, and how fast

Rate

At most one request per domain per 10 seconds, issued sequentially. We never open parallel connections to the same site.

Depth

At most 25 pages per site per pass. We are looking for a menu, not mirroring you.

Frequency

A pass at most once a week. Between passes we serve a cached copy, and we only re-read a page when its content hash has changed.

Crawl-delay

Honoured whenever it is longer than our own 10 seconds.

Errors

429 and 503 back us off and we honour Retry-After. Repeated failures drop your site out of the queue rather than escalating.

robots.txt

We fetch /robots.txt before anything else on a domain and cache it for no more than 24 hours. If it is unreachable or returns a server error, we do not crawl.

To block us completely:

User-agent: EveryvoreBot
Disallow: /

We also obey User-agent: *. Those two tokens — EveryvoreBot and * — are the ones we answer to.

We do not treat rules written for other companies' crawlers as applying to us. If your robots.txt disallows GPTBot or CCBot and says nothing about us, we will still crawl. That is a deliberate choice and we would rather state it than let you assume otherwise: those tokens name crawlers that collect corpora to train models, and that is not what this is. EveryvoreBot reads one menu to answer a question about that menu. Nothing we fetch trains a model — a verdict is a reading of your page, not a copy of it, and never becomes training data.

So if you want us gone, please say so directly rather than relying on a rule aimed at someone else: name EveryvoreBot, or use *, or email us. Any of the three works, and the first two need no reply from us.

What we store, and what we publish

• We keep a private cached snapshot of the pages we fetch, so we can re-read our own copy instead of yours.
• Publicly we show short verbatim quotes, 200 characters at most, each with a link to the page it came from and the date we read it. We never republish a whole menu and we never present ourselves as the source of your words.
• Anything a model concluded is labelled as AI-assisted, and kept distinct from what a menu plainly says. If a quoted phrase is not a verbatim substring of your page, we reject the result and record unknown instead. We would rather tell a user we don't know than tell them something your menu doesn't say.

Removals and corrections

robots.txt is the fastest route — it takes effect on our next pass and needs no conversation with us.

If you would rather write to a person, email crawler@neuralconfig.com. We remove a site on request, without asking why, and we will confirm it within two business days. To correct something we have got wrong about your restaurant rather than remove it, write to everyvore@neuralconfig.com.