# Track AI crawlers on your server: GPTBot, ClaudeBot and the rest

> Track AI crawlers like GPTBot, ChatGPT-User and ClaudeBot on your own server with one open-source package or a plain HTTP call.

Canonical: https://picked.so/integrations/server
Last updated: 2026-10-12

To track GPTBot, ChatGPT-User, ClaudeBot and other AI crawlers on your own server, add our open-source package to your Next.js, Express or Fetch API app, or POST the hits from any language. It sends only AI bot visits, after the response, and never slows down or breaks a request.

```bash
npm install @picked-so/crawler-detect
```

## What does the package do?

`@picked-so/crawler-detect` matches each request's user agent against the AI bots Picked knows. When one matches, it sends the hit (host, path, user agent, IP, status and time) to your project's private ingest address. Every other request is ignored.

**After the response.** The send happens once your page is out, so a visitor never waits for it, and a failed send never touches the request.

**Verified by IP.** Anyone can claim to be `ChatGPT-User`. Picked checks each hit's IP against the ranges OpenAI, Anthropic, Perplexity, Google and Apple publish, and by reverse DNS for Amazonbot, so a fake doesn't count when Verified only is on.

**Visits from AI answers (0.2 and later).** A page view whose referrer is an AI assistant, or whose `utm_source` names one (`utm_source=chatgpt.com`), is sent too: the page, the user agent and the referrer's origin (`https://chatgpt.com`, never the chat's address). No IP address, no cookie. `referrals: false` turns it off.

**A secret address.** The address carries your project's ingest key, so it belongs in server code and an environment variable, never in the browser. Disconnecting issues a new key and the old one stops working.

## How do you add it?

Four steps. The [AI crawlers guide](https://picked.so/docs/ai-crawlers) in the docs has them next to the Cloudflare and Vercel connections.

1. **Open Analytics.** In your project, choose Your server. The dialog shows your address and the code for your stack.
2. **Install the package.** Add it from npm with the command at the top of this page, and set `PICKED_INGEST_URL` to the address from the dialog.
3. **Add one line.** The table below says where it goes for your framework.
4. **Deploy.** The first AI crawler visit shows up once a bot next opens a page.

| Your stack | Where the line goes |
| --- | --- |
| **Next.js** | In `proxy.ts` (Next.js 16), or `middleware.ts` on 13 to 15: call `trackAiCrawler(request, event)` |
| **Express** | Before your routes: `app.use(aiCrawlers())`. Behind a proxy, set `app.set("trust proxy", true)` so the IP is the bot's |
| **Workers, Hono, Bun, Deno, Remix, SvelteKit, Astro** | Anything that handles a Fetch API request: call `trackAiCrawler` with the request, the response and the context |
| **Any other server** | POST up to 200 hits at once as JSON, from any language |

**Working with an AI coding assistant?** The dialog has a **Cursor / Claude** tab with a prompt containing your project's address to install and wire up everything in one go.

## What does the plain HTTP call look like?

Send a hit when the `User-Agent` matches one of the bots Picked knows (the dialog lists the pattern), as JSON to your address:

| Field | What to send |
| --- | --- |
| `host` | Your domain |
| `path` | The path requested |
| `userAgent` | The full user agent string |
| `ip` | The bot's address, not your proxy's |
| `status` | The response status you sent |
| `at` | When it happened, in milliseconds |
| `country` | Optional: a two-letter code, such as the one your CDN gives |

## What should you know before adding it?

**From the day you deploy.** The package sends what happens after it runs. There is no history. Visits from AI answers need version 0.2 or later.

**Your server sees the bot's IP only if the proxy passes it on.** Behind a load balancer or a CDN, make sure the real client address reaches your code, or Picked can't verify it.

**One source per project.** A project reads its crawlers from your server, Cloudflare or Vercel, not two at once, or a visit counts twice.

**A few bots stay unverified.** Meta and some others publish no IP ranges, so their hits show, but not as verified.

**The catalogue is ours.** The package carries its own list of bots. When a new AI crawler appears, we add it and publish a new version, so update the package now and then.

## When this won't work for you

**A static site with no server code.** A static host that runs nothing on the request can't send a hit. Use [Cloudflare](https://picked.so/integrations/cloudflare) if it sits behind it, or [Vercel](https://picked.so/integrations/vercel) if it runs there.

**You don't want to touch the code.** Cloudflare needs no code at all, only a sign-in.

**You need history.** Only [Cloudflare](https://picked.so/integrations/cloudflare) reads the last 30 days back.

## How we checked

**The package.** [@picked-so/crawler-detect on npm](https://www.npmjs.com/package/@picked-so/crawler-detect), version 0.1.0, and its [open-source repository](https://github.com/picked-so/crawler-detect).

**What each bot is for.** [OpenAI](https://developers.openai.com/api/docs/bots), [Anthropic](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) and [Perplexity](https://docs.perplexity.ai/guides/bots), read September 29, 2026.

**What Picked does with a hit.** the product as built on September 30, 2026. The dialog in Analytics has the current code for each stack.

**Setup steps.** [AI crawlers in the docs](https://picked.so/docs/ai-crawlers).

## Frequently asked questions

### Which bots does it detect?

The AI crawlers Picked knows: `GPTBot`, `OAI-SearchBot` and `ChatGPT-User` from OpenAI, `ClaudeBot`, `Claude-SearchBot` and `Claude-User` from Anthropic, `PerplexityBot` and `Perplexity-User`, Google's AI crawlers, Meta's, Applebot, coding agents like Cursor and others: 31 in version 0.4. What each one reads for, and whether to allow it, is on the [AI crawler list](https://picked.so/ai-crawlers).

### Will it slow my site down?

No. The send happens after the response is out, and a failure never touches the request.

### Does it work on Vercel?

Yes, if you'd rather add the package to your `proxy.ts` than use a [log drain](https://picked.so/integrations/vercel). It only sends AI bot visits, so it doesn't carry the drain's volume.

### Is it open source?

Yes. The package and its code are public on GitHub under the Picked organisation.

### Is the address a secret?

Yes. It contains your project's ingest key. Keep it in an environment variable on the server, never in code the browser loads.

### Why can't Picked see a bot's real IP?

Behind a proxy or CDN, your app sees the proxy's address unless it is told to trust the forwarded header. Set the real client address, as in Express's `trust proxy` setting, or the hit can't be verified.
