Streaming server logs — Cloudflare, Vercel, CloudFront, and Signals
Feed your edge or server logs into Maya to unlock AI crawler analytics.
Updated 2026-10-12
AI crawler analytics (LLM Site Visibility) needs to see your real traffic. Maya accepts it from whichever layer you already have — your CDN, your host, or your own servers. All sources feed the same pipeline; pick one.
Cloudflare
In LLM Site Visibility → Config, paste a Cloudflare API Token and your Zone ID, then hit Test Connection. Maya pulls AI-bot traffic for the zone from then on. Use a token scoped to analytics read access for the zone, nothing broader.
Vercel
Connect a Log Drain from your Vercel project. Vercel pushes request logs to Maya as they happen; AI-bot requests are filtered and aggregated on arrival.
AWS CloudFront
CloudFront standard logs (v2) delivered through Amazon Data Firehose push to Maya's logs endpoint with your streaming credentials. One-time setup in AWS; near-real-time after that. The step-by-step CloudFront guide is in the developer docs.
Signals / direct streaming
For everything else there are streaming credentials (project + source + write key) against logs.withmaya.ai:
- Point a log shipper — Vector, Fluent Bit, or similar — at the endpoint from Nginx, Apache, IIS, or any custom stack.
- Or install the lightweight JS collector snippet where server logs aren't available.
- Run the built-in test event to verify the pipe before trusting it.
Per-stack guides (Nginx, Apache, IIS, custom BFF endpoint) with exact filter configs are in the developer docs.
Which should you pick?
| You run | Use |
|---|---|
| Cloudflare in front of the site | Cloudflare token — 2 minutes, no infra |
| The site on Vercel | Log Drain — native push |
| CloudFront CDN | Standard logs v2 + Firehose |
| Your own servers / other CDN | Log shipper to logs.withmaya.ai |
One source is enough; adding a second for the same traffic double-counts. Choose the layer closest to the edge (the one that sees bot requests your origin might never receive).
Privacy
The pipeline is built for bot analytics: guides include source-side filters so you ship crawler/agent traffic, not your users' browsing. What Maya does and doesn't collect is documented in the security docs.