Policy
GridAPM keeps its public website open to search engines and AI assistants. Marketing pages, research articles, tools, standards references, trust pages, and legal notices are published for open web discovery, and AI search crawlers may read and cite them. One boundary applies to every crawler: the internal analytics report route (/analytics-content-report/) is disallowed in robots.txt and noindexed in HTML, and IndexNow submissions cover public www.gridapm.com URLs only.
Nothing sensitive rides on public discovery. Pilot evidence, uploaded files, customer asset data, tool inputs (which run entirely in the browser), and engineering work products are never published on the public site, so no crawler — search or AI — can reach them. GridAPM claims no search-engine partnership or ranking guarantee, and no page grants autonomous engineering authority.
Machine-readable surface
These public files describe the crawl boundary directly, without guessing from page navigation.
| File | Purpose |
|---|---|
| robots.txt | Crawler policy for search and AI discovery agents. |
| sitemap.xml | Canonical XML sitemap for public pages and images. |
| llms.txt | Compact public context file for AI assistants and answer engines. |
| llms-full.txt | Expanded public context file with page summaries and source policy. |
| rss.xml | GridAPM research feed. |
| IndexNow key | Root-hosted key for IndexNow URL submission after deployments. |
Crawlers named explicitly
robots.txt names these agents explicitly. Each is allowed on public pages and disallowed from the internal analytics route; all other crawlers receive the same rule through the wildcard group.
- bingbot
- OAI-SearchBot
- ChatGPT-User
- GPTBot
- Claude-SearchBot
- Claude-User
- ClaudeBot
- PerplexityBot
- Perplexity-User
- Applebot
- Applebot-Extended
- DuckDuckBot
- YandexBot
- Baiduspider
- Yeti