← Insights

Can Google and ChatGPT Actually Read Your Site? Check, Don’t Guess

SEO-Ops is an open-source Agent Skill that checks whether search engines and AI retrieval can fetch, read, index, and cite your site — with deterministic checks and evidence, not LLM guesses.

5 min read
Can Google and ChatGPT Actually Read Your Site? Check, Don’t Guess

Executive summary: SEO-Ops is an open-source Agent Skill from Tigerless Labs that checks whether search engines and AI retrieval can fetch, read, index, and cite your site. Give it a URL and it inspects what a crawler actually sees—robots rules for both search and AI crawlers, sitemaps, canonicals, structured data, rendering, metadata, and redirects—then produces a pass/fail report with evidence. The checks are deterministic code, not LLM judgments; the agent runs them and explains what to fix.

It can run through Claude Code or Codex, but the underlying checker is Python. A basic scan needs no site code and no configuration: just Python 3.9+, requests, PyYAML, network access, and a target URL. (SKILL.md)

Why This Matters

A page can look perfectly normal in a browser and still be wrong from a crawler’s point of view. The canonical may point somewhere else. robots.txt may let Googlebot in but block GPTBot or PerplexityBot—so the page ranks in search but never gets cited in an AI answer. The body may depend on JavaScript that most AI crawlers never execute. A sitemap may contain broken URLs.

You could paste HTML into an LLM and ask, “Is the SEO okay?” But many of these questions have exact answers. SEO-Ops treats them that way: fetch the real HTTP/HTML output, run defined checks, and show the evidence. Its scope is deliberately narrow—the structural SEO/GEO foundation, not content quality, rankings, traffic, UX, or whether the page is persuasive. (SKILL.md)

What SEO-Ops Actually Does

The checklist—30 checks, called the C set—covers crawler-visible structure: robots.txt access for search and AI crawlers, canonical URLs, sitemap health, noindex, JSON-LD, Core Web Vitals, server-rendered body content, headings, Open Graph metadata, hreflang, redirects, mixed content, image attributes, security headers, URL hygiene, and link structure. Each check carries a P0, P1, or P2 priority. (checklist.md)

A few practical details stand out:

  • Framework-agnostic: it evaluates final HTTP/HTML, so React, Vue, Next.js, WordPress, and static sites are treated the same. (README)

  • Evidence, not just scores: failures identify the affected page and reason. (report.example.md)

  • AI crawlers are first-class: robots.txt is checked against a roster of 18 user agents—Googlebot and Bingbot, training crawlers such as GPTBot and ClaudeBot, and retrieval crawlers such as OAI-SearchBot and PerplexityBot—maintained from each vendor’s official docs. (ai-crawlers.yaml)

  • Every check cites its source: each check has its own doc covering why it matters, how to fix it, and the official references behind it. When the agent explains a failure, it reads that doc instead of answering from memory. (checklist references)

  • Two outputs: a human-readable Markdown report plus checks.db, a SQLite store that can accumulate results across runs. (SKILL.md)

  • No credentials for the basic run: CrUX and IndexNow data need optional configuration; most checks work without it. (SKILL.md)

How It Works

At a high level, SEO-Ops separates judgment from measurement. The Agent Skill handles the interaction—confirming the target and explaining failures—while scripts/run.py performs the structural checks. The checklist is the source of truth, and the report records each check as pass, fail, not-applicable, or human review. The agent can also skip the script and review templates or a pull request against the same checklist before anything ships. (SKILL.md)

SEO-Ops pipeline: website URL, fetch as crawler, HTTP/HTML output, deterministic SEO checks, pass/fail with evidence, then a Markdown report and SQLite history
From a URL to a pass/fail report with evidence

The same design makes it CI-friendly: --verify-only runs the repository’s drift guards without network access and exits non-zero when definitions drift. (SKILL.md)

Two companion documents sit beside the machine checks. The T set tells content teams which inputs SEO depends on—titles and descriptions, H2 outlines, image alt text, OG copy, YMYL flags—and maps each to the C checks it feeds, so a red item traces back to whoever owes the fix. The red lines (R1–R8) list what is forbidden—cloaking, fake structured data, link schemes, unreviewed YMYL content—and are enforced by human review, not by the script. (content checklist, red lines)

ApproachSource of truthDeterminismAutomationSetup
SEO-OpsVersioned checklist + official docs per check + live HTTP/HTMLHighAgent runs checks and explains fixes; CI-friendlyPython + 2 packages; no account
Manual LLM promptsPrompt + supplied contextLow to mediumMostly manualVery easy
Typical SEO audit toolsVendor rules + crawler dataUsually high for technical checksHighEasy, often account/SaaS based

Strengths and Limitations

Strengths: the same input is evaluated against explicit rules rather than whatever an LLM happens to infer; it covers AI crawlers and retrieval alongside classic search; it works against live sites or localhost without framework integration; and the report preserves concrete evidence instead of reducing everything to a vague “SEO score.”

Limitations: it is not a complete SEO strategy tool. It does not judge content quality, ranking potential, UX, or traffic, and some checks intentionally remain human reviews. Full-site runs can also take time—the documentation estimates roughly seven minutes for a 271-page site at the default single-worker rate. (SKILL.md)

One gap to note: the repository does not yet include a license file.

Quick Start

# Claude Code
/plugin marketplace add tigerless-labs/seo-ops
/plugin install seo-ops@seo-ops

# Then
/seo-ops check example.com

# Or run the checker directly
python3 scripts/run.py --target https://example.com

Claude Code and Codex installation paths are documented in the project README; direct checker use requires Python 3.9+, requests, and PyYAML.

When to Use It

SEO-Ops makes the most sense for developers, product teams, and open-source maintainers who want a repeatable “can search engines and AI assistants crawl, index, and cite this site?” check before or after shipping. It is especially useful when you want technical SEO to behave more like a test suite: explicit rules, reproducible results, evidence when something fails, and human judgment reserved for the things that actually require it.

Final Takeaway

SEO-Ops turns technical SEO and GEO from “ask an LLM what it thinks” into something much closer to run checks → inspect evidence → fix the failures.

Share this article

Keep reading