indexing
v1.0.1When the user wants to fix indexing issues from Search Console, use noindex, or implement Google Indexing API. Also use when the user mentions "fix indexing,...
⭐ 0· 68·0 current·0 all-time
byKostja Zhang@kostja94
Security Scan
OpenClaw
Suspicious
medium confidencePurpose & Capability
Name and description (fixing indexing, noindex, Google Indexing API) match the SKILL.md content: troubleshooting guidance, noindex guidance, and Indexing API notes. The topics and recommended fixes are appropriate for an SEO indexing helper.
Instruction Scope
The SKILL.md explicitly tells the agent to read local files .claude/project-context.md or .cursor/project-context.md for site URL and goals. The skill metadata declared no required config paths, so the instructions allow the agent to access local project files without that being declared. If the agent runs autonomously it may read these files; users should be aware of that file access. The doc otherwise stays on-topic (GSC, robots.txt, noindex, canonical).
Install Mechanism
No install spec and no code files — instruction-only skill. This has the lowest installation risk because nothing is downloaded or written to disk by the skill itself.
Credentials
The skill declares no environment variables or credentials, which is consistent for an advisory skill. However, it documents how to use the Google Indexing API (service account, Search Console ownership, quota). If the agent were expected to call the Indexing API automatically, it would need sensitive GCP service-account credentials; those credentials are not declared here. That mismatch should be clarified before giving the agent any credentials or automation rights.
Persistence & Privilege
The skill is not always-enabled and does not request any persistent system presence. It does not modify other skills or system-wide settings according to the provided metadata.
What to consider before installing
This is an instruction-only SEO helper that gives reasonable advice about indexing, noindex, and the Google Indexing API. Before installing or enabling it, note two things: (1) the instructions tell the agent to read local project-context files (.claude/project-context.md or .cursor/project-context.md) even though no config paths were declared — review those files for any sensitive information you wouldn't want the agent to read; (2) the Indexing API guidance requires a GCP service account and Search Console ownership to operate — do NOT hand over service-account keys or other credentials unless you explicitly intend the agent to perform indexing actions, and only after you trust the skill. If you only want advice, keep the skill as read-only guidance and avoid providing credentials or enabling automated API calls.Like a lobster shell, security has layers — review code before you run it.
latest
SEO Technical: Indexing
Guides indexing troubleshooting and fix actions. For how to find and diagnose issues in GSC, see google-search-console.
When invoking: On first use, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On subsequent use or when the user asks to skip, go directly to the main output.
Scope (Technical SEO)
- Fix actions: noindex, canonical, content quality, URL Inspection; verify robots.txt does not block (see robots-txt)
- Noindex: Page-level index control; which pages to exclude and how. Complements robots-txt (path-level crawl control) and google-search-console (Coverage diagnosis)
Initial Assessment
Check for project context first: If .claude/project-context.md or .cursor/project-context.md exists, read it for site URL and indexing goals.
Identify issue from GSC (see google-search-console for Coverage report, issue types, diagnosis workflow). Then apply fix below.
Crawled - Currently Not Indexed
| Cause | Action |
|---|---|
| Low quality, duplicate, off-topic | Improve content, fix duplicates, set correct canonical |
| Static assets (CSS/JS) | See below |
| Feed, share URLs with params | Usually OK to ignore; or noindex, canonical to main URL |
| Important content pages | Use URL Inspection, verify canonical/internal links/sitemap, Request indexing |
Static Assets (Next.js / Vercel)
Vercel adds unique dpl= params to static assets per deploy, creating many "Crawled - currently not indexed" URLs.
| Do | Don't |
|---|---|
Keep robots.txt allowing /_next/ | Do not block /_next/ (breaks CSS/JS loading). See robots-txt |
| Accept static assets in GSC as expected | Do not block /_next/static/css/ or ?dpl= |
| Use X-Robots-Tag for static assets | CSS/JS should not be indexed; no SEO impact |
Static assets in "Crawled - currently not indexed" is normal and expected.
Other Issue Types (from GSC Coverage)
| Issue | Fix |
|---|---|
| Excluded by «noindex» tag | Remove noindex if accidental; keep if intentional |
| Blocked by robots.txt | See robots-txt; remove Disallow for important paths |
| Redirect / 404 | Fix URL or add redirect |
| Duplicate / Canonical | Set correct canonical; usually OK |
| Soft-404 | Page returns 200 but content says "not found" or empty—Google may treat as 404. Fix: return 404 status for truly missing pages; or add real content for 200 pages |
Soft-404
A soft-404 occurs when a page returns HTTP 200 but the content indicates the page doesn't exist (e.g. "Page not found" message, empty state). Google may treat it as 404 and exclude from index.
| Fix | When |
|---|---|
| Return 404 | Page truly doesn't exist; use proper 404 status |
| Add content | Page is intentional (e.g. empty search results); ensure substantive content or use noindex |
| Redirect | If URL moved, use 301 to correct destination |
Noindex Usage
- How:
metadata.robots = { index: false }or<meta name="robots" content="noindex">or X-Robots-Tag - Rationale: Not all site content should be indexed; noindex is a valid choice for many pages
- Caution: Avoid noindex on important content pages
- With robots.txt: robots.txt = path-level crawl control; noindex = page-level index control. Do not block noindex pages in robots.txt—crawlers must access the page to read the directive. Use both: robots for /admin/, /api/; noindex for /login/, /thank-you/, etc. See robots-txt for when to use which.
- nofollow ≠ noindex: nofollow controls link equity only; it does not prevent indexing. To exclude from search, use noindex. See page-metadata for meta robots implementation.
Page Types That Typically Need Noindex
| Category | Page Types | Typical Meta | Reason |
|---|---|---|---|
| Auth & Account | Login, Signup, Password reset, Account dashboard | Login: noindex,nofollow; Signup: noindex,follow | No search value; login indexed = security risk; signup follow allows crawl of Privacy/Terms links |
| Admin & Private | Admin, Staging, Test pages, Internal tools | noindex,nofollow | Not for public; avoid discovery |
| Conversion Endpoints | Thank-you, Confirmation, Checkout success, Download gate | noindex,follow | Post-conversion; no SERP value; allow link equity |
| System & Utility | 404, Internal search results, Faceted/filter URLs | noindex,follow or noindex,nofollow | Thin/duplicate; 404 = error state |
| Legal | Privacy, Terms, Cookie Policy (optional) | Often noindex,follow | Low-value indexed; reduces clutter |
| Duplicate & Thin | Printer-friendly, Parameter URLs, Near-duplicate | noindex,follow or canonical | Duplicate content; canonical preferred when possible |
| Low-Value | Media kit, Feedback board (external), Thin press | noindex or index for brand queries | Case-by-case |
noindex,follow vs noindex,nofollow: Use noindex,follow for most cases—excludes from SERP but allows link equity. Use noindex,nofollow only for login (security), staging, or temporary test pages.
Google Indexing API
| Type | Typical use |
|---|---|
| JobPosting | Job boards |
| BroadcastEvent | Live platforms |
Requirements: Enable Indexing API, create service account, add owner in Search Console, request quota (default 200 URLs/day).
Output Format
- Action items: Prioritized fixes
- References: Page indexing report
Related Skills
- google-search-console: Find and diagnose indexing issues in GSC
- robots-txt: Path-level crawl control; when to use robots.txt vs noindex; do not block /_next/ or noindex pages
- page-metadata: Meta robots implementation; noindex vs nofollow
- xml-sitemap: Submit and maintain sitemap
- indexnow: Faster indexing for Bing
- canonical-tag: Resolve duplicate content
Comments
Loading comments...