Search Engines

SpiderPublish gives you per-page control over how search engines see your site, and it keeps your sitemap in sync automatically. Good SEO defaults are built in; the controls are there when you want them.

Per-page SEO

Every page and post has an SEO panel where you set:

  • SEO title — the title shown in search results (distinct from the on-page heading).

  • SEO description — the snippet under the title in results.

  • Open Graph image — the preview image used when the page is shared on social media.

  • Canonical URL — the preferred address for a page, to avoid duplicate-content issues.

Fill these in for any page you care about ranking. If you leave them blank, sensible defaults are used.

The indexing toggle

Each page has an "Allow search engines to index this page" toggle:

  • On (the default) — the page is indexable and is included in your sitemap.xml.

  • Off — the page is marked noindex and is automatically removed from your sitemap.xml and from your llms.txt.

Use off for pages you do not want in search results — thank-you pages, internal utilities, gated content.

System pages are handled for you

Utility pages such as login, signup, password reset, and the 404 page are set to noindex by default, so they never leak into your sitemap or search results. You do not have to remember to hide them.

The sitemap

Your sitemap.xml is generated automatically from your published, indexable content and stays current as you publish and unpublish. Pages set to noindex are excluded. Submit your sitemap URL to Google Search Console once and SpiderPublish keeps it accurate from then on.

Tips

  • Write the description for humans. A clear, benefit-led description earns clicks even at the same ranking.

  • Set an OG image on shareable pages — it dramatically improves how links look in social feeds and chats.

  • Use noindex deliberately. It is the clean way to keep utility pages out of search; just remember a noindexed page also disappears from your sitemap and llms.txt.

Next steps