Skip to main content

The Crawl isn't representing the actual way a user is seeing the site - why and what can be done?

Flat Map - Why it happens, how to solve it.

Jeff avatar
Written by Jeff
Updated this week

What you’re describing is a very common and intentional pattern in modern CMSs: flat, clean URLs that do not mirror how users actually navigate a site.

URL structure ≠ navigational IA (by design)

In many modern setups (WordPress, headless CMSs, JS-driven menus), URLs are intentionally flat:

/article-title

…but users reach those pages via:

/blog → /category → /article

So when pages appear as siblings of /about, that’s not a bug — it’s the literal URL topology. And you’re correct: URL accuracy alone can be misleading for IA.

That’s why VisualSitemaps separates how URLs exist from how information architecture is modeled.


The two structural logics we support today

VisualSitemaps lets you view and organize a site using two complementary structural models:

  1. Directory-based structure
    Groups pages strictly by URL paths. Technically accurate, but limited for flat-URL sites.

  2. Referral-based structure
    Groups pages based on how they are linked and reached (navigation, contextual links). This is usually the correct starting point for flat URLs.


How flat-URL sites are handled in practice (manual IA workflows)

Because there is no standard way menus are coded, no crawler can reliably infer “menu intent” automatically. For flat-URL sites, IA is something you shape intentionally. Today, we support this via (3) fast manual workflows:

These workflows are specifically designed for large flat-URL sites where URL paths don’t express structure.


AI Optimization: generating the intended architecture

On top of that, we also offer AI Optimization for teams who want a future-state IA proposal.

Instead of documenting what exists, the AI analyzes internal links, content semantics, and topical relationships to generate a newly proposed sitemap — complete with logical groupings, priorities, and a cleaner structure that reflects how the site should be organized for users and search engines.

This is intentionally prescriptive, not just descriptive — and it’s especially powerful for flat-URL CMS setups.


Bottom line

You’re right: flat URLs are modern, intentional, and common.
And you’re also right that URL siblings ≠ user mental models.

That’s why VisualSitemaps supports:

  • Multiple structural views (current state)

  • Fast manual IA shaping workflows (practical control)

  • AI-generated optimized maps (future state)

Did this answer your question?