Skip to main content

Woo compliance: sitemaps and robots.txt

OpenCatalogi publishes every Woo catalogue as DiWoo sitemaps, so the national Woo-index (KOOP) and search engines can harvest the published documents, and points crawlers at them from robots.txt.

Spec: openspec/specs/woo-compliance/spec.md, change woo-compliance.

Endpoints​

All three are public (no login) and read only what the public group may read: every search runs with OpenRegister RBAC on.

EndpointAnswers
GET /apps/opencatalogi/api/robots.txtUser-agent: *, the Allow lines for the API, and one Sitemap: line per category for every catalogue with hasWooSitemap: true and a slug
GET /apps/opencatalogi/api/{catalogSlug}/sitemaps/{categoryCode}The sitemap index of one category: one <sitemap> per page of publications
GET /apps/opencatalogi/api/{catalogSlug}/sitemaps/{categoryCode}/publications?page=NOne sitemap page: one <url> per published file, each with its diwoo:Document

{categoryCode} is the sitemap file name of a category, such as sitemapindex-diwoo-infocat014.xml. An unknown code, or a catalogue without a Woo sitemap, answers 400.

Example: GET /apps/opencatalogi/api/woo-publicaties/sitemaps/sitemapindex-diwoo-infocat014.xml

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://example.org/apps/opencatalogi/api/woo-publicaties/sitemaps/sitemapindex-diwoo-infocat014.xml/publications?page=1</loc>
<lastmod>2026-11-02 09:15:00</lastmod>
</sitemap>
</sitemapindex>

A page holds 1000 publications by default (app setting sitemap_max_per_page), newest update first; the index's lastmod for a page is the newest update on that page. A category without publications answers 200 with a <sitemapindex> without children.

hasWooSitemap​

A catalogue publishes Woo sitemaps only with hasWooSitemap: true. robots.txt lists the sitemaps of exactly those catalogues that also have a slug; a catalogue with hasWooSitemap: false, or without the field, adds no Sitemap: line and its sitemap endpoints answer 400.

DiWoo metadata per document​

Each downloadable file of a publication is one document.

DiWoo fieldSourceWhen it is missing
locthe file's downloadUrlempty; files without a download URL are left off the page
lastmodthe latest of the file's modified, the file's published and the publication's @self.updatednow
diwoo:creatiedatumthe publication's @self.created, as a datetoday
diwoo:publisher #textthe name of the publication's organisation, never a userempty, and reported
diwoo:publisher @resourcethe organisation's TOOI identifier (publication tooiIdentifier, a TOOI @self.organisation, or the OpenRegister organisation's TOOI)omitted, and reported
diwoo:format@resource the EU file type of the extension in capitals, #text the extension in lower case—
diwoo:informatiecategoriewooCategory, then category, tooiCategorieUri, tooiCategorieNaam, resolved to the TOOI value listomitted, and reported
diwoo:soortHandelingthe publication's soortHandeling, resolved to the DiWoo value list; ontvangst when it has noneomitted, and reported, when it resolves to nothing
diwoo:atTimethe file's published, then the publication's publicationDatenow
diwoo:officieleTitelthe publication's titleomitted, and reported

An axis that cannot resolve to an official URI is never written as free text: it is left out and reported by GET …/sitemaps/{categoryCode}/validate.

Woo information categories​

The 18 members of the TOOI waardelijst, one sitemap file each. A category an administrator adds locally gets its own file too.

CodeSitemap fileCategory
infocat001sitemapindex-diwoo-infocat001.xmlWetten en algemeen verbindende voorschriften
infocat002sitemapindex-diwoo-infocat002.xmlOverige besluiten van algemene strekking
infocat003sitemapindex-diwoo-infocat003.xmlOntwerpen van wet- en regelgeving met adviesaanvraag
infocat004sitemapindex-diwoo-infocat004.xmlOrganisatie en werkwijze
infocat005sitemapindex-diwoo-infocat005.xmlBereikbaarheidsgegevens
infocat006sitemapindex-diwoo-infocat006.xmlBij vertegenwoordigende organen ingekomen stukken
infocat007sitemapindex-diwoo-infocat007.xmlVergaderstukken Staten-Generaal
infocat008sitemapindex-diwoo-infocat008.xmlVergaderstukken decentrale overheden
infocat009sitemapindex-diwoo-infocat009.xmlAgenda's en besluitenlijsten bestuurscolleges
infocat010sitemapindex-diwoo-infocat010.xmlAdviezen
infocat011sitemapindex-diwoo-infocat011.xmlConvenanten
infocat012sitemapindex-diwoo-infocat012.xmlJaarplannen en jaarverslagen
infocat013sitemapindex-diwoo-infocat013.xmlSubsidieverplichtingen anders dan met beschikking
infocat014sitemapindex-diwoo-infocat014.xmlWoo-verzoeken en -besluiten
infocat015sitemapindex-diwoo-infocat015.xmlOnderzoeksrapporten
infocat016sitemapindex-diwoo-infocat016.xmlBeschikkingen
infocat017sitemapindex-diwoo-infocat017.xmlKlachtoordelen
infocat018sitemapindex-diwoo-infocat018.xmlInspanningsverplichting art 3.1 Woo

Download URLs depend on auto-publishing​

loc is the file's public share link. Files get one when they are published (see the auto-publishing feature); a publication whose files were never shared has no documents on its sitemap page.

Default information categories​

The default category of a kind of object​

Every document in a DiWoo sitemap carries an informatiecategorie from the official TOOI waardelijst. A publication normally names its own category. When it names none, OpenCatalogi looks for a default for its kind of object before it reports the category as missing.

These defaults are WooCategoryMapping objects in the publication register. Each row holds:

PropertyWhat it holds
appThe app that owns the kind of object, for example opencatalogi or decidesk.
objectTypeThat app's schema slug or payload kind, for example publication or decision.
informatiecategorieThe TOOI URI of the category, for example https://identifier.overheid.nl/tooi/def/thes/kern/c_db4862c3.
informatiecategorieLabelThe category's label, shown beside the URI so a person can check it.
activeWhether the default applies. An inactive row is kept but never used.
notesWhy this default was chosen.

The pair (app, objectType) is the key: two apps can both have a decision without one's default touching the other's. OpenRegister cannot enforce that pair as unique, so keep one active row per pair.

How the default is used​

  • A publication's own category always wins. The default is only read when the publication has no wooCategory, category, tooiCategorieUri or tooiCategorieNaam.
  • For OpenCatalogi's own publications the row is looked up with app: opencatalogi and objectType set to the publication's schema slug.
  • The row's URI is checked against the waardelijst again. A row that names no member is treated as if there were no row: the category is left out and reported, never written as free text.
  • A row for a kind of object that does not exist yet does nothing until it does.

Reading the defaults from another app​

There is no separate API. Any app reads the defaults with an ordinary OpenRegister object query on the publication register and the wooCategoryMapping schema, filtered on app and objectType, the same way OpenCatalogi reads them.

The defaults shipped with OpenCatalogi are the eight decidesk rows (meeting agendas, decision lists, minutes, decisions, motions, commitments, council information letters and arrangements). Their categories are resolved to real TOOI URIs.