OCDS Kingfisher Process¶
Kingfisher Process stores and pre-processes OCDS data. An accompanying tool, Kingfisher Collect, downloads OCDS data from sources, and optionally sends it to Kingfisher Process.
If you are viewing this on GitHub, open the full documentation for additional details.
Typical usage¶
Kingfisher Process receives OCDS data either via the web API (used by Kingfisher Collect) or the local-load command.
OCDS data are stored in a PostgreSQL database and organized into “collections”, as described in the Data Model and specified in the Database structure. Once incoming data are stored:
If the Default pre-processing pipeline is enabled, the data are automatically pre-processed into new collections.
If the schema checks are enabled, the data are automatically checked for schema errors.
OCDS data can be published as either releases or records. A release is a point-in-time update about a contracting process, and a record provides an index to all releases for a contracting process.
Kingfisher Process can automatically transform a collection of releases into a collection of compiled releases, containing a single compiled release per contracting process. A compiled release contains the latest value of each field from its original releases.
Once the data are stored, you can query the PostgreSQL database; refer to the Data Model and Database structure for an orientation to the database tables.
A Command-line tool allows you to list collections, add notes to collections, run schema checks, and transform collections.
You can run the web app to view metadata about collections and files.
And that’s it! In short, Kingfisher Process accepts “raw” OCDS data, and then checks and pre-processes it, so that your data analysis can be more predictable and repeatable.