Configuration¶

Setup¶

Create the tool’s configuration directory:

mkdir ~/.config/ocdskingfisher-process

Download the sample main configuration file:

curl https://raw.githubusercontent.com/open-contracting/kingfisher-process/main/samples/config.ini -o ~/.config/ocdskingfisher-process/config.ini

Open the main configuration file at ~/.config/ocdskingfisher-process/config.ini, and follow the instructions below to update it.

PostgreSQL¶

Note

This step is required. All other steps are optional.

Configure the database connection settings:

[DBHOST]
HOSTNAME = localhost
PORT = 5432
USERNAME = ocdskingfisher
PASSWORD =
DBNAME = ocdskingfisher

If you prefer not to store the password in config.ini, you can use the PostgreSQL Password File, ~/.pgpass, which overrides any password in config.ini. Otherwise, if you used the same settings as in the examples during Installation, you only need to set PASSWORD above.

To override config.ini and/or .pgpass, set the KINGFISHER_PROCESS_DB_URI environment variable. This is useful to temporarily use a different database than your default database. For example, in a bash-like shell:

export KINGFISHER_PROCESS_DB_URI='postgresql://user:password@localhost:5432/dbname'

Logging¶

This tool uses the Python logging module. Loggers are in the ocdskingfisher namespace.

Logging from the Command-line tool can be configured with a ~/.config/ocdskingfisher-process/logging.json file. To download the default configuration:

curl https://raw.githubusercontent.com/open-contracting/kingfisher-process/main/samples/logging.json -o ~/.config/ocdskingfisher-process/logging.json

To download a different configuration that includes debug messages:

curl https://raw.githubusercontent.com/open-contracting/kingfisher-process/main/samples/logging-debug.json -o ~/.config/ocdskingfisher-process/logging.json

Web API¶

To allow access to the web API, set API keys, separated by commas. For example, to set 1234 and 5678 as keys (in practice, you should use long, random keys):

[WEB]
API_KEYS = 1234,5678

To override config.ini, set the KINGFISHER_PROCESS_WEB_API_KEYS environment variable.

Collection flags¶

When a new collection is created, flags are set to indicate what operations to perform on them.

All flags are off by default. To turn any on:

[COLLECTION_DEFAULT]
CHECK_DATA = true
CHECK_OLDER_DATA_WITH_SCHEMA_1_1 = false

Default pre-processing pipeline¶

To enable the Default pre-processing pipeline:

[STANDARD_PIPELINE]
RUN = true

Redis¶

Configure the Redis connection settings:

[REDIS]
HOST = localhost
PORT = 6379
DATABASE = 0

Sentry¶

To track crashes, sign up for Sentry, and set the DSN:

[SENTRY]
DSN = https://<key>@sentry.io/<project>

Note

Sentry has its own environment variables. Further reading: Sentry for Python.