transform-collections

This command runs the configured transformers for all collections.

It can be run multiple times on a collection, and data already transformed will not be retransformed.

You should only run one of these at once, as if two are run at once they may try and do the same work.

python ocdskingfisher-process-cli transform-collections

By default, it will run with a single thread. You can have it run multiple threads at once. A single collection will still be processed at the same speed, but multiple collections will be processed at once.

python ocdskingfisher-process-cli transform-collections --threads 10

Running from cron

You can also pass a maximum number of seconds that the process should run for.

python ocdskingfisher-process-cli transform-collections --runforseconds 60

Soon after that number of seconds has passed, the command will exit. (The command will finish the transforming it’s currently doing before stopping, so it may run slightly longer than specified. Allow a minute extra to be safe.)

You can use this option with a cron entry; set a cron entry for this command to run every hour and pass runforseconds as 3540 (60 seconds/minute * 59 minutes).

Then when new data appears in the system, there is no need for someone to run transform-collection by hand - the process run by cron will pick up the new data itself eventually.

The runforseconds option will make sure that only one of these cron processes runs at once.

OCDS Helpdesk deployment

Don’t use this. A cron job runs this once per hour.