Architecture
Runners
Execution engines for parallel and dry-run modes.
OptimusKG provides custom Kedro runners in optimuskg/runners/.
FixedParallelRunner
File: optimuskg/runners/parallel_runner.py
Executes pipeline nodes concurrently. This is the recommended runner for production use.
uv run kedro run \
--to-nodes gold.export_kg \
--runner=optimuskg.runners.FixedParallelRunner \
--asyncUse FixedParallelRunner instead of Kedro's built-in ParallelRunner. The original runner has a bug that prevents validation checks from running.
DryRunner
File: optimuskg/runners/dry_runner.py
Lists nodes in execution order without running them. Useful for verifying pipeline structure and dependencies.
uv run kedro run \
--from-nodes=<node_name> \
--runner=optimuskg.runners.DryRunner