[testbed] TestbedConfig

TOML section: [testbed]

Pydantic model: TestbedConfig defined in hydromodpy.analysis.testbed.config.

Source on GitHub

Validated configuration for one method testbed.

Show fields: User User + Dev All

Fields

config_path

Path required user source

Resolved path of the TOML file that produced this config.

base_dir

Path required user source

Directory used to resolve relative paths.

id

str required user source

Stable testbed identifier.

profile

str required user source

Selected testbed profile.

subject

str required user source

High-level subject covered by the testbed (flow, mesh, transport).

purpose

str required user source

Stated purpose of the testbed (robustness, sensitivity, …).

output_root

Path required user source

Directory where testbed artifacts are written.

execute

bool default = True user source

When false, child configs are materialized but not executed.

continue_on_error

bool default = True user source

When false, the first failure aborts the testbed.

base_config_path

Path | None default = None user source

Optional child workflow TOML used as the case base config.

runner in TOML: [testbed.runner]

TestbedRunnerConfig required user source

Child-runner selection for every case.

Fields of TestbedRunnerConfig

type

str required user source

Runner identifier dispatched for every case (comparison, simulation).

no_display

bool default = True user source

Disable interactive display in child runners.

case in TOML: [[testbed.case]]

tuple[TestbedCaseConfig, …] default = () user source

Explicit executable cases declared in the TOML.

Fields of TestbedCaseConfig

id

str required user source

Stable case identifier.

label

str required user source

Human-readable case label.

axis

str | None default = None user source

Optional axis tag used to group cases in reports.

enabled

bool default = True user source

Toggle to skip a case without removing it from the config.

overlay in TOML: [testbed.case.overlay.<id>]

dict[str, Any] factory user source

TOML overlay merged into the child-runner base config.

catalog in TOML: [testbed.catalog]

TestbedCatalogConfig | None default = None user source

Optional catalog source used to expand cases from rows.

Fields of TestbedCatalogConfig

path

Path required user source

Resolved path to the case catalog (CSV or JSONL).

format

str default = “auto” user source

Catalog format. ‘auto’ infers from the file suffix.

id_field

str default = “case_id” user source

Column carrying the case identifier.

label_field

str | None default = “case_label” user source

Column carrying a human-readable case label.

axis_field

str | None default = “axis” user source

Column carrying the optional axis tag.

enabled_field

str | None default = “enabled” user source

Column flagging whether a row is active.

tags_field

str | None default = “tags” user source

Column carrying free-form tags joined by tag_separator.

required_fields

tuple[str, …] default = () user source

Columns that must be present and non-empty per row.

path_fields

tuple[str, …] default = () user source

Columns whose values are resolved as filesystem paths.

tag_separator

str default = “;” user source

Separator splitting the tags column into individual tags.

field_equals

tuple[tuple[str, str], …] default = () user source

Per-field equality filters applied during catalog selection.

tags

tuple[str, …] default = () user source

Whitelist of tags. Empty disables the filter.

exclude_tags

tuple[str, …] default = () user source

Blacklist of tags applied after the whitelist.

include_disabled

bool default = False user source

When true, rows flagged as disabled are kept.

limit

int | None default = None user source

Optional cap on the number of selected rows.

source_manifest_path

Path | None default = None user source

Optional site-selection manifest used to resolve the catalog path.

source_manifest_output_key

str | None default = None user source

Output key read from the site-selection manifest.

case_from_catalog in TOML: [[testbed.case_from_catalog]]

tuple[TestbedCatalogCaseConfig, …] default = () user source

Case-generation rules applied to catalog rows.

Fields of TestbedCatalogCaseConfig

id_template

str | None default = None user source

Format string evaluated against catalog row fields to derive the case id.

label_template

str | None default = None user source

Format string used to derive the case label.

axis_template

str | None default = None user source

Format string used to derive the optional axis tag.

enabled

bool default = True user source

Toggle to skip the rule without removing it from the config.

overlay in TOML: [testbed.case_from_catalog.overlay.<id>]

dict[str, Any] factory user source

TOML overlay rendered against each catalog row.

required_fields

tuple[str, …] default = () user source

Catalog columns that must be present and non-empty per row.

field_equals

tuple[tuple[str, str], …] default = () user source

Per-field equality filters applied while expanding rows.

tags

tuple[str, …] default = () user source

Whitelist of tags applied during rule expansion.

exclude_tags

tuple[str, …] default = () user source

Blacklist of tags applied during rule expansion.

limit

int | None default = None user source

Optional cap on the number of cases generated by the rule.

metrics in TOML: [[testbed.metrics]]

tuple[TestbedMetricConfig, …] default = () user source

Metrics extracted from each child-runner summary.

Fields of TestbedMetricConfig

name

str required user source

Metric column name.

source

str required user source

Dotted path into the child-runner summary.

required

bool default = False user source

When true, a missing metric fails the case.

Starter TOML snippet

Click to expand a copy-pasteable [testbed] TOML skeleton

Copy this block into your project.toml and uncomment the lines you want to set. Sub-tables ([parent.subfield]) appear in the order Pydantic expects them.

[testbed]
# config_path = ""  # REQUIRED
# base_dir = ""  # REQUIRED
# id = ""  # REQUIRED
# profile = ""  # REQUIRED
# subject = ""  # REQUIRED
# purpose = ""  # REQUIRED
# output_root = ""  # REQUIRED
# execute = true
# continue_on_error = true
# base_config_path = ...  # default = None

[testbed.runner]
# type = ""  # REQUIRED
# no_display = true

[[testbed.case]]
# id = ""  # REQUIRED
# label = ""  # REQUIRED
# axis = ...  # default = None
# enabled = true
# overlay = ...  # factory default

[testbed.catalog]
# path = ""  # REQUIRED
# format = "auto"
# id_field = "case_id"
# label_field = "case_label"
# axis_field = "axis"
# enabled_field = "enabled"
# tags_field = "tags"
# required_fields = []
# path_fields = []
# tag_separator = ";"
# field_equals = []
# tags = []
# exclude_tags = []
# include_disabled = false
# limit = ...  # default = None
# source_manifest_path = ...  # default = None
# source_manifest_output_key = ...  # default = None

[[testbed.case_from_catalog]]
# id_template = ...  # default = None
# label_template = ...  # default = None
# axis_template = ...  # default = None
# enabled = true
# overlay = ...  # factory default
# required_fields = []
# field_equals = []
# tags = []
# exclude_tags = []
# limit = ...  # default = None

[[testbed.metrics]]
# name = ""  # REQUIRED
# source = ""  # REQUIRED
# required = false

Entity-relationship diagram

Click to zoom and pan. Press Esc or click outside to close.