Calibration Architecture#

This page is the code-oriented entry point for the calibration stack. It groups the package map, the runtime classes, the execution-flow diagrams, and the case-vs-core boundary in one place.

For the full operational reference (TOML sections, optimizer catalogue, storage rules, pitfalls, Python API), see Calibration Guide.

Architecture map#

The current calibration stack is split into four layers:

  • hydromodpy/calibration/cli_runner.py owns the hmp run <calibration.toml> workflow entry point: validates the config, builds the engine, runs the optimizer, writes the report. programmatic_runner.py exposes the same flow for in-process use.

  • hydromodpy/simulation/execution/trial.py owns the prepare-once, evaluate-many primitive used by every trial inside the ask/tell loop.

  • hydromodpy/calibration/ (engine, parameters, objective, optimizer, diagnostics, persistence) owns the reusable engine, parameter sets, objective handling, method dispatch, and canonical results.

  • hydromodpy/calibration/cases/ owns runnable scientific cases that exercise the full calibration loop end to end.

Core classes (config)#

This diagram focuses on validated configuration and method-selection objects.

@startuml title Calibration2 Core - Config Classes package "hydromodpy.analysis.calibration.core.engine_config" { class "engine_config.py" as EngineConfigModule <<module>> { +validate_calibration_config_data(config_data) +load_calibration_toml(config_path) +resolve_calibration_settings(config, model_parameter_order) } class CalibrationSectionSchema <<pydantic>> { +objective_metric: str +global_method: str +model_name: str | None } class CalibrationTomlSchema <<pydantic>> { +chronicle: dict +calibration: CalibrationSectionSchema +bounds: dict +calibration_method: dict +output: dict } } package "hydromodpy.analysis.calibration.core.methods_config" { class "methods_config.py" as MethodsConfigModule <<module>> { +canonical_method_name(method) +is_supported_method(method_name) +validate_method_kwargs(method_name, kwargs) +normalize_format_method_kwargs(method, method_kwargs, parameter_names) +validate_calibration_method_section(section) } class GridSearchKwargs <<pydantic>> class RandomSearchKwargs <<pydantic>> class NelderMeadKwargs <<pydantic>> class SimplexKwargs <<pydantic>> class GpMappingKwargs <<pydantic>> class DaMhGpKwargs <<pydantic>> } package "hydromodpy.analysis.calibration.core.parameters" { class CalibrationParameterSet } package "hydromodpy.analysis.calibration.core.objective_function" { class ObjectiveFunction } CalibrationTomlSchema --> CalibrationSectionSchema : contains CalibrationSectionSchema ..> ObjectiveFunction : validate objective_metric CalibrationTomlSchema ..> CalibrationParameterSet : build from bounds EngineConfigModule ..> CalibrationTomlSchema : uses for validation CalibrationTomlSchema ..> GridSearchKwargs : validate [calibration_method] CalibrationTomlSchema ..> RandomSearchKwargs CalibrationTomlSchema ..> NelderMeadKwargs CalibrationTomlSchema ..> SimplexKwargs CalibrationTomlSchema ..> GpMappingKwargs CalibrationTomlSchema ..> DaMhGpKwargs CalibrationTomlSchema ..> MethodsConfigModule : validate method kwargs EngineConfigModule ..> MethodsConfigModule : normalize kwargs note bottom Focus: strict configuration schemas plus key config helper functions. end note @enduml

Core classes (main runtime)#

This diagram focuses on the reusable runtime objects exchanged during one calibration session.

@startuml title Calibration2 Core - Main Runtime Classes package "hydromodpy.analysis.calibration.core.parameters" { class CalibrationParameter { +name: str +lower: float +upper: float } class CalibrationParameterSet { +from_bounds(bounds, parameter_names=None) +names +bounds +dimension +vector_from(values) +mapping_from(values) +contains(values) +clip(values) } CalibrationParameterSet "1" *-- "1..*" CalibrationParameter } package "hydromodpy.analysis.calibration.core.objective_function" { class "objective_function.py" as ObjectiveModule <<module>> { +rmse(observed, simulated) +mae(observed, simulated) +nse(observed, simulated) +nse_log(observed, simulated) +kge(observed, simulated, return_components=False) +objective_function(observed, simulated, metric=\"rmse\") } class ObjectiveFunction { +metric: str +resolve_metric_name(metric) +evaluate(observed, simulated, metric=None, return_components=True) +value_to_cost(value, metric=None) +evaluate_all(observed, simulated) } } package "hydromodpy.analysis.calibration.core.results" { class CalibrationResults { +method: str +x_best: np.ndarray +params_best: dict[str, float] | None +cost_best: float +score_best: float | None +n_evaluations: int +samples: np.ndarray | None +metadata: dict[str, object] -- +from_method_output(method_output, default_method) +attach_context(vector_to_params, score_best) +has_samples +n_samples } } package "hydromodpy.analysis.calibration.core.methods_dispatcher" { class "methods_dispatcher.py" as DispatcherModule <<module>> { +DEFAULT_CALIBRATION_METHOD +_first_docline(func) } class CalibrationMethod { +register(name, calibrator) +available_methods() +grouped_methods() +methods_overview() +calibrate(objective_cost, bounds, method=\"simplex\", **kwargs) } } package "hydromodpy.analysis.calibration.core.engine" { class CalibrationEngine { +observed: np.ndarray +simulator: callable +objective: ObjectiveFunction +parameter_set: CalibrationParameterSet +calibration_method: CalibrationMethod +last_result: CalibrationResults | None -- +params_to_vector(params) +vector_to_params(vector) +simulate(params) +score(params, metric=None) +cost(params, metric=None) +calibrate(method=\"simplex\", **kwargs): CalibrationResults } } package "hydromodpy.analysis.calibration.core.methods" { class "grid_search.py" as GridSearchModule <<module>> { +grid_search_calibrate(objective_cost, bounds, ...) } class "random_search.py" as RandomSearchModule <<module>> { +random_search_calibrate(objective_cost, bounds, ...) } class "nelder_mead.py" as NelderMeadModule <<module>> { +nelder_mead_calibrate(objective_cost, bounds, ...) } class "simplex.py" as SimplexModule <<module>> { +simplex_calibrate(objective_cost, bounds, ...) } class "gp_mapping.py" as GpMappingModule <<module>> { +gp_mapping_calibrate(objective_cost, bounds, ...) } class "da_mh_gp.py" as DaMhGpModule <<module>> { +delayed_acceptance_gp_mh_calibrate(objective_cost, bounds, ...) } } CalibrationEngine --> ObjectiveFunction : uses CalibrationEngine --> CalibrationParameterSet : uses CalibrationEngine --> CalibrationMethod : delegates CalibrationEngine --> CalibrationResults : returns ObjectiveFunction ..> ObjectiveModule : wraps functions CalibrationMethod ..> DispatcherModule : defined in DispatcherModule ..> GridSearchModule : dispatch target DispatcherModule ..> RandomSearchModule : dispatch target DispatcherModule ..> NelderMeadModule : dispatch target DispatcherModule ..> SimplexModule : dispatch target DispatcherModule ..> GpMappingModule : dispatch target DispatcherModule ..> DaMhGpModule : dispatch target @enduml

Calibration activity#

The high-level activity view of one calibration session driven by hmp run.

@startuml title Calibration2 - Calibration Activity Flow start :Load TOML config; :Validate global config\n(engine_config); if (Config valid?) then (yes) else (no) :Raise validation error; stop endif :Validate case chronicle config\n(case_config); if (Chronicle valid?) then (yes) else (no) :Raise validation error; stop endif :Build chronicle\n(observed, forcing, synthetic truth); :Resolve calibration settings\n(method, metric, bounds, kwargs); if (method == da_mh_gp and metric != rmse?) then (yes) :Warn user;\nforce objective_metric = rmse; endif :Create CalibrationEngine; :Dispatch method through CalibrationMethod; repeat :Evaluate objective cost on candidate vector; :Run simulator and compute metric cost; repeat while (Stopping criterion not reached?) :Build CalibrationResults from method output; :Attach params_best and score_best context; :Compute diagnostics and produce plots; stop @enduml

Calibration sequence#

The handoff between the CLI entry point and the generic calibration engine.

@startuml title Calibration2 - Calibration Sequence (Core + Runner) actor User participant "run_calibration.py" as Runner participant "engine_config" as EngineConfig participant "case_config" as CaseConfig participant "chronicle builder" as ChronicleBuilder participant "CalibrationEngine" as Engine participant "CalibrationMethod" as Dispatcher participant "Method Implementation" as MethodImpl participant "CalibrationResults" as Results User -> Runner: execute calibration script Runner -> EngineConfig: load_calibration_toml(config_path) EngineConfig -> EngineConfig: validate_calibration_config_data(...) EngineConfig --> Runner: validated config Runner -> CaseConfig: validate_*_chronicle_config(config["chronicle"]) CaseConfig --> Runner: normalized chronicle params Runner -> ChronicleBuilder: build synthetic chronicle ChronicleBuilder --> Runner: observed + forcing + true params Runner -> EngineConfig: resolve_calibration_settings(config, model_parameter_order) EngineConfig --> Runner: objective_metric, method, parameter_set, method_kwargs Runner -> Engine: __init__(observed, simulator, parameter_set, objective_metric) Runner -> Engine: calibrate(method, **method_kwargs) Engine -> Dispatcher: calibrate(objective_cost, bounds, method, **kwargs) Dispatcher -> MethodImpl: method(objective_cost, bounds, **kwargs) loop expensive evaluations MethodImpl -> Engine: objective_cost(x) Engine -> Engine: cost(x) -> score(x) -> simulate(x) end MethodImpl --> Dispatcher: raw method output (dict) Dispatcher --> Engine: raw result Engine -> Results: from_method_output(raw, default_method) Engine -> Results: attach_context(vector_to_params, score_best) Results --> Engine: structured result Engine --> Runner: CalibrationResults Runner --> User: summary + figures @enduml

Reservoir sequence (case example)#

How one runnable calibration case plugs into the shared core.

@startuml title Calibration2 Reservoir Case - Sequence (Case Implementation + Orchestrator) actor User participant "run_calibration.py" as Runner participant "core.case_orchestrator" as CaseOrchestrator participant "engine_config" as EngineConfig participant "case_implementation.py" as CaseImplementation participant "workflow.py" as Workflow participant "synthetic_data.py" as Synthetic participant "case_config.py" as CaseConfig participant "MODEL_REGISTRY\n(one_reservoir / two_reservoirs)" as Registry participant "CalibrationEngine" as Engine participant "CalibrationMethod" as Dispatcher participant "Selected method\n(simplex, gp_mapping, da_mh_gp, ...)" as MethodImpl participant "CalibrationResults" as Results participant "plotting.py" as Plotting User -> Runner: run script Runner -> CaseOrchestrator: run_calibration_case_from_toml(config, CASE_IMPLEMENTATION) CaseOrchestrator -> EngineConfig: load_calibration_toml(config_calibration_two_reservoir.toml) EngineConfig --> CaseOrchestrator: validated configuration CaseOrchestrator -> CaseImplementation: validate_case_config(chronicle_section, ...) CaseImplementation -> CaseConfig: validate_reservoir_chronicle_config(...) CaseConfig --> CaseImplementation: normalized chronicle config CaseImplementation --> CaseOrchestrator: case_config CaseOrchestrator -> CaseImplementation: build_case(case_config, calibration_section, ...) CaseImplementation -> Synthetic: build_noisy_reservoir_chronicle(...) Synthetic -> Registry: parse_chronicle_parameters(...) Registry --> Synthetic: true_params + initial_state alt one_reservoir Synthetic -> Registry: simulate_outflow(..., forcing = Qin) else two_reservoirs Synthetic -> Registry: simulate_outflow(..., forcing = P) end Synthetic --> CaseImplementation: chronicle (forcing, true flow, noisy observations) CaseImplementation -> Workflow: make_reservoir_simulator(...) CaseImplementation -> Workflow: get_model_parameter_order(...) Workflow --> CaseImplementation: simulator + parameter_order CaseImplementation --> CaseOrchestrator: CalibrationCaseContext CaseOrchestrator -> EngineConfig: resolve_calibration_settings(config, parameter_order) EngineConfig --> CaseOrchestrator: objective_metric, method, parameter_set, method_kwargs CaseOrchestrator -> Engine: __init__(observed, simulator, parameter_set, objective_metric) CaseOrchestrator -> Engine: calibrate(method, **method_kwargs) Engine -> Dispatcher: calibrate(objective_cost, bounds, method, **kwargs) Dispatcher -> MethodImpl: method(objective_cost, bounds, **kwargs) loop full-model evaluations MethodImpl -> Engine: objective_cost(x) Engine -> Engine: cost(x) -> simulate(x) -> value_to_cost(...) end MethodImpl --> Dispatcher: method output (dict) Dispatcher --> Engine: raw result Engine -> Results: from_method_output(...) Engine -> Results: attach_context(vector_to_params, score_best) Results --> Engine: structured result Engine --> CaseOrchestrator: CalibrationResults CaseOrchestrator -> CaseImplementation: build_case_outputs(...) CaseImplementation -> Workflow: evaluate_metrics(...) CaseImplementation --> CaseOrchestrator: metrics + params + q_calib CaseOrchestrator --> Runner: calibration payload Runner -> Runner: print terminal summary Runner -> Plotting: plot_calibration_result(chronicle, calibration, output_png, show_plot) Plotting --> Runner: figure saved/displayed Runner --> User: summary + figure path @enduml

Devkit sequence#

The developer tooling used to scaffold and validate new calibration cases.

@startuml title Calibration2 - Devkit Onboarding Sequence actor Developer participant "devkit.new_case" as NewCase participant "templates/case/*.tmpl" as Templates participant "cases/<new_case>/" as CaseFolder participant "devkit.check_case" as CheckCase participant "core.case_orchestrator" as Orchestrator participant "devkit.doctor" as Doctor participant "devkit.config_reference" as ConfigRef participant "docs/config_reference.md" as ConfigRefDoc Developer -> NewCase: scaffold_case("my_case") NewCase -> Templates: read templates Templates --> NewCase: template text NewCase -> CaseFolder: write skeleton files NewCase --> Developer: scaffold report Developer -> CaseFolder: edit workflow / config / implementation Developer -> CheckCase: check_case("my_case", run_smoke=False) CheckCase -> CheckCase: verify required files CheckCase -> CheckCase: import CASE_IMPLEMENTATION CheckCase -> CheckCase: validate config_calibration*.toml CheckCase --> Developer: structural report (ok/errors/warnings) alt optional smoke run Developer -> CheckCase: check_case("my_case", run_smoke=True) CheckCase -> Orchestrator: run_calibration_case_from_toml(...) Orchestrator --> CheckCase: calibration payload CheckCase --> Developer: smoke report + n_evaluations end Developer -> Doctor: run_doctor() Doctor -> Doctor: check dependencies + methods + discovered cases Doctor --> Developer: health report Developer -> ConfigRef: write_config_reference_markdown() ConfigRef -> ConfigRefDoc: generate schema reference ConfigRefDoc --> Developer: updated markdown @enduml

Case / core structure#

How runnable calibration cases are organized around the shared calibration core: where core/ ends and cases/ begins.

@startuml title Calibration2 - Case Core Structure package "hydromodpy.analysis.calibration.core.case_interface" { class CalibrationCaseContext { +observed +simulator +parameter_order +chronicle +metadata } abstract class AbstractCalibrationCase { +CASE_NAME: str +validate_case_config(...) +build_case(...): CalibrationCaseContext +build_case_outputs(...): Mapping } class "case_interface.py" as CaseInterfaceModule <<module>> { +normalize_case_name(name) +validate_case_implementation(case_implementation) } } package "hydromodpy.analysis.calibration.core.case_orchestrator" { class "case_orchestrator.py" as CaseOrchestratorModule <<module>> { +run_calibration_case(config_data, case_implementation, method_override=None) +run_calibration_case_from_toml(config_path, case_implementation, method_override=None) } } package "hydromodpy.analysis.calibration.cases.<case_name>" { class "case_implementation.py" as CaseImplementationModule <<module>> { +CASE_IMPLEMENTATION } class "ConcreteCase" as ConcreteCase } CaseOrchestratorModule ..> AbstractCalibrationCase : expects interface CaseOrchestratorModule ..> CalibrationCaseContext : consumes CaseOrchestratorModule ..> CaseInterfaceModule : validate_case_implementation() ConcreteCase --|> AbstractCalibrationCase CaseImplementationModule ..> ConcreteCase : defines instance CaseImplementationModule ..> CalibrationCaseContext : builds note bottom This diagram shows the contract layer for case integration: interface definition and orchestration entrypoints. end note @enduml

Notes:

  • Case packages under hydromodpy/calibration/cases are expected to stay thin adapters around the shared engine.

  • CLI-specific concerns such as manifests, reruns, and report persistence are owned by hydromodpy/calibration/cli_runner.py and the reporting helpers in hydromodpy/calibration/persistence.py and report.py.

See also#