E-CVWMD and E-CVWMD-Pairwise: Novel Joint Performance Metrics for Mixed-Type Multivariate Hydroclimatic Models

Evaluating joint predictive performance for multivariate hydroclimatic models requires metrics that simultaneously assess marginal accuracy and cross-variable dependence recovery. Existing metricsthe Energy Score, Variogram Score, and their derivativesdo not adapt to the structural complexity of the residual correlation matrix, treating a single correlated pair identically to a fully dense dependence structure. We propose two novel metric families: Metric E (E-CVWMD: Enhanced Coefficient-of-Variation Weighted Marginal-Dependence) and Metric E2 (E-CVWMD-Pairwise), which are designed for mixed-type multivariate responses combining continuous and binary outcomes within a cross-validation framework. We position Metrics E and E2 as diagnostic ranking tools for comparing competing models rather than as strictly proper scoring rules, and we provide a strictly proper Log-Loss variant (E-LL/E2-LL) for applications that require the full properness guarantee. Metric E assigns variable-level weights proportional to the coefficient of variation (CV) of each outcome on the training partition and adaptively calibrates the marginal-dependence trade-off parameter α ∗ via a global distance-correlation test. Metric E2 refines this by replacing the global test with a pairwise Spearman screening index π ^ , the proportion of variable pairs with significant residual correlationwhich maps linearly to α ∗ ( π ^ ) = 1 − π ^ / 2 ∈ [ 0.5 , 1 ] . Applied to the validation of a Generalized Multivariate Functional Additive Mixed Model (GMFAMM) on 62 Valle del Cauca meteorological stations ( N test ≈ 31,663), the naive significance-based index saturates ( π ^ = 1.0 ) at this large sample sizeevery pair, including correlations as small as | ρ ^ s | ≈ 0.01 , is flagged “significant”which is precisely the sample-size sensitivity we address. Under the effect-size screening ( | ρ ^ s | ≥ 0.05 ), three negligibly correlated pairs are excluded, yielding π ^ = 0.70 and α E 2 ∗ = 0.65 , a better-calibrated weight than Metric E’s α E ∗ ≈ 0.797 under the same data. A large-scale simulation study with 37,440 model evaluations confirms that Metric E inverts the correct ranking at correlation levels ρ ≥ 0.40 (CDR = 0%), while E2 maintains correct discrimination in 14 of 15 simulation conditions (M1 vs. M3). We also delimit the metrics’ scope: E2 degrades under near-saturated uniform dependencea regime in which the strictly proper Energy Score remains preferableand the pairwise index is sensitive to sample size, for which we provide an effect-size-based variant. An R package (mvmetrics v0.2.0) implementing both metrics, the Log-Loss variant, alternative weighting schemes, and the effect-size screening is publicly available.