EAD Accuracy Metric¶

The ead_accuracy metric evaluates the accuracy of predicted EAD (Exposure at Default) values by comparing them to observed EAD values for defaulted accounts.

Metric Type: ead_accuracy

Accuracy Calculation¶

The accuracy is calculated as: 1 - |predicted_ead - observed_ead| / observed_ead

This metric only considers accounts that actually defaulted, comparing the predicted exposure amount to the actual exposure amount at the time of default.

Configuration Fields¶

Record-Level Data Format¶

For individual defaulted loan/account records:

metrics:
  ead_validation:
    metric_type: "ead_accuracy"
    config:
      name: ["ead_model_accuracy"]
      data_format: "record_level"
      predicted_ead: "predicted_exposure" # Column with predicted EAD values
      actual_ead: "actual_exposure" # Column with observed EAD values
      default: "default_flag" # Column with default indicators (must be 1)
      segment: [["product_type"]] # Optional: segmentation columns
      dataset: "defaulted_accounts"

Summary-Level Data Format¶

For pre-aggregated defaulted account data:

metrics:
  summary_ead:
    metric_type: "ead_accuracy"
    config:
      name: ["aggregated_ead_accuracy"]
      data_format: "summary_level"
      predicted_ead: "mean_predicted_ead" # Column with mean predicted EAD
      actual_ead: "mean_actual_ead" # Column with mean observed EAD
      defaults: "default_count" # Column with default counts
      segment: [["risk_grade"]] # Optional: segmentation columns
      dataset: "ead_summary"

Required Fields by Format¶

Record-Level Required¶

name: Metric name(s)
data_format: Must be "record_level"
predicted_ead: predicted_ead column name
actual_ead: Actual EAD column name
default: Default indicator column name
dataset: Dataset reference

Summary-Level Required¶

name: Metric name(s)
data_format: Must be "summary_level"
predicted_ead: Mean predicted EAD column name
actual_ead: Mean actual EAD column name
defaults: Default count column name
dataset: Dataset reference

Optional Fields¶

segment: List of column names for grouping

Output Columns¶

The metric produces the following output columns:

group_key: Segmentation group identifier (struct of segment values)
defaults: Total number of defaults analyzed
observed_ead: Mean observed EAD value
predicted_ead: Mean predicted EAD value
accuracy: Calculated EAD accuracy score

Fan-out Examples¶

Multiple EAD Validations¶

metrics:
  ead_validation_suite:
    metric_type: "ead_accuracy"
    config:
      name: ["overall_ead", "product_ead", "vintage_ead"]
      segment: [null, ["product_type"], ["origination_year"]]
      data_format: "record_level"
      predicted_ead: "model_ead_prediction"
      actual_ead: "realized_ead"
      default: "default_indicator"
      dataset: "historical_defaults"

This creates three EAD accuracy metrics:

Overall EAD accuracy across all defaults
EAD accuracy by product type
EAD accuracy by origination vintage

Comparative Analysis¶

metrics:
  model_comparison:
    metric_type: "ead_accuracy"
    config:
      name: ["model_v1_ead", "model_v2_ead"]
      segment: [null, null]
      data_format: "record_level"
      predicted_ead: "v1_ead_pred" # Note: This would need separate configs
      actual_ead: "actual_ead"
      default: "default_flag"
      dataset: "model_comparison_data"

  # Second metric for model v2
  model_v2_ead:
    metric_type: "ead_accuracy"
    config:
      name: ["model_v2_ead"]
      data_format: "record_level"
      predicted_ead: "v2_ead_pred"
      actual_ead: "actual_ead"
      default: "default_flag"
      dataset: "model_comparison_data"

Data Requirements¶

Record-Level Data¶

One row per defaulted loan/account
predicted_ead: positive numbers or None (negative values not allowed)
Actual EAD: positive numbers or None (negative values not allowed)
Default indicator: must be 1 (only defaulted accounts are analyzed)

Summary-Level Data¶

One row per group/segment of defaulted accounts
Mean predicted EAD: positive numbers or None (negative values not allowed)
Mean actual EAD: positive numbers or None (negative values not allowed)
Default counts: numeric values (any numeric value is allowed)

Important Notes¶

Defaulted Accounts Only: This metric only analyzes accounts that actually defaulted
Positive EAD Values: EAD values (predicted and actual) must be positive numbers or None - negative values are not allowed
Currency Consistency: Ensure predicted and actual EAD values are in the same currency/units
Data Quality: Remove outliers or data quality issues that might skew accuracy calculations