EAD Accuracy Metric

The ead_accuracy metric evaluates the accuracy of predicted EAD (Exposure at Default) values by comparing them to observed EAD values for defaulted accounts.

Metric Type: ead_accuracy

Accuracy Calculation

The accuracy is calculated as: 1 - |predicted_ead - observed_ead| / observed_ead

This metric only considers accounts that actually defaulted, comparing the predicted exposure amount to the actual exposure amount at the time of default.

Configuration Fields

Record-Level Data Format

For individual defaulted loan/account records:

collections:
  ead_validation:
    metrics:
    - name:
      - ead_model_accuracy
      data_format: record
      predicted_ead: predicted_exposure
      actual_ead: actual_exposure
      default: default_flag
      segment:
      - - product_type
      metric_type: ead_accuracy
    dataset: defaulted_accounts

Summary-Level Data Format

For pre-aggregated defaulted account data:

collections:
  summary_ead:
    metrics:
    - name:
      - aggregated_ead_accuracy
      data_format: summary
      predicted_ead: mean_predicted_ead
      actual_ead: mean_actual_ead
      defaults: default_count
      segment:
      - - risk_grade
      metric_type: ead_accuracy
    dataset: ead_summary

Required Fields by Format

Record-Level Required

• name: Metric name(s)
• data_format: Must be "record"
• predicted_ead: predicted_ead column name
• actual_ead: Actual EAD column name
• default: Default indicator column name
• dataset: Dataset reference

Summary-Level Required

• name: Metric name(s)
• data_format: Must be "summary"
• predicted_ead: Mean predicted EAD column name
• actual_ead: Mean actual EAD column name
• defaults: Default count column name
• dataset: Dataset reference

Optional Fields

• segment: List of column names for grouping

Output Columns

The metric produces the following output columns:

• group_key: Segmentation group identifier (struct of segment values)
• defaults: Total number of defaults analyzed
• observed_ead: Mean observed EAD value
• predicted_ead: Mean predicted EAD value
• accuracy: Calculated EAD accuracy score

Fan-out Examples

Multiple EAD Validations

collections:
  ead_validation_suite:
    metrics:
    - name:
      - overall_ead
      - product_ead
      - vintage_ead
      segment:
      - null
      - - product_type
      - - origination_year
      data_format: record
      predicted_ead: model_ead_prediction
      actual_ead: realized_ead
      default: default_indicator
      metric_type: ead_accuracy
    dataset: historical_defaults

This creates three EAD accuracy metrics:

1. Overall EAD accuracy across all defaults
2. EAD accuracy by product type
3. EAD accuracy by origination vintage

Comparative Analysis

collections:
  model_comparison:
    metrics:
    - name:
      - model_v1_ead
      - model_v2_ead
      segment:
      - null
      - null
      data_format: record
      predicted_ead: v1_ead_pred
      actual_ead: actual_ead
      default: default_flag
      metric_type: ead_accuracy
    dataset: model_comparison_data
  model_v2_ead:
    metrics:
    - name:
      - model_v2_ead
      data_format: record
      predicted_ead: v2_ead_pred
      actual_ead: actual_ead
      default: default_flag
      metric_type: ead_accuracy
    dataset: model_comparison_data

Data Requirements

Record-Level Data

• One row per defaulted loan/account
• predicted_ead: positive numbers or None (negative values not allowed)
• Actual EAD: positive numbers or None (negative values not allowed)
• Default indicator: must be 1 (only defaulted accounts are analyzed)

Summary-Level Data

• One row per group/segment of defaulted accounts
• Mean predicted EAD: positive numbers or None (negative values not allowed)
• Mean actual EAD: positive numbers or None (negative values not allowed)
• Default counts: numeric values (any numeric value is allowed)

Important Notes

1. Defaulted Accounts Only: This metric only analyzes accounts that actually defaulted
2. Positive EAD Values: EAD values (predicted and actual) must be positive numbers or None - negative values are not allowed
3. Currency Consistency: Ensure predicted and actual EAD values are in the same currency/units
4. Data Quality: Remove outliers or data quality issues that might skew accuracy calculations