Skip to content

TNP Statistic Library

A comprehensive Python library for statistical calculations and metrics, particularly focused on financial risk modeling and validation.

Overview

The TNP Statistic Library provides statistical metrics for risk modeling with two distinct approaches:

  1. Interactive Function-based Usage - For exploratory analysis and direct calculations
  2. Batch Processing with YAML Workflows - For production pipelines and standardized reporting

Available Metrics

  • Accuracy Metrics: Default accuracy, EAD accuracy, Hosmer-Lemeshow test, Jeffreys test, Binomial test, MAPE (Mean Absolute Percentage Error), RMSE (Root Mean Squared Error), T-Test
  • Discrimination Metrics: AUC (Area Under Curve), Gini coefficient, Kolmogorov-Smirnov statistic, F1 score, F2 score
  • Normality Testing: Shapiro-Wilk test for assessing data distribution normality
  • Summary Statistics: Mean, median calculations with segmentation support

Key Features

  • Support for both record-level and summary-level data formats
  • Segmentation capabilities for group-based analysis
  • Built on Polars for high-performance data processing
  • Type-safe interfaces with comprehensive overloads
  • Consistent API across all metrics

Installation

pip install tnp-statistic-library

For advanced installation options including package mirrors and wheel files, see the Getting Started guide.

Approach Comparison

Feature Interactive Functions YAML Workflows
Use Case Data exploration, ad-hoc analysis Production pipelines, standardized reporting
Type Safety Full IDE support and type checking Schema validation
Batch Processing Manual iteration required Built-in fan-out expansion
Configuration Code-based parameters Declarative YAML files
Flexibility Maximum flexibility Structured, repeatable configurations

Quick Start Examples

Interactive Function Usage

import polars as pl
from tnp_statistic_library.metrics import default_accuracy, auc, mape, f1_score, f2_score, binomial_test, shapiro_wilk

# Create sample data
df = pl.DataFrame({
    "probability": [0.1, 0.3, 0.7, 0.2, 0.9, 0.4],
    "default_flag": [0, 0, 1, 0, 1, 0],
    "actual_values": [0.08, 0.35, 0.65, 0.25, 0.85, 0.45],
    "predicted_values": [0.1, 0.3, 0.7, 0.2, 0.9, 0.4],
    "region": ["North", "North", "South", "East", "South", "East"]
})

# Calculate model accuracy
accuracy_result = default_accuracy(
    name="model_validation",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag"
)

# Calculate AUC by region
auc_result = auc(
    name="discrimination_power",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag",
    segment=["region"]
)

# Calculate MAPE for prediction accuracy
mape_result = mape(
    name="prediction_accuracy",
    dataset=df,
    data_format="record_level",
    observed="actual_values",
    predicted="predicted_values",
    segment=["region"]
)

# Calculate F1 score for classification performance
f1_result = f1_score(
    name="classification_f1",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag",
    segment=["region"]
)

# Test if observed default rate differs from expected
binomial_result = binomial_test(
    name="default_rate_test",
    dataset=df,
    data_format="record_level",
    default="default_flag",
    expected_probability=0.35,  # Expected 35% default rate
    segment=["region"]
)

# Calculate F2 score emphasizing recall
f2_result = f2_score(
    name="classification_f2",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag",
    threshold=0.3,  # Lower threshold for better recall
    segment=["region"]
)

# Test data normality assumption for statistical tests
normality_result = shapiro_wilk(
    name="data_normality_test",
    dataset=df,
    data_format="record_level",
    data_column="actual_values",
    segment=["region"]
)

YAML Workflow Usage

# config.yaml
datasets:
  portfolio_data:
    location: "data.csv"

metrics:
  model_validation:
    metric_type: default_accuracy
    config:
      name: ["accuracy_check"]
      dataset: "portfolio_data"
      data_format: "record_level"
      prob_def: "probability"
      default: "default_flag"

Start Here

  • Getting Started - Complete tutorial with step-by-step examples and installation options
  • Examples - Comprehensive reference for all metric types
  • Metrics by Model Type - Guide to selecting metrics for PD, EAD, and LGD models

Interactive Usage (Function-based)

Batch Processing (YAML-based)

Requirements

  • Python 3.11+
  • Polars for high-performance data processing
  • NumPy, SciPy for statistical calculations

Support

For questions, issues, or contributions, please refer to the project repository.