Discrimination Metrics¶

Discrimination metrics for assessing model's ability to distinguish between classes.

discrimination ¶

Discrimination metrics helper functions.

This module provides convenient helper functions for discrimination-related statistical metrics.

auc ¶

auc(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level"],
    *,
    prob_def: str,
    default: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

auc(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["summary_level"],
    *,
    mean_pd: str,
    defaults: str,
    volume: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

auc(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level", "summary_level"],
    **kwargs: Any,
) -> pl.DataFrame

Calculate the Area Under the ROC Curve (auc) for record-level or summary-level data.

Record-level usage (data_format="record_level"): Required parameters: prob_def, default

Summary-level usage (data_format="summary_level"): Required parameters: mean_pd, defaults, volume

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the metric.	required
`dataset`	`LazyFrame \| DataFrame`	Dataset to compute the auc on.	required
`data_format`	`Literal['record_level', 'summary_level']`	Format of the input data ("record_level" or "summary_level").	required
`**kwargs`	`Any`	Additional keyword arguments specific to the data format. For record_level: prob_def (str), default (str), segment (optional) For summary_level: mean_pd (str), defaults (str), volume (str), segment (optional)	`{}`

Returns:

Type	Description
`DataFrame`	DataFrame containing the auc result and associated metadata.

Examples:

Record-level usage:

result = auc(
    name="model_auc",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag"
)

Summary-level usage:

result = auc(
    name="portfolio_auc",
    dataset=summary_df,
    data_format="summary_level",
    mean_pd="mean_pd",
    defaults="defaults",
    volume="volume"
)

kolmogorov_smirnov ¶

kolmogorov_smirnov(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level"],
    *,
    prob_def: str,
    default: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

kolmogorov_smirnov(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["summary_level"],
    *,
    mean_pd: str,
    defaults: str,
    volume: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

kolmogorov_smirnov(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level", "summary_level"],
    **kwargs: Any,
) -> pl.DataFrame

Calculate the Kolmogorov-Smirnov statistic for record-level or summary-level data.

The Kolmogorov-Smirnov statistic measures the maximum difference between the cumulative distribution functions of predicted scores for defaulters vs non-defaulters. It ranges from 0 to 1, where higher values indicate better discrimination.

Record-level usage (data_format="record_level"): Required parameters: prob_def, default

Summary-level usage (data_format="summary_level"): Required parameters: mean_pd, defaults, volume

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the metric.	required
`dataset`	`LazyFrame \| DataFrame`	Dataset to compute the KS statistic on.	required
`data_format`	`Literal['record_level', 'summary_level']`	Format of the input data ("record_level" or "summary_level").	required
`**kwargs`	`Any`	Additional keyword arguments specific to the data format. For record_level: prob_def (str), default (str), segment (optional) For summary_level: mean_pd (str), defaults (str), volume (str), segment (optional)	`{}`

Returns:

Type	Description
`DataFrame`	DataFrame containing the KS statistic, p-value, and associated metadata.

Examples:

Record-level usage:

result = kolmogorov_smirnov(
    name="model_ks",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag"
)

Summary-level usage:

result = kolmogorov_smirnov(
    name="portfolio_ks",
    dataset=summary_df,
    data_format="summary_level",
    mean_pd="mean_pd",
    defaults="defaults",
    volume="volume"
)

gini ¶

gini(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level"],
    *,
    prob_def: str,
    default: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

gini(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["summary_level"],
    *,
    mean_pd: str,
    defaults: str,
    volume: str,
    segment: list[str] | None = None,
) -> pl.DataFrame

gini(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level", "summary_level"],
    **kwargs: Any,
) -> pl.DataFrame

Calculate the Gini coefficient for record-level or summary-level data.

The Gini coefficient is calculated as 2*AUC - 1, where AUC is the Area Under the ROC Curve. It ranges from -1 to 1, where: - 1 indicates perfect discrimination - 0 indicates no discrimination (random) - -1 indicates perfectly inverse discrimination

Record-level usage (data_format="record_level"): Required parameters: prob_def, default

Summary-level usage (data_format="summary_level"): Required parameters: mean_pd, defaults, volume

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the metric.	required
`dataset`	`LazyFrame \| DataFrame`	Dataset to compute the Gini coefficient on.	required
`data_format`	`Literal['record_level', 'summary_level']`	Format of the input data ("record_level" or "summary_level").	required
`**kwargs`	`Any`	Additional keyword arguments specific to the data format. For record_level: prob_def (str), default (str), segment (optional) For summary_level: mean_pd (str), defaults (str), volume (str), segment (optional)	`{}`

Returns:

Type	Description
`DataFrame`	DataFrame containing the Gini coefficient result and associated metadata.

Examples:

Record-level usage:

result = gini(
    name="model_gini",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag"
)

Summary-level usage:

result = gini(
    name="portfolio_gini",
    dataset=summary_df,
    data_format="summary_level",
    mean_pd="mean_pd",
    defaults="defaults",
    volume="volume"
)

f1_score ¶

f1_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level"],
    *,
    prob_def: str,
    default: str,
    threshold: float = 0.5,
    segment: list[str] | None = None,
) -> pl.DataFrame

f1_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["summary_level"],
    *,
    mean_pd: str,
    defaults: str,
    volume: str,
    threshold: float = 0.5,
    segment: list[str] | None = None,
) -> pl.DataFrame

f1_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level", "summary_level"],
    **kwargs: Any,
) -> pl.DataFrame

Calculate the F1 score for record-level or summary-level data.

The F1 score is the harmonic mean of precision and recall, providing a balanced measure of classification performance.

Record-level usage (data_format="record_level"): Required parameters: prob_def, default Optional parameters: threshold (default 0.5)

Summary-level usage (data_format="summary_level"): Required parameters: mean_pd, defaults, volume Optional parameters: threshold (default 0.5)

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the metric.	required
`dataset`	`LazyFrame \| DataFrame`	Dataset to compute the F1 score on.	required
`data_format`	`Literal['record_level', 'summary_level']`	Format of the input data ("record_level" or "summary_level").	required
`**kwargs`	`Any`	Additional keyword arguments specific to the data format. For record_level: prob_def (str), default (str), threshold (float), segment (optional) For summary_level: mean_pd (str), defaults (str), volume (str), threshold (float), segment (optional)	`{}`

Returns:

Type	Description
`DataFrame`	DataFrame containing the F1 score and associated metrics for each group.

Examples:

Record-level usage:

result = f1_score(
    name="model_f1",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag",
    threshold=0.6
)

Summary-level usage:

result = f1_score(
    name="portfolio_f1",
    dataset=summary_df,
    data_format="summary_level",
    mean_pd="mean_pd",
    defaults="defaults",
    volume="volume",
    threshold=0.4
)

f2_score ¶

f2_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level"],
    *,
    prob_def: str,
    default: str,
    threshold: float = 0.5,
    segment: list[str] | None = None,
) -> pl.DataFrame

f2_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["summary_level"],
    *,
    mean_pd: str,
    defaults: str,
    volume: str,
    threshold: float = 0.5,
    segment: list[str] | None = None,
) -> pl.DataFrame

f2_score(
    name: str,
    dataset: LazyFrame | DataFrame,
    data_format: Literal["record_level", "summary_level"],
    **kwargs: Any,
) -> pl.DataFrame

Calculate the F2 score for record-level or summary-level data.

The F2 score weights recall higher than precision, making it suitable for scenarios where missing positive cases (false negatives) is more costly than false positives.

Record-level usage (data_format="record_level"): Required parameters: prob_def, default Optional parameters: threshold (default 0.5)

Summary-level usage (data_format="summary_level"): Required parameters: mean_pd, defaults, volume Optional parameters: threshold (default 0.5)

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the metric.	required
`dataset`	`LazyFrame \| DataFrame`	Dataset to compute the F2 score on.	required
`data_format`	`Literal['record_level', 'summary_level']`	Format of the input data ("record_level" or "summary_level").	required
`**kwargs`	`Any`	Additional keyword arguments specific to the data format. For record_level: prob_def (str), default (str), threshold (float), segment (optional) For summary_level: mean_pd (str), defaults (str), volume (str), threshold (float), segment (optional)	`{}`

Returns:

Type	Description
`DataFrame`	DataFrame containing the F2 score and associated metrics for each group.

Examples:

Record-level usage:

result = f2_score(
    name="model_f2",
    dataset=df,
    data_format="record_level",
    prob_def="probability",
    default="default_flag",
    threshold=0.3
)

Summary-level usage:

result = f2_score(
    name="portfolio_f2",
    dataset=summary_df,
    data_format="summary_level",
    mean_pd="mean_pd",
    defaults="defaults",
    volume="volume",
    threshold=0.7
)

options: show_source: false heading_level: 2 members_order: source