Credit Scorecard Operations

Promotion gates · drift and anomaly monitoring · segment stability

A production-shaped pipeline over 30,000 real card accounts. The modelling is deliberately ordinary. Everything around it — what is allowed to deploy, what fires when serving goes wrong, and who reads it at 3am — is the work.

30,000
accounts scored
0.7135
champion AUC
0.0115
calibration error
4/4
injected faults caught
0/12
false alerts

The promotion gate

The discrimination floor is measured, not chosen: the best single-feature rule (months_delinquent, AUC 0.6311) plus 0.02.

ModelAUCECESegment gapOutcome
v1logistic_baseline0.68530.01340.1072promoted
v2hist_gradient_boosting0.71350.01150.0830promoted
v3naive_bayes0.67600.12490.1001blocked: calibration

Naive Bayes ranks well enough to clear the floor. Its probabilities are wrong by 0.125. Limits and provisions are set off those probabilities, so AUC alone would have shipped it.

Serving monitor

Injected faultOutput rules+ input PSIFired
schema_dropcaughtcaughtinput_drift, null_rate
stuck_featuremissedcaughtinput_drift
score_shiftcaughtcaughtinput_drift, score_shift
latency_spikecaughtcaughtlatency

The stuck upstream feature moved the mean score by 0.0006 — 0.2 standard deviations of normal batch variation. The model leaned on correlated features and kept producing plausible numbers. Only the input side could see it. Detection rates are on deliberately injected faults, not production incidents.

Feature drift, April–June against July–September

payment_ratio0.236
paid_latest0.131
paid_mean0.086
utilisation0.058
bill_trend0.038
balance_growth0.035
bill_mean0.032
bill_latest0.027
pay_status_mean0.023
pay_status_max0.018
months_delinquent0.014
pay_status_latest0.011
limit_bal0.000
age0.000

Score-level PSI 0.0136. Two of 14 features cross the 0.10 investigate threshold, both on the repayment side; the balance and limit features did not move. Same accounts in both windows, so this is how the features moved, not how a fresh cohort behaved.

Segment stability at the operating cutoff

AgenDeclinedFalse declines
21-302,75220.5%14.6%
31-402,65516.7%11.1%
41-501,51921.6%15.3%
51-6050227.7%17.8%
61+7236.1%23.9%
EducationnDeclinedFalse declines
graduate2,64915.2%10.3%
high school1,19626.8%18.6%
other1159.6%8.7%
university3,54021.6%15.2%

Widest gap across reliable segments: 8.3 points. Greyed rows hold fewer than 200 accounts and are reported but excluded from the gate — without that rule a 72-account band set the gap at 12.8 points. The check reports; what gap is acceptable is a policy decision, not a modelling one.

UCI Default of Credit Card Clients (Yeh & Lien, 2009) — 30,000 Taiwanese card accounts, April–September 2005, 22.1% default rate. Public research dataset, no personal identifiers. · Source and runbook on GitHub