GitHub’s secret detector accepts 11.5% more alerts and scans 86% faster

What changed

GitHub describes a two-stage pipeline for secret detection: regular expressions first identify likely candidates, then a fine-tuned ModernBERT classifier reviews them. The goal is to catch credentials developers may have accidentally committed while filtering out harmless strings that waste security teams’ time.

In GitHub’s production A/B test, the company reports customer acceptance rose 11.5% and median post-push scan execution time fell 86%. Serving costs were more than 98% lower than the previous LLM-based second stage. GitHub says the classifier also helps detect generic secrets, expanding protection beyond a list of known token patterns.

What the result means

These are GitHub’s measured results from its own service, not a public third-party benchmark. Still, the design lesson is concrete: use a cheaper specialized model for a narrow classification step, and reserve a general-purpose model for problems that need broader reasoning. For a developer, that can mean faster feedback and fewer low-value warnings without removing the first-pass pattern checks.