A Hybrid Approach for Automated Security Pattern Recovery

From security blueprint to code map

Security patterns describe reusable architectural solutions to recurring security problems. The paper treats each pattern as a blueprint: identify its prescribed roles, locate the code that performs them, then reconstruct how those pieces interact. For password authentication, those roles might include the component enforcing a login check, the verification manager and the password store.

The distinction matters because one function can carry several responsibilities, while another role may be implemented by a library outside the project. SecPatRec labels roles as present in the repository, externally implemented or missing from the selected flow. Its scope is deliberately narrower than the whole application: for password authentication, the reference specification selects the login flow rather than registration or password reset.

Two tools for two different jobs

The first stage uses a large language model agent that repeatedly searches and reads code. It receives a structured description of the target pattern and uses both meaning-based search and exact text search to locate relevant functions, methods and other symbols. The output maps architectural roles to specific code locations.

The second stage uses Joern to construct a code property graph, a representation of source-code structure and relationships. A traversal follows the mapped entities to recover their interactions. The authors also add rules for links that Joern misses, including imports and selected framework mechanisms. This is static reconstruction, not observation of a running application. Some added links assume documented framework behaviour without checking the project's configuration.

What the evaluation found

The authors evaluated 100 pattern instances across 93 open-source repositories: 49 Python projects and 44 Java projects, selected for their use of FastAPI or Spring Security. The five patterns cover password-based authentication, verifiable-token authentication, obscure-token access control, session-based access control and rate limiting.

Across four language models, the reported average role-detection accuracy was 0.84, mapping-localization score 0.84 and interaction-recovery score 0.66. These numbers measure different things. Detection checks whether a role's presence status agrees with the reference annotations. Localization rewards proximity to the reference code location, so 0.84 does not mean 84% of mappings were exact. Interaction scoring considers both recovered relationships and their order.

The comparison was a one-shot language model given the pattern and the packed repository, without search tools or graph analysis. Sixteen instances exceeded its configured 900,000-token input limit and received zero scores. That affects the aggregate comparison. On the 84 instances both approaches could process, SecPatRec still reported higher interaction scores for every model, but it did not win every metric.

The authors also report lower estimated language-model API costs per run. Those estimates use public API prices even for two models hosted on their own cloud infrastructure; they do not represent the actual cost of that hosting.

Where the evidence stops

Framework-driven behaviour remains a substantial blind spot despite the added traversal rules. In one Spring session-access-control example, authentication and authorization components were connected at runtime rather than by an explicit repository call. One model correctly detected all six roles, yet the recovered interaction sequence scored zero. Recognizing the pieces did not guarantee reconstructing their behaviour.

The reference annotations also require caution. Claude Code proposed role mappings, and graph traversal generated candidate interaction sequences before human review. The two annotators were experienced web developers, not the systems' architects. On 114 roles initially annotated independently, agreement was low: Cohen's kappa was 0.17. They resolved disagreements together and annotated the remaining roles jointly. The paper also uses the same graph and traversal machinery for reference and pipeline interaction sequences, making human validation especially important.

Each model ran only once per instance and setting, leaving run-to-run variability unresolved. Public repositories and catalogue material may also have appeared in model training. Results cover two languages, two frameworks and five patterns from one catalogue, not arbitrary software architectures.

The supported takeaway is architectural inspection assistance: SecPatRec produces code mappings and interaction evidence that reviewers can examine. The preprint evaluates recovery against annotations, not whether the recovered application is secure.