LatentTriad screens AI agent skills before they act

Reusable skills give language-model agents instructions, scripts and configuration for specialized tasks. They can also introduce instructions that exceed what the user authorized. The security question is contextual: reading project files may be appropriate for a backup, while sending those files to an unapproved destination is not. A package’s stated purpose does not itself grant permission.

In the under-review preprint LatentTriad: Pre-Action Risk Representations for Malicious Agent Skill Detection, Zhe Yu and colleagues investigate whether that distinction can be detected inside a model before it generates an action. Their results support pre-action screening under the tested conditions, not a general guarantee that malicious skills can be identified without execution.

Reading the model before action

LatentTriad examines hidden states: numerical representations formed as a model processes its input. It uses the initial processing stage, called prefill, when the model reads the trusted task, tool interface and assembled skill package. The detector generates no output tokens, invokes no tools and executes no skill-provided code.

The method extracts three kinds of states. Declaration states cover the skill’s introductory description. Operation states cover the remaining instructions and readable auxiliary files, including scripts and configuration. Boundary states come from the final prompt position before generation. These are contextual representations from one assembled input, rather than separately encoded file summaries.

Three analysis branches use those states. Boundary reads the complete context. Operation measures projections onto learned risk directions while reducing some fitted group-specific variation. Geometry measures departures from relationships learned on benign packages, such as how declarations relate to operations. Those relationships are statistical references, not causal models, and an unusual relationship alone does not establish maliciousness.

The fixed decision rule allows Boundary to trigger a block independently. An additional Operation alert requires Geometry agreement. All branches reuse the same prefill, although their readouts require training on labeled packages.

Strong benchmark results, uneven coverage

The main evaluation contains 5,830 packages from five sources, arranged into 2,915 malicious–benign pairs. Related content and paired packages are grouped to prevent them from crossing training and test partitions. The authors evaluate seven model backbones, fitting detectors separately for the evaluation splits.

They report pooled malicious-class F1 scores of 99.0–99.2%. F1 combines precision and recall; it is not the percentage of all packages classified correctly. When each source receives equal weight, F1 falls to 93.0–94.6%. That distinction matters because one source supplies nearly 90% of the packages.

False alarms show a similar imbalance. Pooled false-positive rates are 1.1–1.4%, but equal-source averages are 8.0–11.2%. The headline pooled result therefore does not describe benign acceptance equally well across all five ecosystems. With an entire source excluded from training and development, mean source-macro F1 drops further to 73.2%. LatentTriad still exceeds the tested same-backbone representation detectors on that metric, but its within-source performance does not transfer intact.

The component experiments also qualify the architecture’s benefits. Geometry alone produces higher F1 than the full system at the tested threshold, while a simpler Boundary-or-Operation rule achieves higher recall and F1 than the fixed three-branch rule. The findings support useful information across multiple states without establishing that all three prediction heads, or their chosen combination, are necessary.

What the internal signals establish

Controlled experiments test task–operation relationships expressed as natural language, Python, JSON and mixed text. Risk remains readable when an expression form and test domains are withheld from fitting. Separate interventions replace Operation states with matched donor states and observe shifts in later Boundary representations and fixed detector decisions.

These interventions provide evidence of functional influence inside the tested computations. They do not show an agent executing the substituted operation or establish Boundary as the necessary route through which the detector reaches its verdict.

More demanding structural tests reveal a limit. When the full architecture is trained on synthetic tasks and tested with new opaque symbol assignments, performance remains near chance. These are separate synthetic fits, not retests of the natural-package checkpoints. Strong composition-benchmark results also coexist with strong workspace-only controls, so benchmark success does not by itself demonstrate general reasoning about cross-file dependencies.

Adversarial results require caution too. Valid feedback-guided rewrites cover few initially blocked paths. Separate white-box rewrites often preserve detection but substantially increase false alarms on their selected benign subset.

A screen, not a security boundary

A separately measured compact implementation completes prefill and detection in a median 259–284 milliseconds on the reported single-GPU setup. It uses different extraction and scoring settings from the main detector, whose end-to-end runtime was not measured.

Deployment requires access to model internals and a validated input contract. Binary contents, later downloads and environment-dependent behavior may remain outside the inspected context. The authors therefore position LatentTriad alongside permission enforcement and runtime monitoring: an early warning mechanism, not proof that a passing skill is safe.