CARE

Canonicalization · Attribution · Resolution Engine —
Pre-Execution Command Verification for Shell-Executing LLM Agents

Yu Liu^1* · Wenxiao Zhang^2* · Zhiwei Yang^1* · Zhongyi Zhang^1† · Hanqi Feng³ · Xinyu Wang² · Peng Qiu³ · Yanbing Liu¹ · Barnabas Poczos³ · Jin B. Hong^2†
¹Institute of Information Engineering, Chinese Academy of Sciences · ²The University of Western Australia · ³Carnegie Mellon University
^*Equal contribution ^†Corresponding authors

LLM agents now run your shell. CARE checks every command before it executes: deterministic, shell-aware evidence decides the common case in a fraction of a millisecond — and only the genuinely ambiguous 4% ever wake an LLM judge.

📄 Paper (ISSRE 2026) 💻 Code 📚 BibTeX

🎉 News · 2026-06-16 — CARE is accepted at the IEEE International Symposium on Software Reliability Engineering (ISSRE 2026)!

ISSRE 2026 · Accepted Pre-Execution Shell Guard Static-First · 2.32 ms MIT License

agent → CARE → host shell

$ rm -rf /var/log/*
✗ DENY  score 0.765 · L2+L3+L4 · MITRE T1485 · 0.3 ms

$ cat /etc/os-release
✓ ALLOW  read-context on system path · 0.2 ms

$ bash -c 'trap "nohup sh …/echo-art.sh" EXIT'
⚠ WARN score 0.18 → LLM judge → DENY  persistence hook

$ x=rm; y=-rf; eval "$x$IFS$y$IFS/tmp/*"
✗ DENY  canonicalized first — obfuscation unwrapped

TL;DR

Fast like a rule engine. Careful like a judge.

On the leakage-controlled main split (549 commands, 12 baselines), CARE occupies the region no prior guard reaches: high F1, sub-1% false positives, and millisecond latency — with a full evidence trace for every decision.

85.64%

F1 on the main split

best LLM judge: 73.09% · best static baseline: 72.62%

0.91%

false-positive rate

1 deny in 300 benign NL2SH tasks

2.32ms

mean latency

median 0.24 ms — LLM judges: 17–460 ms

4.2%

commands reach the LLM

23 of 549 — everything else stays static

WHY

Command dispatch is the new attack surface

Coding agents like Claude Code and Codex don't just write text — they run commands. One unsafe dispatch can delete files, leak secrets, or plant persistence. CVE-2025-66032 showed Claude Code's command validation bypassed with a simple $IFS rewrite. Yet existing guards force a bad trade: regex-style rules are fast but blind (29–73% F1); always-on LLM judges are smart but slow, costly, and erratic (17–460 ms, up to 46% FPR).

Bubble chart of eleven guards: F1 versus false-positive rate, bubble size is latency. CARE's green star sits alone in the high-F1, low-FPR, low-latency corner. — **Nobody was in the top-left corner.** F1 vs. FPR (log) across eleven guards; bubble area = mean latency (log). Static rules (blue) stay fast but plateau at 72.6% F1; LLM guards (pink) live in the 17–460 ms band with high FPR. CARE (★) reaches 85.64% F1 at 0.91% FPR and 2.32 ms — and its static profile (●) nearly matches at 0.34 ms. *(click to zoom)*

KEY IDEA

Three moves, in order — and the LLM goes last

Canonicalize before you judge

A bounded, non-executing normalizer unwraps shell wrappers, resolves $IFS/variable splitting, decodes base64/printf tricks — turning an obfuscated command into a stable verification target. The exact bypass class behind CVE-2025-66032 dies here, before any analysis runs.

Proof: best obfuscation-suite F1 of all 12 guards — 88.62% on 250 obfuscated attacks.

Evidence with provenance, not vibes

Five deterministic views — AST structure, command semantics, read/write-aware path sensitivity, and 139 provenance-tagged rules (92 MITRE ATT&CK · 31 GTFOBins · 16 curated) — aggregate into ALLOW / WARN / DENY. Every decision ships an auditable trace: which layer fired, which rule, which catalog it came from.

Proof: ablating any one view costs 1.8–26.7 pp F1 — all five earn their keep.

Escalate only true ambiguity

High-confidence ALLOW/DENY never touch a model. Skip predicates keep catalog-attested and sensitive-path denies fully static; only the residual WARN band — 23 of 549 commands — goes to a safety-biased LLM judge. Judge errors fail closed to DENY.

Proof: Resolution halves FPR (1.82→0.91%) at unchanged recall, for +2 ms mean.

CARE pipeline: Stage 1 canonicalization, Stage 2 five-layer attribution, Stage 3 resolution with verdict routing, escalation policy and LLM judge. — **The pipeline.** Stage 1 normalizes the raw command; Stage 2 builds multi-view static evidence (L1 syntax → L2 semantics, L3 paths, L4 pattern+provenance → L5 policy triage); Stage 3 finalizes confident verdicts directly and routes only underdetermined WARN cases through the escalation policy to the judge. *(click to zoom)*

RESULTS

The numbers that matter

Guard (main split, 549 cmds)	F1 % ↑	DR % ↑	FPR % ↓	Latency ↓
Best static baseline (OpenClaw4Layer)	72.62	57.27	0.30	0.02 ms
Best LLM judge (LLMJudge, Qwen3-30B)	73.09	67.27	11.25	45.9 ms
Best guard model (Granite Guardian 2B)	65.58	82.27	45.90	17.3 ms
CARE (w/o Resolution) — static only	84.99	75.91	1.82	0.34 ms
CARE — full pipeline	85.64	75.91	0.91	2.32 ms

Significant over PromptOnly, LLMJudge, ClawGuardSim, OpenClaw4Layer and Llama Guard 3 under McNemar's test (all p < 10⁻⁵).

🧑‍💻 Developers barely notice it

NL2SH-ALFA benign tasks: 57.00% resolved with CARE vs 57.33% with no guard at all — a single deny in 300 tasks. Over 5-turn agent loops: identical 65.33% resolve rate, 4 cumulative denies (Granite Guardian: 529).

🐳 Real harm, really reduced

600 attack commands generated by an LLM agent, executed in a Docker sandbox: realised harm drops from 74.8% (unguarded) to 37.3% with CARE's static profile — the most conservative non-LLM operating point, at a 38.7% deny rate instead of Granite's 71.8% blanket blocking.

🎛️ One engine, two deployment profiles

Full CARE when benign recovery matters: the judge rescues dual-use commands like rsync/scp, halving FPR. Static-only when conservatism matters: 84.99% F1 at 0.34 ms, zero LLM dependency, strongest harm reduction on LLM-paraphrased attacks.

Threat model: adversarial influence reaches the LLM agent; CARE guards the command dispatch boundary seeing only the command and bounded path context. — **Scope, honestly stated.** CARE mediates *single commands* at the dispatch boundary, seeing only the command string and bounded path context — never prompts or agent reasoning. It is a complement to sandboxing and host hardening, not a replacement; session-level attack chains remain future work. *(click to zoom)*

CITATION

BibTeX

@inproceedings{liu2026care,
  author    = {Liu, Yu and Zhang, Wenxiao and Yang, Zhiwei and Zhang, Zhongyi and
               Feng, Hanqi and Wang, Xinyu and Qiu, Peng and Liu, Yanbing and
               Poczos, Barnabas and Hong, Jin B.},
  title     = {{CARE}: Pre-Execution Command Verification for Shell-Executing {LLM} Agents},
  booktitle = {IEEE International Symposium on Software Reliability Engineering (ISSRE)},
  year      = {2026}
}