Back to skill

Security audit

alibabacloud-agent-observability

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent read-only Alibaba Cloud trace-analysis skill, but review is warranted because it caches sensitive telemetry locally by default in a predictable temp directory and retains some cache entries indefinitely.

Install only in an environment where Alibaba Cloud CLI updates/plugins are approved. Prefer setting AGENT_OBS_CACHE_DIR to a private 0700 directory or disabling cache with AGENT_OBS_NO_CACHE=1, and periodically delete generated reports/facts because they may contain enterprise trace messages, tool arguments, infrastructure metadata, and secrets captured in telemetry.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/obs_core.py:59
Finding

Raw Trace Telemetry Stored in an Insecure Shared Temporary Cache

Content
View full analysis

Vulnerability Details

File Location: scripts/obs_core.py, lines 59–60 and 104–113
Vulnerability Type: Insecure temporary storage and sensitive data exposure
Risk Level: Medium

Complete Code Snippet

python
def cache_dir():
    return os.environ.get("AGENT_OBS_CACHE_DIR") or os.path.join(
        tempfile.gettempdir(), "agent_obs_cache")

def cache_put(key_dict, data, stable=False):
    """Store a response atomically; prune the directory occasionally."""
    if not cache_enabled():
        return
    try:
        os.makedirs(cache_dir(), exist_ok=True)
        entry = {"v": 1, "ts": int(time.time()), "stable": stable,
                 "key": key_dict, "data": data}
        path = _cache_path(key_dict)
        tmp = path + ".tmp"
        with open(tmp, "w", encoding="utf-8") as fh:
            json.dump(entry, fh, ensure_ascii=False)
        os.replace(tmp, path)

Technical Analysis

When AGENT_OBS_CACHE_DIR is not configured, the Skill stores cached cloud responses in the predictable shared temporary directory agent_obs_cache under the system temporary directory, normally /tmp/agent_obs_cache.

The directory and files are created without explicit restrictive permissions. Their effective permissions therefore depend on the process umask. The implementation also does not verify that the cache directory is owned by the current user, is not a symbolic link, or was not pre-created by another local user.

Raw responses from authenticated CMS and SLS requests are passed into this cache. These responses can contain enterprise trace attributes, conversation excerpts, tool arguments, tool results, infrastructure metadata, and secrets accidentally recorded in telemetry. Redaction performed during report rendering does not protect the raw cache.

Historical windows marked as stable do not expire, increasing the duration of exposure.

Trigger conditions:

  1. Caching is enabled, which is the default.
  2. AGENT_OBS_CACHE_DIR is unset ...[truncated 2293 chars]
Remediation
View remediation

Remediation Suggestions

  1. Use a private per-user cache directory, such as an application directory under the user's home or XDG cache directory.
  2. Create the directory with mode 0700 and verify that it is owned by the current effective UID.
  3. Reject cache paths that are symbolic links or resolve to directories owned by another user.
  4. Create cache files with mode 0600, preferably using os.open() with O_CREAT | O_EXCL | O_NOFOLLOW where supported.
  5. Use uniquely named temporary files in the verified cache directory before performing an atomic replacement.
  6. Validate that a cache entry's stored key exactly matches the requested key before returning its data.
  7. Apply bounded retention to stable and recent cache entries rather than retaining stable-window responses indefinitely.
  8. Avoid caching message bodies, tool results, or other sensitive fields unless the user explicitly enables sensitive-data caching.
  9. Document the local retention behavior and provide a supported command or option to securely clear cached telemetry.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • YARA SignaturesMalware Match, Webshell Match, Cryptominer Match
Findings (80)

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

The skill markets trace_id-focused observability but also includes time-window aggregation and drift analysis behavior. Expanding from a single-trace model to broader telemetry aggregation changes the privacy and scope profile and should be disclosed clearly.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The skill markets trace_id-focused observability but also includes time-window aggregation and drift analysis behavior. Expanding from a single-trace model to broader telemetry aggregation changes the privacy and scope profile and should be disclosed clearly.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The skill markets trace_id-focused observability but also includes time-window aggregation and drift analysis behavior. Expanding from a single-trace model to broader telemetry aggregation changes the privacy and scope profile and should be disclosed clearly.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
86% confidence
Finding

The skill markets trace_id-focused observability but also includes time-window aggregation and drift analysis behavior. Expanding from a single-trace model to broader telemetry aggregation changes the privacy and scope profile and should be disclosed clearly.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The skill markets trace_id-focused observability but also includes time-window aggregation and drift analysis behavior. Expanding from a single-trace model to broader telemetry aggregation changes the privacy and scope profile and should be disclosed clearly.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 53)May include surrounding context.

md
For correct usage see §8 Core Workflow; forbidden practices: directly calling `preflight.py` / `span_search.py` / `trace_chain.py` / `decision_evidence.py` / `r

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 125)May include surrounding context.

md
For correct usage see §8 Core Workflow; forbidden practices: directly calling `preflight.py` / `span_search.py` / `trace_chain.py` / `decision_evidence.py` / `r

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 196)May include surrounding context.

md
For correct usage see §8 Core Workflow; forbidden practices: directly calling `preflight.py` / `span_search.py` / `trace_chain.py` / `decision_evidence.py` / `r

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 397)May include surrounding context.

md
For correct usage see §8 Core Workflow; forbidden practices: directly calling `preflight.py` / `span_search.py` / `trace_chain.py` / `decision_evidence.py` / `r

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 417)May include surrounding context.

md
For correct usage see §8 Core Workflow; forbidden practices: directly calling `preflight.py` / `span_search.py` / `trace_chain.py` / `decision_evidence.py` / `r

Self-Modification

High
Category
Rogue Agent
Confidence
96% confidence
Finding

The skill directs self-update behavior via a remote install script and built-in upgrade path. Even if intended for dependency management, self-modifying or self-updating instructions increase supply-chain risk and can alter the runtime environment in ways unrelated to the requested analysis.

Content

Scanner excerpt · SKILL.md (reported line 62)May include surrounding context.

md
> [MUST] Verify: `aliyun version` — must be >= 3.3.3.
>
> - **First install or major upgrade:** `/bin/bash -c "$(curl -fsSL --connect-timeout 10 --max-time 120 https://aliyuncli.alicdn.com/setup.sh)"`
> - **Routine update (CLI >= 3.3.5):** `aliyun upgrade` — prefer this built-in self-update over re-running the install script.
> - See `references/cli-installation-guide.md` for full installation instructions.

**Pre-check: Aliyun CLI plugin update required**

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
80% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · SKILL.md (reported line 106)May include surrounding context.

md
>
> - **NEVER** read, echo, print, or dump AK/SK/STS token values in any form. The following commands are **explicitly FORBIDDEN** because they expose credential secrets to the terminal transcript:
>   - `echo $ALIBABA_CLOUD_ACCESS_KEY_ID` / `echo $ALIBABA_CLOUD_ACCESS_KEY_SECRET` / `echo $ALIBABA_CLOUD_SECURITY_TOKEN`
>   - `env | grep -i access` / `env | grep -i secret` / `env | grep -i token` / `printenv`
>   - `cat ~/.aliyun/config.json` / `cat ~/.alibabacloud/credentials` (these files contain plaintext `access_key_secret` and `sts_token`)
>   - Any other command whose output includes credential **values** (not just type or status)
> - **NEVER** ask the user to input AK/SK directly in the conversation or command line

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
80% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · SKILL.md (reported line 106)May include surrounding context.

md
Use `aliyun configure list` to check profile status.

#### ❌ INCORRECT
`echo $ALIBABA_CLOUD_ACCESS_KEY_ID` / `echo $ALIBABA_CLOUD_ACCESS_KEY_SECRET`; `env | grep -i access` / `env | grep -i secret` / `printenv` (dumps credential values to transcript); `cat ~/.aliyun/config.json` / `cat ~/.alibabacloud/credentials` (plaintext `access_key_secret` and `sts_token`); any command whose output contains credential **values**; printing/echoing AK/SK; requesting AK/SK in conversation; `aliyun configure set` with plaintext credentials.

## 5. Query Patterns (Tested Constraints)

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
80% confidence
Finding

Code enumerates, copies, or searches environment variables for secrets. Bulk environment access can collect credentials unrelated to the skill's stated purpose.

Content

Scanner excerpt · references/acceptance-criteria.md (reported line 57)May include surrounding context.

md
Use `aliyun configure list` to check profile status.

#### ❌ INCORRECT
`echo $ALIBABA_CLOUD_ACCESS_KEY_ID` / `echo $ALIBABA_CLOUD_ACCESS_KEY_SECRET`; `env | grep -i access` / `env | grep -i secret` / `printenv` (dumps credential values to transcript); `cat ~/.aliyun/config.json` / `cat ~/.alibabacloud/credentials` (plaintext `access_key_secret` and `sts_token`); any command whose output contains credential **values**; printing/echoing AK/SK; requesting AK/SK in conversation; `aliyun configure set` with plaintext credentials.

## 5. Query Patterns (Tested Constraints)

YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]

High
Category
YARA Match
Confidence
80% confidence
Finding

YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Content

Scanner excerpt · references/behavior-insight-playbook.md (reported line 19)May include surrounding context.

md
nding, and any "this application's error rate/latency" conclusion is invalid; `service_filter_unavailable: true` = LoongSuit side is narrowed but eBPF side cannot be narrowed — the two sides have different calibers and must be labeled with their source separately.

| Scope | Data Source | Record Granularity | Evidence Identifier |
|---|---|---|---|
| `loongsuit` | CMS 2.0 workspace trajectory (UModel `GetEntityStoreData`, including cross-Agent traces) | Single operation span | `trace_id` + `span_id` |
| `ebpf` | SLS `ebpf-event` (raw retrieval + client-side aggregation) | Single runtime fact (process/network/HTTP; **no Session/Turn/Step hierarchy**) | `event.id` + `http.exchange.id` |

Field dictionaries: LoongSuit side the LoongSuit field dictionary; eBPF side the eBPF field dictionary.

Unique material per source (complementary roles):
- loongsuit: `reasoning` parts in `gen_ai.input.messages` (model chain-of-thought), span duration/TTFT, cross-Agent `serviceName` view -> **preferre

Instruction Override

High
Category
Prompt Injection
Confidence
80% confidence
Finding

This pattern attempts to override system instructions or ignore safety constraints. Without LLM analysis, manual review is recommended.

Content

Scanner excerpt · references/behavior-insight-playbook.md (reported line 130)May include surrounding context.

md
- eBPF side has no per-call context sequence (ebpf-event has no messages and no turn/step hierarchy); this view is LoongSuit side only; cross-reference with hang signature to determine whether long task failure is a hang or context pressure

**Security & Hijack Analysis (content retrieval -> forensics)**
- Entry: `span_search.py --match "<injection characteristic text>" --from <ts> --to <ts>` (e.g., "ignore previous instructions", unauthorized instruction fragments; **LoongSuit side only**, ebpf-event has no message payloads). The slice must first be confirmed with the user (the specific time period when the attack occurred) and **span <= maximum window limit** (default 4h); do not scan directly with wide spans like 168h — the script will not reject it, it will just be extremely slow
- Forensics chain: hit span (`hits[].trace_id`/`span_id`) -> `decision_evidence.py --span-id <hit span>` to read input/output messages and tool arguments -> determine whether the target was hijacked (subsequent calls deviate from `entry_task`, sensitive path reads / external address exfiltration / credential-type parameters appear)
- Tool-side signal: sensitive tools (file read, HTTP exfiltration) appear in `trace_overview.by_tool` with parameters pointing to sensitive resources; outbound calls unrelated to `entry_task` are high-weight evidence
- Boundary: when the payload only stays in the tool result and is not consumed by the model (not entering input/output messages), `--match` will not hit; whether it constitutes an attack is a qualitative conclusion and must cite messages/arguments fragments from the forensics chain

Self-Modification

High
Category
Rogue Agent
Confidence
90% confidence
Finding

Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Content

Scanner excerpt · references/cli-installation-guide.md (reported line 71)May include surrounding context.

aliyun version

text

### Self-Update (CLI >= 3.3.5)

Once the CLI is at version 3.3.5 or newer, routine updates can use the built-in self-update subcommand instead of re-running the install script:

Self-Modification

High
Category
Rogue Agent
Confidence
90% confidence
Finding

Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Content

Scanner excerpt · references/cli-installation-guide.md (reported line 73)May include surrounding context.

Self-Update (CLI >= 3.3.5)

Once the CLI is at version 3.3.5 or newer, routine updates can use the built-in self-update subcommand instead of re-running the install script:

bash
aliyun upgrade

Hidden Instructions

High
Category
Prompt Injection
Confidence
70% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · references/report-template.html (reported line 8)May include surrounding context.

html
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Agent链路观测报告</title>
<!--
  ─────────────────────────────────────────────────────────────
  Agent链路观测报告 · 单文件模版
  ─────────────────────────────────────────────────────────────

Hidden Instructions

High
Category
Prompt Injection
Confidence
70% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · references/report-template.html (reported line 8)May include surrounding context.

html
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Agent链路观测报告</title>
<!--
  ─────────────────────────────────────────────────────────────
  Agent链路观测报告 · 单文件模版
  ─────────────────────────────────────────────────────────────

Hidden Instructions

High
Category
Prompt Injection
Confidence
70% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · references/report-template.html (reported line 853)May include surrounding context.

html
</div>
      </div>
      <p class="li__tidrow"><span class="li__lbl">Trace ID</span><span class="li__tid" id="liTraceId">—</span></p>
      <div class="pills" id="liPills"><!-- JS 渲染 --></div>
      <div class="io-cards">
        <div class="io-card">
          <span class="io-card__badge">输入</span>

Hidden Instructions

High
Category
Prompt Injection
Confidence
70% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · references/report-template.html (reported line 911)May include surrounding context.

html
</div>
  </section>

  <!-- ══════════ 板块 4 · LLM调用分析 ══════════ -->
  <section class="blk" id="sec-llm" aria-labelledby="llm-h">
    <h2 class="blk__title" id="llm-h">
      <button class="blk__fold" type="button" data-fold="#sec-llm" aria-expanded="true" aria-label="折叠 LLM调用分析">

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
85% confidence
Finding

The workflow forwards a full copy of the parent process environment to every child script, which can expose cloud credentials, tokens, proxy secrets, or other sensitive runtime state to downstream components. In a skill that orchestrates multiple scripts and processes user-selected traces, this expands the blast radius if any child script is compromised, logs its environment, or is later modified unsafely.

Content

Scanner excerpt · scripts/workflow.py (reported line 175)May include surrounding context.

python
decides."""
    script = os.path.join(SCRIPTS_DIR, CHILD_SCRIPTS[name])
    cmd = [sys.executable, script, *argv]
    env = os.environ.copy()
    if env_extra:
        env.update(env_extra)
    start = time.time()

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The skill requires shell, network, file read/write, and environment access but declares no explicit tool scope or permission boundary. That makes the effective attack surface much larger than users and hosts can easily reason about, especially because the document also instructs installation, upgrade, and report-writing actions.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

Broad triggers like "Trajectory" and "trace analysis" can cause the skill to activate in contexts far outside the intended Alibaba Cloud observability use case. Unintended activation matters here because the skill has shell/network capabilities and may prompt environment-changing setup behavior.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.prompt_injection_instructions

Prompt-injection style instruction pattern detected.

Warn
Code
suspicious.prompt_injection_instructions
Location
references/behavior-insight-playbook.md:130