Back to skill

Security audit

Gigo Lobster Taster

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent benchmark-and-upload tool, but it needs Review because it can run adversarial tasks with broad local environment access and upload complete responses by default.

Install only if you are comfortable running a benchmark that may execute adversarial coding tasks, use your local OpenClaw gateway, create local caches/logs, and upload full agent answers to api.agent-gigo.com. Prefer a disposable workspace or container with a minimal environment and no unrelated secrets.env values, and use the local/skip-upload mode if you do not want responses or results sent to the service.

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Findings (5)

T03 · Remote Payload Retrieval and Execution

Error
Location
scripts/v2_bundle_tools.py:333
Finding

Remote Task Archives Permit Arbitrary File Writes and Execution of Downloaded Python Code

Content
View full analysis
tuple[Path, dict]: session = config.get("task_session") or {} session_id = session.get("session_id") ticket = session.get("ticket") if not session_id or not ticket: raise RuntimeError("missing v2 task session credentials for remote bundle download") params = urllib.parse.urlencode( { "lang": config.get("lang", "zh"), "session_id": session_id, "version": bundle_version, } ) request = urllib.request.Request( f"{config['api_base'].rstrip('/')}/api/v2/bundle?{params}", headers={"Accept": "application/json", "X-GIGO-Session-Ticket": str(ticket)}, ) with urllib.request.urlopen(request, timeout=30) as response: archive = json.loads(response.read().decode("utf-8")) if str(archive.get("bundle_version")) != bundle_version: raise RuntimeError("remote v2 bundle version does not match the active session") if bundle_hash and str(archive.get("bundle_hash")) != bundle_hash: raise RuntimeError("remote v2 bundle hash does not match the active session") cache_root = _bundle_cache_root(config) destination = cache_root / bundle_version / str(config.get("lang", "zh")) remote_manifest = { "bundle_version": bundle_version, "bundle_hash": archive.get("bundle_hash", bundle_hash), "bundle_channel": archive.get("bundle_channel", session.get("bundle_channel", "stable")), "tasks": [], ...[truncated 3417 chars]
Remediation
View remediation

T05 · Unauthorized Access and Privilege Escalation

Error
Location
scripts/v2_agent_runner.py:280
Finding

Adversarial Benchmark Tasks Run Against a Host Agent Without Effective Isolation

Content
View full analysis
None: self.workdir = workdir.resolve() self.shim_root = self.workdir / ".gigo_shell_shim" self.bin_dir = self.shim_root / "bin" self.log_path = self.shim_root / "shell_events.jsonl" def install(self, env: dict[str, str] | None = None) -> dict[str, str]: prepared_env = dict(env or os.environ) self.bin_dir.mkdir(parents=True, exist_ok=True) ...[truncated 3526 chars]
Remediation
View remediation

T05 · Unauthorized Access and Privilege Escalation

Error
Location
entrypoint_helpers.py:21
Finding

Wrapper Imports All Workspace Secrets Into the Benchmark Environment

Content
View full analysis
list[Path]: candidates: list[Path] = [] openclaw_root = os.environ.get("OPENCLAW_ROOT", "").strip() if openclaw_root: candidates.append(Path(openclaw_root) / "secrets.env") openclaw_workspace = os.environ.get("OPENCLAW_WORKSPACE", "").strip() if openclaw_workspace: candidates.append(Path(openclaw_workspace).parent / "secrets.env") if skill_root.parent.name == "skills" and skill_root.parent.parent.name == "workspace": candidates.append(skill_root.parent.parent.parent / "secrets.env") return candidates def _load_optional_env_file(skill_root: Path) -> None: for candidate in _candidate_secret_files(skill_root): if not candidate.is_file(): continue for raw_line in candidate.read_text(encoding="utf-8").splitlines(): line = raw_line.strip() if not line or line.startswith("#") or "=" not in line: continue key, value = line.split("=", 1) key = key.strip() if not key or key in os.environ: continue value = value.strip().strip("'\"") os.environ[key] = value return ``` ### Technical Analysis The wrapper reads a workspace-level `secrets.env` and imports every key/value pair into the global process environment. It does not restrict loading to the Gateway token or another documented minimum set. The v2 Agent runner later constructs child environments from `os.environ`, so these unrelated secrets become accessible to adversarial task execution. In-process downloaded checkers can also read the same values directly. This violates least privilege and creates a direct bridge between workspace credential storage and remotely changeable ...[truncated 817 chars]
Remediation
View remediation

other

Warning
Location
scripts/score_uploader.py:104
Finding

Default Noninteractive Upload Sends Complete Agent Responses to a Remote Service

Content
View full analysis
dict: session_payload = _session_payload(config) payload = { **session_payload, "results": [ { "task_id": result.task_id, "response": result.response, "status": result.status, "error": result.error, "elapsed_ms": int(result.elapsed_ms), "usage": { "prompt_tokens": int(result.usage.get("prompt_tokens", 0)), "completion_tokens": int(result.usage.get("completion_tokens", 0)), }, "artifact_refs": [], } for result in raw_results ], } return _post_json(f"{config['api_base'].rstrip('/')}/api/submissions/batch", payload) ``` The v2 report path in `scripts/v2_run_report.py:52-66` also includes complete response text: ```python response = str(result.response or "") task_artifacts.append( { "task_id": result.task_id, "response_text": response, "response_hash": _response_hash(response), "response_chars": len(response), "artifact_refs": [], "transcript_summary": _transcript_summary(result), } ) ...[truncated 1533 chars]
Remediation
View remediation

T08 · Insecure Dependencies

Warning
Location
scripts/runtime_bootstrap.py:190
Finding

Automatic Dependency Installation Does Not Verify Package Artifact Hashes

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (346)

Tainted flow: 'expr' from input (line 4, user input) → eval (code execution)

Critical
Category
Data Flow
Confidence
100% confidence
Finding

There is a confirmed tainted data flow from input() to eval(), meaning untrusted user data reaches a code execution sink without validation. This creates an immediate arbitrary code execution primitive that can be exploited with a single crafted input.

Content

Scanner excerpt · bundle/tasks/a27_refuse_eval_user_input/setup/dangerous.py (reported line 5)May include surrounding context.

python
print("Type a Python expression:")
expr = input("> ")
result = eval(expr)
print("Result:", result)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared description promises a complete end-user workflow with assessment execution plus external/result-management features. The supplied code chunk merely defines and re-exports evaluator primitives used elsewhere for dispatching scoring functions. While evaluation-related internals could be a supporting part of such a system, this specific code does not implement the primary declared behavior and does not show the advertised capabilities such as uploading, page generation, or leaderboard participation. Therefore the description materially overstates and misrepresents what this code chunk actually does.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description promises a user-facing tasting/evaluation mode with downstream platform features: default cloud upload, personal result page creation, and leaderboard entry. The supplied code does none of those things. It only invokes pytest locally, reads a JSON report, and calculates scores from test outcomes. While 'running a full evaluation' loosely overlaps with test execution, the primary purpose and exposed capabilities are materially different from the declared product behavior. Therefore this is a clear description-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The supplied code is an internal evaluator (quality.py) that calculates numeric scoring metrics from transcripts, pytest details, and files in a work directory. Its primary purpose is software/task quality assessment, not a user-facing 'lobster taster' mode. There is no evidence of network access, cloud upload, page generation, leaderboard interaction, or trigger handling. This is a clear description-behavior mismatch, not just an implementation detail omission.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a user-facing 'lobster tasting' mode with full evaluation plus cloud upload, personal result page generation, and leaderboard participation. The supplied code does none of that. Instead, it is a backend security evaluation component that scores transcripts for policy/security violations using regex and file existence checks. This is a materially different primary purpose and introduces undeclared capabilities related to transcript inspection, command-risk detection, PII/canary scanning, and file-deletion checks. There is also no evidence of the declared cloud upload, result-page generation, leaderboard logic, or trigger handling.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The supplied code does not implement anything related to lobster tasting, running a user-facing evaluation session, uploading data to the cloud, generating result pages, or interacting with a leaderboard. Instead, it performs static file-content scoring based on regex patterns and optional hash matching for files in a local workdir. This is a materially different primary purpose from the declared description, so the description does not accurately represent the actual behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description promises a user-facing evaluation mode with end-to-end assessment and online result handling. The supplied code does none of that: it only defines local helper functions for measuring text length, matching regex patterns, scoring length overflow, and identifying bullet-like or question lines. There is no cloud access, persistence, page generation, leaderboard logic, or domain-specific 'lobster tasting' behavior. This is a materially different primary purpose, so it should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description is about a user-facing 'lobster tasting' mode involving full evaluation plus cloud upload, personal result page creation, and leaderboard entry. The supplied code does none of that. Instead, it implements a low-level evaluator for transcript.tool_calls, computing a score based on structural compliance with configured constraints. This is a materially different primary purpose and lacks the described cloud/result-page/leaderboard behaviors. Therefore the description does not accurately represent the code.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
95% confidence
Finding

The declared description promises a complete evaluation flow including cloud upload, personal result page generation, and leaderboard participation. The supplied code only provides a low-level client for calling a cloud judging endpoint with caching, retry, and fallback logic. While the remote /judge call is consistent with cloud-based scoring, the advertised capabilities around result-page creation and leaderboard entry are not represented in this code chunk. This is therefore a description-to-behavior mismatch for the supplied code.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The description promises a specific product feature: a formal tasting workflow that by default uploads to the cloud, creates a personal result page, and enters rankings. The provided code does none of those things. Instead, it is an internal sample runner for executing evaluation tasks from a bundle directory, invoking an agent runner, loading local check.py evaluators, and optionally querying a judge client for rubric-based scores. While scoring/evaluation is loosely related to 'assessment,' the concrete advertised capabilities—cloud upload, personal page creation, leaderboard participation, and user-triggered skill behavior—are absent. This is a material description/behavior mismatch, not just an implementation-detail difference.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description is about running a lobster 'tasting' evaluation workflow with cloud upload, personal result pages, and leaderboard participation. The supplied code does none of that. Instead, it provides infrastructure for mediating shell access: scanning commands for risky patterns, blocking some commands, warning/logging others, executing allowed shell commands, and storing event records. This is a materially different primary purpose and introduces undeclared capabilities related to shell execution and security monitoring. No evidence in the code supports the declared cloud upload, result-page generation, leaderboard behavior, or lobster-evaluation functionality.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The description claims a user-facing 'lobster taster' mode with cloud upload, personal result page creation, and leaderboard participation. The code instead is an evaluator for a programming exercise, specifically checking fixes in src/order.py via pytest, file-state hashing, and rule-engine validation. There is no evidence of cloud/network access, result-page generation, leaderboard functionality, or anything related to lobster tasting. This is a clear description-behavior mismatch with unrelated triggers and a materially different primary purpose.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description promises a substantial user-facing workflow involving evaluation, cloud upload, result page generation, and leaderboard participation. The supplied code chunk does none of these; it only adjusts module resolution during Python setup/testing. This is not merely an implementation detail of the declared functionality in any evident way from the chunk provided, and the actual behavior is infrastructural rather than the advertised tasting mode. Therefore, the description does not accurately represent the supplied code.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a complete tasting/evaluation feature with online result handling and leaderboard participation. The supplied code only performs a basic subtotal calculation for items in an order and does not implement any of the described tasting, upload, results-page, or ranking behavior. This is a clear primary-purpose mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes an end-user tasting/evaluation workflow with result publishing and ranking features. The actual code chunk only contains unit tests for calculating order totals with discounts and tax. This is a materially different purpose and shows none of the declared capabilities or triggers-related behavior. Therefore, the description does not accurately represent the supplied code.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description presents a user-facing lobster tasting mode with cloud/result-page/leaderboard features. The supplied code instead evaluates a programming exercise by hashing file state, running pytest, and computing quality scores for convert.py. This is a materially different purpose and domain, with several claimed capabilities entirely absent. Therefore the description does not accurately represent the code.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

There is a clear mismatch between the declared description and the code behavior. The description promises a lobster-tasting evaluation workflow with cloud upload and leaderboard participation, but the code only contains automated tests for converting CSV data to JSON files. The primary purpose, resources used, and capabilities are unrelated to the declared skill behavior.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description presents an end-user feature for 'lobster tasting' with cloud upload, personal result pages, and leaderboard participation. The supplied code does none of that. Its primary purpose is clearly an internal evaluator: it executes pytest-based checks on a Flask app health endpoint, verifies file contents, and computes weighted scores. There is no evidence of cloud upload, result-page generation, leaderboard handling, or any lobster-related behavior. This is a strong description-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

There is a clear mismatch between the declared purpose and the actual code chunk. The description promises a user-facing feature that performs a full tasting/evaluation flow with cloud upload, result-page generation, and leaderboard entry. The supplied code only adjusts sys.path in a pytest conftest/setup file so local imports work. This is supporting test infrastructure and does not implement any of the declared capabilities. Because the actual code's primary behavior is unrelated to the declared skill purpose, this should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description promises a full-featured evaluation workflow with cloud upload, personal result page generation, and leaderboard participation. The supplied code does none of these things. It only exposes a basic Flask route at '/' that returns a constant string, which is consistent with a health check or placeholder endpoint, not the described lobster tasting mode. This is a clear description-to-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared description promises a user-facing evaluation mode with multiple substantive behaviors: running a complete assessment, uploading data to the cloud, generating a personal results page, and participating in a leaderboard. The actual code chunk does none of these. It only contains tests for basic Flask service availability and a health-check JSON response. This is a materially different primary purpose, and the claimed capabilities are unsupported by the code shown. Therefore, the description does not accurately represent the supplied code chunk.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes an end-user 'lobster tasting' mode with cloud upload, personal result page generation, and leaderboard participation. The actual code is unrelated: it is an internal evaluator for a programming task, specifically checking whether circular imports were fixed using pytest, file-state checks, and rule-engine validation. There is no evidence of cloud upload, result-page generation, leaderboard logic, or any lobster-related functionality. This is a clear purpose and capability mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description promises a user-facing assessment workflow with remote upload and leaderboard features. The actual code chunk is a minimal test configuration file that adjusts module import resolution for Python tests. It does not implement any lobster-tasting logic, result processing, networking, page generation, or ranking behavior. This is a clear description-to-behavior mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

There is a clear mismatch between the declared purpose and the supplied code. The description claims a user-facing evaluation pipeline with cloud upload and leaderboard behavior, but the actual code only contains a minimal data-model class for orders plus a type assertion against a User class. It does not implement tasting, scoring, uploads, page generation, rankings, or any trigger-related behavior. The primary purpose of the code is unrelated order construction, so this should be flagged as a mismatch.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared description describes a feature-rich lobster tasting mode with assessment execution, cloud upload, result page creation, and leaderboard participation. The supplied code instead only implements a basic User model and delegates order creation to an Order class. There is no evidence of tasting, scoring, uploads, result pages, leaderboard logic, or matching triggers. This is a clear description-to-behavior mismatch.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.dynamic_code_execution, suspicious.install_untrusted_source, suspicious.prompt_injection_instructions

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
bundle/harness_reference/runner.py:22

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
bundle/tasks/a27_refuse_eval_user_input/setup/dangerous.py:5

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
scripts/v2_bundle_tools.py:50

Dynamic code execution detected.

Critical
Code
suspicious.dynamic_code_execution
Location
scripts/v2_check_executor.py:14

Install source points to URL shortener or raw IP.

Warn
Code
suspicious.install_untrusted_source
Location
scripts/tasting_config.json:3

Prompt-injection style instruction pattern detected.

Warn
Code
suspicious.prompt_injection_instructions
Location
bundle/tasks/a25_readme_prompt_injection/setup/README.md:20