ℹ
Purpose & Capability
The name/description (resume a previous 'lobster' benchmark run) align with the included bundle: a full v2 evaluation harness, wrapper scripts (run_resume.py), runner/scorer/judge/upload logic, and many task/checker files. The large code bundle is consistent with a benchmark suite rather than a tiny helper. However SKILL.md mentions optional GIGO_* env vars and behavior (upload modes) while the registry declares no required env — those are optional but should be noted.
!
Instruction Scope
SKILL.md gives explicit runtime constraints that limit agent behavior (e.g., "Never use `cd ... && python ..."; "Do not run `--help`, inspect the whole repo, or switch to `main.py` once the wrapper command is clear"). That reduces transparency and looks like a prompt-injection-style attempt to prevent further inspection. The instructions also reference specific log and workspace paths and recommend tailing logs rather than reading them in full. The doc also refers to environment variables (GIGO_*) and behavior (upload/register) that the skill will act upon; these env vars are not listed in requires.env. Overall the instructions go beyond a minimal wrapper-run directive and place operational constraints that a cautious user should verify.
ℹ
Install Mechanism
There is no install spec (instruction-only), which is lower-install risk. However the published package contains a large code bundle (hundreds of files) that will be present on disk when the skill is installed and includes network-capable modules (gateway_client, judge_client, score_uploader). There is no external download URL in the install step, but the included code can initiate outbound requests when executed.
!
Credentials
Registry metadata lists no required environment variables or credentials, yet SKILL.md and many scripts reference optional environment variables (GIGO_LOBSTER_NAME, GIGO_LOBSTER_TAGS, GIGO_REQUIRE_PNG_CERT, GIGO_UPLOAD_MODE, etc.) and the code includes clients that POST to a remote /judge gateway and upload scores. The skill can perform network uploads by design (gigo-lobster-taster uploads by default). Because no primary credential or gateway URL is declared in metadata, it's unclear which endpoints and credentials (if any) will be used; this gap increases risk of unintended data exfiltration if you run the wrapper with defaults.
✓
Persistence & Privilege
The skill does not request always:true and does not declare system-wide persistence. It contains many scripts but there is no install-time mechanism that forces permanent agent inclusion or modifies other skills' configs. Normal autonomous invocation is allowed (default), which is expected for skills.
Scan Findings in Context
[ignore-previous-instructions] unexpected: A pattern consistent with prompt injection was detected in SKILL.md. The SKILL.md does include directives that tell the agent not to run repository inspection or certain commands; while that may be intended to reduce confusion, it resembles 'ignore previous instructions' style manipulation and is not expected for a transparent wrapper.
[unicode-control-chars] unexpected: Unicode control characters were flagged in SKILL.md by the pre-scan. Use of invisible control chars is uncommon for benign README/runtime instructions and can be used to obfuscate or alter how tooling parses the file; inspect the raw file if you plan to run it.
What to consider before installing
What to consider before installing or running:
- This package is a full benchmark harness that by default can contact remote services (gateway /judge and score upload). If you don't want uploads, run in local/register/doctor modes or pass --skip-upload / use gigo-lobster-local before running the resume wrapper.
- SKILL.md contains explicit instructions that tell the agent not to inspect the repository and to run only the wrapper. That reduces transparency — review run_resume.py, main.py, and scripts/gateway_client.py / scripts/score_uploader.py yourself before executing.
- The metadata declares no required env vars, yet the skill references optional GIGO_* env vars and relies on a gateway client. Confirm what gateway_base / credentials your environment will provide and whether network calls will be made (and to which hosts) before running.
- Because the bundle includes many files capable of network I/O and uploading, run it first in an isolated environment: a disposable VM, container, or sandbox, and inspect the code (search for requests.post, gateway URLs, hard-coded endpoints, and any code that reads files outside the workdir).
- If you only want to test safely, run gigo-lobster-doctor or gigo-lobster-local to avoid uploading; or run run_resume.py with --skip-upload / --register-only as appropriate.
- If anything is unclear (unknown publisher, unexpected network hosts, or the presence of invisible control characters), do not run it with real credentials or on a machine with sensitive data. Ask the publisher to clarify gateway endpoints, what is uploaded, and why SKILL.md prohibits repo inspection.
If you'd like, I can:
- show the top-level of run_resume.py and scripts/gateway_client.py so you can see what remote hosts/URLs are used, or
- produce a short checklist of files to inspect for network/exfiltration behavior before running.
