Arxiv Gamedevbench Evaluating Agentic Capabili

Learned from arXiv paper GameDevBench: Evaluating Agentic Capabilities Through Game Development. Use this skill to scaffold Node.js experiments based on the...

Security Scan

The skill is an auto-generated Node.js scaffold that matches its description (a runnable paper-based scaffold); it requests only the node binary and contains no network calls, secret access, or surprising install steps.

Purpose & Capability

Name/description claim Node.js scaffolding for the GameDevBench paper and the skill only requires the node binary; included files (index.js, scripts/run.js, package.json, paper.json) are consistent with that purpose.

Instruction Scope

SKILL.md instructs running scripts/run.js. The included run.js only logs metadata and a truncated abstract; there are no instructions to read unrelated files, access environment variables, or send data to external endpoints.

Install Mechanism

No install spec provided (instruction-only). There are local code files bundled, but no downloads, package installs, or extract steps. package.json has no dependencies, so nothing is pulled at runtime beyond node.

Credentials

The skill declares no required environment variables or credentials and the code does not access process.env; requested privileges are minimal and appropriate for a local Node.js scaffold.

Persistence & Privilege

always is false and the skill does not persist configuration or modify other skills/system settings. It only exposes a main() that prints to stdout.

Assessment

This skill appears coherent and low-risk: it’s an auto-generated Node.js scaffold that only prints a paper summary and a TODO. Before running, inspect the bundled files (already included) to confirm there are no added network calls or secret reads. Execute in a sandbox or isolated environment if you want additional caution. If you plan to extend the scaffold, review package.json before adding dependencies and avoid running it with elevated privileges.

Source

• Paper key: 44f3ad505bee7a5c25a60d2a3686cb7e
• Title: GameDevBench: Evaluating Agentic Capabilities Through Game Development
• Categories: cs.AI,cs.CL,cs.SE

Learned insight

Despite rapid progress on coding agents, progress on their multimodal counterparts has lagged behind. A key challenge is the scarcity of evaluation testbeds that combine the complexity of software development with the need for deep multimodal understanding. Game development provides such a testbed as agents must navigate large, dense codebases while manipulating intrinsically multimodal assets such as shaders, sprites, and animations within a visual game scene. We present GameDevBench, the first

Node.js implementation entry

node {baseDir}/scripts/run.js