Back to skill

Security audit

Amazing PsyCoder

Security checks for vulnerabilities and agentic risk

Overview

The skill has a coherent psychology-code purpose, but it bundles runnable reference experiments with unsafe code patterns and data-upload behavior that should be reviewed before use.

Review this skill before installing in a research environment. Do not run or copy bundled raw demos directly, validate all condition files from collaborators, require the reviewer step before data collection, and make any Pavlovia/Liaison upload behavior explicit to participants and researchers. If using install.sh manually, note that it overwrites existing skill directories with the same names.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
psy-exp-coder/psychopy/demo/_raw/Choice_Reaction_Time_Task__choiceRTT_lastrun.py:452
Finding

Arbitrary Python Execution Through Unvalidated Condition-File Column Names

Content
View full analysis

Vulnerability Details

File Location: psy-exp-coder/psychopy/demo/_raw/Choice_Reaction_Time_Task__choiceRTT_lastrun.py:452-470
Additional Locations: The same legacy pattern appears in multiple PsychoPy raw demos and paradigm references, including psy-exp-coder/psychopy/paradigms/choice-reaction-time.md:497-504.
Vulnerability Type: Dynamic code execution using spreadsheet-controlled identifiers
Risk Level: High

Vulnerable Code

python
PracticeLoop = data.TrialHandler(nReps=1.0, method='random',
    extraInfo=expInfo, originPath=-1,
    trialList=data.importConditions('RTtimeConditions.xlsx'),
    seed=None, name='PracticeLoop')
thisExp.addLoop(PracticeLoop)
thisPracticeLoop = PracticeLoop.trialList[0]

# abbreviate parameter names if possible
if thisPracticeLoop != None:
    for paramName in thisPracticeLoop:
        exec('{} = thisPracticeLoop[paramName]'.format(paramName))

for thisPracticeLoop in PracticeLoop:
    currentLoop = PracticeLoop
    if thisPracticeLoop != None:
        for paramName in thisPracticeLoop:
            exec('{} = thisPracticeLoop[paramName]'.format(paramName))

Technical Analysis

data.importConditions() loads column names from an external Excel condition file. Each column name becomes paramName, which is interpolated directly into Python source code and passed to exec().

The program assumes every column name is a harmless Python identifier, but it performs no identifier validation or allowlisting. A maliciously constructed column name can terminate the intended assignment and insert additional Python statements. The resulting statements execute with the full privileges of the user running PsychoPy.

The repository itself identifies this legacy pattern as unsafe in psy-exp-coder/psychopy/spec/README.md:694 and prohibits it in psy-exp-coder/psychopy/mapping/README.md:288-296. Nevertheless, executable demos containing the pattern remain ...[truncated 1512 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove every use of exec() for condition-variable injection from demos and paradigm references.

  2. Access condition fields explicitly:

    python
    this_image = thisPracticeLoop["thisImage"]
    correct_answer = thisPracticeLoop["corrAns"]
    
  3. If generic field processing is necessary, validate fields against a fixed allowlist:

    python
    ALLOWED_FIELDS = {"thisImage", "corrAns", "duration"}
    
    unexpected = set(thisPracticeLoop) - ALLOWED_FIELDS
    if unexpected:
        raise ValueError(f"Unexpected condition fields: {sorted(unexpected)}")
    
  4. Validate workbook provenance and integrity before execution.

  5. Add an automated repository check that rejects exec() and equivalent dynamic-evaluation patterns in executable demos and generated output.

  6. Clearly mark legacy raw demos as non-production references and ensure the generation pipeline cannot copy them verbatim.

  7. Replace vulnerable code in the duplicated paradigm Markdown examples so unsafe patterns are not reintroduced by generated experiments.

T09 · Insecure Skill Coding Practices

Warning
Location
psy-exp-coder/psychopy/demo/_raw/ultimatum_game__ultimatum_game_lastrun.py:1170
Finding

Conditional Transmission of Directly Identifiable Participant Data Without an Explicit Privacy Gate

Content
View full analysis

Vulnerability Details

File Location: psy-exp-coder/psychopy/demo/_raw/ultimatum_game__ultimatum_game_lastrun.py:1170-1172, 1542-1551, 2840-2842
Additional Location: psy-exp-coder/psychopy/paradigms/ultimatum-game.md:1211-1212, 1583-1592, 2881-2883
Vulnerability Type: Insufficient minimization and disclosure for remote research-data transmission
Risk Level: Medium

Vulnerable Code

python
thisExp.addData('welcome.stopped', welcome.tStop)
thisExp.addData('player_name.text', player_name.text)

if thisSession is not None:
    # if running in a Session with a Liaison client, send data up to now
    thisSession.sendExperimentData()

for thisTrial in trials:
    currentLoop = trials
    thisExp.timestampOnFlip(win, 'thisRow.t', format=globalClock.format)
    if thisSession is not None:
        # if running in a Session with a Liaison client, send data up to now
        thisSession.sendExperimentData()

A further transmission occurs after the trial loop:

python
if thisSession is not None:
    # if running in a Session with a Liaison client, send data up to now
    thisSession.sendExperimentData()

Technical Analysis

The experiment requests a participant-entered name and stores it in the ExperimentHandler as player_name.text. The same handler also accumulates behavioral records such as offers, decisions, mouse activity, earnings, timestamps, participant identifiers, and session metadata.

When a PsychoPy Session with a Liaison client is supplied, the experiment calls sendExperimentData() before the trial loop, during trial processing, and after the loop. Consequently, accumulated experiment records may be transmitted to the destination configured by the external session.

This is conditional behavior rather than an unconditional exfiltration channel: the standalone entry point invokes run() without a thisSession argument, so ordinary direct script execution ...[truncated 1813 chars]

Remediation
View remediation

Remediation Suggestions

  1. Do not collect real names by default. Use randomly generated or institution-issued pseudonymous participant identifiers.

  2. Keep direct identifiers in a separate, access-controlled linkage file when identity collection is scientifically necessary.

  3. Make remote synchronization explicitly disabled by default and require a documented configuration flag:

    python
    ENABLE_REMOTE_SYNC = False
    
    if ENABLE_REMOTE_SYNC and thisSession is not None:
        thisSession.sendExperimentData()
    
  4. Before enabling synchronization, disclose the receiving service, data fields, transmission timing, retention policy, and responsible data controller.

  5. Obtain appropriate participant consent and institutional approval for remote transfer.

  6. Remove direct identifiers from the ExperimentHandler before calling sendExperimentData().

  7. Add a deployment-time allowlist for approved session destinations and fail closed when the destination cannot be verified.

  8. Modify the paradigm reference so it describes optional session synchronization rather than embedding transmission calls in reusable code.

  9. Add reviewer checks for direct identifiers combined with sendExperimentData() or equivalent upload APIs.

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (521)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 24)May include surrounding context.

md
the [agentskills.io](https://agentskills.io) open standard. See [PLATFORMS.md](PLATFORMS.md) for platform-specific installation and tool mapping.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 253)May include surrounding context.

md
he full template with detailed per-step requirements is in the [coder SKILL.md](psy-exp-coder/SKILL.md#code-template).

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 259)May include surrounding context.

md
he full template with detailed per-step requirements is in the [coder SKILL.md](psy-exp-coder/SKILL.md#code-template).

Direct Prompt Extraction

High
Category
System Prompt Leakage
Confidence
85% confidence
Finding

Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Content

Scanner excerpt · psy-exp-coder/SKILL.md (reported line 40)May include surrounding context.

md
## Language Consistency (Red Line)

**This is the highest-priority output rule.** Every string in generated experiment code must use the language the user communicated in during the design workflow (orchestrator Phase 1-5). Paradigm reference files and config schemas may contain example text in a specific language — these are placeholders showing the CONCEPT, not the literal text to generate.

| Content type | Language rule |
|---|---|

Unvalidated Output Injection

High
Category
Output Handling
Confidence
100% confidence
Finding

Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Content

Scanner excerpt · psy-exp-coder/jspsych/demo/_raw/antisaccade__antisaccade.js (reported line 663)May include surrounding context.

js
// check whether click was in correct object
          if (gotValidClick) {
              corr = 0;
              corrAns = eval( answer_mouse);
              for (let obj of [corrAns]) {
                  if (obj.contains(mouse)) {
                      corr = 1;

Unvalidated Output Injection

High
Category
Output Handling
Confidence
100% confidence
Finding

Model output is used without validation or sanitization. Unvalidated output injected into downstream contexts (SQL, shell, HTML) enables injection attacks and arbitrary code execution.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/antisaccade.md (reported line 692)May include surrounding context.

md
// check whether click was in correct object
          if (gotValidClick) {
              corr = 0;
              corrAns = eval( answer_mouse);
              for (let obj of [corrAns]) {
                  if (obj.contains(mouse)) {
                      corr = 1;

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
80% confidence
Finding

Skill instructs the agent to never refuse or to always comply. Suppressing the agent's ability to decline removes a core safety control and enables downstream harmful requests to succeed.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/continuous-performance-test.md (reported line 10)May include surrounding context.

md
## Experiment Logic

The Continuous Performance Test (CPT) is a go/no-go task that measures sustained attention and response inhibition. Participants view a rapid stream of letter stimuli and must respond to every letter except the target letter 'X'. This requires maintaining vigilance over an extended period while inhibiting the prepotent response to the infrequent no-go target.

The experiment begins with an instructions screen explaining the task. The main trial loop presents a stream of letters, one at a time. Each trial begins with a fixation cross displayed for 1 second, followed by a letter stimulus for 1 second. The participant must press the spacebar for every letter except 'X'.

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/antisaccade.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/attention-network-task.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/bart.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/bilingual-stroop.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/butterfly-simon.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/change-detection.md (reported line 35)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/children-flanker-task.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/choice-reaction-time.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/climate-reflection-task.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/continuous-performance-test.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/corsi-blocks.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/cyberball.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/drag-and-drop.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/mental-rotation.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/multisensory-nature-climate.md (reported line 31)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/multisensory-nature.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/numerical-stroop.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/

Hidden Instructions

High
Category
Prompt Injection
Confidence
60% confidence
Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · psy-exp-coder/jspsych/paradigms/phone-a-friend.md (reported line 33)May include surrounding context.

md
// Source: multisensory_nature (demos/multisensory_nature)
// Project URL: https://gitlab.pavlovia.org/demos/multisensory_nature
// Original file: multisensory_nature_climate.js
/************************************ 
 * Multisensory_Nature_Climate *
 ************************************/