Back to skill

Security audit

Paper To Table

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly does its stated paper-to-table job, but users should review it because it can retain paper excerpts in logs and write untrusted spreadsheet values without formula neutralization.

Install only if you are comfortable reviewing outputs before opening them in spreadsheet software and storing logs that may contain excerpts from processed papers. Avoid using it on confidential or untrusted papers unless formula neutralization and reduced logging are added.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Warning
Location
references/extraction-patterns.md:26
Finding

Untrusted Paper Content Is Embedded Directly into the LLM Instruction Prompt

Content
View full analysis

Vulnerability Details

File Location: references/extraction-patterns.md:26-35
Vulnerability Type: Indirect prompt injection
Risk Level: Medium

Vulnerable Code

text
## HEADER DEFINITIONS
{header_definitions}

## PAPER CONTENT
{paper_text}

## EXTRACTION INSTRUCTIONS
For each header:
1. Search the paper content thoroughly for explicit mentions
2. Extract the exact or closely paraphrased information

Technical Analysis

The prompt template inserts untrusted paper text directly into the same instruction context used to direct the LLM. Although the template contains extraction quality rules, it does not explicitly classify the paper as untrusted data or tell the model to disregard instructions appearing inside the document.

A malicious PDF, DOCX, or TXT file can therefore include instructions that attempt to override the extraction rules, manipulate the generated JSON, fabricate field values, or request disclosure of prompt context. This is an instruction-data boundary failure. The generated data may subsequently reach the table-writing workflow because validation does not verify that every value is supported by cited paper evidence.

Attack Path

  1. An attacker supplies a crafted academic-paper file containing hidden or visible model instructions.
  2. extract_paper.py extracts those instructions as ordinary document text.
  3. The extracted text is substituted into the {paper_text} placeholder.
  4. The LLM receives attacker-controlled instructions in the same context as trusted extraction instructions.
  5. The LLM may follow the embedded instructions and return manipulated JSON.
  6. The manipulated values can be passed to write_table.py and persisted in the output table.

Impact Assessment

Exploitation can compromise the integrity of extracted research data, produce fabricated or attacker-selected table values, corrupt the expected JSON structure, and potentially disclose ...[truncated 236 chars]

Remediation
View remediation

Remediation Suggestions

  • Explicitly state that paper content is untrusted data and that instructions appearing within it must never be followed.
  • Place document content in a separate, data-only message or structured attachment when the LLM API supports role separation.
  • Use strong delimiters around the document and state that delimiter contents may only be analyzed as evidence.
  • Require the output to conform to a strict JSON Schema containing only expected header names.
  • Reject unexpected keys, invalid confidence values, missing sources, and incorrect value types.
  • Require every non-N/A value to include a verifiable source quotation or location from the paper.
  • Add detection and human-review handling for documents containing instruction-like phrases directed at an AI system.
  • Do not allow extracted document content to invoke tools, alter system prompts, or select output destinations.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/write_table.py:151
Finding

Untrusted Extracted Values Are Written to CSV and XLSX Without Formula Neutralization

Content
View full analysis

Vulnerability Details

File Location: scripts/write_table.py:151-158 and scripts/write_table.py:200-210
Vulnerability Type: Spreadsheet formula injection
Risk Level: Medium

Vulnerable Code

python
# Write data matching headers
for col_idx, header in enumerate(headers, 1):
    if header:
        # Handle nested dict (with confidence) or direct value
        if header in row_data:
            value = row_data[header]
            if isinstance(value, dict):
                ws.cell(row=next_row, column=col_idx, value=value.get('value', ''))
            else:
                ws.cell(row=next_row, column=col_idx, value=value)
python
# Append row
with open(table_path, 'a', encoding=encoding, newline='') as f:
    writer = csv.writer(f)
    row = []
    for h in headers:
        h_stripped = h.strip() if h else ''
        if h_stripped in row_data:
            value = row_data[h_stripped]
            if isinstance(value, dict):
                row.append(value.get('value', ''))
            else:
                row.append(value)
        else:
            row.append('')
    writer.writerow(row)

Technical Analysis

Values derived from untrusted papers and LLM output are written directly into spreadsheet cells. The code does not escape or reject strings beginning with spreadsheet formula markers such as =, +, -, or @. It also does not account for leading tab or carriage-return characters that some spreadsheet applications may ignore before interpreting a formula.

The existing validate_data function only searches for selected inference-related phrases. It does not sanitize spreadsheet formulas or enforce that extracted content is stored as literal text.

CSV files are particularly exposed because spreadsheet applications infer cell types when the file is opened. XLSX values may also be interpreted as formulas when formula-leading strings are assign ...[truncated 1167 chars]

Remediation
View remediation

Remediation Suggestions

  • Introduce one centralized function that converts every externally derived cell value to safe literal text.
  • Neutralize values whose first effective character is =, +, -, or @, including values prefixed by tabs, carriage returns, or other ignorable whitespace.
  • For CSV output, prefix potentially dangerous values with an apostrophe or use another neutralization method compatible with the supported spreadsheet applications.
  • For XLSX output, explicitly set cells to text format and assign a string value that cannot be interpreted as a formula.
  • Apply sanitization to nested value properties and direct scalar values alike.
  • Preserve the original unsanitized value only in a separate, non-executable audit representation if necessary.
  • Add tests covering formula markers, leading whitespace, tabs, carriage returns, Unicode variants, and nested extraction objects.
  • Treat formula neutralization as mandatory rather than tying it to the optional hallucination-validation flag.

T09 · Insecure Skill Coding Practices

Note
Location
scripts/batch_process.py:59
Finding

Batch Logs Persist Unnecessary Paper Content in Plaintext

Content
View full analysis

Vulnerability Details

File Location: scripts/batch_process.py:59-64 and scripts/batch_process.py:132-144
Vulnerability Type: Excessive sensitive-data retention
Risk Level: Low

Vulnerable Code

python
result['llm_input'] = {
    'headers': headers,
    'domain': domain,
    'paper_text': extraction_data.get('full_text', '')[:5000]  # First 5000 chars for preview
}

return result
python
# Save progress log after each paper
with open(log_file, 'w', encoding='utf-8') as f:
    json.dump({
        'batch_info': {
            'total': len(papers),
            'success': success_count,
            'failed': failed_count,
            'timestamp': datetime.now().isoformat()
        },
        'results': results
    }, f, ensure_ascii=False, indent=2)

Technical Analysis

The batch result retains the first 5,000 characters of every processed paper under llm_input.paper_text. The complete result list is repeatedly serialized to a persistent JSON log. Operational progress logging does not require retaining document excerpts.

The log is created using normal process defaults, without explicitly restrictive file permissions, encryption, access-control checks, or a retention policy. Papers may contain unpublished research, licensed text, personal information, participant details, or other confidential material. Persisting excerpts creates an additional copy with potentially broader exposure than the source document.

Attack Path

  1. A user batch-processes confidential or restricted papers.
  2. The first 5,000 characters of each paper are stored in the in-memory result.
  3. The result is serialized into batch_log_YYYYMMDD_HHMMSS.json.
  4. Another local user, backup process, synchronization service, or unintended recipient gains access to the output directory.
  5. The party reads the retained paper excerpts from the log without needing access to the original documen ...[truncated 380 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove paper_text from persistent batch results and logs by default.
  • Log only status, source identifier, cryptographic file hash, text length, detected sections, timestamps, warnings, and errors.
  • If diagnostic excerpts are explicitly enabled, make the feature opt-in and clearly warn users that document content will be retained.
  • Create log files with owner-only permissions where the operating system supports it.
  • Store sensitive diagnostics in a user-selected secure directory rather than under a broadly accessible output folder.
  • Add configurable retention limits and securely delete diagnostic logs when they are no longer needed.
  • Avoid recording unnecessary absolute paths if paths may reveal sensitive project or user information.
  • Document the logging behavior and expected confidentiality controls.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (2)

Ae4

Medium
Category
analysis-evasion
Confidence
80% confidence
Finding

Suspicious Unicode normalization or mixed-script content

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The OCR call hard-codes lang='chi_sim+eng', which imposes a specific language/locale behavior. Under the policy, language constraints should either be user-selectable or clearly justified as region-specific; neither is present here.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.