Parameter Golf Monitor

Monitor the openai/parameter-golf competition leaderboard by fetching PR data from GitHub. Use this skill whenever the user asks about the parameter-golf com...

Security Scan

The skill's code, instructions, and requirements are consistent with its stated purpose of reading public GitHub PRs to build a leaderboard; it requests no credentials or unusual system access.

Purpose & Capability

The name/description claim to monitor openai/parameter-golf PRs and the included script directly implements that by calling GitHub's public API and extracting val_bpb scores. There are no unrelated environment variables, binaries, or config paths requested.

Instruction Scope

SKILL.md instructs the agent/user to run the bundled Python script and describes flags (--me, --watch, --json, etc.). The runtime instructions and the script only access GitHub's public API and local CLI arguments; they do not read local secrets, other files, or send data to unexpected endpoints.

Install Mechanism

This is an instruction-only skill with a bundled script and no install spec. No external downloads, package installs, or extract operations are used. The script relies only on Python stdlib (urllib), matching README and SKILL.md.

Credentials

The skill declares no required environment variables or credentials and the script uses unauthenticated GitHub API calls (60 req/hour). Nothing in the code attempts to read environment variables or credentials.

Persistence & Privilege

The skill is not always-enabled, does not request persistent platform privileges, and does not modify other skills or system-wide settings. It merely fetches public data when invoked and (optionally) polls while running.

Assessment

This skill appears coherent and safe for its stated purpose. It makes unauthenticated requests to GitHub's public API (subject to a 60 requests/hour limit), so avoid using very frequent poll intervals unless you understand rate limits. The script does network I/O (to api.github.com) and will print or emit JSON of PR data — if you need higher rate limits or private repo access you would need to provide a GitHub token, which this skill currently does not request. As with any third-party code, review the bundled script before running in your environment if you have stricter policies (e.g., offline systems or environments that restrict outbound network access).

A skill for tracking the openai/parameter-golf competition leaderboard in real time by fetching open and merged PRs from GitHub's public API.

What this skill does

The monitor script at scripts/monitor.py queries the GitHub API for PRs on openai/parameter-golf, extracts val_bpb scores from titles and bodies, classifies submissions (record vs non-record), and displays a ranked leaderboard table.

It requires zero authentication — it uses the unauthenticated GitHub API (60 requests/hour rate limit).

When to use this skill

• User asks "what's the current leaderboard?" or "who's winning parameter golf?"
• User wants to check their ranking: "where do I stand?" or "how's my PR doing?"
• User asks about competitor scores or techniques
• User wants to monitor for new submissions or changes
• User says "check the competition" or "show me the standings"
• User wants to compare their score against the field

How to run

The script is at scripts/monitor.py relative to this skill's directory. Run it with Python 3 (no dependencies beyond stdlib).

One-shot leaderboard

python3 <skill-dir>/scripts/monitor.py

Shows all open PRs ranked by val_bpb (lower is better).

Highlight a specific user

python3 <skill-dir>/scripts/monitor.py --me <github-username>

Marks the user's PR(s) with <-- and shows their rank + gap to #1 in the summary. Ask the user for their GitHub username if they want highlighting and you don't already know it.

Common flag combinations

Goal	Command
Full leaderboard	python3 scripts/monitor.py
Top 10 records only	python3 scripts/monitor.py --top 10 --records-only
Highlight a user	python3 scripts/monitor.py --me USERNAME
Merged records (SOTA history)	python3 scripts/monitor.py --merged
Everything (open + merged)	python3 scripts/monitor.py --all
Today's PRs only	python3 scripts/monitor.py --since YYYY-MM-DD
JSON for piping	python3 scripts/monitor.py --json
Live polling every 5 min	python3 scripts/monitor.py --watch 5

All flags

• --me USER — Highlight a GitHub username. Shows rank and gap to #1.
• --top N — Show only the top N scored entries (highlighted user always shown even if outside top N).
• --records-only — Exclude non-record submissions.
• --merged — Show only merged PRs.
• --all — Show both open and merged PRs.
• --since YYYY-MM-DD — Filter to PRs created on or after this date.
• --json — Output as JSON instead of a table (useful for piping to jq or other tools).
• --watch MIN — Poll every N minutes continuously (Ctrl+C to stop).

Interpreting results

• Rank is by val_bpb ascending (lower = better). Only scored PRs get a numeric rank.
• Status shows open, merged, or non-rec (non-record submission).
• Scores marked ? mean no val_bpb could be extracted from the PR title or body.
• Some very low scores (e.g. < 1.10) may be evaluation-only or use non-standard setups — check the PR for details.
• The gap to #1 in the summary tells the highlighted user how far they are from the top.

Rate limits

The GitHub API allows 60 unauthenticated requests per hour. Each run_once call makes 1 request (open PRs) or 2 requests (if --merged or --all). The --watch mode respects this — polling every 5 minutes uses ~12-24 requests/hour.

If you hit the rate limit, the script will show an error and retry on the next interval.

Example output

[2026-03-20 02:17:15] Parameter Golf Leaderboard (open PRs)

Rank   val_bpb     PR  Status    Author              Date        Title
----------------------------------------------------------------------------------------------------
   1   1.15390  #135   open      unnir               2026-03-19  Record: OrthoInit + Int6 MLP3x + BigramHash + SmearGate
   2   1.15800  #106   open      krammnic            2026-03-19  record: 1.158
   3   1.15850  #122   open      mtybadger           2026-03-19  Record: Sliding Window Eval, 2048 Vocab Size
   4   1.16019  #156   open      dexhunter           2026-03-20  Int6 STE + NorMuon + SWA + Sliding Window  <--

Best: val_bpb=1.15390 (#135 by unnir)
Total: 56 PRs, 40 with scores

You (dexhunter): rank #4, val_bpb=1.16019, gap to #1: +0.00629