Scrask

When the user sends a screenshot via Telegram, parse it using Gemini (fast, default) with automatic Claude fallback when confidence is low. Saves results to Google Calendar (events) or Google Tasks (reminders and tasks). Saves silently if confidence is high; asks for confirmation if uncertain.

Audits

Pass

ClawScanReview

Agentic behavior and permission review.

Static analysisPass

Pattern checks against bundled files.

VirusTotalPass

Multi-engine malware detections and file reputation.

Install

openclaw skills install scrask-bot

Scrask Bot

Overview

This skill activates when the user sends a screenshot via Telegram. It uses vision AI to extract actionable information from the image, then:

High confidence (≥ 0.75): Saves immediately and replies with a brief confirmation.
Low confidence (< 0.75): Shows a structured preview in Telegram and asks for confirmation before saving.

Provider behaviour (auto mode, default):

Step	What happens
1	Gemini 2.0 Flash parses the screenshot (fast, cheap)
2	If any item confidence < 0.60, Claude Opus reruns the parse
3	Whichever provider scores higher average confidence wins
4	Output includes `provider`, `fallback_triggered`, and confidence delta

Set vision_provider to "claude" or "gemini" to lock a specific provider.

Output destinations (AI-decided by content type):

Detected type	Destination
Event (has date+time, venue, or invite link)	Google Calendar
Reminder (deadline, due date, personal action)	Google Tasks (with due date)
Task (no date, pure action item)	Google Tasks (no due date)

Trigger Conditions

Activate when:

The user sends a message in Telegram that contains an image attachment
The image appears to be a screenshot — not a photo of a person, place, or physical object
No other skill has already claimed the image

Do not activate for:

Photos of people, places, food, scenery
Screenshots of code, errors, or UI bugs (leave for other skills)
Images the user explicitly asks to edit, describe, or analyze for another purpose

Step-by-Step Instructions

Step 1: Acknowledge Immediately

Reply in Telegram right away so the user knows the skill is working:

"📸 Got it — analyzing your screenshot..."

Do not make the user wait silently.

Step 2: Run the Parser

python3 ~/.openclaw/skills/scrask-bot/scripts/scrask_bot.py \
  --image-path "<path-to-temp-image>" \
  --provider "$CONFIG_VISION_PROVIDER" \
  --timezone "$CONFIG_TIMEZONE" \
  --google-credentials "$GOOGLE_CREDENTIALS"

The script auto-resolves the API key from ANTHROPIC_API_KEY or GEMINI_API_KEY
depending on the provider — no need to pass it explicitly.

The script returns a JSON object with:

success — whether parsing worked
no_actionable_content — true if nothing found
results[] — one entry per detected item, each with confidence, type, destination, needs_confirmation, action_taken
telegram_reply — the pre-formatted message to send back to the user

Step 3: Handle the Output

If no_actionable_content is true: Reply: "🤷 I couldn't find any event, reminder, or task info in that screenshot. Could you describe what you'd like to add?"

If success is true: Send the telegram_reply value directly back to the user in Telegram. The script has already:

Saved high-confidence items silently
Formatted confirmation prompts for low-confidence items

Do not rephrase or reformat the telegram_reply — send it as-is.

Step 4: Handle Confirmation Responses

If the script returned items with needs_confirmation: true, wait for the user's reply.

"yes" or "save" or "add": Re-run the script for that specific item with confirmed=true, or use the calendar_create / tasks_create tools directly with the extracted fields.

"edit": Ask what to change, update the relevant field, then save.

"skip" or "no": Reply: "Got it, skipped ✓"

Step 5: Confirm Saves

For items saved silently (high confidence), the telegram_reply from the script already contains the confirmation message. Examples of what the user will see:

📅 Added to Calendar: **Team Standup** — 2026-03-01 at 09:00
🔔 Added to Tasks: **Pay electricity bill** (due 2026-02-28)
✅ Added to Tasks: **Review PR for Arjun**

Edge Cases

Scenario	Behavior
Screenshot is in Hindi, Tamil, or another language	Extract and translate silently; save title in English
Recurring event ("every Monday")	Set RRULE on the calendar event; mention it in the reply
Date has already passed	Flag in the reply: "⚠️ This date has already passed (Feb 10). Save anyway?"
Multiple items in one screenshot	Process each independently; confirm per item if needed
Screenshot of someone's calendar	Detect `already_in_calendar_hint`; reply: "Looks like this event is already in your calendar 🗓️"
Google API auth failure	Reply with the specific error and suggest re-checking GOOGLE_CREDENTIALS
Zoom/Meet link found	Add to Calendar as both location and description

Configuration

{
  "skills": {
    "entries": {
      "scrask-bot": {
        "enabled": true,
        "env": {
          "GEMINI_API_KEY": "AIza-your-gemini-key",
          "ANTHROPIC_API_KEY": "sk-ant-your-key-here",
          "GOOGLE_CREDENTIALS": "/home/user/.openclaw/google-creds.json"
        },
        "config": {
          "vision_provider": "auto",
          "fallback_threshold": 0.60,
          "timezone": "Asia/Kolkata",
          "confidence_threshold": 0.75,
          "reminder_minutes_before": 30
        }
      }
    }
  }
}

Permissions Required

image:read — to access the screenshot from Telegram
network:outbound — to call Anthropic API and Google APIs
telegram:reply — to send confirmation messages back to the user