Back to skill

Security audit

KallyAI Executive Assistant

Security checks for vulnerabilities and agentic risk

Overview

KallyAI is a legitimate-looking executive assistant skill, but it delegates many sensitive real-world actions through a broad natural-language interface and an unpinned external CLI.

Review this carefully before installing. Use it only if you trust KallyAI with calls, messages, email, bookings, billing-related tasks, phone numbers, and stored OAuth access. Prefer explicit direct commands, confirm any action that spends money or contacts third parties, and revoke/logout if you stop using it. A pinned and verifiable CLI release plus explicit confirmation rules would materially reduce the risk.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:4
Finding
Unpinned and Unverifiable Third-Party CLI Dependency## Vulnerability Details **File Location**: `SKILL.md`, line 4 **Vulnerability Type**: `T08: Insecure Dependencies` **Risk Level**: Medium **Complete Code Snippet**: ```yaml metadata: {"clawdbot":{"emoji":"📞","requires":{"bins":["kallyai"]},"install":[{"id":"pip","kind":"pip","package":"kallyai-cli","bins":["kallyai"],"label":"Install via pip"}]}} ``` ### Technical Analysis The Skill directs the platform to install the `kallyai-cli` package from the pip package ecosystem without specifying an exact version or cryptographic hash. The project contains no dependency lockfile, checksum, vendored implementation, or other integrity constraint that would bind installation to the artifact reviewed during this audit. As a result, the effective executable can change independently of `SKILL.md`. The package's implementation cannot be audited from this project, despite the CLI being expected to process authentication tokens and perform sensitive operations involving calls, email, messages, contacts, calendars, bookings, and phone numbers. This is a supply-chain weakness rather than evidence that the current package is malicious. Exploitation requires compromise, replacement, or unsafe publication of a package release selected during installation. ### Attack Path 1. An attacker compromises the package publisher, package repository account, release pipeline, or another component capable of publishing a malicious `kallyai-cli` release. 2. The attacker publishes a modified release containing malicious installation-time or runtime behavior. 3. Because the dependency has no version or hash constraint, a subsequent Skill installation resolves the attacker-controlled release. 4. Package installation hooks or the installed `kallyai` executable run with the privileges of the installing user. 5. The malicious implementation can attempt to read locally accessible credentials and data or intercept sensitive information and delegated actions sup ...[truncated 727 chars]
Remediation
## Remediation Suggestions 1. Pin `kallyai-cli` to an explicitly reviewed, immutable version rather than resolving the latest available release. 2. Enforce cryptographic hashes through a hash-locked requirements file or equivalent package-integrity mechanism. 3. Publish and reference a verifiable source repository, release commit, and build provenance for the CLI. 4. Use signed releases and verify package signatures or trusted provenance attestations before installation. 5. Permit upgrades only after reviewing the new source and updating the approved version and hashes. 6. Run the CLI with least privilege and restrict its access to unrelated local files, environment variables, and credentials. 7. Grant narrowly scoped service authorization and provide revocation and credential-rotation procedures in case of dependency compromise.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Findings (6)

Vague Triggers

High
Confidence
96% confidence
Finding
The skill description is extremely broad and covers many unrelated, high-impact actions such as calls, email, bookings, budgeting, inbox access, and errands. This can cause the agent to invoke the skill for ambiguous user requests, leading to unintended external actions or disclosure of private data without sufficiently specific user intent.

Missing User Warnings

High
Confidence
97% confidence
Finding
The skill enables numerous privacy- and action-impacting capabilities—reading messages, sending email, making calls, handling bills, ordering services, and managing channels—yet the description does not prominently warn users about data access, third-party transmission, costs, or autonomous side effects. In this context, missing warnings materially increase the risk of uninformed consent and unsafe use.

Credential Access

High
Category
Privilege Escalation
Content
https://api.kallyai.com/v1/auth/authorize?response_type=code&client_id=YOUR_CLIENT_ID&redirect_uri=YOUR_REDIRECT_URI&scope=calls.write
```

User signs in with Google or Apple → receives access token.

For CLI, authentication is automatic (first API call opens browser).
Confidence
70% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Vague Triggers

Medium
Confidence
91% confidence
Finding
Telling users to use `ask` for 'most requests' delegates routing to a generic natural-language interface without clear safety boundaries. In a skill capable of emailing, calling, ordering, booking, and managing accounts, vague NL routing increases the chance of overbroad interpretation and accidental execution of impactful actions.

External Transmission

Medium
Category
Data Exfiltration
Content
For OAuth integration, redirect user to:
```
https://api.kallyai.com/v1/auth/authorize?response_type=code&client_id=YOUR_CLIENT_ID&redirect_uri=YOUR_REDIRECT_URI&scope=calls.write
```

User signs in with Google or Apple → receives access token.
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
For OAuth integration, redirect user to:
```
https://api.kallyai.com/v1/auth/authorize?response_type=code&client_id=YOUR_CLIENT_ID&redirect_uri=YOUR_REDIRECT_URI&scope=calls.write
```

User signs in with Google or Apple → receives access token.
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Static analysis

No suspicious patterns detected.