Vendor Performance Audit

Conduct quarterly vendor reviews using KPI scoring across delivery, quality, communication, cost, and alignment to guide renewal, improvement, or offboarding...

Security Scan

The skill is a coherent vendor-audit checklist, but its runtime instructions expect access to internal ticketing/log systems and evidence sources without declaring how credentials or data access will be provided — the scope is vague and could lead to broad data access if the agent is allowed to fetch records automatically.

Purpose & Capability

The name, description, and scoring framework align with a vendor-performance audit. However, the SKILL.md repeatedly requires the agent to "pull ticket data, delivery logs, or incident records" and to review incident logs — capabilities that normally require credentials or integrations (JIRA, ServiceNow, Zendesk, cloud logging, ticket DBs). The skill does not declare any required credentials or integrations, so there is a mild mismatch between expected data access and declared requirements.

Instruction Scope

Instructions are operational and actionable (scorecard, weighted calculation, incident severity modifiers, improvement-plan template). They explicitly direct the agent to obtain evidence from ticket systems and logs and to review incident histories. Those directives are useful for the audit purpose but are vague about which systems/sources to use and grant the agent broad discretion to access any available records — this is scope creep that could result in the agent reading sensitive internal files or services if permitted.

Install Mechanism

No install spec and no code files; the skill is instruction-only. This minimizes filesystem/remote-code risk — nothing is downloaded or executed by the skill package itself.

Credentials

The skill declares no required environment variables or credentials, yet its runtime instructions require access to potentially sensitive systems (ticketing/incident logs, delivery logs). The absence of declared credentials or integration requirements is disproportionate to the data the skill asks for and leaves unclear how the agent should be given access (and whether that access will be scoped/read-only).

Persistence & Privilege

The skill is not set to always: true, and model invocation is not disabled (normal). It does not request persistence or system-level configuration changes. There is no explicit privilege escalation or modification of other skills' configs.

What to consider before installing

This skill is essentially a human-facing audit template with clear scoring rules — useful — but it expects the agent to "pull ticket data, delivery logs, or incident records" without specifying how to connect to those systems. Before installing or enabling the skill: - Confirm which ticketing/log systems (JIRA, ServiceNow, Zendesk, PagerDuty, S3/cloud logs, internal DB) the agent will access and how credentials will be provided. - Prefer read-only, scoped credentials and explicit allowlists (only vendor X records for date range Y). - Ask the publisher for details on what data the agent will read/store and where any generated reports are persisted or transmitted. - Test on a non-production or sample vendor dataset first. - Ensure audit logging is enabled (who/when the agent accessed records). If the publisher cannot explain how data access will be scoped, treat the skill as higher risk — do not grant wide access to internal ticketing or logs.

Framework: Vendor Performance Scorecard (VPS) Output: Scored vendor review, improvement plan or offboarding recommendation

Most vendor relationships drift because nobody's measuring them. This quarterly audit system gives you a structured way to evaluate every significant vendor, surface problems before they escalate, and make data-driven decisions about renewing, renegotiating, or replacing.

---

When to Run This Audit

• Quarterly for all Priority vendors (ACV > $10K or operationally critical)
• Semi-annually for Standard vendors
• Triggered any time a major incident occurs (SLA breach, security issue, delivery failure)
• Pre-renewal (minimum 60 days before contract end)

---

Phase 1: KPI Scorecard

Rate each dimension 1-5. Be honest — this is for your decision-making, not the vendor's feelings.

Dimension 1: Delivery & SLA Performance (Weight: 30%)

Score	Criteria
5	Consistently exceeds SLA. Proactive communication on any hiccup. Zero surprise failures.
4	Meets SLA >95% of the time. Issues are rare and resolved quickly.
3	Meets SLA most of the time. Occasional misses with reasonable resolution.
2	Frequent SLA misses. Resolution is slow or requires escalation.
1	Regular delivery failures. SLA is aspirational, not operational.

Evidence required: Pull ticket data, delivery logs, or incident records. Don't score from memory.

Dimension 2: Quality of Output (Weight: 25%)

Score	Criteria
5	Output exceeds expectations. Error rate near zero. Rework is essentially unheard of.
4	Output meets quality bar consistently. Minor issues handled proactively.
3	Generally acceptable quality. Some rework required.
2	Quality is inconsistent. Rework is common. Internal team spends time fixing vendor output.
1	Output frequently doesn't meet standards. Significant internal overhead to compensate.

Dimension 3: Responsiveness & Communication (Weight: 20%)

Score	Criteria
5	Always reachable. Proactively surfaces issues. Communication is clear and timely.
4	Responsive within agreed SLA. Communicates proactively most of the time.
3	Generally responsive but reactive. Sometimes requires chasing.
2	Slow to respond. You often initiate all communication. Escalations required.
1	Unreliable contact. Incidents discovered by you, not surfaced by them.

Dimension 4: Value vs. Cost (Weight: 15%)

Score	Criteria
5	Clear ROI. Cost is at or below market for quality delivered. Strong value demonstrated.
4	Good value. Cost is reasonable given output and relationship quality.
3	Market rate. Neither a bargain nor obviously overpriced.
2	Starting to feel overpriced relative to value delivered or market alternatives.
1	Overpriced for what we get. Alternatives would deliver more for less.

Dimension 5: Strategic Alignment & Roadmap (Weight: 10%)

Score	Criteria
5	Deeply aligned. They understand our business and proactively help us get where we're going.
4	Good alignment. They know our goals and adjust accordingly.
3	Transactional but functional. Delivers what's scoped, no more.
2	Misaligned in places. Their direction and ours are diverging.
1	No alignment. Product/service is moving away from our needs.

---

Phase 2: Composite Score & Tier Classification

Weighted score calculation:

VPS = (D1 × 0.30) + (D2 × 0.25) + (D3 × 0.20) + (D4 × 0.15) + (D5 × 0.10)

Max score = 5.0

VPS Range	Tier	Recommended Action
4.0 – 5.0	🟢 Green — Trusted Partner	Renew, consider expanding scope or strategic partnership
3.0 – 3.9	🟡 Yellow — Watch	Renew with conditions; issue improvement plan for lowest-scoring dimension
2.0 – 2.9	🟠 Orange — At Risk	Renegotiate terms or begin sourcing alternatives; 60-day improvement window
1.0 – 1.9	🔴 Red — Replace	Begin active replacement process; do not renew

---

Phase 3: Issue Log Review

Before finalizing the score, review your incident/ticket log for this vendor over the review period:

• How many incidents were opened? How many are still open?
• What was the average resolution time? Compare to SLA.
• Were any incidents flagged as critical/high-impact?
• Did any incidents result in downstream business impact (revenue loss, client complaints, compliance exposure)?

Incident severity modifier:

• 1+ critical incident with unresolved root cause → drop tier by one level
• 3+ medium incidents unresolved → flag for improvement plan regardless of VPS score

---

Phase 4: Improvement Plan Template (Yellow & Orange Tiers)

If VPS < 4.0, issue a formal improvement plan:

Improvement Plan — [Vendor Name] — [Quarter]

• Review Period: [start] – [end]
• VPS Score: [X.X] / 5.0
• Tier: Yellow / Orange
• Review Date: [90 days from today]

Key Issues Identified:

1. [Specific issue with evidence]
2. [Specific issue with evidence]

Required Improvements:

1. [Specific, measurable change required] — Target: [metric] by [date]
2. [Specific, measurable change required] — Target: [metric] by [date]

Consequences if not met:

• Yellow: Move to Orange tier; begin parallel sourcing
• Orange: Contract not renewed; active replacement begins

Acknowledgment: Share this plan with the vendor. Get written acknowledgment.

---

Phase 5: Offboarding Trigger Criteria

Initiate replacement when ANY of the following are true:

• VPS score < 2.0
• Two consecutive quarters in Orange tier
• Critical incident with material business impact and no credible root cause fix
• Vendor signals they are discontinuing the product/service
• Market alternative offers >30% better value at equivalent quality
• Compliance or security failure

When trigger is met: immediately move to replacement sourcing and set a hard cutover date.

---

Audit Schedule Template

Vendor	Category	ACV	Tier	Last Audit	Next Audit	Owner
[Name]	Software	$X	Green	[date]	[date]	[name]

Run this as a quarterly review in your ops calendar.