Performance Monitoring

监控Agent调度、执行及内存性能指标，自动记录日志，分析趋势，识别瓶颈并提供优化建议。

Security Scan

The skill's stated monitoring purpose aligns with its instructions, but it references sending metrics/alerts to Feishu and reading agent-internal data (memories, logs) without declaring required credentials or detailing data access — that mismatch raises concern and needs clarification before install.

Purpose & Capability

The name/description (performance monitoring of Agent scheduling, execution, memory) matches the metric calculations and alerting behavior described in SKILL.md and config.json. However, the skill declares no required env vars or credentials while config.json lists alert_channel: "feishu" and SKILL.md mentions recording to 飞书表格 (Feishu). Sending data to Feishu normally requires webhooks/tokens — those are not declared, which is disproportionate to the stated manifest (missing auth requirements).

Instruction Scope

SKILL.md instructs collecting internal agent metrics and 'memories' and recording logs and alerts (including to Feishu). The document implies reading agent-internal data structures (dispatch logs, task logs, memory lists). The instructions (as provided) do not specify what exact data fields will be collected, any filtering/sanitization, or where logs are stored. They also do not explain how Feishu integration is authenticated or whether data is transformed before external transmission. This open-endedness grants the skill broad discretion to read and transmit potentially sensitive agent data.

Install Mechanism

Instruction-only skill with no install spec and no code files — lowest file-based risk. Nothing will be downloaded or written to disk by an installer as part of this package.

Credentials

Requires no environment variables according to the registry, but config and SKILL.md indicate external alerting to Feishu and access to internal memories/logs. External posting to Feishu would normally require a webhook URL or API token (sensitive). The absence of declared credentials or guidance on required secrets is a proportionality mismatch. Also, collecting 'memories' and detailed logs can expose sensitive user data — the skill does not specify scoping or redaction rules.

Persistence & Privilege

always:false (normal) and model invocation not disabled (normal). Autonomous invocation is allowed by default; combined with the above concerns (external posting and broad data access), autonomous runs could exfiltrate data if credentials are provided later by the operator. The skill does not request persistent installation privileges or modify other skills' configs.

What to consider before installing

Before installing, confirm these points with the skill author or maintainer: (1) How will Feishu alerts be authenticated? Demand explicit environment variables (webhook URL or API token) and review them before providing. (2) Exactly which agent logs and 'memories' will be collected, stored, and transmitted? Ask for a data schema, retention policy, and any redaction/filters for sensitive fields. (3) Where are logs stored and who can access them? Verify retention_days and make sure it meets your privacy policy. (4) Because the skill's source and homepage are unknown, prefer running it in a sandboxed or test agent first and require explicit approval before enabling autonomous invocation in production. (5) If you must use it, provide minimal-scoped credentials (dedicated Feishu webhook limited to a test space) and monitor network activity and outputs for unexpected data exfiltration.

Performance Monitoring Skill

版本: v1.0 创建日期: 2026-03-26 作者: 象腿 (main agent) 用途: 监控Agent性能，收集关键指标，优化系统表现

---

🎯 核心功能

Performance Monitoring skill负责：

1. 指标收集: 收集调度、执行、性能等关键指标
2. 数据记录: 将指标记录到日志和飞书表格
3. 趋势分析: 分析性能趋势，识别瓶颈
4. 告警机制: 性能异常时自动告警
5. 优化建议: 基于数据提供优化建议

---

📋 监控指标体系

1. Agent调度指标

1.1 调度成功率 (Dispatch Success Rate)

定义: 成功调度的任务数 / 总调度任务数

目标: >95%

计算:

def calculate_dispatch_success_rate(dispatch_logs):
    """
    计算调度成功率

    Args:
        dispatch_logs: 调度日志列表

    Returns:
        float: 成功率 (0.0-1.0)
    """
    total = len(dispatch_logs)
    if total == 0:
        return 1.0

    success = sum(1 for log in dispatch_logs if log['success'])
    return success / total

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "dispatch_success_rate",
  "value": 0.95,
  "total": 100,
  "success": 95,
  "failed": 5
}

---

1.2 平均调度耗时 (Avg Dispatch Time)

定义: 从收到请求到agent开始执行的平均时间

目标: <5秒

计算:

def calculate_avg_dispatch_time(dispatch_logs):
    """
    计算平均调度耗时

    Args:
        dispatch_logs: 调度日志列表

    Returns:
        float: 平均耗时（秒）
    """
    times = [log['dispatch_time'] for log in dispatch_logs if 'dispatch_time' in log]

    if not times:
        return 0.0

    return sum(times) / len(times)

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "avg_dispatch_time",
  "value": 3.45,
  "unit": "seconds",
  "min": 1.2,
  "max": 8.5,
  "p50": 3.2,
  "p95": 6.8
}

---

1.3 重试率 (Retry Rate)

定义: 需要重试的任务数 / 总调度任务数

目标: <10%

计算:

def calculate_retry_rate(dispatch_logs):
    """
    计算重试率

    Args:
        dispatch_logs: 调度日志列表

    Returns:
        float: 重试率 (0.0-1.0)
    """
    total = len(dispatch_logs)
    if total == 0:
        return 0.0

    retried = sum(1 for log in dispatch_logs if log.get('retry_count', 0) > 0)
    return retried / total

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "retry_rate",
  "value": 0.08,
  "total": 100,
  "retried": 8
}

---

1.4 Fallback率 (Fallback Rate)

定义: Fallback到main agent的任务数 / 总调度任务数

目标: <5%

计算:

def calculate_fallback_rate(dispatch_logs):
    """
    计算fallback率

    Args:
        dispatch_logs: 调度日志列表

    Returns:
        float: fallback率 (0.0-1.0)
    """
    total = len(dispatch_logs)
    if total == 0:
        return 0.0

    fallback = sum(1 for log in dispatch_logs if log.get('fallback', False))
    return fallback / total

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "fallback_rate",
  "value": 0.03,
  "total": 100,
  "fallback": 3
}

---

2. 并行处理指标

2.1 并行加速比 (Parallel Speedup)

定义: 串行总时间 / 并行总时间

目标: >1.3x

计算:

def calculate_parallel_speedup(serial_time, parallel_time):
    """
    计算并行加速比

    Args:
        serial_time: 串行执行时间（秒）
        parallel_time: 并行执行时间（秒）

    Returns:
        float: 加速比
    """
    if parallel_time == 0:
        return 1.0

    return serial_time / parallel_time

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "parallel_speedup",
  "value": 1.5,
  "serial_time": 30.0,
  "parallel_time": 20.0
}

---

2.2 任务完成率 (Task Completion Rate)

定义: 成功完成的任务数 / 总任务数

目标: >95%

计算:

def calculate_task_completion_rate(task_logs):
    """
    计算任务完成率

    Args:
        task_logs: 任务日志列表

    Returns:
        float: 完成率 (0.0-1.0)
    """
    total = len(task_logs)
    if total == 0:
        return 1.0

    completed = sum(1 for log in task_logs if log['success'])
    return completed / total

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "task_completion_rate",
  "value": 0.97,
  "total": 100,
  "completed": 97,
  "failed": 3
}

---

3. Memory指标

3.1 Memory数量 (Memory Count)

定义: 当前memory总数量

目标: <=200

计算:

def calculate_memory_count(memories):
    """
    计算memory数量

    Args:
        memories: 记忆列表

    Returns:
        int: memory数量
    """
    return len(memories)

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "memory_count",
  "value": 180,
  "window": 200,
  "utilization": 0.9
}

---

3.2 平均相关度 (Avg Relevance Score)

定义: 所有memory的平均相关度分数

目标: >0.7

计算:

def calculate_avg_relevance_score(memories):
    """
    计算平均相关度分数

    Args:
        memories: 记忆列表

    Returns:
        float: 平均相关度分数
    """
    if not memories:
        return 0.0

    scores = [m.get('relevance_score', 0.5) for m in memories]
    return sum(scores) / len(scores)

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "avg_relevance_score",
  "value": 0.75,
  "min": 0.3,
  "max": 0.98,
  "distribution": {
    "high": 120,
    "medium": 50,
    "low": 10
  }
}

---

3.3 检索准确率 (Retrieval Accuracy)

定义: 相关结果数 / 总返回结果数

目标: >80%

计算:

def calculate_retrieval_accuracy(retrieval_logs):
    """
    计算检索准确率

    Args:
        retrieval_logs: 检索日志列表

    Returns:
        float: 准确率 (0.0-1.0)
    """
    total_results = sum(log['total_results'] for log in retrieval_logs)
    if total_results == 0:
        return 1.0

    relevant_results = sum(log['relevant_results'] for log in retrieval_logs)
    return relevant_results / total_results

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "retrieval_accuracy",
  "value": 0.85,
  "total_results": 100,
  "relevant_results": 85
}

---

3.4 Token效率 (Token Efficiency)

定义: 保留价值 / 总token数

目标: >95%

计算:

def calculate_token_efficiency(original_tokens, current_tokens):
    """
    计算Token效率

    Args:
        original_tokens: 原始token数
        current_tokens: 当前token数

    Returns:
        float: 效率 (0.0-1.0)
    """
    if original_tokens == 0:
        return 1.0

    return 1.0 - (current_tokens / original_tokens)

记录格式:

{
  "timestamp": "2026-03-26T12:00:00+08:00",
  "metric": "token_efficiency",
  "value": 0.96,
  "original_tokens": 100000,
  "current_tokens": 4000,
  "saved_tokens": 96000
}

---

🛠️ PowerShell实现

指标收集器

# 指标收集器
function Collect-PerformanceMetrics {
    param(
        [string]$LogDir = "C:\Users\Administrator\.openclaw\workspace-main\logs",
        [string]$MetricsFile = "performance-metrics.json"
    )

    $metrics = @{}

    # 1. 收集调度指标
    $dispatchLogs = Get-DispatchLogs -LogDir $LogDir
    $metrics['dispatch_success_rate'] = Calculate-DispatchSuccessRate -Logs $dispatchLogs
    $metrics['avg_dispatch_time'] = Calculate-AvgDispatchTime -Logs $dispatchLogs
    $metrics['retry_rate'] = Calculate-RetryRate -Logs $dispatchLogs
    $metrics['fallback_rate'] = Calculate-FallbackRate -Logs $dispatchLogs

    # 2. 收集并行指标
    $parallelLogs = Get-ParallelLogs -LogDir $LogDir
    $metrics['parallel_speedup'] = Calculate-ParallelSpeedup -Logs $parallelLogs
    $metrics['task_completion_rate'] = Calculate-TaskCompletionRate -Logs $parallelLogs

    # 3. 收集memory指标
    $memories = Load-Memories
    $metrics['memory_count'] = Calculate-MemoryCount -Memories $memories
    $metrics['avg_relevance_score'] = Calculate-AvgRelevanceScore -Memories $memories
    $metrics['retrieval_accuracy'] = Calculate-RetrievalAccuracy -Logs (Get-RetrievalLogs -LogDir $LogDir)
    $metrics['token_efficiency'] = Calculate-TokenEfficiency -Memories $memories

    # 4. 添加时间戳
    $metrics['timestamp'] = (Get-Date).ToString("yyyy-MM-ddTHH:mm:ssK")

    # 5. 保存到文件
    $metricsJson = $metrics | ConvertTo-Json -Depth 10
    $metricsPath = Join-Path $LogDir $MetricsFile
    $metricsJson | Out-File -FilePath $metricsPath -Encoding UTF8

    return $metrics
}

# 获取调度日志
function Get-DispatchLogs {
    param([string]$LogDir)

    $logFile = Join-Path $LogDir "dispatch-*.log"
    if (Test-Path $logFile) {
        $logs = Get-Content $logFile | ConvertFrom-Json
        return $logs
    }

    return @()
}

# 计算调度成功率
function Calculate-DispatchSuccessRate {
    param([array]$Logs)

    $total = $Logs.Count
    if ($total -eq 0) {
        return @{value = 1.0; total = 0; success = 0; failed = 0}
    }

    $success = ($Logs | Where-Object { $_.success -eq $true }).Count

    return @{
        value = [math]::Round($success / $total, 2)
        total = $total
        success = $success
        failed = $total - $success
    }
}

# 计算平均调度耗时
function Calculate-AvgDispatchTime {
    param([array]$Logs)

    $times = $Logs | Where-Object { $_.dispatch_time } | ForEach-Object { $_.dispatch_time }

    if ($times.Count -eq 0) {
        return @{value = 0; unit = "seconds"}
    }

    $avg = ($times | Measure-Object -Average).Average

    return @{
        value = [math]::Round($avg, 2)
        unit = "seconds"
        min = [math]::Round(($times | Measure-Object -Minimum).Minimum, 2)
        max = [math]::Round(($times | Measure-Object -Maximum).Maximum, 2)
    }
}

# 计算重试率
function Calculate-RetryRate {
    param([array]$Logs)

    $total = $Logs.Count
    if ($total -eq 0) {
        return @{value = 0; total = 0; retried = 0}
    }

    $retried = ($Logs | Where-Object { $_.retry_count -gt 0 }).Count

    return @{
        value = [math]::Round($retried / $total, 2)
        total = $total
        retried = $retried
    }
}

# 计算fallback率
function Calculate-FallbackRate {
    param([array]$Logs)

    $total = $Logs.Count
    if ($total -eq 0) {
        return @{value = 0; total = 0; fallback = 0}
    }

    $fallback = ($Logs | Where-Object { $_.fallback -eq $true }).Count

    return @{
        value = [math]::Round($fallback / $total, 2)
        total = $total
        fallback = $fallback
    }
}

---

📊 数据记录

日志格式

所有指标记录到统一格式的日志文件：

logs/performance-metrics-YYYY-MM-DD.json

格式:

{
  "date": "2026-03-26",
  "metrics": {
    "dispatch_success_rate": 0.95,
    "avg_dispatch_time": 3.45,
    "retry_rate": 0.08,
    "fallback_rate": 0.03,
    "parallel_speedup": 1.5,
    "task_completion_rate": 0.97,
    "memory_count": 180,
    "avg_relevance_score": 0.75,
    "retrieval_accuracy": 0.85,
    "token_efficiency": 0.96
  },
  "alerts": [
    {
      "metric": "dispatch_success_rate",
      "threshold": 0.95,
      "actual": 0.92,
      "severity": "warning"
    }
  ]
}

---

🚨 告警机制

告警规则

def check_alerts(metrics):
    """
    检查指标是否触发告警

    Args:
        metrics: 指标字典

    Returns:
        list: 告警列表
    """
    alerts = []

    # 1. 调度成功率告警
    if metrics['dispatch_success_rate']['value'] < 0.95:
        alerts.append({
            'metric': 'dispatch_success_rate',
            'threshold': 0.95,
            'actual': metrics['dispatch_success_rate']['value'],
            'severity': 'warning' if metrics['dispatch_success_rate']['value'] > 0.9 else 'critical'
        })

    # 2. 平均调度耗时告警
    if metrics['avg_dispatch_time']['value'] > 5.0:
        alerts.append({
            'metric': 'avg_dispatch_time',
            'threshold': 5.0,
            'actual': metrics['avg_dispatch_time']['value'],
            'severity': 'warning' if metrics['avg_dispatch_time']['value'] < 10.0 else 'critical'
        })

    # 3. 重试率告警
    if metrics['retry_rate']['value'] > 0.10:
        alerts.append({
            'metric': 'retry_rate',
            'threshold': 0.10,
            'actual': metrics['retry_rate']['value'],
            'severity': 'warning'
        })

    # 4. Memory数量告警
    if metrics['memory_count']['value'] > 200:
        alerts.append({
            'metric': 'memory_count',
            'threshold': 200,
            'actual': metrics['memory_count']['value'],
            'severity': 'warning'
        })

    return alerts

---

📈 趋势分析

7天趋势

def analyze_7day_trend():
    """
    分析过去7天的性能趋势

    Returns:
        dict: 趋势分析结果
    """
    # 获取过去7天的指标
    metrics_history = load_metrics_history(days=7)

    trend = {}
    for metric_name in ['dispatch_success_rate', 'avg_dispatch_time', 'retry_rate']:
        values = [m[metric_name] for m in metrics_history]

        # 计算趋势
        if len(values) >= 2:
            if values[-1] > values[-2]:
                direction = 'up'
            elif values[-1] < values[-2]:
                direction = 'down'
            else:
                direction = 'stable'

            # 计算变化率
            change_rate = (values[-1] - values[0]) / values[0] if values[0] != 0 else 0

            trend[metric_name] = {
                'direction': direction,
                'change_rate': change_rate,
                'current': values[-1],
                'previous': values[-2]
            }

    return trend

---

⚙️ 配置文件

performance-monitoring-config.json

{
  "version": "1.0",
  "config": {
    "collection_interval": 3600,
    "retention_days": 30,
    "enable_alerts": true,
    "alert_channel": "feishu"
  },
  "thresholds": {
    "dispatch_success_rate": {
      "warning": 0.90,
      "critical": 0.85
    },
    "avg_dispatch_time": {
      "warning": 5.0,
      "critical": 10.0
    },
    "retry_rate": {
      "warning": 0.10,
      "critical": 0.20
    },
    "fallback_rate": {
      "warning": 0.05,
      "critical": 0.10
    },
    "memory_count": {
      "warning": 180,
      "critical": 200
    }
  },
  "targets": {
    "dispatch_success_rate": 0.95,
    "avg_dispatch_time": 5.0,
    "retry_rate": 0.10,
    "fallback_rate": 0.05,
    "parallel_speedup": 1.3,
    "task_completion_rate": 0.95,
    "memory_count": 200,
    "avg_relevance_score": 0.7,
    "retrieval_accuracy": 0.8,
    "token_efficiency": 0.95
  }
}

---

🚀 未来优化

短期 (1-2周)

• 实现自动化性能报告
• 添加性能基线对比
• 实现异常检测算法

中期 (1个月)

• 实现性能预测模型
• 添加根因分析
• 实现自动优化建议

长期 (3个月)

• 引入AI驱动的性能优化
• 实现自适应阈值调整
• 构建性能知识库

---

Skill版本: v1.0 最后更新: 2026-03-26 维护者: 象腿 (main agent)