agentscope-model-compare

Compare multiple LLM models for use with AgentScope 2.0. Use this whenever the user wants to evaluate models, benchmark responses, or choose the best model for their agent task.

This skill runs the same AgentScope task against multiple LLM models and compares latency, token usage, and output style.

What this skill does

• Configures multiple AgentScope chat models
• Runs a shared agent task against each model
• Reports latency, token counts, and output length
• Highlights which models completed vs failed

Usage

Run the comparison script from this skill folder:

bash

python3 scripts/compare_models.py --task "Translate this into Chinese: Hello world"

The script uses environment variables for API keys and prints a comparison table.

Safety

• API keys are read from .env, which is gitignored.
• No keys are hardcoded.
• Costs may apply; limit concurrent calls if needed.