Localized reading guide
m365-agent-evaluatorPurpose, usage and source
Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".
Source retrieved:
- Source repository
- microsoft/skills
- SKILL.md
.github/plugins/microsoft-365-agents-toolkit/skills/m365-agent-evaluator/SKILL.md- Repository stars
- 3,157
- Repository updated
- 2026-10-31
- Categories
- Cloud
- Last collected
License: MIT
Workflow excerpts are reading aids, not executable instructions from Codez. No third-party Skill was run to produce this page.
The purpose and workflow passages below are upstream material, organized and translated for reading. They are distinct from the separately labeled Codez editorial analysis.
What this Skill does
Use this skill when a user wants to create, run, or analyze evaluation suites for Microsoft 365 Copilot declarative agents with the public @microsoft/m365-copilot-eval CLI. Trigger on intents such as "evaluate my agent", "test my agent", "run my evals", "create eval prompts", "add multi-turn tests", "tune evaluator thresholds", "why is my agent failing", or "set up eval environment variables".
How the upstream workflow works
Activation workflow
Identify the user goal: setup, dataset authoring, running evals, analyzing results, or updating an existing eval suite. Load only the reference needed for the current goal: references/workflow.md for the end-to-end operator workflow and CLI commands. references/azure-setup.md for prerequisites, env files, and secret handling.
M365 Agent Evaluator
Use this skill to help users evaluate Microsoft 365 Copilot declarative agents with @microsoft/m365-copilot-eval. The skill designs schema-compatible eval datasets, runs the public preview CLI, analyzes results, and recommends targeted fixes. Default to Microsoft 365 Agents Toolkit (ATK) projects when detected, but do not hard-stop solely because the current directory is not ATK. The CLI can also evaluate deployed agents with an explicit M365_AGENT_ID or --m365-agent-id.
npx -y --package @microsoft/m365-copilot-eval@latest runevalsRequirements and setup
No short requirements passage was selected from this file. This does not mean the Skill has no dependencies. Check the repository setup instructions and referenced files.
Limits and quality checks
Version and PATH safety
Before diagnosing agent behavior, confirm which executable is running: If bare runevals prints This version of the M365 Evals CLI has stopped working and must be updated, treat it as a stale PATH/global install. Re-run with the npx --package ...@latest command above, then ask before removing global shims with npm uninstall -g @microsoft/m365-copilot-eval.
Always use this CLI invocation
Do not recommend the old private aka.ms installer, global installs, bare runevals, bare npx runevals, --input, or --html.
Start with a safe trial
- Check the supported host and the repository installation instructions. Keep the Skill directory and its referenced scripts, assets and documentation together; SKILL.md alone may omit dependencies.
- Give the agent your real input, desired output and acceptance criteria. Invocation syntax and available tools depend on the host; this site does not assume a universal install command.
- Try the source workflow in a disposable project or on a copy. Review operations that change files, remote services or billable resources before allowing them.
- Validate the output against the source quality checks and your own requirements. A successful trigger or a high repository Star count does not establish a correct result.
Read further in the source
Related Skills
Translation and scope
This reading guide selects passages from the pinned upstream SKILL.md. Translation is assisted. It is not a complete manual or a claim of tested compatibility. Read the original and its referenced files before use.
This introduction refers to the pinned version below; current repository activity and collection time are shown separately.
- Pinned source
- Read the original ↗
- Source retrieved
- Source commit
00d66a64919edca5bca2c673373444e36a4b5461- Source SHA-256
5a1da9a62088eb7518937e1cba1d66e5a644083d82a22aa51f242f5915ca21ee