Skip to content

Prompt Files (Slash Commands)

Prompt files are reusable prompts that users invoke as slash commands (e.g. /review, /explain). VS Code uses .prompt.md files in .github/prompts/; Claude Code uses .md files in .claude/commands/. Testing them means verifying the LLM behaves correctly when the slash command is invoked.

Loading a Prompt File

Use load_prompt_file() to load the body of a single prompt file:

from pytest_skill_engineering import load_prompt_file
from pytest_skill_engineering.copilot import CopilotEval

agent = CopilotEval(name="prompt-test")

async def test_review_command(copilot_eval):
    """The /review slash command produces actionable feedback."""
    prompt = load_prompt_file(".github/prompts/review.prompt.md")
    result = await copilot_eval(
        agent,
        prompt["body"],
        prompt_name="review",  # tracked in the report
    )
    assert result.success
    assert result.prompt_name == "review"

load_prompt_file() returns a dict with:

Key Description
name Derived from filename (e.g. review)
body The markdown body — what gets sent to the LLM
description From frontmatter description: field, or None
metadata All other frontmatter fields

Prompt File Format

.github/prompts/review.prompt.md
---
description: Review code for quality and security issues
mode: agent
---

Review the current file for:
- Security vulnerabilities
- Performance issues
- Code style problems

Provide specific line numbers and suggested fixes.

The body is the content below the frontmatter separator.

Testing All Prompt Files

Use load_prompt_files() to load every prompt file in a directory and parametrize over them:

import pytest
from pytest_skill_engineering import load_prompt_files
from pytest_skill_engineering.copilot import CopilotEval

PROMPTS = load_prompt_files(".github/prompts/")

agent = CopilotEval(name="prompt-test")

@pytest.mark.parametrize("prompt", PROMPTS, ids=lambda p: p["name"])
async def test_prompt_files(copilot_eval, prompt):
    """All slash commands produce a successful response."""
    result = await copilot_eval(
        agent,
        prompt["body"],
        prompt_name=prompt["name"],
    )
    assert result.success
    assert result.prompt_name == prompt["name"]

VS Code vs Claude Code

Format Location Extension Invoked as
VS Code .github/prompts/ .prompt.md /review in Copilot Chat
Claude Code .claude/commands/ .md /review in Claude Code

load_prompt_files() handles both — .prompt.md files take precedence if both exist with the same name.

Tracking Prompt Names in Reports

The prompt_name kwarg on copilot_eval tags the result so reports can group tests by slash command:

result = await copilot_eval(agent, prompt["body"], prompt_name=prompt["name"])
# result.prompt_name == "review"

This appears in the HTML report's per-prompt breakdown, letting you compare how different slash commands perform across models.

Combining with MCP Servers

Prompt files often reference tools. Test them with the appropriate MCP servers:

from pytest_skill_engineering import load_prompt_file
from pytest_skill_engineering.copilot import CopilotEval

async def test_explain_command(copilot_eval):
    """The /explain command reads the file before explaining."""
    prompt = load_prompt_file(".github/prompts/explain.prompt.md")
    agent = CopilotEval(name="explain-test")
    result = await copilot_eval(agent, prompt["body"], prompt_name="explain")

    assert result.success
    assert result.tool_was_called("read_file")
    assert result.prompt_name == "explain"

Next Steps