testzeus-test-authoring#

Create and edit TestZeus test definitions in Gherkin, including AI-assisted test generation.

Category: core · Version: 1.0.0 · Requires: testzeus-auth

Triggers: create test, write test, author test, generate test, new test case

Triggers#

See triggers.md for pattern matching rules.

Overview#

Guides users through conversational test case creation using a multi-step wizard approach. The wizard collects all necessary information to generate comprehensive test cases using AI-powered generation.

Design Philosophy#

Hybrid Approach#

Combines conversational guidance with structured input components:

  • Conversational for intent understanding and guidance

  • Structured components for specific inputs (selections, counts)

  • Natural language for complex inputs (descriptions, scenarios)

Review-First Workflow#

All generated content goes through a review step before saving:

  • User can accept, modify, or reject generated test cases

  • Allows refinement through additional prompting

  • Ensures quality before committing to the test suite

Refinement Loop#

Users can iteratively refine generated content:

  • Edit generated feature files inline

  • Regenerate specific scenarios

  • Add or remove test cases before saving

Capabilities#

1. Guided Test Creation Wizard#

Multi-step conversational wizard that collects:

  • Test name (required)

  • Test description/goal (required)

  • Target environment (optional)

  • Test data selection (optional)

  • Tags for organization (optional)

  • Specific scenarios to cover (optional)

  • Number of test cases to generate

  • Reasoning complexity level

2. AI-Powered Test Generation#

Leverages the tests_ai_generator infrastructure:

  • Generates Gherkin-style test scenarios

  • Creates step definitions

  • Handles edge cases based on complexity setting

  • Supports knowledge base integration

3. Generated Content Review#

Review interface allows:

  • View all generated test cases

  • Edit individual test cases

  • Remove unwanted test cases

  • Regenerate specific scenarios

  • Regenerate all with different parameters

4. Test Saving Options#

After review:

  • Save all generated tests

  • Save selected tests only

  • Save as draft for later editing

  • Discard and start over

Workflow States#

idle
  │
  ▼
understanding_goal ──────────► cancel
  │ (user wants to create test)
  ▼
collecting_name
  │
  ▼
collecting_description
  │
  ▼
collecting_environment ──────► skip_environment
  │
  ▼
collecting_test_data ───────► skip_test_data
  │
  ▼
collecting_tags ────────────► skip_tags
  │
  ▼
collecting_scenarios ───────► skip_scenarios
  │
  ▼
collecting_count
  │
  ▼
collecting_complexity
  │
  ▼
confirming_settings
  │
  ▼
generating ──────────────────► regenerate
  │
  ▼
reviewing ──────────────────► edit_generated
  │                           │
  │                           ▼
  │                       generating
  ▼
saving ─────────────────────► discard
  │
  ▼
complete ────────────────────► idle

Conversation Flow#

Step 1: Understand Intent#

Agent asks: “What would you like to test?”

User provides: High-level goal like “login functionality” or “checkout flow”

Agent follows up: “Got it! What should I call this test?”

Step 2: Collect Name#

Agent asks: “What should I call this test?”

User provides: A descriptive name like “login_validation” or “checkout_payment”

Validation: Name must be unique, alphanumeric with underscores

Agent follows up: “Perfect! Now briefly describe what this test should verify.”

Step 3: Collect Description#

Agent asks: “Describe what this test should verify.”

User provides: A description like “Verify users can log in with valid credentials and see appropriate errors for invalid ones.”

Agent follows up: “Should this test run against a specific environment?”

Step 4: Environment (Optional)#

Agent asks: “Select an environment or skip.”

Shows: Available environments as selectable options

User selects: e.g., “staging” or “skip”

Agent follows up: “Do you want to use any test data for this test?”

Step 5: Test Data (Optional)#

Agent asks: “Select test data or skip.”

Shows: Available test data sets

User selects: e.g., “QA Test Users” or “skip”

Agent follows up: “Add any tags to organize this test?”

Step 6: Tags (Optional)#

Agent asks: “Add tags or skip.”

User provides: Comma-separated tags like “authentication, smoke” or “skip”

Agent follows up: “Describe the specific scenarios to test.”

Step 7: Scenarios (Optional)#

Agent asks: “What scenarios should I cover? Describe the paths you want tested.”

User provides: Natural language description of scenarios

Examples:

  • “Log in with valid email and password”

  • “Log in with incorrect password shows error”

  • “Log in with non-existent email”

  • “Session timeout after inactivity”

Agent follows up: “How many test cases should I generate?”

Step 8: Count#

Agent asks: “How many test cases? (1-20)”

Shows: Slider component for easy selection

User provides: Number like “8” or uses slider

Agent follows up: “What reasoning complexity level?”

Step 9: Complexity#

Agent asks: “Select complexity level:”

Shows: Options with descriptions

  • Low: Basic happy path tests

  • Medium: Common edge cases included

  • High: Thorough coverage with edge cases

User selects: e.g., “medium”

Agent confirms: “Let me show you the summary before generating.”

Step 10: Confirm Settings#

Agent shows: Summary component with all collected settings

Summary:
- Name: login_validation
- Description: Verify login functionality
- Environment: staging
- Test Data: QA Test Users
- Tags: authentication, smoke
- Scenarios: [user input]
- Test Cases: 8
- Complexity: medium

Ready to generate?

User confirms or requests changes

Step 11: Generate#

Agent shows: Progress indicator

{
  "component": "StatusBadge",
  "props": { "status": "generating", "label": "Generating 8 test cases..." }
}

When complete: Show generated tests for review

Step 12: Review#

Agent shows: List of generated test cases with edit options

{
  "component": "AuthoringReview",
  "props": {
    "testCases": [
      { "id": "1", "name": "Valid login", "status": "generated" },
      { "id": "2", "name": "Invalid password", "status": "generated" },
      ...
    ],
    "actions": ["save_all", "edit", "regenerate", "discard"]
  }
}

User can:

  • Save all test cases

  • Edit specific test cases

  • Regenerate all or specific

  • Discard and start over

Step 13: Save#

On save: Create tests via API

Agent confirms: “Created 8 test cases successfully!”

Response Formatting#

Status Display#

{
  "component": "StatusBadge",
  "props": {
    "status": "generating",
    "label": "Generating test cases..."
  }
}

Wizard Progress#

{
  "component": "WizardProgress",
  "props": {
    "currentStep": 5,
    "totalSteps": 10,
    "stepName": "Collecting scenarios",
    "canGoBack": true
  }
}

Settings Summary#

{
  "component": "AuthoringSummary",
  "props": {
    "name": "login_validation",
    "description": "Verify login functionality",
    "environment": { "id": "env_1", "name": "staging" },
    "testData": [{ "id": "td_1", "name": "QA Test Users" }],
    "tags": ["authentication", "smoke"],
    "scenarioCount": 8,
    "complexity": "medium"
  }
}

Test Case List#

{
  "component": "TestCaseList",
  "props": {
    "testCases": [
      {
        "id": "gen_1",
        "name": "Valid credentials login",
        "feature": "Given a user on the login page...",
        "selected": true
      }
    ],
    "totalGenerated": 8
  }
}

Confirmation Prompt#

{
  "component": "ConfirmDialog",
  "props": {
    "title": "Save Test Cases",
    "message": "Save 8 test cases to your suite?",
    "confirmLabel": "Save All",
    "cancelLabel": "Review Again"
  }
}

Best Practices#

  1. Be conversational, not robotic - Use natural language transitions between steps

  2. Provide sensible defaults - Pre-select based on context when possible

  3. Show, don’t just tell - Use components to display options visually

  4. Allow skipping - Optional fields should be skippable

  5. Summarize frequently - Help users track what they’ve entered

  6. Make edits easy - Allow corrections at any step

  7. Be patient - Some users need more guidance than others

  8. Confirm before major actions - Always confirm before generating or saving

  9. Handle errors gracefully - Explain what went wrong and how to fix it

  10. Offer alternatives - If generation fails, offer manual test creation

Error Handling#

Validation Errors#

  • Show inline error messages with component schema validation

  • Highlight problematic fields

  • Suggest corrections

Generation Errors#

  • Explain the error in plain language

  • Offer to retry with simpler parameters

  • Offer manual test creation as fallback

Save Errors#

  • Show error details

  • Offer to retry

  • Offer to save as draft for later

State Persistence#

Store wizard state in session memory:

{
  "currentStep": 5,
  "collectedData": {
    "name": "login_validation",
    "description": "...",
    "environment": { "id": "...", "name": "..." },
    "testData": [...],
    "tags": [...],
    "scenarios": "...",
    "count": 8,
    "complexity": "medium"
  },
  "generatedTests": [...],
  "reviewState": {
    "editedTests": [...],
    "selectedForSave": ["1", "2", "3"]
  }
}

Examples#

See examples/conversation.md for detailed examples.