Test logic update
This commit is contained in:
@@ -1,17 +1,7 @@
|
||||
№ **speckit.tasks.md**
|
||||
### Modified Workflow
|
||||
---
|
||||
|
||||
description: Generate tests, manage test documentation, and ensure maximum code coverage
|
||||
|
||||
```markdown
|
||||
description: Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts.
|
||||
handoffs:
|
||||
- label: Analyze For Consistency
|
||||
agent: speckit.analyze
|
||||
prompt: Run a project analysis for consistency
|
||||
send: true
|
||||
- label: Implement Project
|
||||
agent: speckit.implement
|
||||
prompt: Start the implementation in phases
|
||||
send: true
|
||||
---
|
||||
|
||||
## User Input
|
||||
@@ -22,95 +12,167 @@ $ARGUMENTS
|
||||
|
||||
You **MUST** consider the user input before proceeding (if not empty).
|
||||
|
||||
## Outline
|
||||
## Goal
|
||||
|
||||
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute.
|
||||
Execute full testing cycle: analyze code for testable modules, write tests with proper coverage, maintain test documentation, and ensure no test duplication or deletion.
|
||||
|
||||
2. **Load design documents**: Read from FEATURE_DIR:
|
||||
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities), ux_reference.md (experience source of truth)
|
||||
- **Optional**: data-model.md (entities), contracts/ (API endpoints), research.md (decisions)
|
||||
## Operating Constraints
|
||||
|
||||
3. **Execute task generation workflow**:
|
||||
- **Architecture Analysis (CRITICAL)**: Scan existing codebase for patterns (DI, Auth, ORM).
|
||||
- Load plan.md/spec.md.
|
||||
- Generate tasks organized by user story.
|
||||
- **Apply Fractal Co-location**: Ensure all unit tests are mapped to `__tests__` subdirectories relative to the code.
|
||||
- Validate task completeness.
|
||||
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
||||
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
||||
3. **Use TEST_DATA fixtures** - For CRITICAL tier modules, read @TEST_DATA from semantic_protocol.md
|
||||
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
||||
|
||||
4. **Generate tasks.md**: Use `.specify/templates/tasks-template.md` as structure.
|
||||
- Phase 1: Context & Setup.
|
||||
- Phase 2: Foundational tasks.
|
||||
- Phase 3+: User Stories (Priority order).
|
||||
- Final Phase: Polish.
|
||||
- **Strict Constraint**: Ensure tasks follow the Co-location and Mocking rules below.
|
||||
## Execution Steps
|
||||
|
||||
5. **Report**: Output path to generated tasks.md and summary.
|
||||
### 1. Analyze Context
|
||||
|
||||
Context for task generation: $ARGUMENTS
|
||||
Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS.
|
||||
|
||||
## Task Generation Rules
|
||||
Determine:
|
||||
- FEATURE_DIR - where the feature is located
|
||||
- TASKS_FILE - path to tasks.md
|
||||
- Which modules need testing based on task status
|
||||
|
||||
**CRITICAL**: Tasks MUST be actionable, specific, architecture-aware, and context-local.
|
||||
### 2. Load Relevant Artifacts
|
||||
|
||||
### Implementation & Testing Constraints (ANTI-LOOP & CO-LOCATION)
|
||||
**From tasks.md:**
|
||||
- Identify completed implementation tasks (not test tasks)
|
||||
- Extract file paths that need tests
|
||||
|
||||
To prevent infinite debugging loops and context fragmentation, apply these rules:
|
||||
**From semantic_protocol.md:**
|
||||
- Read @TIER annotations for modules
|
||||
- For CRITICAL modules: Read @TEST_DATA fixtures
|
||||
|
||||
1. **Fractal Co-location Strategy (MANDATORY)**:
|
||||
- **Rule**: Unit tests MUST live next to the code they verify.
|
||||
- **Forbidden**: Do NOT create unit tests in root `tests/` or `backend/tests/`. Those are for E2E/Integration only.
|
||||
- **Pattern (Python)**:
|
||||
- Source: `src/domain/order/processing.py`
|
||||
- Test Task: `Create tests in src/domain/order/__tests__/test_processing.py`
|
||||
- **Pattern (Frontend)**:
|
||||
- Source: `src/lib/components/UserCard.svelte`
|
||||
- Test Task: `Create tests in src/lib/components/__tests__/UserCard.test.ts`
|
||||
**From existing tests:**
|
||||
- Scan `__tests__` directories for existing tests
|
||||
- Identify test patterns and coverage gaps
|
||||
|
||||
2. **Semantic Relations**:
|
||||
- Test generation tasks must explicitly instruct to add the relation header: `# @RELATION: VERIFIES -> [TargetComponent]`
|
||||
### 3. Test Coverage Analysis
|
||||
|
||||
3. **Strict Mocking for Unit Tests**:
|
||||
- Any task creating Unit Tests MUST specify: *"Use `unittest.mock.MagicMock` for heavy dependencies (DB sessions, Auth). Do NOT instantiate real service classes."*
|
||||
Create coverage matrix:
|
||||
|
||||
4. **Schema/Model Separation**:
|
||||
- Explicitly separate tasks for ORM Models (SQLAlchemy) and Pydantic Schemas.
|
||||
| Module | File | Has Tests | TIER | TEST_DATA Available |
|
||||
|--------|------|-----------|------|-------------------|
|
||||
| ... | ... | ... | ... | ... |
|
||||
|
||||
### UX Preservation (CRITICAL)
|
||||
### 4. Write Tests (TDD Approach)
|
||||
|
||||
- **Source of Truth**: `ux_reference.md` is the absolute standard.
|
||||
- **Verification Task**: You **MUST** add a specific task at the end of each User Story phase: `- [ ] Txxx [USx] Verify implementation matches ux_reference.md (Happy Path & Errors)`
|
||||
For each module requiring tests:
|
||||
|
||||
### Checklist Format (REQUIRED)
|
||||
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
||||
2. **Read TEST_DATA**: If CRITICAL tier, read @TEST_DATA from semantic_protocol.md
|
||||
3. **Write test**: Follow co-location strategy
|
||||
- Python: `src/module/__tests__/test_module.py`
|
||||
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
||||
4. **Use mocks**: Use `unittest.mock.MagicMock` for external dependencies
|
||||
|
||||
Every task MUST strictly follow this format:
|
||||
### 4a. UX Contract Testing (Frontend Components)
|
||||
|
||||
```text
|
||||
- [ ] [TaskID] [P?] [Story?] Description with file path
|
||||
For Svelte components with `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY` tags:
|
||||
|
||||
1. **Parse UX tags**: Read component file and extract all `@UX_*` annotations
|
||||
2. **Generate UX tests**: Create tests for each UX state transition
|
||||
```javascript
|
||||
// Example: Testing @UX_STATE: Idle -> Expanded
|
||||
it('should transition from Idle to Expanded on toggle click', async () => {
|
||||
render(Sidebar);
|
||||
const toggleBtn = screen.getByRole('button', { name: /toggle/i });
|
||||
await fireEvent.click(toggleBtn);
|
||||
expect(screen.getByTestId('sidebar')).toHaveClass('expanded');
|
||||
});
|
||||
```
|
||||
3. **Test @UX_FEEDBACK**: Verify visual feedback (toast, shake, color changes)
|
||||
4. **Test @UX_RECOVERY**: Verify error recovery mechanisms (retry, clear input)
|
||||
5. **Use @UX_TEST fixtures**: If component has `@UX_TEST` tags, use them as test specifications
|
||||
|
||||
**UX Test Template:**
|
||||
```javascript
|
||||
// [DEF:__tests__/test_Component:Module]
|
||||
// @RELATION: VERIFIES -> ../Component.svelte
|
||||
// @PURPOSE: Test UX states and transitions
|
||||
|
||||
describe('Component UX States', () => {
|
||||
// @UX_STATE: Idle -> {action: click, expected: Active}
|
||||
it('should transition Idle -> Active on click', async () => { ... });
|
||||
|
||||
// @UX_FEEDBACK: Toast on success
|
||||
it('should show toast on successful action', async () => { ... });
|
||||
|
||||
// @UX_RECOVERY: Retry on error
|
||||
it('should allow retry on error', async () => { ... });
|
||||
});
|
||||
```
|
||||
|
||||
**Examples**:
|
||||
- ✅ `- [ ] T005 [US1] Create unit tests for OrderService in src/services/__tests__/test_order.py (Mock DB)`
|
||||
- ✅ `- [ ] T006 [US1] Implement OrderService in src/services/order.py`
|
||||
- ❌ `- [ ] T005 [US1] Create tests in backend/tests/test_order.py` (VIOLATION: Wrong location)
|
||||
### 5. Test Documentation
|
||||
|
||||
### Task Organization & Phase Structure
|
||||
Create/update documentation in `specs/<feature>/tests/`:
|
||||
|
||||
**Phase 1: Context & Setup**
|
||||
- **Goal**: Prepare environment and understand existing patterns.
|
||||
- **Mandatory Task**: `- [ ] T001 Analyze existing project structure, auth patterns, and `conftest.py` location`
|
||||
```
|
||||
tests/
|
||||
├── README.md # Test strategy and overview
|
||||
├── coverage.md # Coverage matrix and reports
|
||||
└── reports/
|
||||
└── YYYY-MM-DD-report.md
|
||||
```
|
||||
|
||||
**Phase 2: Foundational (Data & Core)**
|
||||
- Database Models (ORM).
|
||||
- Pydantic Schemas (DTOs).
|
||||
- Core Service interfaces.
|
||||
### 6. Execute Tests
|
||||
|
||||
**Phase 3+: User Stories (Iterative)**
|
||||
- **Step 1: Isolation Tests (Co-located)**:
|
||||
- `- [ ] Txxx [USx] Create unit tests for [Component] in [Path]/__tests__/test_[name].py`
|
||||
- *Note: Specify using MagicMock for external deps.*
|
||||
- **Step 2: Implementation**: Services -> Endpoints.
|
||||
- **Step 3: Integration**: Wire up real dependencies (if E2E tests requested).
|
||||
- **Step 4: UX Verification**.
|
||||
Run tests and report results:
|
||||
|
||||
**Final Phase: Polish**
|
||||
- Linting, formatting, final manual verify.
|
||||
**Backend:**
|
||||
```bash
|
||||
cd backend && .venv/bin/python3 -m pytest -v
|
||||
```
|
||||
|
||||
**Frontend:**
|
||||
```bash
|
||||
cd frontend && npm run test
|
||||
```
|
||||
|
||||
### 7. Update Tasks
|
||||
|
||||
Mark test tasks as completed in tasks.md with:
|
||||
- Test file path
|
||||
- Coverage achieved
|
||||
- Any issues found
|
||||
|
||||
## Output
|
||||
|
||||
Generate test execution report:
|
||||
|
||||
```markdown
|
||||
# Test Report: [FEATURE]
|
||||
|
||||
**Date**: [YYYY-MM-DD]
|
||||
**Executed by**: Tester Agent
|
||||
|
||||
## Coverage Summary
|
||||
|
||||
| Module | Tests | Coverage % |
|
||||
|--------|-------|------------|
|
||||
| ... | ... | ... |
|
||||
|
||||
## Test Results
|
||||
|
||||
- Total: [X]
|
||||
- Passed: [X]
|
||||
- Failed: [X]
|
||||
- Skipped: [X]
|
||||
|
||||
## Issues Found
|
||||
|
||||
| Test | Error | Resolution |
|
||||
|------|-------|------------|
|
||||
| ... | ... | ... |
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [ ] Fix failed tests
|
||||
- [ ] Add more coverage for [module]
|
||||
- [ ] Review TEST_DATA fixtures
|
||||
```
|
||||
|
||||
## Context for Testing
|
||||
|
||||
$ARGUMENTS
|
||||
|
||||
Reference in New Issue
Block a user