workflows update
This commit is contained in:
@@ -6,7 +6,7 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
||||||
|
|
||||||
**INPUT:**
|
**INPUT:**
|
||||||
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_`).
|
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`).
|
||||||
2. GENERATED TEST CODE.
|
2. GENERATED TEST CODE.
|
||||||
|
|
||||||
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
||||||
@@ -17,7 +17,7 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
|
|
||||||
2. **The Logic Mirror (Echoing):**
|
2. **The Logic Mirror (Echoing):**
|
||||||
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
||||||
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_CONTRACT`, @TEST_FIXTURE, @TEST_EDGE, @TEST_INVARIANT or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT` or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
||||||
|
|
||||||
3. **The "Happy Path" Illusion:**
|
3. **The "Happy Path" Illusion:**
|
||||||
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
||||||
@@ -26,26 +26,78 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
4. **Missing Post-Condition Verification:**
|
4. **Missing Post-Condition Verification:**
|
||||||
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
||||||
|
|
||||||
### II. AUDIT CHECKLIST
|
5. **Missing Edge Case Coverage:**
|
||||||
|
- *Definition:* The test suite ignores `@TEST_EDGE` scenarios defined in the contract.
|
||||||
|
- *Rule:* Every `@TEST_EDGE` in the source contract MUST have a corresponding test case.
|
||||||
|
|
||||||
|
6. **Missing Invariant Verification:**
|
||||||
|
- *Definition:* The test suite does not verify `@TEST_INVARIANT` conditions.
|
||||||
|
- *Rule:* Every `@TEST_INVARIANT` MUST be verified by at least one test that attempts to break it.
|
||||||
|
|
||||||
|
7. **Missing UX State Testing (Svelte Components):**
|
||||||
|
- *Definition:* For Svelte components with `@UX_STATE`, the test suite does not verify state transitions.
|
||||||
|
- *Rule:* Every `@UX_STATE` transition MUST have a test verifying the visual/behavioral change.
|
||||||
|
- *Check:* `@UX_FEEDBACK` mechanisms (toast, shake, color) must be tested.
|
||||||
|
- *Check:* `@UX_RECOVERY` mechanisms (retry, clear input) must be tested.
|
||||||
|
|
||||||
|
### II. SEMANTIC PROTOCOL COMPLIANCE
|
||||||
|
|
||||||
|
Verify the test file follows GRACE-Poly semantics:
|
||||||
|
|
||||||
|
1. **Anchor Integrity:**
|
||||||
|
- Test file MUST start with `[DEF:__tests__/test_name:Module]`
|
||||||
|
- Test file MUST end with `[/DEF:__tests__/test_name:Module]`
|
||||||
|
|
||||||
|
2. **Required Tags:**
|
||||||
|
- `@RELATION: VERIFIES -> <path_to_source>` must be present
|
||||||
|
- `@PURPOSE:` must describe what is being tested
|
||||||
|
|
||||||
|
3. **TIER Alignment:**
|
||||||
|
- If source is `@TIER: CRITICAL`, test MUST cover all `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`
|
||||||
|
- If source is `@TIER: STANDARD`, test MUST cover `@PRE` and `@POST`
|
||||||
|
- If source is `@TIER: TRIVIAL`, basic smoke test is acceptable
|
||||||
|
|
||||||
|
### III. AUDIT CHECKLIST
|
||||||
|
|
||||||
Evaluate the test code against these criteria:
|
Evaluate the test code against these criteria:
|
||||||
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
||||||
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
||||||
3. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_`?
|
3. **Test Contract Compliance:** Does the test follow the interface defined in `@TEST_CONTRACT`?
|
||||||
4. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
4. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_FIXTURE`?
|
||||||
|
5. **Edge Coverage:** Are all `@TEST_EDGE` scenarios tested?
|
||||||
|
6. **Invariant Coverage:** Are all `@TEST_INVARIANT` conditions verified?
|
||||||
|
7. **UX Coverage (if applicable):** Are all `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY` tested?
|
||||||
|
8. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
||||||
|
9. **Semantic Anchor:** Does the test file have proper `[DEF]` and `[/DEF]` anchors?
|
||||||
|
|
||||||
### III. OUTPUT FORMAT
|
### IV. OUTPUT FORMAT
|
||||||
|
|
||||||
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
||||||
|
|
||||||
{
|
{
|
||||||
"verdict": "APPROVED" | "REJECTED",
|
"verdict": "APPROVED" | "REJECTED",
|
||||||
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "NONE",
|
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "MISSING_EDGES" | "MISSING_INVARIANTS" | "MISSING_UX_TESTS" | "SEMANTIC_VIOLATION" | "NONE",
|
||||||
"audit_details": {
|
"audit_details": {
|
||||||
"target_invoked": true/false,
|
"target_invoked": true/false,
|
||||||
"pre_conditions_tested": true/false,
|
"pre_conditions_tested": true/false,
|
||||||
"post_conditions_tested": true/false,
|
"post_conditions_tested": true/false,
|
||||||
"test_data_used": true/false
|
"test_fixture_used": true/false,
|
||||||
|
"edges_covered": true/false,
|
||||||
|
"invariants_verified": true/false,
|
||||||
|
"ux_states_tested": true/false,
|
||||||
|
"semantic_anchors_present": true/false
|
||||||
|
},
|
||||||
|
"coverage_summary": {
|
||||||
|
"total_edges": number,
|
||||||
|
"edges_tested": number,
|
||||||
|
"total_invariants": number,
|
||||||
|
"invariants_tested": number,
|
||||||
|
"total_ux_states": number,
|
||||||
|
"ux_states_tested": number
|
||||||
|
},
|
||||||
|
"tier_compliance": {
|
||||||
|
"source_tier": "CRITICAL" | "STANDARD" | "TRIVIAL",
|
||||||
|
"meets_tier_requirements": true/false
|
||||||
},
|
},
|
||||||
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
---
|
---
|
||||||
description: USE SEMANTIC
|
description: USE SEMANTIC
|
||||||
---
|
---
|
||||||
Прочитай .specify/memory/semantics.md (или .ai/standards/semantics.md, если не найден). ОБЯЗАТЕЛЬНО используй его при разработке
|
Прочитай .ai/standards/semantics.md. ОБЯЗАТЕЛЬНО используй его при разработке
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ Load only the minimal necessary context from each artifact:
|
|||||||
**From constitution:**
|
**From constitution:**
|
||||||
|
|
||||||
- Load `.ai/standards/constitution.md` for principle validation
|
- Load `.ai/standards/constitution.md` for principle validation
|
||||||
|
- Load `.ai/standards/semantics.md` for technical standard validation
|
||||||
|
|
||||||
### 3. Build Semantic Models
|
### 3. Build Semantic Models
|
||||||
|
|
||||||
|
|||||||
@@ -53,6 +53,15 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||||
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||||
|
|
||||||
|
3. Load and analyze the implementation context:
|
||||||
|
- **REQUIRED**: Read `.ai/standards/semantics.md` for strict coding standards and contract requirements
|
||||||
|
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
|
||||||
|
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
|
||||||
|
- **IF EXISTS**: Read data-model.md for entities and relationships
|
||||||
|
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
|
||||||
|
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||||
|
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||||
|
|
||||||
4. **Project Setup Verification**:
|
4. **Project Setup Verification**:
|
||||||
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
||||||
|
|
||||||
@@ -111,7 +120,13 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- **Validation checkpoints**: Verify each phase completion before proceeding
|
- **Validation checkpoints**: Verify each phase completion before proceeding
|
||||||
|
|
||||||
7. Implementation execution rules:
|
7. Implementation execution rules:
|
||||||
- **Setup first**: Initialize project structure, dependencies, configuration
|
- **Strict Adherence**: Apply `.ai/standards/semantics.md` rules:
|
||||||
|
- Every file MUST start with a `[DEF:id:Type]` header and end with a closing `[/DEF:id:Type]` anchor.
|
||||||
|
- Include `@TIER` and define contracts (`@PRE`, `@POST`).
|
||||||
|
- For Svelte components, use `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY`, and explicitly declare reactivity with `@UX_REATIVITY: State: $state, Derived: $derived`.
|
||||||
|
- **Molecular Topology Logging**: Use prefixes `[EXPLORE]`, `[REASON]`, `[REFLECT]` in logs to trace logic.
|
||||||
|
- **CRITICAL Contracts**: If a task description contains a contract summary (e.g., `CRITICAL: PRE: ..., POST: ...`), these constraints are **MANDATORY** and must be strictly implemented in the code using guards/assertions (if applicable per protocol).
|
||||||
|
- **Setup first**: Initialize project structure, dependencies, configuration
|
||||||
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
||||||
- **Core development**: Implement models, services, CLI commands, endpoints
|
- **Core development**: Implement models, services, CLI commands, endpoints
|
||||||
- **Integration work**: Database connections, middleware, logging, external services
|
- **Integration work**: Database connections, middleware, logging, external services
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
|
|
||||||
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
2. **Load context**: Read FEATURE_SPEC and `.ai/standards/constitution.md`. Load IMPL_PLAN template (already copied).
|
2. **Load context**: Read `.ai/ROOT.md` and `.ai/PROJECT_MAP.md` to understand the project structure and navigation. Then read required standards: `.ai/standards/constitution.md` and `.ai/standards/semantics.md`. Load IMPL_PLAN template.
|
||||||
|
|
||||||
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
||||||
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
||||||
@@ -64,16 +64,30 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
|
|
||||||
**Prerequisites:** `research.md` complete
|
**Prerequisites:** `research.md` complete
|
||||||
|
|
||||||
1. **Extract entities from feature spec** → `data-model.md`:
|
0. **Validate Design against UX Reference**:
|
||||||
- Entity name, fields, relationships
|
- Check if the proposed architecture supports the latency, interactivity, and flow defined in `ux_reference.md`.
|
||||||
- Validation rules from requirements
|
- **Linkage**: Ensure key UI states from `ux_reference.md` map to Component Contracts (`@UX_STATE`).
|
||||||
- State transitions if applicable
|
- **CRITICAL**: If the technical plan compromises the UX (e.g. "We can't do real-time validation"), you **MUST STOP** and warn the user.
|
||||||
|
|
||||||
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
|
1. **Extract entities from feature spec** → `data-model.md`:
|
||||||
- Identify what interfaces the project exposes to users or other systems
|
- Entity name, fields, relationships, validation rules.
|
||||||
- Document the contract format appropriate for the project type
|
|
||||||
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
|
2. **Design & Verify Contracts (Semantic Protocol)**:
|
||||||
- Skip if project is purely internal (build scripts, one-off tools, etc.)
|
- **Drafting**: Define `[DEF:id:Type]` Headers, Contracts, and closing `[/DEF:id:Type]` for all new modules based on `.ai/standards/semantics.md`.
|
||||||
|
- **TIER Classification**: Explicitly assign `@TIER: [CRITICAL|STANDARD|TRIVIAL]` to each module.
|
||||||
|
- **CRITICAL Requirements**: For all CRITICAL modules, define full `@PRE`, `@POST`, and (if UI) `@UX_STATE` contracts. **MUST** also define testing contracts: `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, and `@TEST_INVARIANT`.
|
||||||
|
- **Self-Review**:
|
||||||
|
- *Completeness*: Do `@PRE`/`@POST` cover edge cases identified in Research? Are test contracts present for CRITICAL?
|
||||||
|
- *Connectivity*: Do `@RELATION` tags form a coherent graph?
|
||||||
|
- *Compliance*: Does syntax match `[DEF:id:Type]` exactly and is it closed with `[/DEF:id:Type]`?
|
||||||
|
- **Output**: Write verified contracts to `contracts/modules.md`.
|
||||||
|
|
||||||
|
3. **Simulate Contract Usage**:
|
||||||
|
- Trace one key user scenario through the defined contracts to ensure data flow continuity.
|
||||||
|
- If a contract interface mismatch is found, fix it immediately.
|
||||||
|
|
||||||
|
4. **Generate API contracts**:
|
||||||
|
- Output OpenAPI/GraphQL schema to `/contracts/` for backend-frontend sync.
|
||||||
|
|
||||||
3. **Agent context update**:
|
3. **Agent context update**:
|
||||||
- Run `.specify/scripts/bash/update-agent-context.sh agy`
|
- Run `.specify/scripts/bash/update-agent-context.sh agy`
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
2. **Load design documents**: Read from FEATURE_DIR:
|
2. **Load design documents**: Read from FEATURE_DIR:
|
||||||
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
|
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities), ux_reference.md (experience source of truth)
|
||||||
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
||||||
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
||||||
|
|
||||||
@@ -70,6 +70,12 @@ The tasks.md should be immediately executable - each task must be specific enoug
|
|||||||
|
|
||||||
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
||||||
|
|
||||||
|
### UX Preservation (CRITICAL)
|
||||||
|
|
||||||
|
- **Source of Truth**: `ux_reference.md` is the absolute standard for the "feel" of the feature.
|
||||||
|
- **Violation Warning**: If any task would inherently violate the UX (e.g. "Remove progress bar to simplify code"), you **MUST** flag this to the user immediately.
|
||||||
|
- **Verification Task**: You **MUST** add a specific task at the end of each User Story phase: `- [ ] Txxx [USx] Verify implementation matches ux_reference.md (Happy Path & Errors)`
|
||||||
|
|
||||||
### Checklist Format (REQUIRED)
|
### Checklist Format (REQUIRED)
|
||||||
|
|
||||||
Every task MUST strictly follow this format:
|
Every task MUST strictly follow this format:
|
||||||
@@ -113,9 +119,12 @@ Every task MUST strictly follow this format:
|
|||||||
- If tests requested: Tests specific to that story
|
- If tests requested: Tests specific to that story
|
||||||
- Mark story dependencies (most stories should be independent)
|
- Mark story dependencies (most stories should be independent)
|
||||||
|
|
||||||
2. **From Contracts**:
|
2. **From Contracts (CRITICAL TIER)**:
|
||||||
- Map each interface contract → to the user story it serves
|
- Identify components marked as `@TIER: CRITICAL` in `contracts/modules.md`.
|
||||||
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
|
- For these components, **MUST** append the summary of `@PRE`, `@POST`, `@UX_STATE`, and test contracts (`@TEST_FIXTURE`, `@TEST_EDGE`) directly to the task description.
|
||||||
|
- Example: `- [ ] T005 [P] [US1] Implement Auth (CRITICAL: PRE: token exists, POST: returns User, TESTS: 2 edges) in src/auth.py`
|
||||||
|
- Map each contract/endpoint → to the user story it serves
|
||||||
|
- If tests requested: Each contract → contract test task [P] before implementation in that story's phase
|
||||||
|
|
||||||
3. **From Data Model**:
|
3. **From Data Model**:
|
||||||
- Map each entity to the user story(ies) that need it
|
- Map each entity to the user story(ies) that need it
|
||||||
|
|||||||
@@ -249,6 +249,7 @@ component/__tests__/Component.test.js
|
|||||||
# [DEF:__tests__/test_module:Module]
|
# [DEF:__tests__/test_module:Module]
|
||||||
# @RELATION: VERIFIES -> ../module.py
|
# @RELATION: VERIFIES -> ../module.py
|
||||||
# @PURPOSE: Contract testing for module
|
# @PURPOSE: Contract testing for module
|
||||||
|
# [/DEF:__tests__/test_module:Module]
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
||||||
|
|
||||||
**INPUT:**
|
**INPUT:**
|
||||||
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_`).
|
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`).
|
||||||
2. GENERATED TEST CODE.
|
2. GENERATED TEST CODE.
|
||||||
|
|
||||||
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
||||||
@@ -17,7 +17,7 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
|
|
||||||
2. **The Logic Mirror (Echoing):**
|
2. **The Logic Mirror (Echoing):**
|
||||||
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
||||||
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_` or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT` or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
||||||
|
|
||||||
3. **The "Happy Path" Illusion:**
|
3. **The "Happy Path" Illusion:**
|
||||||
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
||||||
@@ -26,26 +26,78 @@ description: Audit AI-generated unit tests. Your goal is to aggressively search
|
|||||||
4. **Missing Post-Condition Verification:**
|
4. **Missing Post-Condition Verification:**
|
||||||
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
||||||
|
|
||||||
### II. AUDIT CHECKLIST
|
5. **Missing Edge Case Coverage:**
|
||||||
|
- *Definition:* The test suite ignores `@TEST_EDGE` scenarios defined in the contract.
|
||||||
|
- *Rule:* Every `@TEST_EDGE` in the source contract MUST have a corresponding test case.
|
||||||
|
|
||||||
|
6. **Missing Invariant Verification:**
|
||||||
|
- *Definition:* The test suite does not verify `@TEST_INVARIANT` conditions.
|
||||||
|
- *Rule:* Every `@TEST_INVARIANT` MUST be verified by at least one test that attempts to break it.
|
||||||
|
|
||||||
|
7. **Missing UX State Testing (Svelte Components):**
|
||||||
|
- *Definition:* For Svelte components with `@UX_STATE`, the test suite does not verify state transitions.
|
||||||
|
- *Rule:* Every `@UX_STATE` transition MUST have a test verifying the visual/behavioral change.
|
||||||
|
- *Check:* `@UX_FEEDBACK` mechanisms (toast, shake, color) must be tested.
|
||||||
|
- *Check:* `@UX_RECOVERY` mechanisms (retry, clear input) must be tested.
|
||||||
|
|
||||||
|
### II. SEMANTIC PROTOCOL COMPLIANCE
|
||||||
|
|
||||||
|
Verify the test file follows GRACE-Poly semantics:
|
||||||
|
|
||||||
|
1. **Anchor Integrity:**
|
||||||
|
- Test file MUST start with `[DEF:__tests__/test_name:Module]`
|
||||||
|
- Test file MUST end with `[/DEF:__tests__/test_name:Module]`
|
||||||
|
|
||||||
|
2. **Required Tags:**
|
||||||
|
- `@RELATION: VERIFIES -> <path_to_source>` must be present
|
||||||
|
- `@PURPOSE:` must describe what is being tested
|
||||||
|
|
||||||
|
3. **TIER Alignment:**
|
||||||
|
- If source is `@TIER: CRITICAL`, test MUST cover all `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`
|
||||||
|
- If source is `@TIER: STANDARD`, test MUST cover `@PRE` and `@POST`
|
||||||
|
- If source is `@TIER: TRIVIAL`, basic smoke test is acceptable
|
||||||
|
|
||||||
|
### III. AUDIT CHECKLIST
|
||||||
|
|
||||||
Evaluate the test code against these criteria:
|
Evaluate the test code against these criteria:
|
||||||
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
||||||
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
||||||
3. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_`?
|
3. **Test Contract Compliance:** Does the test follow the interface defined in `@TEST_CONTRACT`?
|
||||||
4. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
4. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_FIXTURE`?
|
||||||
|
5. **Edge Coverage:** Are all `@TEST_EDGE` scenarios tested?
|
||||||
|
6. **Invariant Coverage:** Are all `@TEST_INVARIANT` conditions verified?
|
||||||
|
7. **UX Coverage (if applicable):** Are all `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY` tested?
|
||||||
|
8. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
||||||
|
9. **Semantic Anchor:** Does the test file have proper `[DEF]` and `[/DEF]` anchors?
|
||||||
|
|
||||||
### III. OUTPUT FORMAT
|
### IV. OUTPUT FORMAT
|
||||||
|
|
||||||
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
||||||
|
|
||||||
{
|
{
|
||||||
"verdict": "APPROVED" | "REJECTED",
|
"verdict": "APPROVED" | "REJECTED",
|
||||||
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "NONE",
|
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "MISSING_EDGES" | "MISSING_INVARIANTS" | "MISSING_UX_TESTS" | "SEMANTIC_VIOLATION" | "NONE",
|
||||||
"audit_details": {
|
"audit_details": {
|
||||||
"target_invoked": true/false,
|
"target_invoked": true/false,
|
||||||
"pre_conditions_tested": true/false,
|
"pre_conditions_tested": true/false,
|
||||||
"post_conditions_tested": true/false,
|
"post_conditions_tested": true/false,
|
||||||
"test_data_used": true/false
|
"test_fixture_used": true/false,
|
||||||
|
"edges_covered": true/false,
|
||||||
|
"invariants_verified": true/false,
|
||||||
|
"ux_states_tested": true/false,
|
||||||
|
"semantic_anchors_present": true/false
|
||||||
|
},
|
||||||
|
"coverage_summary": {
|
||||||
|
"total_edges": number,
|
||||||
|
"edges_tested": number,
|
||||||
|
"total_invariants": number,
|
||||||
|
"invariants_tested": number,
|
||||||
|
"total_ux_states": number,
|
||||||
|
"ux_states_tested": number
|
||||||
|
},
|
||||||
|
"tier_compliance": {
|
||||||
|
"source_tier": "CRITICAL" | "STANDARD" | "TRIVIAL",
|
||||||
|
"meets_tier_requirements": true/false
|
||||||
},
|
},
|
||||||
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
||||||
}
|
}
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
---
|
---
|
||||||
description: USE SEMANTIC
|
description: USE SEMANTIC
|
||||||
---
|
---
|
||||||
Прочитай .specify/memory/semantics.md (или .ai/standards/semantics.md, если не найден). ОБЯЗАТЕЛЬНО используй его при разработке
|
Прочитай .ai/standards/semantics.md. ОБЯЗАТЕЛЬНО используй его при разработке
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ Load only the minimal necessary context from each artifact:
|
|||||||
**From constitution:**
|
**From constitution:**
|
||||||
|
|
||||||
- Load `.ai/standards/constitution.md` for principle validation
|
- Load `.ai/standards/constitution.md` for principle validation
|
||||||
|
- Load `.ai/standards/semantics.md` for technical standard validation
|
||||||
|
|
||||||
### 3. Build Semantic Models
|
### 3. Build Semantic Models
|
||||||
|
|
||||||
|
|||||||
@@ -53,6 +53,15 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||||
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||||
|
|
||||||
|
3. Load and analyze the implementation context:
|
||||||
|
- **REQUIRED**: Read `.ai/standards/semantics.md` for strict coding standards and contract requirements
|
||||||
|
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
|
||||||
|
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
|
||||||
|
- **IF EXISTS**: Read data-model.md for entities and relationships
|
||||||
|
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
|
||||||
|
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
||||||
|
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
||||||
|
|
||||||
4. **Project Setup Verification**:
|
4. **Project Setup Verification**:
|
||||||
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
||||||
|
|
||||||
@@ -111,7 +120,13 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- **Validation checkpoints**: Verify each phase completion before proceeding
|
- **Validation checkpoints**: Verify each phase completion before proceeding
|
||||||
|
|
||||||
7. Implementation execution rules:
|
7. Implementation execution rules:
|
||||||
- **Setup first**: Initialize project structure, dependencies, configuration
|
- **Strict Adherence**: Apply `.ai/standards/semantics.md` rules:
|
||||||
|
- Every file MUST start with a `[DEF:id:Type]` header and end with a closing `[/DEF:id:Type]` anchor.
|
||||||
|
- Include `@TIER` and define contracts (`@PRE`, `@POST`).
|
||||||
|
- For Svelte components, use `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY`, and explicitly declare reactivity with `@UX_REATIVITY: State: $state, Derived: $derived`.
|
||||||
|
- **Molecular Topology Logging**: Use prefixes `[EXPLORE]`, `[REASON]`, `[REFLECT]` in logs to trace logic.
|
||||||
|
- **CRITICAL Contracts**: If a task description contains a contract summary (e.g., `CRITICAL: PRE: ..., POST: ...`), these constraints are **MANDATORY** and must be strictly implemented in the code using guards/assertions (if applicable per protocol).
|
||||||
|
- **Setup first**: Initialize project structure, dependencies, configuration
|
||||||
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
||||||
- **Core development**: Implement models, services, CLI commands, endpoints
|
- **Core development**: Implement models, services, CLI commands, endpoints
|
||||||
- **Integration work**: Database connections, middleware, logging, external services
|
- **Integration work**: Database connections, middleware, logging, external services
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
|
|
||||||
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
2. **Load context**: Read FEATURE_SPEC and `.ai/standards/constitution.md`. Load IMPL_PLAN template (already copied).
|
2. **Load context**: Read `.ai/ROOT.md` and `.ai/PROJECT_MAP.md` to understand the project structure and navigation. Then read required standards: `.ai/standards/constitution.md` and `.ai/standards/semantics.md`. Load IMPL_PLAN template.
|
||||||
|
|
||||||
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
||||||
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
||||||
@@ -64,16 +64,30 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
|
|
||||||
**Prerequisites:** `research.md` complete
|
**Prerequisites:** `research.md` complete
|
||||||
|
|
||||||
1. **Extract entities from feature spec** → `data-model.md`:
|
0. **Validate Design against UX Reference**:
|
||||||
- Entity name, fields, relationships
|
- Check if the proposed architecture supports the latency, interactivity, and flow defined in `ux_reference.md`.
|
||||||
- Validation rules from requirements
|
- **Linkage**: Ensure key UI states from `ux_reference.md` map to Component Contracts (`@UX_STATE`).
|
||||||
- State transitions if applicable
|
- **CRITICAL**: If the technical plan compromises the UX (e.g. "We can't do real-time validation"), you **MUST STOP** and warn the user.
|
||||||
|
|
||||||
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
|
1. **Extract entities from feature spec** → `data-model.md`:
|
||||||
- Identify what interfaces the project exposes to users or other systems
|
- Entity name, fields, relationships, validation rules.
|
||||||
- Document the contract format appropriate for the project type
|
|
||||||
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
|
2. **Design & Verify Contracts (Semantic Protocol)**:
|
||||||
- Skip if project is purely internal (build scripts, one-off tools, etc.)
|
- **Drafting**: Define `[DEF:id:Type]` Headers, Contracts, and closing `[/DEF:id:Type]` for all new modules based on `.ai/standards/semantics.md`.
|
||||||
|
- **TIER Classification**: Explicitly assign `@TIER: [CRITICAL|STANDARD|TRIVIAL]` to each module.
|
||||||
|
- **CRITICAL Requirements**: For all CRITICAL modules, define full `@PRE`, `@POST`, and (if UI) `@UX_STATE` contracts. **MUST** also define testing contracts: `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, and `@TEST_INVARIANT`.
|
||||||
|
- **Self-Review**:
|
||||||
|
- *Completeness*: Do `@PRE`/`@POST` cover edge cases identified in Research? Are test contracts present for CRITICAL?
|
||||||
|
- *Connectivity*: Do `@RELATION` tags form a coherent graph?
|
||||||
|
- *Compliance*: Does syntax match `[DEF:id:Type]` exactly and is it closed with `[/DEF:id:Type]`?
|
||||||
|
- **Output**: Write verified contracts to `contracts/modules.md`.
|
||||||
|
|
||||||
|
3. **Simulate Contract Usage**:
|
||||||
|
- Trace one key user scenario through the defined contracts to ensure data flow continuity.
|
||||||
|
- If a contract interface mismatch is found, fix it immediately.
|
||||||
|
|
||||||
|
4. **Generate API contracts**:
|
||||||
|
- Output OpenAPI/GraphQL schema to `/contracts/` for backend-frontend sync.
|
||||||
|
|
||||||
3. **Agent context update**:
|
3. **Agent context update**:
|
||||||
- Run `.specify/scripts/bash/update-agent-context.sh agy`
|
- Run `.specify/scripts/bash/update-agent-context.sh agy`
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
||||||
|
|
||||||
2. **Load design documents**: Read from FEATURE_DIR:
|
2. **Load design documents**: Read from FEATURE_DIR:
|
||||||
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
|
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities), ux_reference.md (experience source of truth)
|
||||||
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
||||||
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
||||||
|
|
||||||
@@ -70,6 +70,12 @@ The tasks.md should be immediately executable - each task must be specific enoug
|
|||||||
|
|
||||||
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
||||||
|
|
||||||
|
### UX Preservation (CRITICAL)
|
||||||
|
|
||||||
|
- **Source of Truth**: `ux_reference.md` is the absolute standard for the "feel" of the feature.
|
||||||
|
- **Violation Warning**: If any task would inherently violate the UX (e.g. "Remove progress bar to simplify code"), you **MUST** flag this to the user immediately.
|
||||||
|
- **Verification Task**: You **MUST** add a specific task at the end of each User Story phase: `- [ ] Txxx [USx] Verify implementation matches ux_reference.md (Happy Path & Errors)`
|
||||||
|
|
||||||
### Checklist Format (REQUIRED)
|
### Checklist Format (REQUIRED)
|
||||||
|
|
||||||
Every task MUST strictly follow this format:
|
Every task MUST strictly follow this format:
|
||||||
@@ -113,9 +119,12 @@ Every task MUST strictly follow this format:
|
|||||||
- If tests requested: Tests specific to that story
|
- If tests requested: Tests specific to that story
|
||||||
- Mark story dependencies (most stories should be independent)
|
- Mark story dependencies (most stories should be independent)
|
||||||
|
|
||||||
2. **From Contracts**:
|
2. **From Contracts (CRITICAL TIER)**:
|
||||||
- Map each interface contract → to the user story it serves
|
- Identify components marked as `@TIER: CRITICAL` in `contracts/modules.md`.
|
||||||
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
|
- For these components, **MUST** append the summary of `@PRE`, `@POST`, `@UX_STATE`, and test contracts (`@TEST_FIXTURE`, `@TEST_EDGE`) directly to the task description.
|
||||||
|
- Example: `- [ ] T005 [P] [US1] Implement Auth (CRITICAL: PRE: token exists, POST: returns User, TESTS: 2 edges) in src/auth.py`
|
||||||
|
- Map each contract/endpoint → to the user story it serves
|
||||||
|
- If tests requested: Each contract → contract test task [P] before implementation in that story's phase
|
||||||
|
|
||||||
3. **From Data Model**:
|
3. **From Data Model**:
|
||||||
- Map each entity to the user story(ies) that need it
|
- Map each entity to the user story(ies) that need it
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ Execute full testing cycle: analyze code for testable modules, write tests with
|
|||||||
|
|
||||||
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
||||||
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
||||||
3. **Use TEST_DATA fixtures** - For CRITICAL tier modules, read @TEST_ from semantics header
|
3. **Use TEST_FIXTURE fixtures** - For CRITICAL tier modules, read @TEST_FIXTURE from semantics header
|
||||||
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
||||||
|
|
||||||
## Execution Steps
|
## Execution Steps
|
||||||
@@ -40,7 +40,7 @@ Determine:
|
|||||||
- Identify completed implementation tasks (not test tasks)
|
- Identify completed implementation tasks (not test tasks)
|
||||||
- Extract file paths that need tests
|
- Extract file paths that need tests
|
||||||
|
|
||||||
**From .specify/memory/semantics.md:**
|
**From .ai/standards/semantics.md:**
|
||||||
- Read @TIER annotations for modules
|
- Read @TIER annotations for modules
|
||||||
- For CRITICAL modules: Read @TEST_ fixtures
|
- For CRITICAL modules: Read @TEST_ fixtures
|
||||||
|
|
||||||
@@ -52,8 +52,8 @@ Determine:
|
|||||||
|
|
||||||
Create coverage matrix:
|
Create coverage matrix:
|
||||||
|
|
||||||
| Module | File | Has Tests | TIER | TEST_DATA Available |
|
| Module | File | Has Tests | TIER | TEST_FIXTURE Available |
|
||||||
|--------|------|-----------|------|-------------------|
|
|--------|------|-----------|------|----------------------|
|
||||||
| ... | ... | ... | ... | ... |
|
| ... | ... | ... | ... | ... |
|
||||||
|
|
||||||
### 4. Write Tests (TDD Approach)
|
### 4. Write Tests (TDD Approach)
|
||||||
@@ -61,7 +61,7 @@ Create coverage matrix:
|
|||||||
For each module requiring tests:
|
For each module requiring tests:
|
||||||
|
|
||||||
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
||||||
2. **Read TEST_DATA**: If CRITICAL tier, read @TEST_ from semantic header
|
2. **Read TEST_FIXTURE**: If CRITICAL tier, read @TEST_FIXTURE from semantic header
|
||||||
3. **Write test**: Follow co-location strategy
|
3. **Write test**: Follow co-location strategy
|
||||||
- Python: `src/module/__tests__/test_module.py`
|
- Python: `src/module/__tests__/test_module.py`
|
||||||
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
||||||
@@ -102,6 +102,7 @@ describe('Component UX States', () => {
|
|||||||
// @UX_RECOVERY: Retry on error
|
// @UX_RECOVERY: Retry on error
|
||||||
it('should allow retry on error', async () => { ... });
|
it('should allow retry on error', async () => { ... });
|
||||||
});
|
});
|
||||||
|
// [/DEF:__tests__/test_Component:Module]
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Test Documentation
|
### 5. Test Documentation
|
||||||
@@ -170,7 +171,7 @@ Generate test execution report:
|
|||||||
|
|
||||||
- [ ] Fix failed tests
|
- [ ] Fix failed tests
|
||||||
- [ ] Add more coverage for [module]
|
- [ ] Add more coverage for [module]
|
||||||
- [ ] Review TEST_DATA fixtures
|
- [ ] Review TEST_FIXTURE fixtures
|
||||||
```
|
```
|
||||||
|
|
||||||
## Context for Testing
|
## Context for Testing
|
||||||
|
|||||||
103
.kilocode/workflows/audit-test.md
Normal file
103
.kilocode/workflows/audit-test.md
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
---
|
||||||
|
description: Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
||||||
|
---
|
||||||
|
|
||||||
|
**ROLE:** Elite Quality Assurance Architect and Red Teamer.
|
||||||
|
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
||||||
|
|
||||||
|
**INPUT:**
|
||||||
|
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`).
|
||||||
|
2. GENERATED TEST CODE.
|
||||||
|
|
||||||
|
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
||||||
|
|
||||||
|
1. **The Tautology (Self-Fulfilling Prophecy):**
|
||||||
|
- *Definition:* The test asserts hardcoded values against hardcoded values without executing the core business logic, or mocks the actual function being tested.
|
||||||
|
- *Example of Failure:* `assert 2 + 2 == 4` or mocking the class under test so that it returns exactly what the test asserts.
|
||||||
|
|
||||||
|
2. **The Logic Mirror (Echoing):**
|
||||||
|
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
||||||
|
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT` or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
||||||
|
|
||||||
|
3. **The "Happy Path" Illusion:**
|
||||||
|
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
||||||
|
- *Rule:* Every `@PRE` tag in the source contract MUST have a corresponding test that deliberately violates it and asserts the correct Exception/Error state.
|
||||||
|
|
||||||
|
4. **Missing Post-Condition Verification:**
|
||||||
|
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
||||||
|
|
||||||
|
5. **Missing Edge Case Coverage:**
|
||||||
|
- *Definition:* The test suite ignores `@TEST_EDGE` scenarios defined in the contract.
|
||||||
|
- *Rule:* Every `@TEST_EDGE` in the source contract MUST have a corresponding test case.
|
||||||
|
|
||||||
|
6. **Missing Invariant Verification:**
|
||||||
|
- *Definition:* The test suite does not verify `@TEST_INVARIANT` conditions.
|
||||||
|
- *Rule:* Every `@TEST_INVARIANT` MUST be verified by at least one test that attempts to break it.
|
||||||
|
|
||||||
|
7. **Missing UX State Testing (Svelte Components):**
|
||||||
|
- *Definition:* For Svelte components with `@UX_STATE`, the test suite does not verify state transitions.
|
||||||
|
- *Rule:* Every `@UX_STATE` transition MUST have a test verifying the visual/behavioral change.
|
||||||
|
- *Check:* `@UX_FEEDBACK` mechanisms (toast, shake, color) must be tested.
|
||||||
|
- *Check:* `@UX_RECOVERY` mechanisms (retry, clear input) must be tested.
|
||||||
|
|
||||||
|
### II. SEMANTIC PROTOCOL COMPLIANCE
|
||||||
|
|
||||||
|
Verify the test file follows GRACE-Poly semantics:
|
||||||
|
|
||||||
|
1. **Anchor Integrity:**
|
||||||
|
- Test file MUST start with `[DEF:__tests__/test_name:Module]`
|
||||||
|
- Test file MUST end with `[/DEF:__tests__/test_name:Module]`
|
||||||
|
|
||||||
|
2. **Required Tags:**
|
||||||
|
- `@RELATION: VERIFIES -> <path_to_source>` must be present
|
||||||
|
- `@PURPOSE:` must describe what is being tested
|
||||||
|
|
||||||
|
3. **TIER Alignment:**
|
||||||
|
- If source is `@TIER: CRITICAL`, test MUST cover all `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, `@TEST_INVARIANT`
|
||||||
|
- If source is `@TIER: STANDARD`, test MUST cover `@PRE` and `@POST`
|
||||||
|
- If source is `@TIER: TRIVIAL`, basic smoke test is acceptable
|
||||||
|
|
||||||
|
### III. AUDIT CHECKLIST
|
||||||
|
|
||||||
|
Evaluate the test code against these criteria:
|
||||||
|
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
||||||
|
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
||||||
|
3. **Test Contract Compliance:** Does the test follow the interface defined in `@TEST_CONTRACT`?
|
||||||
|
4. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_FIXTURE`?
|
||||||
|
5. **Edge Coverage:** Are all `@TEST_EDGE` scenarios tested?
|
||||||
|
6. **Invariant Coverage:** Are all `@TEST_INVARIANT` conditions verified?
|
||||||
|
7. **UX Coverage (if applicable):** Are all `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY` tested?
|
||||||
|
8. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
||||||
|
9. **Semantic Anchor:** Does the test file have proper `[DEF]` and `[/DEF]` anchors?
|
||||||
|
|
||||||
|
### IV. OUTPUT FORMAT
|
||||||
|
|
||||||
|
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
||||||
|
|
||||||
|
{
|
||||||
|
"verdict": "APPROVED" | "REJECTED",
|
||||||
|
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "MISSING_EDGES" | "MISSING_INVARIANTS" | "MISSING_UX_TESTS" | "SEMANTIC_VIOLATION" | "NONE",
|
||||||
|
"audit_details": {
|
||||||
|
"target_invoked": true/false,
|
||||||
|
"pre_conditions_tested": true/false,
|
||||||
|
"post_conditions_tested": true/false,
|
||||||
|
"test_fixture_used": true/false,
|
||||||
|
"edges_covered": true/false,
|
||||||
|
"invariants_verified": true/false,
|
||||||
|
"ux_states_tested": true/false,
|
||||||
|
"semantic_anchors_present": true/false
|
||||||
|
},
|
||||||
|
"coverage_summary": {
|
||||||
|
"total_edges": number,
|
||||||
|
"edges_tested": number,
|
||||||
|
"total_invariants": number,
|
||||||
|
"invariants_tested": number,
|
||||||
|
"total_ux_states": number,
|
||||||
|
"ux_states_tested": number
|
||||||
|
},
|
||||||
|
"tier_compliance": {
|
||||||
|
"source_tier": "CRITICAL" | "STANDARD" | "TRIVIAL",
|
||||||
|
"meets_tier_requirements": true/false
|
||||||
|
},
|
||||||
|
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
||||||
|
}
|
||||||
@@ -117,7 +117,11 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- **Validation checkpoints**: Verify each phase completion before proceeding
|
- **Validation checkpoints**: Verify each phase completion before proceeding
|
||||||
|
|
||||||
7. Implementation execution rules:
|
7. Implementation execution rules:
|
||||||
- **Strict Adherence**: Apply `.ai/standards/semantics.md` rules - every file must start with [DEF] header, include @TIER, and define contracts.
|
- **Strict Adherence**: Apply `.ai/standards/semantics.md` rules:
|
||||||
|
- Every file MUST start with a `[DEF:id:Type]` header and end with a closing `[/DEF:id:Type]` anchor.
|
||||||
|
- Include `@TIER` and define contracts (`@PRE`, `@POST`).
|
||||||
|
- For Svelte components, use `@UX_STATE`, `@UX_FEEDBACK`, `@UX_RECOVERY`, and explicitly declare reactivity with `@UX_REATIVITY: State: $state, Derived: $derived`.
|
||||||
|
- **Molecular Topology Logging**: Use prefixes `[EXPLORE]`, `[REASON]`, `[REFLECT]` in logs to trace logic.
|
||||||
- **CRITICAL Contracts**: If a task description contains a contract summary (e.g., `CRITICAL: PRE: ..., POST: ...`), these constraints are **MANDATORY** and must be strictly implemented in the code using guards/assertions (if applicable per protocol).
|
- **CRITICAL Contracts**: If a task description contains a contract summary (e.g., `CRITICAL: PRE: ..., POST: ...`), these constraints are **MANDATORY** and must be strictly implemented in the code using guards/assertions (if applicable per protocol).
|
||||||
- **Setup first**: Initialize project structure, dependencies, configuration
|
- **Setup first**: Initialize project structure, dependencies, configuration
|
||||||
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
||||||
|
|||||||
@@ -73,13 +73,13 @@ You **MUST** consider the user input before proceeding (if not empty).
|
|||||||
- Entity name, fields, relationships, validation rules.
|
- Entity name, fields, relationships, validation rules.
|
||||||
|
|
||||||
2. **Design & Verify Contracts (Semantic Protocol)**:
|
2. **Design & Verify Contracts (Semantic Protocol)**:
|
||||||
- **Drafting**: Define [DEF] Headers and Contracts for all new modules based on `.ai/standards/semantics.md`.
|
- **Drafting**: Define `[DEF:id:Type]` Headers, Contracts, and closing `[/DEF:id:Type]` for all new modules based on `.ai/standards/semantics.md`.
|
||||||
- **TIER Classification**: Explicitly assign `@TIER: [CRITICAL|STANDARD|TRIVIAL]` to each module.
|
- **TIER Classification**: Explicitly assign `@TIER: [CRITICAL|STANDARD|TRIVIAL]` to each module.
|
||||||
- **CRITICAL Requirements**: For all CRITICAL modules, define full `@PRE`, `@POST`, and (if UI) `@UX_STATE` contracts.
|
- **CRITICAL Requirements**: For all CRITICAL modules, define full `@PRE`, `@POST`, and (if UI) `@UX_STATE` contracts. **MUST** also define testing contracts: `@TEST_CONTRACT`, `@TEST_FIXTURE`, `@TEST_EDGE`, and `@TEST_INVARIANT`.
|
||||||
- **Self-Review**:
|
- **Self-Review**:
|
||||||
- *Completeness*: Do `@PRE`/`@POST` cover edge cases identified in Research?
|
- *Completeness*: Do `@PRE`/`@POST` cover edge cases identified in Research? Are test contracts present for CRITICAL?
|
||||||
- *Connectivity*: Do `@RELATION` tags form a coherent graph?
|
- *Connectivity*: Do `@RELATION` tags form a coherent graph?
|
||||||
- *Compliance*: Does syntax match `[DEF:id:Type]` exactly?
|
- *Compliance*: Does syntax match `[DEF:id:Type]` exactly and is it closed with `[/DEF:id:Type]`?
|
||||||
- **Output**: Write verified contracts to `contracts/modules.md`.
|
- **Output**: Write verified contracts to `contracts/modules.md`.
|
||||||
|
|
||||||
3. **Simulate Contract Usage**:
|
3. **Simulate Contract Usage**:
|
||||||
|
|||||||
@@ -121,8 +121,8 @@ Every task MUST strictly follow this format:
|
|||||||
|
|
||||||
2. **From Contracts (CRITICAL TIER)**:
|
2. **From Contracts (CRITICAL TIER)**:
|
||||||
- Identify components marked as `@TIER: CRITICAL` in `contracts/modules.md`.
|
- Identify components marked as `@TIER: CRITICAL` in `contracts/modules.md`.
|
||||||
- For these components, **MUST** append the summary of `@PRE`, `@POST`, and `@UX_STATE` contracts directly to the task description.
|
- For these components, **MUST** append the summary of `@PRE`, `@POST`, `@UX_STATE`, and test contracts (`@TEST_FIXTURE`, `@TEST_EDGE`) directly to the task description.
|
||||||
- Example: `- [ ] T005 [P] [US1] Implement Auth (CRITICAL: PRE: token exists, POST: returns User) in src/auth.py`
|
- Example: `- [ ] T005 [P] [US1] Implement Auth (CRITICAL: PRE: token exists, POST: returns User, TESTS: 2 edges) in src/auth.py`
|
||||||
- Map each contract/endpoint → to the user story it serves
|
- Map each contract/endpoint → to the user story it serves
|
||||||
- If tests requested: Each contract → contract test task [P] before implementation in that story's phase
|
- If tests requested: Each contract → contract test task [P] before implementation in that story's phase
|
||||||
|
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ Execute full testing cycle: analyze code for testable modules, write tests with
|
|||||||
|
|
||||||
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
1. **NEVER delete existing tests** - Only update if they fail due to bugs in the test or implementation
|
||||||
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
2. **NEVER duplicate tests** - Check existing tests first before creating new ones
|
||||||
3. **Use TEST_DATA fixtures** - For CRITICAL tier modules, read @TEST_DATA from .ai/standards/semantics.md
|
3. **Use TEST_FIXTURE fixtures** - For CRITICAL tier modules, read @TEST_FIXTURE from .ai/standards/semantics.md
|
||||||
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
4. **Co-location required** - Write tests in `__tests__` directories relative to the code being tested
|
||||||
|
|
||||||
## Execution Steps
|
## Execution Steps
|
||||||
@@ -52,8 +52,8 @@ Determine:
|
|||||||
|
|
||||||
Create coverage matrix:
|
Create coverage matrix:
|
||||||
|
|
||||||
| Module | File | Has Tests | TIER | TEST_DATA Available |
|
| Module | File | Has Tests | TIER | TEST_FIXTURE Available |
|
||||||
|--------|------|-----------|------|-------------------|
|
|--------|------|-----------|------|----------------------|
|
||||||
| ... | ... | ... | ... | ... |
|
| ... | ... | ... | ... | ... |
|
||||||
|
|
||||||
### 4. Write Tests (TDD Approach)
|
### 4. Write Tests (TDD Approach)
|
||||||
@@ -61,7 +61,7 @@ Create coverage matrix:
|
|||||||
For each module requiring tests:
|
For each module requiring tests:
|
||||||
|
|
||||||
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
1. **Check existing tests**: Scan `__tests__/` for duplicates
|
||||||
2. **Read TEST_DATA**: If CRITICAL tier, read @TEST_ from semantics header
|
2. **Read TEST_FIXTURE**: If CRITICAL tier, read @TEST_FIXTURE from semantics header
|
||||||
3. **Write test**: Follow co-location strategy
|
3. **Write test**: Follow co-location strategy
|
||||||
- Python: `src/module/__tests__/test_module.py`
|
- Python: `src/module/__tests__/test_module.py`
|
||||||
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
- Svelte: `src/lib/components/__tests__/test_component.test.js`
|
||||||
@@ -102,6 +102,7 @@ describe('Component UX States', () => {
|
|||||||
// @UX_RECOVERY: Retry on error
|
// @UX_RECOVERY: Retry on error
|
||||||
it('should allow retry on error', async () => { ... });
|
it('should allow retry on error', async () => { ... });
|
||||||
});
|
});
|
||||||
|
// [/DEF:__tests__/test_Component:Module]
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Test Documentation
|
### 5. Test Documentation
|
||||||
@@ -170,7 +171,7 @@ Generate test execution report:
|
|||||||
|
|
||||||
- [ ] Fix failed tests
|
- [ ] Fix failed tests
|
||||||
- [ ] Add more coverage for [module]
|
- [ ] Add more coverage for [module]
|
||||||
- [ ] Review TEST_DATA fixtures
|
- [ ] Review TEST_FIXTURE fixtures
|
||||||
```
|
```
|
||||||
|
|
||||||
## Context for Testing
|
## Context for Testing
|
||||||
|
|||||||
183429
backend/logs/app.log.1
183429
backend/logs/app.log.1
File diff suppressed because it is too large
Load Diff
@@ -57,7 +57,10 @@ class GitStatus(BaseModel):
|
|||||||
# [DEF:LastTask:DataClass]
|
# [DEF:LastTask:DataClass]
|
||||||
class LastTask(BaseModel):
|
class LastTask(BaseModel):
|
||||||
task_id: Optional[str] = None
|
task_id: Optional[str] = None
|
||||||
status: Optional[str] = Field(None, pattern="^RUNNING|SUCCESS|ERROR|WAITING_INPUT$")
|
status: Optional[str] = Field(
|
||||||
|
None,
|
||||||
|
pattern="^PENDING|RUNNING|SUCCESS|FAILED|ERROR|AWAITING_INPUT|WAITING_INPUT|AWAITING_MAPPING$",
|
||||||
|
)
|
||||||
# [/DEF:LastTask:DataClass]
|
# [/DEF:LastTask:DataClass]
|
||||||
|
|
||||||
# [DEF:DashboardItem:DataClass]
|
# [DEF:DashboardItem:DataClass]
|
||||||
|
|||||||
@@ -10,6 +10,7 @@
|
|||||||
|
|
||||||
# [SECTION: IMPORTS]
|
# [SECTION: IMPORTS]
|
||||||
from typing import List, Dict, Optional, Any
|
from typing import List, Dict, Optional, Any
|
||||||
|
from datetime import datetime
|
||||||
from ..core.superset_client import SupersetClient
|
from ..core.superset_client import SupersetClient
|
||||||
from ..core.task_manager.models import Task
|
from ..core.task_manager.models import Task
|
||||||
from ..services.git_service import GitService
|
from ..services.git_service import GitService
|
||||||
@@ -39,7 +40,7 @@ class ResourceService:
|
|||||||
# @RETURN: List[Dict] - Dashboards with git_status and last_task fields
|
# @RETURN: List[Dict] - Dashboards with git_status and last_task fields
|
||||||
# @RELATION: CALLS -> SupersetClient.get_dashboards_summary
|
# @RELATION: CALLS -> SupersetClient.get_dashboards_summary
|
||||||
# @RELATION: CALLS -> self._get_git_status_for_dashboard
|
# @RELATION: CALLS -> self._get_git_status_for_dashboard
|
||||||
# @RELATION: CALLS -> self._get_last_task_for_resource
|
# @RELATION: CALLS -> self._get_last_llm_task_for_dashboard
|
||||||
async def get_dashboards_with_status(
|
async def get_dashboards_with_status(
|
||||||
self,
|
self,
|
||||||
env: Any,
|
env: Any,
|
||||||
@@ -60,10 +61,11 @@ class ResourceService:
|
|||||||
git_status = self._get_git_status_for_dashboard(dashboard_id)
|
git_status = self._get_git_status_for_dashboard(dashboard_id)
|
||||||
dashboard_dict['git_status'] = git_status
|
dashboard_dict['git_status'] = git_status
|
||||||
|
|
||||||
# Get last task status
|
# Show status of the latest LLM validation for this dashboard.
|
||||||
last_task = self._get_last_task_for_resource(
|
last_task = self._get_last_llm_task_for_dashboard(
|
||||||
f"dashboard-{dashboard_id}",
|
dashboard_id,
|
||||||
tasks
|
env.id,
|
||||||
|
tasks,
|
||||||
)
|
)
|
||||||
dashboard_dict['last_task'] = last_task
|
dashboard_dict['last_task'] = last_task
|
||||||
|
|
||||||
@@ -72,6 +74,59 @@ class ResourceService:
|
|||||||
logger.info(f"[ResourceService][Coherence:OK] Fetched {len(result)} dashboards with status")
|
logger.info(f"[ResourceService][Coherence:OK] Fetched {len(result)} dashboards with status")
|
||||||
return result
|
return result
|
||||||
# [/DEF:get_dashboards_with_status:Function]
|
# [/DEF:get_dashboards_with_status:Function]
|
||||||
|
|
||||||
|
# [DEF:_get_last_llm_task_for_dashboard:Function]
|
||||||
|
# @PURPOSE: Get most recent LLM validation task for a dashboard in an environment
|
||||||
|
# @PRE: dashboard_id is a valid integer identifier
|
||||||
|
# @POST: Returns the newest llm_dashboard_validation task summary or None
|
||||||
|
# @PARAM: dashboard_id (int) - The dashboard ID
|
||||||
|
# @PARAM: env_id (Optional[str]) - Environment ID to match task params
|
||||||
|
# @PARAM: tasks (Optional[List[Task]]) - List of tasks to search
|
||||||
|
# @RETURN: Optional[Dict] - Task summary with task_id and status
|
||||||
|
def _get_last_llm_task_for_dashboard(
|
||||||
|
self,
|
||||||
|
dashboard_id: int,
|
||||||
|
env_id: Optional[str],
|
||||||
|
tasks: Optional[List[Task]] = None,
|
||||||
|
) -> Optional[Dict[str, Any]]:
|
||||||
|
if not tasks:
|
||||||
|
return None
|
||||||
|
|
||||||
|
dashboard_id_str = str(dashboard_id)
|
||||||
|
matched_tasks = []
|
||||||
|
|
||||||
|
for task in tasks:
|
||||||
|
if getattr(task, "plugin_id", None) != "llm_dashboard_validation":
|
||||||
|
continue
|
||||||
|
|
||||||
|
params = getattr(task, "params", {}) or {}
|
||||||
|
if str(params.get("dashboard_id")) != dashboard_id_str:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if env_id is not None:
|
||||||
|
task_env = params.get("environment_id") or params.get("env")
|
||||||
|
if str(task_env) != str(env_id):
|
||||||
|
continue
|
||||||
|
|
||||||
|
matched_tasks.append(task)
|
||||||
|
|
||||||
|
if not matched_tasks:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _task_time(task_obj: Any) -> datetime:
|
||||||
|
return (
|
||||||
|
getattr(task_obj, "started_at", None)
|
||||||
|
or getattr(task_obj, "finished_at", None)
|
||||||
|
or getattr(task_obj, "created_at", None)
|
||||||
|
or datetime.min
|
||||||
|
)
|
||||||
|
|
||||||
|
last_task = max(matched_tasks, key=_task_time)
|
||||||
|
return {
|
||||||
|
"task_id": str(getattr(last_task, "id", "")),
|
||||||
|
"status": str(getattr(last_task, "status", "")),
|
||||||
|
}
|
||||||
|
# [/DEF:_get_last_llm_task_for_dashboard:Function]
|
||||||
|
|
||||||
# [DEF:get_datasets_with_status:Function]
|
# [DEF:get_datasets_with_status:Function]
|
||||||
# @PURPOSE: Fetch datasets from environment with mapping progress and last task status
|
# @PURPOSE: Fetch datasets from environment with mapping progress and last task status
|
||||||
|
|||||||
@@ -664,10 +664,14 @@
|
|||||||
switch (status.toLowerCase()) {
|
switch (status.toLowerCase()) {
|
||||||
case "running":
|
case "running":
|
||||||
return '<svg class="animate-spin" width="16" height="16" viewBox="0 0 24 24"><path fill="currentColor" d="M12 2a10 10 0 1 0 10 10A10 10 0 0 0 12 2zm0 18a8 8 0 1 1 8-8 8 0 0 1-8 8z"/></svg>';
|
return '<svg class="animate-spin" width="16" height="16" viewBox="0 0 24 24"><path fill="currentColor" d="M12 2a10 10 0 1 0 10 10A10 10 0 0 0 12 2zm0 18a8 8 0 1 1 8-8 8 0 0 1-8 8z"/></svg>';
|
||||||
|
case "pending":
|
||||||
|
return '<svg class="animate-spin" width="16" height="16" viewBox="0 0 24 24"><path fill="currentColor" d="M12 2a10 10 0 1 0 10 10A10 10 0 0 0 12 2zm0 18a8 8 0 1 1 8-8 8 0 0 1-8 8z"/></svg>';
|
||||||
case "success":
|
case "success":
|
||||||
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M9 16.17L4.83 12l-1.42 1.41L9 19 21 7l-1.41-1.41L9 16.17z"/></svg>';
|
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M9 16.17L4.83 12l-1.42 1.41L9 19 21 7l-1.41-1.41L9 16.17z"/></svg>';
|
||||||
|
case "failed":
|
||||||
case "error":
|
case "error":
|
||||||
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z"/></svg>';
|
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z"/></svg>';
|
||||||
|
case "awaiting_input":
|
||||||
case "waiting_input":
|
case "waiting_input":
|
||||||
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z"/></svg>';
|
return '<svg width="16" height="16" viewBox="0 0 24 24" fill="currentColor"><path d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z"/></svg>';
|
||||||
default:
|
default:
|
||||||
@@ -928,10 +932,16 @@
|
|||||||
<span class="ml-1">
|
<span class="ml-1">
|
||||||
{#if dashboard.lastTask.status.toLowerCase() === "running"}
|
{#if dashboard.lastTask.status.toLowerCase() === "running"}
|
||||||
{$t.dashboard?.task_running}
|
{$t.dashboard?.task_running}
|
||||||
|
{:else if dashboard.lastTask.status.toLowerCase() === "pending"}
|
||||||
|
{$t.dashboard?.task_running}
|
||||||
{:else if dashboard.lastTask.status.toLowerCase() === "success"}
|
{:else if dashboard.lastTask.status.toLowerCase() === "success"}
|
||||||
{$t.dashboard?.task_done}
|
{$t.dashboard?.task_done}
|
||||||
|
{:else if dashboard.lastTask.status.toLowerCase() === "failed"}
|
||||||
|
{$t.dashboard?.task_failed}
|
||||||
{:else if dashboard.lastTask.status.toLowerCase() === "error"}
|
{:else if dashboard.lastTask.status.toLowerCase() === "error"}
|
||||||
{$t.dashboard?.task_failed}
|
{$t.dashboard?.task_failed}
|
||||||
|
{:else if dashboard.lastTask.status.toLowerCase() === "awaiting_input"}
|
||||||
|
{$t.dashboard?.task_waiting}
|
||||||
{:else if dashboard.lastTask.status.toLowerCase() === "waiting_input"}
|
{:else if dashboard.lastTask.status.toLowerCase() === "waiting_input"}
|
||||||
{$t.dashboard?.task_waiting}
|
{$t.dashboard?.task_waiting}
|
||||||
{/if}
|
{/if}
|
||||||
|
|||||||
@@ -94,28 +94,35 @@ OUTPUT_COMPRESSED_MD = ".ai/PROJECT_MAP.md"
|
|||||||
OUTPUT_MODULE_MAP_MD = ".ai/MODULE_MAP.md"
|
OUTPUT_MODULE_MAP_MD = ".ai/MODULE_MAP.md"
|
||||||
REPORTS_DIR = "semantics/reports"
|
REPORTS_DIR = "semantics/reports"
|
||||||
|
|
||||||
# Tier-based mandatory tags
|
# Tier-based mandatory tags aligned with .ai/standards/semantics.md
|
||||||
TIER_MANDATORY_TAGS = {
|
TIER_MANDATORY_TAGS = {
|
||||||
Tier.CRITICAL: {
|
Tier.CRITICAL: {
|
||||||
"Module": ["PURPOSE", "LAYER", "SEMANTICS", "TIER", "INVARIANT"],
|
"Module": ["PURPOSE", "LAYER", "SEMANTICS", "TIER", "INVARIANT"],
|
||||||
"Component": ["PURPOSE", "LAYER", "SEMANTICS", "TIER", "INVARIANT"],
|
"Component": ["PURPOSE", "LAYER", "SEMANTICS", "TIER", "INVARIANT", "UX_STATE", "UX_FEEDBACK", "UX_RECOVERY", "UX_REATIVITY"],
|
||||||
"Function": ["PURPOSE", "PRE", "POST"],
|
"Function": ["PURPOSE", "PRE", "POST", "TEST_CONTRACT", "TEST_FIXTURE", "TEST_EDGE", "TEST_INVARIANT"],
|
||||||
"Class": ["PURPOSE", "TIER"]
|
"Class": ["PURPOSE", "TIER", "INVARIANT"],
|
||||||
|
"Store": ["PURPOSE", "TIER", "INVARIANT"]
|
||||||
},
|
},
|
||||||
Tier.STANDARD: {
|
Tier.STANDARD: {
|
||||||
"Module": ["PURPOSE", "LAYER", "SEMANTICS", "TIER"],
|
"Module": ["PURPOSE", "LAYER", "SEMANTICS", "TIER"],
|
||||||
"Component": ["PURPOSE", "LAYER", "SEMANTICS", "TIER"],
|
"Component": ["PURPOSE", "LAYER", "SEMANTICS", "TIER", "UX_STATE"],
|
||||||
"Function": ["PURPOSE", "PRE", "POST"],
|
"Function": ["PURPOSE", "PRE", "POST"],
|
||||||
"Class": ["PURPOSE", "TIER"]
|
"Class": ["PURPOSE", "TIER"],
|
||||||
|
"Store": ["PURPOSE", "TIER"]
|
||||||
},
|
},
|
||||||
Tier.TRIVIAL: {
|
Tier.TRIVIAL: {
|
||||||
"Module": ["PURPOSE", "TIER"],
|
"Module": ["PURPOSE"],
|
||||||
"Component": ["PURPOSE", "TIER"],
|
"Component": ["PURPOSE"],
|
||||||
"Function": ["PURPOSE"],
|
"Function": ["PURPOSE"],
|
||||||
"Class": ["PURPOSE", "TIER"]
|
"Class": ["PURPOSE"],
|
||||||
|
"Store": ["PURPOSE"]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
ALLOWED_RELATION_PREDICATES = {
|
||||||
|
"DEPENDS_ON", "CALLS", "INHERITS", "IMPLEMENTS", "DISPATCHES", "BINDS_TO"
|
||||||
|
}
|
||||||
|
|
||||||
# Tier-based belief state requirements
|
# Tier-based belief state requirements
|
||||||
TIER_BELIEF_REQUIRED = {
|
TIER_BELIEF_REQUIRED = {
|
||||||
Tier.CRITICAL: True,
|
Tier.CRITICAL: True,
|
||||||
@@ -252,7 +259,17 @@ class SemanticEntity:
|
|||||||
self.start_line
|
self.start_line
|
||||||
))
|
))
|
||||||
|
|
||||||
# 3. Check for Belief State Logging based on TIER
|
# 3. Validate relation predicates against GRACE-Poly allowlist
|
||||||
|
for rel in self.relations:
|
||||||
|
rel_type = rel.get("type", "").upper()
|
||||||
|
if rel_type and rel_type not in ALLOWED_RELATION_PREDICATES:
|
||||||
|
self.compliance_issues.append(ComplianceIssue(
|
||||||
|
f"Invalid @RELATION predicate: {rel_type}. Allowed: {', '.join(sorted(ALLOWED_RELATION_PREDICATES))}",
|
||||||
|
Severity.ERROR if tier == Tier.CRITICAL else Severity.WARNING,
|
||||||
|
self.start_line
|
||||||
|
))
|
||||||
|
|
||||||
|
# 4. Check for Belief State Logging based on TIER
|
||||||
if self.type == "Function":
|
if self.type == "Function":
|
||||||
belief_required = TIER_BELIEF_REQUIRED.get(tier, False)
|
belief_required = TIER_BELIEF_REQUIRED.get(tier, False)
|
||||||
if belief_required:
|
if belief_required:
|
||||||
@@ -270,7 +287,7 @@ class SemanticEntity:
|
|||||||
self.start_line
|
self.start_line
|
||||||
))
|
))
|
||||||
|
|
||||||
# 4. Check for @INVARIANT in CRITICAL tier
|
# 5. Check for @INVARIANT in CRITICAL tier
|
||||||
if tier == Tier.CRITICAL and self.type in ["Module", "Component", "Class"]:
|
if tier == Tier.CRITICAL and self.type in ["Module", "Component", "Class"]:
|
||||||
if "INVARIANT" not in [k.upper() for k in self.tags.keys()]:
|
if "INVARIANT" not in [k.upper() for k in self.tags.keys()]:
|
||||||
self.compliance_issues.append(ComplianceIssue(
|
self.compliance_issues.append(ComplianceIssue(
|
||||||
@@ -338,7 +355,7 @@ def get_patterns(lang: str) -> Dict[str, Pattern]:
|
|||||||
if lang == "python":
|
if lang == "python":
|
||||||
return {
|
return {
|
||||||
"anchor_start": re.compile(r"#\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
"anchor_start": re.compile(r"#\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
||||||
"anchor_end": re.compile(r"#\s*\[/DEF:(?P<name>[\w\.]+)(?::\w+)?\]"),
|
"anchor_end": re.compile(r"#\s*\[/DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
||||||
"tag": re.compile(r"#\s*@(?P<tag>[A-Z_]+):\s*(?P<value>.*)"),
|
"tag": re.compile(r"#\s*@(?P<tag>[A-Z_]+):\s*(?P<value>.*)"),
|
||||||
"relation": re.compile(r"#\s*@RELATION:\s*(?P<type>\w+)\s*->\s*(?P<target>.*)"),
|
"relation": re.compile(r"#\s*@RELATION:\s*(?P<type>\w+)\s*->\s*(?P<target>.*)"),
|
||||||
"func_def": re.compile(r"^\s*(async\s+)?def\s+(?P<name>\w+)"),
|
"func_def": re.compile(r"^\s*(async\s+)?def\s+(?P<name>\w+)"),
|
||||||
@@ -348,11 +365,11 @@ def get_patterns(lang: str) -> Dict[str, Pattern]:
|
|||||||
else:
|
else:
|
||||||
return {
|
return {
|
||||||
"html_anchor_start": re.compile(r"<!--\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]\s*-->"),
|
"html_anchor_start": re.compile(r"<!--\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]\s*-->"),
|
||||||
"html_anchor_end": re.compile(r"<!--\s*\[/DEF:(?P<name>[\w\.]+)(?::\w+)?\]\s*-->"),
|
"html_anchor_end": re.compile(r"<!--\s*\[/DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]\s*-->"),
|
||||||
"js_anchor_start": re.compile(r"//\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
"js_anchor_start": re.compile(r"//\s*\[DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
||||||
"js_anchor_end": re.compile(r"//\s*\[/DEF:(?P<name>[\w\.]+)(?::\w+)?\]"),
|
"js_anchor_end": re.compile(r"//\s*\[/DEF:(?P<name>[\w\.]+):(?P<type>\w+)\]"),
|
||||||
"html_tag": re.compile(r"@(?P<tag>[A-Z_]+):\s*(?P<value>.*)"),
|
"html_tag": re.compile(r"@(?P<tag>[A-Z_]+):\s*(?P<value>.*)"),
|
||||||
"jsdoc_tag": re.compile(r"\*\s*@(?P<tag>[a-zA-Z]+)\s+(?P<value>.*)"),
|
"jsdoc_tag": re.compile(r"\*\s*@(?P<tag>[A-Za-z_]+)\s*:?\s*(?P<value>.*)"),
|
||||||
"relation": re.compile(r"//\s*@RELATION:\s*(?P<type>\w+)\s*->\s*(?P<target>.*)"),
|
"relation": re.compile(r"//\s*@RELATION:\s*(?P<type>\w+)\s*->\s*(?P<target>.*)"),
|
||||||
"func_def": re.compile(r"^\s*(export\s+)?(async\s+)?function\s+(?P<name>\w+)"),
|
"func_def": re.compile(r"^\s*(export\s+)?(async\s+)?function\s+(?P<name>\w+)"),
|
||||||
"console_log": re.compile(r"console\.log\s*\(\s*['\"`]\[[\w_]+\]\[[A-Za-z0-9_:]+\]"),
|
"console_log": re.compile(r"console\.log\s*\(\s*['\"`]\[[\w_]+\]\[[A-Za-z0-9_:]+\]"),
|
||||||
@@ -550,22 +567,23 @@ def parse_file(full_path: str, rel_path: str, lang: str) -> Tuple[List[SemanticE
|
|||||||
|
|
||||||
if match_end:
|
if match_end:
|
||||||
name = match_end.group("name")
|
name = match_end.group("name")
|
||||||
|
type_ = match_end.group("type")
|
||||||
|
|
||||||
if not stack:
|
if not stack:
|
||||||
issues.append(ComplianceIssue(
|
issues.append(ComplianceIssue(
|
||||||
f"{rel_path}:{lineno} Found closing anchor [/DEF:{name}] without opening anchor.",
|
f"{rel_path}:{lineno} Found closing anchor [/DEF:{name}:{type_}] without opening anchor.",
|
||||||
Severity.ERROR,
|
Severity.ERROR,
|
||||||
lineno
|
lineno
|
||||||
))
|
))
|
||||||
continue
|
continue
|
||||||
|
|
||||||
top = stack[-1]
|
top = stack[-1]
|
||||||
if top.name == name:
|
if top.name == name and top.type == type_:
|
||||||
top.end_line = lineno
|
top.end_line = lineno
|
||||||
stack.pop()
|
stack.pop()
|
||||||
else:
|
else:
|
||||||
issues.append(ComplianceIssue(
|
issues.append(ComplianceIssue(
|
||||||
f"{rel_path}:{lineno} Mismatched closing anchor. Expected [/DEF:{top.name}], found [/DEF:{name}].",
|
f"{rel_path}:{lineno} Mismatched closing anchor. Expected [/DEF:{top.name}:{top.type}], found [/DEF:{name}:{type_}].",
|
||||||
Severity.ERROR,
|
Severity.ERROR,
|
||||||
lineno
|
lineno
|
||||||
))
|
))
|
||||||
@@ -1039,7 +1057,7 @@ class SemanticMapGenerator:
|
|||||||
|
|
||||||
# Write Relations
|
# Write Relations
|
||||||
for rel in entity.relations:
|
for rel in entity.relations:
|
||||||
if rel['type'] in ['DEPENDS_ON', 'CALLS', 'INHERITS', 'IMPLEMENTS', 'DISPATCHES']:
|
if rel['type'] in ['DEPENDS_ON', 'CALLS', 'INHERITS', 'IMPLEMENTS', 'DISPATCHES', 'BINDS_TO']:
|
||||||
f.write(f"{indent} - 🔗 {rel['type']} -> `{rel['target']}`\n")
|
f.write(f"{indent} - 🔗 {rel['type']} -> `{rel['target']}`\n")
|
||||||
|
|
||||||
if level < 3:
|
if level < 3:
|
||||||
|
|||||||
65
run.sh
65
run.sh
@@ -61,6 +61,71 @@ validate_env() {
|
|||||||
|
|
||||||
validate_env
|
validate_env
|
||||||
|
|
||||||
|
# Database connectivity preflight
|
||||||
|
check_database() {
|
||||||
|
# Keep resolution order aligned with backend/src/core/database.py defaults.
|
||||||
|
local DB_URL="${DATABASE_URL:-${POSTGRES_URL:-postgresql+psycopg2://postgres:postgres@localhost:5432/ss_tools}}"
|
||||||
|
|
||||||
|
# SQLite does not require external service.
|
||||||
|
if [[ "$DB_URL" == sqlite* ]]; then
|
||||||
|
echo "Database preflight: sqlite detected, skipping PostgreSQL connectivity check."
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
local DB_HOST DB_PORT
|
||||||
|
read -r DB_HOST DB_PORT < <(
|
||||||
|
python3 - "$DB_URL" <<'PY'
|
||||||
|
import sys
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
url = sys.argv[1]
|
||||||
|
if "://" not in url:
|
||||||
|
print("localhost 5432")
|
||||||
|
raise SystemExit(0)
|
||||||
|
|
||||||
|
# Support SQLAlchemy schemes like postgresql+psycopg2://...
|
||||||
|
scheme, rest = url.split("://", 1)
|
||||||
|
parsed = urlparse(f"{scheme.split('+', 1)[0]}://{rest}")
|
||||||
|
host = parsed.hostname or "localhost"
|
||||||
|
port = parsed.port or 5432
|
||||||
|
print(f"{host} {port}")
|
||||||
|
PY
|
||||||
|
)
|
||||||
|
|
||||||
|
local check_cmd
|
||||||
|
check_cmd='import socket,sys; socket.create_connection((sys.argv[1], int(sys.argv[2])), timeout=1).close()'
|
||||||
|
|
||||||
|
if python3 -c "$check_cmd" "$DB_HOST" "$DB_PORT" >/dev/null 2>&1; then
|
||||||
|
echo "Database preflight: reachable at ${DB_HOST}:${DB_PORT}."
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Database preflight: cannot connect to ${DB_HOST}:${DB_PORT}."
|
||||||
|
|
||||||
|
# For local development defaults, attempt to auto-start bundled PostgreSQL.
|
||||||
|
if [ "$DB_HOST" = "localhost" ] && [ "$DB_PORT" = "5432" ] && command -v docker >/dev/null 2>&1; then
|
||||||
|
if [ -f "docker-compose.yml" ]; then
|
||||||
|
echo "Attempting to start local PostgreSQL via docker compose (service: db)..."
|
||||||
|
docker compose up -d db || true
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
for _ in {1..20}; do
|
||||||
|
if python3 -c "$check_cmd" "$DB_HOST" "$DB_PORT" >/dev/null 2>&1; then
|
||||||
|
echo "Database preflight: reachable at ${DB_HOST}:${DB_PORT}."
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Error: PostgreSQL is unavailable at ${DB_HOST}:${DB_PORT}."
|
||||||
|
echo "Run: docker compose up -d db"
|
||||||
|
echo "Or set DATABASE_URL/POSTGRES_URL to a reachable database."
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
check_database
|
||||||
|
|
||||||
# Backend dependency management
|
# Backend dependency management
|
||||||
setup_backend() {
|
setup_backend() {
|
||||||
if [ "$SKIP_INSTALL" = true ]; then
|
if [ "$SKIP_INSTALL" = true ]; then
|
||||||
|
|||||||
Reference in New Issue
Block a user