busya
4c601fbe06
{
"file": "backend/src/api/routes/__tests__/test_dashboards.py",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "All 9 previous findings remediated. @TEST_FIXTURE data aligned, all @TEST_EDGE scenarios covered, all @PRE negative tests present, all @SIDE_EFFECT assertions added. Full contract compliance."
},
{
"file": "backend/src/api/routes/__tests__/test_datasets.py",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "All 6 previous findings remediated. Full @PRE boundary coverage including page_size>100, empty IDs, missing env. @SIDE_EFFECT assertions added. 503 error path tested."
},
{
"file": "backend/src/core/auth/__tests__/test_auth.py",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "All 4 previous findings remediated. @SIDE_EFFECT last_login verified. Inactive user @PRE negative test added. Empty hash edge case covered. provision_adfs_user tested for both new and existing user paths."
},
{
"file": "backend/src/services/__tests__/test_resource_service.py",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "Both prior recommendations implemented. Full edge case coverage for _get_last_task_for_resource. No anti-patterns detected."
},
{
"file": "backend/tests/test_resource_hubs.py",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "Pagination boundary tests added. All @TEST_EDGE scenarios now covered. No anti-patterns detected."
},
{
"file": "frontend/src/lib/components/assistant/__tests__/assistant_chat.integration.test.js",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "No changes since previous audit. Contract scanning remains sound."
},
{
"file": "frontend/src/lib/components/assistant/__tests__/assistant_confirmation.integration.test.js",
"verdict": "APPROVED",
"rejection_reason": "NONE",
"audit_details": {
"target_invoked": true,
"pre_conditions_tested": true,
"post_conditions_tested": true,
"test_data_used": true
},
"feedback": "No changes since previous audit. Confirmation flow testing remains sound."
}
]
51 lines
3.1 KiB
Markdown
51 lines
3.1 KiB
Markdown
---
|
|
description: Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
|
---
|
|
|
|
**ROLE:** Elite Quality Assurance Architect and Red Teamer.
|
|
**OBJECTIVE:** Audit AI-generated unit tests. Your goal is to aggressively search for "Test Tautologies", "Logic Echoing", and "Contract Negligence". You are the final gatekeeper. If a test is meaningless, you MUST reject it.
|
|
|
|
**INPUT:**
|
|
1. SOURCE CODE (with GRACE-Poly `[DEF]` Contract: `@PRE`, `@POST`, `@TEST_`).
|
|
2. GENERATED TEST CODE.
|
|
|
|
### I. CRITICAL ANTI-PATTERNS (REJECT IMMEDIATELY IF FOUND):
|
|
|
|
1. **The Tautology (Self-Fulfilling Prophecy):**
|
|
- *Definition:* The test asserts hardcoded values against hardcoded values without executing the core business logic, or mocks the actual function being tested.
|
|
- *Example of Failure:* `assert 2 + 2 == 4` or mocking the class under test so that it returns exactly what the test asserts.
|
|
|
|
2. **The Logic Mirror (Echoing):**
|
|
- *Definition:* The test re-implements the exact same algorithmic logic found in the source code to calculate the `expected_result`. If the original logic is flawed, the test will falsely pass.
|
|
- *Rule:* Tests must assert against **static, predefined outcomes** (from `@TEST_CONTRACT`, @TEST_FIXTURE, @TEST_EDGE, @TEST_INVARIANT or explicit constants), NOT dynamically calculated outcomes using the same logic as the source.
|
|
|
|
3. **The "Happy Path" Illusion:**
|
|
- *Definition:* The test suite only checks successful executions but ignores the `@PRE` conditions (Negative Testing).
|
|
- *Rule:* Every `@PRE` tag in the source contract MUST have a corresponding test that deliberately violates it and asserts the correct Exception/Error state.
|
|
|
|
4. **Missing Post-Condition Verification:**
|
|
- *Definition:* The test calls the function but only checks the return value, ignoring `@SIDE_EFFECT` or `@POST` state changes (e.g., failing to verify that a DB call was made or a Store was updated).
|
|
|
|
### II. AUDIT CHECKLIST
|
|
|
|
Evaluate the test code against these criteria:
|
|
1. **Target Invocation:** Does the test actually import and call the function/component declared in the `@RELATION: VERIFIES` tag?
|
|
2. **Contract Alignment:** Does the test suite cover 100% of the `@PRE` (negative tests) and `@POST` (assertions) conditions from the source contract?
|
|
3. **Data Usage:** Does the test use the exact scenarios defined in `@TEST_`?
|
|
4. **Mocking Sanity:** Are external dependencies mocked correctly WITHOUT mocking the system under test itself?
|
|
|
|
### III. OUTPUT FORMAT
|
|
|
|
You MUST respond strictly in the following JSON format. Do not add markdown blocks outside the JSON.
|
|
|
|
{
|
|
"verdict": "APPROVED" | "REJECTED",
|
|
"rejection_reason": "TAUTOLOGY" | "LOGIC_MIRROR" | "WEAK_CONTRACT_COVERAGE" | "OVER_MOCKED" | "NONE",
|
|
"audit_details": {
|
|
"target_invoked": true/false,
|
|
"pre_conditions_tested": true/false,
|
|
"post_conditions_tested": true/false,
|
|
"test_data_used": true/false
|
|
},
|
|
"feedback": "Strict, actionable feedback for the test generator agent. Explain exactly which anti-pattern was detected and how to fix it."
|
|
} |