busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(llm-plugin): switch to environment API for log retrieval

Browse Source 
- Replace local backend.log reading with Superset API /log/ fetch
- Update DashboardValidationPlugin to use SupersetClient
- Filter logs by dashboard_id and last 24 hours
- Update spec FR-006 to reflect API usage
This commit is contained in:
busya
2026-02-06 17:57:25 +03:00
parent f018b97ed2
commit 7de96c17c4
2 changed files with 88 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
specs/017-llm-analysis-plugin/spec.md
View File

@@ -82,6 +82,8 @@ As a Developer, I want the system to suggest commit messages based on changes di
- What happens if the dashboard screenshot cannot be generated? (System should proceed with logs only or fail depending on configuration).
- What happens if the context (logs/metadata) exceeds the LLM's token limit? (System should truncate or summarize input).
- How does the system handle missing API keys? (Task should fail immediately with a configuration error).
- What happens if the dashboard has multiple tabs with lazy-loaded charts? (System must switch through all tabs recursively to trigger chart rendering before capture).
- What happens if Playwright encounters font loading timeouts in headless mode? (System must use CDP `Page.captureScreenshot` to bypass Playwright's internal timeout mechanism).
## Requirements *(mandatory)*
@@ -92,9 +94,11 @@ As a Developer, I want the system to suggest commit messages based on changes di
- **FR-028**: The system MUST mask all API keys in the UI and logs, displaying only the last 4 characters (e.g., `sk-...1234`). [Security]
- **FR-003**: System MUST implement a `DashboardValidationPlugin` that integrates with the existing `PluginBase` architecture.
- **FR-004**: `DashboardValidationPlugin` MUST accept a dashboard identifier as input.
- **FR-005**: `DashboardValidationPlugin` MUST be capable of retrieving a visual representation (screenshot) of the dashboard. The visual representation MUST be a PNG or JPEG image with a minimum resolution of 1280x720px to ensure legibility for the LLM. [Clarity]
- **FR-005**: `DashboardValidationPlugin` MUST be capable of retrieving a visual representation (screenshot) of the dashboard. The visual representation MUST be a PNG image with a resolution of 1920px width and full page height to ensure all dashboard content is captured. [Clarity]
- **FR-016**: System MUST support configurable screenshot strategies: 'Headless Browser' (default, high accuracy) and 'API Thumbnail' (fallback/fast).
- **FR-006**: `DashboardValidationPlugin` MUST retrieve recent execution logs associated with the dashboard, limited to the last 100 lines or 24 hours (whichever is smaller) to prevent token overflow. [Reliability]
- **FR-030**: The screenshot capture MUST use Playwright with Chrome DevTools Protocol (CDP) to avoid font loading timeouts in headless mode.
- **FR-031**: The screenshot capture MUST implement recursive tab switching to trigger lazy-loaded chart rendering on multi-tab dashboards before capturing.
- **FR-006**: `DashboardValidationPlugin` MUST retrieve recent execution logs associated with the dashboard from the Environment API (e.g., `/api/v1/log/`), limited to the last 100 lines or 24 hours (whichever is smaller) to prevent token overflow. [Reliability]
- **FR-007**: `DashboardValidationPlugin` MUST combine visual and text data to prompt a Multimodal LLM for analysis. The analysis output MUST be structured as a JSON object containing `status` (Pass/Fail), `issues` (list of strings), and `summary` (text) to enable structured UI presentation. [Clarity]
- **FR-008**: System MUST implement a `DocumentationPlugin` (or similar) for documenting datasets and dashboards.
- **FR-009**: `DocumentationPlugin` MUST retrieve schema and metadata for the target asset.
@@ -130,4 +134,6 @@ As a Developer, I want the system to suggest commit messages based on changes di
- **SC-001**: Users can successfully configure and validate a connection to at least one LLM provider.
- **SC-002**: A dashboard validation task completes within 90 seconds (assuming standard LLM latency).
- **SC-003**: The system successfully processes a multimodal prompt (image + text) and returns a structured analysis.
- **SC-004**: Generated documentation for a standard dataset contains descriptions for at least 80% of the columns (based on LLM capability, but pipeline must support it).
- **SC-004**: Generated documentation for a standard dataset contains descriptions for at least 80% of the columns (based on LLM capability, but pipeline must support it).
- **SC-005**: Screenshots capture full dashboard content including all tabs (1920px width, full height) without font loading timeouts.
- **SC-006**: Analysis results are displayed in task logs with clear `[ANALYSIS_SUMMARY]` and `[ANALYSIS_ISSUE]` markers for easy parsing.