119 lines
6.4 KiB
Markdown
119 lines
6.4 KiB
Markdown
# Implementation Plan: LLM Analysis & Documentation Plugins
|
|
|
|
**Branch**: `017-llm-analysis-plugin` | **Date**: 2026-01-28 | **Spec**: [specs/017-llm-analysis-plugin/spec.md](spec.md)
|
|
**Input**: Feature specification from `/specs/017-llm-analysis-plugin/spec.md`
|
|
|
|
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
|
|
|
|
## Summary
|
|
|
|
This feature implements two new plugins for the `ss-tools` platform: `DashboardValidationPlugin` and `DocumentationPlugin`. The `DashboardValidationPlugin` uses multimodal LLMs to analyze dashboard health by combining screenshots (via headless browser or API) and execution logs. The `DocumentationPlugin` automates the generation of metadata descriptions for datasets and columns using LLMs. Additionally, the feature integrates LLM capabilities into the Git plugin for commit message generation and adds a configuration interface for managing multiple LLM providers (OpenAI, OpenRouter, Kilo).
|
|
|
|
## Technical Context
|
|
|
|
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
|
|
**Primary Dependencies**:
|
|
- Backend: `openai` (or compatible SDK), `playwright` (for headless screenshots), `fastapi` (existing), `pydantic` (existing), `sqlalchemy` (existing)
|
|
- Frontend: `svelte` (existing), `tailwindcss` (existing)
|
|
**Storage**: SQLite (existing `tasks.db` for results, `auth.db` for permissions, `mappings.db` or new `plugins.db` for provider config/metadata)
|
|
**Testing**: `pytest` (Backend), `vitest` (Frontend)
|
|
**Target Platform**: Linux server (existing environment)
|
|
**Project Type**: Web application (FastAPI + SvelteKit)
|
|
**Performance Goals**: Dashboard validation < 90s, Documentation generation < 30s per asset
|
|
**Constraints**: Must integrate with existing `PluginBase` and `TaskManager`. Secure storage for API keys.
|
|
**Scale/Scope**: Support for configurable LLM providers, scheduled tasks, and notification integration.
|
|
|
|
## Implementation Notes
|
|
|
|
### Screenshot Capture Implementation
|
|
The screenshot service uses Playwright with Chrome DevTools Protocol (CDP) to avoid font loading timeouts in headless mode. Key implementation details:
|
|
- **Full Page Capture**: Uses CDP `Page.captureScreenshot` with `captureBeyondViewport: true` and `fromSurface: true`
|
|
- **Tab Switching**: Implements recursive tab switching to trigger lazy-loaded chart rendering on multi-tab dashboards
|
|
- **Authentication**: Uses direct UI login flow (navigating to `/login/` and filling credentials) instead of API cookie injection for better reliability
|
|
- **Resolution**: 1920px width with dynamic full page height calculation
|
|
|
|
### Authentication Flow
|
|
The service authenticates via Playwright UI login rather than API authentication:
|
|
1. Navigate to `/login/` page
|
|
2. Fill username/password fields (supports multiple Superset versions)
|
|
3. Click login button
|
|
4. Verify successful redirect to `/superset/welcome/`
|
|
5. Navigate to dashboard with valid session
|
|
|
|
This approach is more reliable than API-to-UI cookie injection which was causing 403 Forbidden errors.
|
|
|
|
## Constitution Check
|
|
|
|
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
|
|
|
- [x] **I. Semantic Protocol Compliance**: Will follow `[DEF]`, `@RELATION`, and file structure standards.
|
|
- [x] **II. Causal Validity**: Contracts/Interfaces will be defined before implementation.
|
|
- [x] **III. Immutability of Architecture**: No changes to core immutable layers planned; extending via Plugin architecture.
|
|
- [x] **IV. Design by Contract**: All new plugins and services will use DbC.
|
|
- [x] **V. Belief State Logging**: Will use standard logging for belief state.
|
|
- [x] **VI. Fractal Complexity Limit**: Plugins will be modular and adhere to complexity limits.
|
|
- [x] **VII. Everything is a Plugin**: Implemented as `DashboardValidationPlugin` and `DocumentationPlugin`.
|
|
- [x] **VIII. Unified Frontend Experience**: Will use existing Svelte components and i18n.
|
|
- [x] **IX. Security & Access Control**: Will define permissions (e.g., `plugin:llm:validate`, `plugin:llm:configure`).
|
|
|
|
## Project Structure
|
|
|
|
### Documentation (this feature)
|
|
|
|
```text
|
|
specs/017-llm-analysis-plugin/
|
|
├── plan.md # This file (/speckit.plan command output)
|
|
├── research.md # Phase 0 output (/speckit.plan command)
|
|
├── data-model.md # Phase 1 output (/speckit.plan command)
|
|
├── quickstart.md # Phase 1 output (/speckit.plan command)
|
|
├── contracts/ # Phase 1 output (/speckit.plan command)
|
|
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
|
|
```
|
|
|
|
### Source Code (repository root)
|
|
|
|
```text
|
|
backend/
|
|
├── src/
|
|
│ ├── plugins/
|
|
│ │ ├── llm_analysis/ # New Plugin Directory
|
|
│ │ │ ├── __init__.py
|
|
│ │ │ ├── plugin.py # Implements DashboardValidationPlugin & DocumentationPlugin
|
|
│ │ │ ├── service.py # LLM interaction logic + ScreenshotService (CDP, tab switching)
|
|
│ │ │ ├── scheduler.py # Scheduled validation task handler
|
|
│ │ │ └── models.py # Pydantic models for LLM config/results
|
|
│ │ └── git/ # Existing Git Plugin
|
|
│ │ └── ... # Update for commit message generation
|
|
│ ├── api/
|
|
│ │ └── routes/
|
|
│ │ └── llm.py # New routes for LLM config/proxy if needed
|
|
│ └── services/
|
|
│ └── llm_provider.py # Shared service for LLM provider management
|
|
└── tests/
|
|
└── plugins/
|
|
└── test_llm_analysis.py
|
|
|
|
frontend/
|
|
├── src/
|
|
│ ├── components/
|
|
│ │ ├── llm/
|
|
│ │ │ ├── ProviderConfig.svelte # Configuration UI
|
|
│ │ │ └── ValidationReport.svelte # Result viewer
|
|
│ │ └── git/
|
|
│ │ └── CommitMessageGen.svelte # UI component for commit generation
|
|
│ └── routes/
|
|
│ └── admin/
|
|
│ └── settings/
|
|
│ └── llm/ # Settings page route
|
|
```
|
|
|
|
**Structure Decision**: Web application structure with backend plugins and frontend components, consistent with existing project layout.
|
|
|
|
## Complexity Tracking
|
|
|
|
> **Fill ONLY if Constitution Check has violations that must be justified**
|
|
|
|
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
|
|-----------|------------|-------------------------------------|
|
|
| N/A | | |
|