Implementation Plan: LLM Analysis & Documentation Plugins

Branch: 017-llm-analysis-plugin | Date: 2026-01-28 | Spec: specs/017-llm-analysis-plugin/spec.md Input: Feature specification from /specs/017-llm-analysis-plugin/spec.md

Note: This template is filled in by the /speckit.plan command. See .specify/templates/commands/plan.md for the execution workflow.

Summary

This feature implements two new plugins for the ss-tools platform: DashboardValidationPlugin and DocumentationPlugin. The DashboardValidationPlugin uses multimodal LLMs to analyze dashboard health by combining screenshots (via headless browser or API) and execution logs. The DocumentationPlugin automates the generation of metadata descriptions for datasets and columns using LLMs. Additionally, the feature integrates LLM capabilities into the Git plugin for commit message generation and adds a configuration interface for managing multiple LLM providers (OpenAI, OpenRouter, Kilo).

Technical Context

Language/Version: Python 3.9+ (Backend), Node.js 18+ (Frontend) Primary Dependencies:

Backend: openai (or compatible SDK), playwright (for headless screenshots), fastapi (existing), pydantic (existing), sqlalchemy (existing)
Frontend: svelte (existing), tailwindcss (existing) Storage: SQLite (existing tasks.db for results, auth.db for permissions, mappings.db or new plugins.db for provider config/metadata) Testing: pytest (Backend), vitest (Frontend) Target Platform: Linux server (existing environment) Project Type: Web application (FastAPI + SvelteKit) Performance Goals: Dashboard validation < 90s, Documentation generation < 30s per asset Constraints: Must integrate with existing PluginBase and TaskManager. Secure storage for API keys. Scale/Scope: Support for configurable LLM providers, scheduled tasks, and notification integration.

Implementation Notes

Screenshot Capture Implementation

The screenshot service uses Playwright with Chrome DevTools Protocol (CDP) to avoid font loading timeouts in headless mode. Key implementation details:

Full Page Capture: Uses CDP Page.captureScreenshot with captureBeyondViewport: true and fromSurface: true
Tab Switching: Implements recursive tab switching to trigger lazy-loaded chart rendering on multi-tab dashboards
Authentication: Uses direct UI login flow (navigating to /login/ and filling credentials) instead of API cookie injection for better reliability
Resolution: 1920px width with dynamic full page height calculation

Authentication Flow

The service authenticates via Playwright UI login rather than API authentication:

Navigate to /login/ page
Fill username/password fields (supports multiple Superset versions)
Click login button
Verify successful redirect to /superset/welcome/
Navigate to dashboard with valid session

This approach is more reliable than API-to-UI cookie injection which was causing 403 Forbidden errors.

Constitution Check

GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.

I. Semantic Protocol Compliance: Will follow [DEF], @RELATION, and file structure standards.
II. Causal Validity: Contracts/Interfaces will be defined before implementation.
III. Immutability of Architecture: No changes to core immutable layers planned; extending via Plugin architecture.
IV. Design by Contract: All new plugins and services will use DbC.
V. Belief State Logging: Will use standard logging for belief state.
VI. Fractal Complexity Limit: Plugins will be modular and adhere to complexity limits.
VII. Everything is a Plugin: Implemented as DashboardValidationPlugin and DocumentationPlugin.
VIII. Unified Frontend Experience: Will use existing Svelte components and i18n.
IX. Security & Access Control: Will define permissions (e.g., plugin:llm:validate, plugin:llm:configure).

Project Structure

Documentation (this feature)

specs/017-llm-analysis-plugin/
├── plan.md              # This file (/speckit.plan command output)
├── research.md          # Phase 0 output (/speckit.plan command)
├── data-model.md        # Phase 1 output (/speckit.plan command)
├── quickstart.md        # Phase 1 output (/speckit.plan command)
├── contracts/           # Phase 1 output (/speckit.plan command)
└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)

Source Code (repository root)

backend/
├── src/
│   ├── plugins/
│   │   ├── llm_analysis/       # New Plugin Directory
│   │   │   ├── __init__.py
│   │   │   ├── plugin.py       # Implements DashboardValidationPlugin & DocumentationPlugin
│   │   │   ├── service.py      # LLM interaction logic + ScreenshotService (CDP, tab switching)
│   │   │   ├── scheduler.py    # Scheduled validation task handler
│   │   │   └── models.py       # Pydantic models for LLM config/results
│   │   └── git/                # Existing Git Plugin
│   │       └── ...             # Update for commit message generation
│   ├── api/
│   │   └── routes/
│   │       └── llm.py          # New routes for LLM config/proxy if needed
│   └── services/
│       └── llm_provider.py     # Shared service for LLM provider management
└── tests/
    └── plugins/
        └── test_llm_analysis.py

frontend/
├── src/
│   ├── components/
│   │   ├── llm/
│   │   │   ├── ProviderConfig.svelte   # Configuration UI
│   │   │   └── ValidationReport.svelte # Result viewer
│   │   └── git/
│   │       └── CommitMessageGen.svelte # UI component for commit generation
│   └── routes/
│       └── admin/
│           └── settings/
│               └── llm/                # Settings page route

Structure Decision: Web application structure with backend plugins and frontend components, consistent with existing project layout.

Complexity Tracking

Fill ONLY if Constitution Check has violations that must be justified

Violation	Why Needed	Simpler Alternative Rejected Because
N/A

6.4 KiB Raw Blame History