tasks ready

2026-01-28 18:30:23 +03:00
parent 0e0e26e2f7
commit b2bbd73439
14 changed files with 664 additions and 2 deletions
--- a/.kilocode/rules/specify-rules.md
+++ b/.kilocode/rules/specify-rules.md
@@ -32,6 +32,7 @@ Auto-generated from all feature plans. Last updated: 2025-12-19
 - Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend) (015-frontend-nav-redesign)
 - N/A (UI reorganization and API integration) (015-frontend-nav-redesign)
 - SQLite (`auth.db`) for Users, Roles, Permissions, and Mappings. (016-multi-user-auth)
+- SQLite (existing `tasks.db` for results, `auth.db` for permissions, `mappings.db` or new `plugins.db` for provider config/metadata) (017-llm-analysis-plugin)

 - Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui)

@@ -52,9 +53,9 @@ cd src; pytest; ruff check .
 Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions

 ## Recent Changes
+- 017-llm-analysis-plugin: Added Python 3.9+ (Backend), Node.js 18+ (Frontend)
 - 016-multi-user-auth: Added Python 3.9+ (Backend), Node.js 18+ (Frontend)
 - 015-frontend-nav-redesign: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
- 014-file-storage-ui: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend)


 <!-- MANUAL ADDITIONS START -->
--- a/backend/backend/auth.db
+++ b/backend/backend/auth.db
--- a/backend/backend/git_repos/12
+++ b/backend/backend/git_repos/12
--- a/specs/017-llm-analysis-plugin/checklists/data-integrity.md
+++ b/specs/017-llm-analysis-plugin/checklists/data-integrity.md
@@ -0,0 +1,20 @@
+# Checklist: Data Integrity Requirements Quality
+
+**Purpose**: Validate requirement quality for data generation and persistence.
+**Feature**: `017-llm-analysis-plugin`
+**Created**: 2026-01-28
+
+## Requirement Completeness
+- [ ] CHK001 Are requirements defined for mapping generated documentation to the correct metadata fields? [Completeness, Spec §FR-017]
+- [ ] CHK002 Are requirements specified for handling schema changes during documentation generation? [Completeness, Gap]
+- [ ] CHK003 Are requirements defined for validating the format of the generated commit message? [Completeness, Gap]
+- [ ] CHK004 Are requirements specified for concurrent documentation updates? [Completeness, Gap]
+
+## Requirement Clarity
+- [ ] CHK005 Is "structured description" defined with specific markdown or text constraints? [Clarity, Spec §SC-004]
+- [ ] CHK006 Are "recent commit history" requirements defined by count or time window? [Clarity, Spec §FR-012]
+
+## Edge Case Coverage
+- [ ] CHK007 Are requirements defined for when the LLM generates hallucinated column names? [Edge Case, Gap]
+- [ ] CHK008 Are rollback requirements defined if a metadata update fails partially? [Edge Case, Gap]
+- [ ] CHK009 Are requirements defined for handling empty or null schema inputs? [Edge Case, Gap]
--- a/specs/017-llm-analysis-plugin/checklists/reliability-security.md
+++ b/specs/017-llm-analysis-plugin/checklists/reliability-security.md
@@ -0,0 +1,24 @@
+# Checklist: Reliability & Security Requirements Quality
+
+**Purpose**: Validate requirement quality for LLM integration reliability and security.
+**Feature**: `017-llm-analysis-plugin`
+**Created**: 2026-01-28
+
+## Requirement Completeness
+- [ ] CHK001 Are retry strategies (count, backoff) defined for all external LLM API calls? [Completeness, Spec §FR-018]
+- [ ] CHK002 Are timeout thresholds specified for long-running validation tasks? [Completeness, Gap]
+- [ ] CHK003 Are encryption requirements defined for storing API keys at rest? [Completeness, Spec §FR-002]
+- [ ] CHK004 Are masking requirements defined for displaying API keys in the UI? [Completeness, Gap]
+- [ ] CHK005 Is the fallback behavior defined when the primary screenshot method (Headless) fails? [Completeness, Spec §FR-016]
+- [ ] CHK006 Are requirements defined for handling rate limits from LLM providers? [Completeness, Gap]
+- [ ] CHK007 Are data privacy requirements specified regarding what dashboard data (screenshots, logs) is sent to the LLM? [Completeness, Gap]
+
+## Requirement Clarity
+- [ ] CHK008 Is "securely store" quantified with specific encryption standards (e.g., AES-256)? [Clarity, Spec §FR-002]
+- [ ] CHK009 Are "recent execution logs" defined by specific time window or line count? [Clarity, Spec §FR-006]
+- [ ] CHK010 Is "automatic retry logic" defined with specific backoff parameters? [Clarity, Spec §FR-018]
+
+## Edge Case Coverage
+- [ ] CHK011 Are requirements defined for scenarios where the LLM provider is completely unreachable? [Edge Case, Gap]
+- [ ] CHK012 Are requirements defined for handling "token limit exceeded" errors with large logs? [Edge Case, Gap]
+- [ ] CHK013 Are requirements defined for invalid/expired API keys during task execution? [Edge Case, Gap]
--- a/specs/017-llm-analysis-plugin/checklists/requirements.md
+++ b/specs/017-llm-analysis-plugin/checklists/requirements.md
@@ -0,0 +1,34 @@
+# Specification Quality Checklist: LLM Analysis & Documentation Plugins
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-01-28
+**Feature**: [specs/017-llm-analysis-plugin/spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
--- a/specs/017-llm-analysis-plugin/checklists/ux-usability.md
+++ b/specs/017-llm-analysis-plugin/checklists/ux-usability.md
@@ -0,0 +1,24 @@
+# Checklist: UX & Usability Requirements Quality
+
+**Purpose**: Validate requirement quality for user interaction and workflows.
+**Feature**: `017-llm-analysis-plugin`
+**Created**: 2026-01-28
+
+## Requirement Completeness
+- [ ] CHK001 Are requirements defined for the "Validate" button state during active execution (loading/disabled)? [Completeness, Gap]
+- [ ] CHK002 Are feedback requirements defined for successful/failed connection tests in Settings? [Completeness, Gap]
+- [ ] CHK003 Are requirements specified for viewing historical validation reports? [Completeness, Spec §US1]
+- [ ] CHK004 Are requirements defined for the notification content layout (summary vs. link)? [Completeness, Spec §FR-015]
+- [ ] CHK005 Are requirements defined for the "Generate Commit" interaction flow (modal vs. inline)? [Completeness, Spec §US4]
+
+## Requirement Clarity
+- [ ] CHK006 Is "visual representation" defined with resolution or format constraints? [Clarity, Spec §FR-005]
+- [ ] CHK007 Are "structured analysis" outputs defined with specific UI presentation requirements? [Clarity, Spec §SC-003]
+
+## Consistency
+- [ ] CHK008 Do the new "Validate" and "Generate Docs" actions follow existing UI patterns (icons, placement)? [Consistency, Gap]
+- [ ] CHK009 Are error messages consistent with the application's existing error handling standards? [Consistency, Gap]
+
+## Edge Case Coverage
+- [ ] CHK010 Are requirements defined for when a dashboard cannot be rendered (e.g., 404)? [Edge Case, Gap]
+- [ ] CHK011 Are requirements defined for when the generated commit message is empty or invalid? [Edge Case, Gap]
--- a/specs/017-llm-analysis-plugin/contracts/llm-api.yaml
+++ b/specs/017-llm-analysis-plugin/contracts/llm-api.yaml
@@ -0,0 +1,96 @@
+openapi: 3.0.0
+info:
+  title: LLM Analysis Plugin API
+  version: 1.0.0
+paths:
+  /api/settings/llm/providers:
+    get:
+      summary: List configured LLM providers
+      responses:
+        '200':
+          description: List of providers
+          content:
+            application/json:
+              schema:
+                type: array
+                items:
+                  $ref: '#/components/schemas/LLMProviderConfig'
+    post:
+      summary: Create or update a provider
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/LLMProviderConfigInput'
+      responses:
+        '200':
+          description: Created/Updated provider
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/LLMProviderConfig'
+
+  /api/git/generate-message:
+    post:
+      summary: Generate commit message from diff
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                diff:
+                  type: string
+                files:
+                  type: array
+                  items:
+                    type: string
+                recent_commits:
+                  type: array
+                  items:
+                    type: string
+      responses:
+        '200':
+          description: Generated message
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+
+components:
+  schemas:
+    LLMProviderConfig:
+      type: object
+      properties:
+        id:
+          type: string
+          format: uuid
+        provider_type:
+          type: string
+          enum: [openai, openrouter, kilo]
+        name:
+          type: string
+        base_url:
+          type: string
+        default_model:
+          type: string
+        is_active:
+          type: boolean
+    LLMProviderConfigInput:
+      type: object
+      properties:
+        provider_type:
+          type: string
+        name:
+          type: string
+        base_url:
+          type: string
+        api_key:
+          type: string
+        default_model:
+          type: string
--- a/specs/017-llm-analysis-plugin/data-model.md
+++ b/specs/017-llm-analysis-plugin/data-model.md
@@ -0,0 +1,51 @@
+# Data Model: LLM Analysis Plugin
+
+**Feature**: `017-llm-analysis-plugin`
+
+## Entities
+
+### LLMProviderConfig
+Stores configuration for different LLM providers.
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| id | UUID | Yes | Unique identifier |
+| provider_type | Enum | Yes | `openai`, `openrouter`, `kilo` |
+| name | String | Yes | Display name (e.g., "Production GPT-4") |
+| base_url | String | Yes | API Endpoint URL |
+| api_key | String | Yes | Encrypted API Key |
+| default_model | String | Yes | Model identifier (e.g., `gpt-4o`) |
+| is_active | Boolean | Yes | Whether this provider is currently enabled |
+
+### ValidationResult
+Stores the outcome of a dashboard validation task.
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| id | UUID | Yes | Unique identifier |
+| dashboard_id | String | Yes | Reference to the dashboard |
+| timestamp | DateTime | Yes | When the validation ran |
+| status | Enum | Yes | `PASS`, `WARN`, `FAIL` |
+| screenshot_path | String | No | Path to the captured screenshot (if stored) |
+| issues | JSON | Yes | List of detected issues `[{severity, message, location}]` |
+| raw_response | Text | No | Full LLM response for debugging |
+
+### TaskConfiguration (Extension)
+Extends existing Task model to support validation-specific params.
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| task_type | String | Yes | `dashboard_validation` |
+| parameters | JSON | Yes | `{dashboard_id, provider_id, screenshot_strategy}` |
+
+## API Contracts
+
+See `contracts/` directory for OpenAPI specifications.
+
+### Key Interactions
+
+1. **Configure Provider**: `POST /api/settings/llm/providers`
+2. **Trigger Validation**: `POST /api/tasks/dispatch` (payload: `{type: "dashboard_validation", ...}`)
+3. **Get Results**: `GET /api/tasks/{task_id}/result` (or via WebSocket stream)
+4. **Generate Documentation**: `POST /api/tasks/dispatch` (payload: `{type: "dataset_documentation", ...}`)
+5. **Generate Commit**: `POST /api/git/generate-message`
--- a/specs/017-llm-analysis-plugin/plan.md
+++ b/specs/017-llm-analysis-plugin/plan.md
@@ -0,0 +1,98 @@
+# Implementation Plan: LLM Analysis & Documentation Plugins
+
+**Branch**: `017-llm-analysis-plugin` | **Date**: 2026-01-28 | **Spec**: [specs/017-llm-analysis-plugin/spec.md](spec.md)
+**Input**: Feature specification from `/specs/017-llm-analysis-plugin/spec.md`
+
+**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
+
+## Summary
+
+This feature implements two new plugins for the `ss-tools` platform: `DashboardValidationPlugin` and `DocumentationPlugin`. The `DashboardValidationPlugin` uses multimodal LLMs to analyze dashboard health by combining screenshots (via headless browser or API) and execution logs. The `DocumentationPlugin` automates the generation of metadata descriptions for datasets and columns using LLMs. Additionally, the feature integrates LLM capabilities into the Git plugin for commit message generation and adds a configuration interface for managing multiple LLM providers (OpenAI, OpenRouter, Kilo).
+
+## Technical Context
+
+**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
+**Primary Dependencies**:
+- Backend: `openai` (or compatible SDK), `playwright` (for headless screenshots), `fastapi` (existing), `pydantic` (existing), `sqlalchemy` (existing)
+- Frontend: `svelte` (existing), `tailwindcss` (existing)
+**Storage**: SQLite (existing `tasks.db` for results, `auth.db` for permissions, `mappings.db` or new `plugins.db` for provider config/metadata)
+**Testing**: `pytest` (Backend), `vitest` (Frontend)
+**Target Platform**: Linux server (existing environment)
+**Project Type**: Web application (FastAPI + SvelteKit)
+**Performance Goals**: Dashboard validation < 90s, Documentation generation < 30s per asset
+**Constraints**: Must integrate with existing `PluginBase` and `TaskManager`. Secure storage for API keys.
+**Scale/Scope**: Support for configurable LLM providers, scheduled tasks, and notification integration.
+
+## Constitution Check
+
+*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
+
+- [x] **I. Semantic Protocol Compliance**: Will follow `[DEF]`, `@RELATION`, and file structure standards.
+- [x] **II. Causal Validity**: Contracts/Interfaces will be defined before implementation.
+- [x] **III. Immutability of Architecture**: No changes to core immutable layers planned; extending via Plugin architecture.
+- [x] **IV. Design by Contract**: All new plugins and services will use DbC.
+- [x] **V. Belief State Logging**: Will use standard logging for belief state.
+- [x] **VI. Fractal Complexity Limit**: Plugins will be modular and adhere to complexity limits.
+- [x] **VII. Everything is a Plugin**: Implemented as `DashboardValidationPlugin` and `DocumentationPlugin`.
+- [x] **VIII. Unified Frontend Experience**: Will use existing Svelte components and i18n.
+- [x] **IX. Security & Access Control**: Will define permissions (e.g., `plugin:llm:validate`, `plugin:llm:configure`).
+
+## Project Structure
+
+### Documentation (this feature)
+
+```text
+specs/017-llm-analysis-plugin/
+├── plan.md              # This file (/speckit.plan command output)
+├── research.md          # Phase 0 output (/speckit.plan command)
+├── data-model.md        # Phase 1 output (/speckit.plan command)
+├── quickstart.md        # Phase 1 output (/speckit.plan command)
+├── contracts/           # Phase 1 output (/speckit.plan command)
+└── tasks.md             # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
+```
+
+### Source Code (repository root)
+
+```text
+backend/
+├── src/
+│   ├── plugins/
+│   │   ├── llm_analysis/       # New Plugin Directory
+│   │   │   ├── __init__.py
+│   │   │   ├── plugin.py       # Implements DashboardValidationPlugin & DocumentationPlugin
+│   │   │   ├── service.py      # LLM interaction logic
+│   │   │   └── models.py       # Pydantic models for LLM config/results
+│   │   └── git/                # Existing Git Plugin
+│   │       └── ...             # Update for commit message generation
+│   ├── api/
+│   │   └── routes/
+│   │       └── llm.py          # New routes for LLM config/proxy if needed
+│   └── services/
+│       └── llm_provider.py     # Shared service for LLM provider management
+└── tests/
+    └── plugins/
+        └── test_llm_analysis.py
+
+frontend/
+├── src/
+│   ├── components/
+│   │   ├── llm/
+│   │   │   ├── ProviderConfig.svelte   # Configuration UI
+│   │   │   └── ValidationReport.svelte # Result viewer
+│   │   └── git/
+│   │       └── CommitMessageGen.svelte # UI component for commit generation
+│   └── routes/
+│       └── admin/
+│           └── settings/
+│               └── llm/                # Settings page route
+```
+
+**Structure Decision**: Web application structure with backend plugins and frontend components, consistent with existing project layout.
+
+## Complexity Tracking
+
+> **Fill ONLY if Constitution Check has violations that must be justified**
+
+| Violation | Why Needed | Simpler Alternative Rejected Because |
+|-----------|------------|-------------------------------------|
+| N/A | | |
--- a/specs/017-llm-analysis-plugin/quickstart.md
+++ b/specs/017-llm-analysis-plugin/quickstart.md
@@ -0,0 +1,42 @@
+# Quickstart: LLM Analysis Plugin
+
+## Prerequisites
+
+1. An API Key from OpenAI, OpenRouter, or Kilo.
+2. (Optional) Playwright browsers installed for high-fidelity screenshots:
+   ```bash
+   playwright install chromium
+   ```
+
+## Configuration
+
+1. Navigate to **Settings > LLM Providers**.
+2. Click **"Add Provider"**.
+3. Select type (e.g., `OpenRouter`).
+4. Enter your API Key and Base URL (e.g., `https://openrouter.ai/api/v1`).
+5. Set a default model (e.g., `anthropic/claude-3.5-sonnet`).
+6. Click **Save**.
+
+## Usage
+
+### Dashboard Validation
+
+1. Go to **Environments**.
+2. Select a dashboard from the list.
+3. Click **"Validate"**.
+4. Wait for the task to complete and view the report in the **Task History** panel.
+
+### Dataset Documentation
+
+1. Go to **Tools > Documentation**.
+2. Select a dataset.
+3. Click **"Generate Docs"**.
+4. Review the generated descriptions and changes will be applied to the metadata.
+
+### Git Commit Generation
+
+1. Make changes to files in the repository.
+2. Go to the **Git** plugin tab.
+3. Stage your files.
+4. Click the **"✨" (Magic)** button next to the commit message box.
+5. Review and edit the suggested message before committing.
--- a/specs/017-llm-analysis-plugin/research.md
+++ b/specs/017-llm-analysis-plugin/research.md
@@ -0,0 +1,48 @@
+# Research: LLM Analysis & Documentation Plugins
+
+**Feature**: `017-llm-analysis-plugin`
+
+## 1. LLM Provider Integration
+
+**Decision**: Use a unified `LLMProviderService` that abstracts OpenAI-compatible APIs.
+**Rationale**: OpenRouter, Kilo, and OpenAI all support the standard OpenAI API format. This simplifies implementation by using the `openai` Python SDK and changing the `base_url` and `api_key` dynamically based on configuration.
+**Alternatives Considered**: LangChain (too heavy/complex for this specific scope), custom HTTP requests (reinventing the wheel).
+
+## 2. Dashboard Screenshot Capture
+
+**Decision**: Implement a `ScreenshotService` with a strategy pattern supporting `Playwright` (primary) and `Superset API` (fallback).
+**Rationale**:
+- **Playwright**: Provides the most accurate "user-view" render, handling JS-heavy charts that API thumbnails might miss or render poorly. Requires a browser binary.
+- **Superset API**: Faster, lightweight, but relies on Superset's internal thumbnail cache which can be stale.
+**Implementation Detail**: The service will check configuration. If 'Headless' is selected, it launches a Playwright context, logs in (using a service account or session cookie), navigates to the dashboard, waits for network idle, and captures.
+
+## 3. Multimodal Analysis Prompting
+
+**Decision**: Use a structured prompt template that accepts base64-encoded images and text logs.
+**Rationale**: Models like GPT-4o and Claude 3.5 Sonnet (via OpenRouter) support this natively.
+**Prompt Structure**:
+- System: "You are a Data Observability Expert..."
+- User Image: [Base64 Screenshot]
+- User Text: "Recent Logs: \n[Log Snippets]..."
+- Output Format: JSON (Status, Issues[], Recommendations[])
+
+## 4. Documentation Persistence
+
+**Decision**: Update `Dataset` and `Column` models in the existing metadata database (likely `mappings.db` or the main application DB if integrated).
+**Rationale**: Keeps documentation co-located with the assets.
+**Mechanism**: The `DocumentationPlugin` will fetch schema, generate markdown, and execute an `UPDATE` operation on the relevant tables/fields.
+
+## 5. Git Commit Integration
+
+**Decision**: Add a REST endpoint `/api/git/generate-message` used by the frontend Git component.
+**Rationale**: Keeps the heavy lifting (LLM call, diff processing) on the backend. The frontend simply sends the list of staged files and a diff summary (truncated if necessary).
+
+## 6. Security & Storage
+
+**Decision**: Encrypt API keys at rest using the existing `Fernet` or similar encryption mechanism used for database credentials.
+**Rationale**: API keys are sensitive. They should not be stored in plain text in `config.json` or the DB.
+
+## 7. Retry Logic
+
+**Decision**: Use `tenacity` library for decorators on LLM service methods.
+**Rationale**: Standard, robust, declarative retry logic (exponential backoff) as required by FR-018.
--- a/specs/017-llm-analysis-plugin/spec.md
+++ b/specs/017-llm-analysis-plugin/spec.md
@@ -0,0 +1,122 @@
+# Feature Specification: LLM Analysis & Documentation Plugins
+
+**Feature Branch**: `017-llm-analysis-plugin`
+**Created**: 2026-01-28
+**Status**: Draft
+**Input**: User description: "LLM Dashboard Validation Plugin для интеграции LLM в ss-tools. Плагин должен поддерживать анализ корректности работы дашбордов через мультимодальную LLM (скриншот + логи). Второй плагин должен заниматься документированием датасетов и дашбордов, используя LLM. Возможно включение в создание текста коммитов в плагине Git Поддержка провайдеров: OpenRouter, Kilo Provider, OpenAI API. Интеграция с существующей PluginBase архитектурой, Task Manager, WebSocket логами."
+
+## Clarifications
+
+### Session 2026-01-28
+- Q: Notification Content Strategy → A: **Summary with Link**: Include status (Pass/Fail), key issues count, and a direct link to the full report in the UI.
+- Q: Dashboard Screenshot Source → A: **Hybrid (Configurable)**: Support both Headless Browser (accurate) and API/Thumbnail (fast) methods, allowing admin configuration.
+- Q: Dataset Documentation Output Format → A: **Direct Object Update**: Update the description fields of the dataset and its columns directly within the dataset object (persisted to backend/metadata).
+- Q: Git Commit Message Context → A: **Diff + Recent History**: Send diff, file names, and the last 3 commit messages to match style.
+- Q: LLM Failure Handling → A: **Retry then Fail**: Automatically retry 3 times with exponential backoff before failing.
+
+## User Scenarios & Testing *(mandatory)*
+
+### User Story 1 - Dashboard Health Analysis (Priority: P1)
+
+As a Data Engineer, I want to automatically analyze a dashboard's status using visual and log data directly from the Environments interface so that I can identify rendering issues or data errors without manual inspection.
+
+**Why this priority**: Core value proposition of the feature. Enables automated quality assurance.
+
+**Independent Test**: Can be tested by selecting a dashboard in the Environment list and clicking "Validate", or by scheduling a validation task.
+
+**Acceptance Scenarios**:
+
+1. **Given** I am on the Environments page (Dashboard list), **When** I select a dashboard and click "Validate", **Then** the system triggers a validation task with the dashboard's context.
+2. **Given** the validation task is running, **When** it completes, **Then** I see the analysis report (visual + logs) in the task output history.
+3. **Given** I want regular checks, **When** I configure a schedule for the validation task (similar to Backup plugin), **Then** the system runs the check automatically at the specified interval.
+4. **Given** a validation issue is found, **When** the task completes, **Then** the system sends a notification (Email/Pulse) if configured.
+
+---
+
+### User Story 2 - Automated Dataset Documentation (Priority: P1)
+
+As a Data Steward, I want to generate documentation for datasets and dashboards using LLMs so that I can maintain up-to-date metadata with minimal manual effort.
+
+**Why this priority**: significantly reduces maintenance overhead for data governance.
+
+**Independent Test**: Can be tested by selecting a dataset and triggering the documentation task, then checking if a description is generated.
+
+**Acceptance Scenarios**:
+
+1. **Given** a dataset identifier, **When** I run the Documentation task, **Then** the system fetches the dataset's schema and metadata.
+2. **Given** the metadata is fetched, **When** sent to the LLM, **Then** a structured description/documentation text is returned.
+3. **Given** the documentation is generated, **When** the task completes, **Then** the result is available for review (e.g., in the task log or saved to a file/db).
+
+---
+
+### User Story 3 - LLM Provider Configuration (Priority: P1)
+
+As an Administrator, I want to configure different LLM providers (OpenAI, OpenRouter, Kilo) so that I can switch between models based on cost or capability.
+
+**Why this priority**: Prerequisite for any LLM functionality.
+
+**Independent Test**: Can be tested by entering API keys in settings and verifying a connection/test call.
+
+**Acceptance Scenarios**:
+
+1. **Given** the Settings page, **When** I select an LLM provider (e.g., OpenAI) and enter an API key, **Then** the system saves the configuration.
+2. **Given** a configured provider, **When** I run an analysis task, **Then** the system uses the selected provider for API calls.
+
+---
+
+### User Story 4 - Git Commit Message Suggestion (Priority: P3)
+
+As a Developer, I want the system to suggest commit messages based on changes directly within the Git plugin interface so that I can maintain consistent history with minimal effort.
+
+**Why this priority**: Enhances the existing Git workflow and improves commit quality.
+
+**Independent Test**: Can be tested by staging files in the Git plugin and clicking the "Generate Message" button.
+
+**Acceptance Scenarios**:
+
+1. **Given** staged changes in the Git plugin, **When** I click "Generate Message", **Then** the system analyzes the diff using the configured LLM and populates the commit message field with a suggested summary.
+
+### Edge Cases
+
+- What happens when the LLM provider API is down or times out? (System should retry or fail gracefully with a clear error message).
+- What happens if the dashboard screenshot cannot be generated? (System should proceed with logs only or fail depending on configuration).
+- What happens if the context (logs/metadata) exceeds the LLM's token limit? (System should truncate or summarize input).
+- How does the system handle missing API keys? (Task should fail immediately with a configuration error).
+
+## Requirements *(mandatory)*
+
+### Functional Requirements
+
+- **FR-001**: System MUST allow configuration of multiple LLM providers, specifically supporting OpenAI API, OpenRouter, and Kilo Provider.
+- **FR-002**: System MUST securely store API keys for these providers.
+- **FR-003**: System MUST implement a `DashboardValidationPlugin` that integrates with the existing `PluginBase` architecture.
+- **FR-004**: `DashboardValidationPlugin` MUST accept a dashboard identifier as input.
+- **FR-005**: `DashboardValidationPlugin` MUST be capable of retrieving a visual representation (screenshot) of the dashboard.
+- **FR-016**: System MUST support configurable screenshot strategies: 'Headless Browser' (default, high accuracy) and 'API Thumbnail' (fallback/fast).
+- **FR-006**: `DashboardValidationPlugin` MUST retrieve recent execution logs associated with the dashboard.
+- **FR-007**: `DashboardValidationPlugin` MUST combine visual and text data to prompt a Multimodal LLM for analysis.
+- **FR-008**: System MUST implement a `DocumentationPlugin` (or similar) for documenting datasets and dashboards.
+- **FR-009**: `DocumentationPlugin` MUST retrieve schema and metadata for the target asset.
+- **FR-017**: `DocumentationPlugin` MUST apply generated descriptions directly to the target object's metadata fields (dataset description, column descriptions).
+- **FR-010**: All LLM interactions MUST be executed as asynchronous tasks via the Task Manager.
+- **FR-018**: System MUST implement automatic retry logic (3 attempts with exponential backoff) for failed LLM API calls.
+- **FR-011**: Task execution logs and results MUST be streamed via the existing WebSocket infrastructure.
+- **FR-012**: System SHOULD expose an interface to generate text summaries for Git diffs, utilizing the diff, file list, and recent commit history as context.
+- **FR-013**: System MUST support scheduling of validation tasks for dashboards (leveraging existing scheduler architecture).
+- **FR-014**: System SHOULD support notification dispatch (Email, Pulse) upon validation failure or completion.
+- **FR-015**: Notifications MUST contain a summary of results (Status, Issue Count) and a direct link to the full report, avoiding sensitive full details in the message body.
+
+### Key Entities
+
+- **LLMProviderConfig**: Stores provider type (OpenAI, etc.), base URL, model name, and API key.
+- **ValidationResult**: Stores the analysis output, timestamp, and reference to the dashboard.
+- **AutoDocumentation**: Stores the generated documentation text for an asset.
+
+## Success Criteria *(mandatory)*
+
+### Measurable Outcomes
+
+- **SC-001**: Users can successfully configure and validate a connection to at least one LLM provider.
+- **SC-002**: A dashboard validation task completes within 90 seconds (assuming standard LLM latency).
+- **SC-003**: The system successfully processes a multimodal prompt (image + text) and returns a structured analysis.
+- **SC-004**: Generated documentation for a standard dataset contains descriptions for at least 80% of the columns (based on LLM capability, but pipeline must support it).
--- a/specs/017-llm-analysis-plugin/tasks.md
+++ b/specs/017-llm-analysis-plugin/tasks.md
@@ -0,0 +1,103 @@
+# Tasks: LLM Analysis & Documentation Plugins
+
+**Feature**: `017-llm-analysis-plugin`
+**Status**: Pending
+**Spec**: [specs/017-llm-analysis-plugin/spec.md](spec.md)
+
+## Dependencies
+
+- **Phase 1 (Setup)**: Must run first.
+- **Phase 2 (Foundational)**: Must complete before any User Stories.
+- **Phase 3 (US1 - Dashboard Analysis)**: Depends on Phase 2.
+- **Phase 4 (US2 - Documentation)**: Depends on Phase 2. Parallel with Phase 3.
+- **Phase 5 (US3 - Configuration)**: Depends on Phase 2. Recommended before Phase 3/4 for easier testing, but technically parallelizable if config is mocked.
+- **Phase 6 (US4 - Git Integration)**: Independent of other stories, depends on Phase 2.
+
+## Phase 1: Setup
+
+**Goal**: Initialize project structure and dependencies.
+
+- [ ] T001 Install backend dependencies (openai, playwright, tenacity) in `backend/requirements.txt`
+- [ ] T002 Create plugin directory structure `backend/src/plugins/llm_analysis/`
+- [ ] T003 Initialize `backend/src/plugins/llm_analysis/__init__.py`
+- [ ] T004 Create `backend/src/plugins/llm_analysis/models.py` for Pydantic models (LLMProviderConfig, ValidationResult)
+- [ ] T005 Update `backend/src/core/plugin_loader.py` to recognize new plugin type if necessary
+- [ ] T006 Create `backend/src/api/routes/llm.py` placeholder
+- [ ] T007 Register new route in `backend/src/app.py`
+
+## Phase 2: Foundational
+
+**Goal**: Implement core services and shared infrastructure.
+
+- [ ] T008 Implement `LLMProviderService` in `backend/src/services/llm_provider.py` (CRUD for providers, encryption)
+- [ ] T009 Implement `ScreenshotService` in `backend/src/plugins/llm_analysis/service.py` (Playwright + API strategies)
+- [ ] T010 Implement `LLMClient` in `backend/src/plugins/llm_analysis/service.py` (OpenAI SDK wrapper, retry logic with tenacity)
+- [ ] T011 Create `backend/src/plugins/llm_analysis/plugin.py` with `PluginBase` implementation stubs
+- [ ] T012 Define database schema updates for `LLMProviderConfig` in `backend/src/models/llm.py` (or appropriate location)
+- [ ] T013 Run migration to create new tables (if using SQLAlchemy/Alembic) or update SQLite schema
+
+## Phase 3: Dashboard Health Analysis (US1)
+
+**Goal**: Enable automated dashboard validation with multimodal analysis.
+
+- [ ] T014 [US1] Implement `validate_dashboard` task logic in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T015 [US1] Implement log retrieval logic (fetch recent logs) in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T016 [US1] Construct multimodal prompt (image + text) in `backend/src/plugins/llm_analysis/service.py`
+- [ ] T017 [US1] Implement result parsing and persistence (ValidationResult) in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T018 [US1] Add `validate` endpoint trigger in `backend/src/api/routes/tasks.py` (or reuse existing dispatch)
+- [ ] T019 [US1] Implement notification dispatch (Email/Pulse) on failure in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T020 [US1] Create `frontend/src/components/llm/ValidationReport.svelte` for viewing results
+- [ ] T021 [US1] Add "Validate" button to `frontend/src/components/DashboardGrid.svelte` (or Environments list)
+- [ ] T022 [US1] Enable scheduling for validation tasks in `backend/src/core/scheduler.py` (if not automatic via TaskManager)
+
+## Phase 4: Automated Dataset Documentation (US2)
+
+**Goal**: Generate and persist dataset documentation.
+
+- [ ] T023 [US2] Implement `generate_documentation` task logic in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T024 [US2] Implement metadata fetching (schema, columns) in `backend/src/plugins/llm_analysis/plugin.py`
+- [ ] T025 [US2] Construct documentation prompt in `backend/src/plugins/llm_analysis/service.py`
+- [ ] T026 [US2] Implement metadata update logic (write back to DB) in `backend/src/services/mapping_service.py` (or relevant service)
+- [ ] T027 [US2] Add "Generate Docs" button to `frontend/src/components/tools/MapperTool.svelte` (or Dataset view)
+- [ ] T028 [US2] Create feedback/preview UI component `frontend/src/components/llm/DocPreview.svelte` (optional but recommended)
+
+## Phase 5: LLM Provider Configuration (US3)
+
+**Goal**: Manage LLM providers via UI.
+
+- [ ] T029 [US3] Implement CRUD endpoints for providers in `backend/src/api/routes/llm.py`
+- [ ] T030 [US3] Create `frontend/src/components/llm/ProviderConfig.svelte` form
+- [ ] T031 [US3] Create `frontend/src/routes/admin/settings/llm/+page.svelte` settings page
+- [ ] T032 [US3] Implement "Test Connection" button logic in frontend/backend
+- [ ] T033 [US3] Ensure API keys are masked in frontend responses
+
+## Phase 6: Git Commit Message Suggestion (US4)
+
+**Goal**: AI-assisted commit messages.
+
+- [ ] T034 [US4] Implement `generate_commit_message` logic in `backend/src/plugins/git_plugin.py` (or `llm_analysis` if preferred, but Git context lives in Git plugin)
+- [ ] T035 [US4] Create endpoint `POST /api/git/generate-message` in `backend/src/api/routes/git.py`
+- [ ] T036 [US4] Construct commit generation prompt (Diff + History) in `backend/src/services/git_service.py`
+- [ ] T037 [US4] Add "Generate" button to `frontend/src/components/git/CommitModal.svelte`
+- [ ] T038 [US4] Wire up frontend to call generation endpoint and populate textarea
+
+## Phase 7: Polish & Cross-Cutting
+
+**Goal**: Finalize and verify.
+
+- [ ] T039 Verify all permissions (`plugin:llm:validate`, etc.) are registered and enforceable
+- [ ] T040 Verify i18n strings for all new UI components
+- [ ] T041 Run full end-to-end test of Dashboard Validation flow
+- [ ] T042 Run full end-to-end test of Documentation flow
+- [ ] T043 Update `README.md` with new plugin capabilities
+
+## Implementation Strategy
+
+1. **MVP**: Focus on Phase 1, 2, 3, and 5. This delivers the core value (Dashboard Validation) and the necessary configuration.
+2. **Increment 1**: Add Phase 4 (Documentation).
+3. **Increment 2**: Add Phase 6 (Git Integration).
+
+## Parallel Execution Examples
+
+- **Backend/Frontend Split**: T020 (Frontend Report) can be built while T014-T017 (Backend Logic) are in progress, using mock data.
+- **Story Parallelism**: US2 (Documentation) and US4 (Git) are largely independent and can be worked on simultaneously by different developers (or sequentially without blocking).