ss-tools/specs/017-llm-analysis-plugin/tasks.md

Tasks: LLM Analysis & Documentation Plugins

Feature: 017-llm-analysis-plugin Status: Pending Spec: specs/017-llm-analysis-plugin/spec.md

Dependencies

• Phase 1 (Setup): Must run first.
• Phase 2 (Foundational): Must complete before any User Stories.
• Phase 3 (US1 - Dashboard Analysis): Depends on Phase 2.
• Phase 4 (US2 - Documentation): Depends on Phase 2. Parallel with Phase 3.
• Phase 5 (US3 - Configuration): Depends on Phase 2. Recommended before Phase 3/4 for easier testing, but technically parallelizable if config is mocked.
• Phase 6 (US4 - Git Integration): Independent of other stories, depends on Phase 2.

Phase 1: Setup

Goal: Initialize project structure and dependencies.

• T001 Install backend dependencies (openai, playwright, tenacity) in backend/requirements.txt
• T002 Create plugin directory structure backend/src/plugins/llm_analysis/
• T003 Initialize backend/src/plugins/llm_analysis/__init__.py
• T004 Create backend/src/plugins/llm_analysis/models.py for Pydantic models (LLMProviderConfig, ValidationResult)
• T005 Update backend/src/core/plugin_loader.py to recognize new plugin type if necessary
• T006 Create backend/src/api/routes/llm.py placeholder
• T007 Register new route in backend/src/app.py

Phase 2: Foundational

Goal: Implement core services and shared infrastructure.

• T008 Implement LLMProviderService in backend/src/services/llm_provider.py (CRUD for providers, AES-256 encryption)
• T009 Implement ScreenshotService in backend/src/plugins/llm_analysis/service.py (Playwright + API strategies, fallback logic)
• T010 Implement LLMClient in backend/src/plugins/llm_analysis/service.py (OpenAI SDK wrapper, retry logic with tenacity, rate limit handling)
• T011 Create backend/src/plugins/llm_analysis/plugin.py with PluginBase implementation stubs
• T012 Define database schema updates for LLMProviderConfig in backend/src/models/llm.py (or appropriate location)
• T013 Run migration to create new tables (if using SQLAlchemy/Alembic) or update SQLite schema

Phase 3: Dashboard Health Analysis (US1)

Goal: Enable automated dashboard validation with multimodal analysis.

• T014 [US1] Implement validate_dashboard task logic in backend/src/plugins/llm_analysis/plugin.py
• T015 [US1] Implement log retrieval logic (fetch recent logs) in backend/src/plugins/llm_analysis/plugin.py
• T016 [US1] Construct multimodal prompt (image + text) in backend/src/plugins/llm_analysis/service.py (ensure data privacy/masking)
• T017 [US1] Implement result parsing and persistence (ValidationResult) in backend/src/plugins/llm_analysis/plugin.py
• T018 [US1] Add validate endpoint trigger in backend/src/api/routes/tasks.py (or reuse existing dispatch)
• T019 [US1] Implement notification dispatch (Email/Pulse) on failure in backend/src/plugins/llm_analysis/plugin.py (Summary + Link format)
• T020 [US1] Create frontend/src/components/llm/ValidationReport.svelte for viewing results
• T021 [US1] Add "Validate" button to frontend/src/components/DashboardGrid.svelte (or Environments list)
• T022 [US1] Enable scheduling for validation tasks in backend/src/core/scheduler.py (if not automatic via TaskManager)

Phase 4: Automated Dataset Documentation (US2)

Goal: Generate and persist dataset documentation.

• T023 [US2] Implement generate_documentation task logic in backend/src/plugins/llm_analysis/plugin.py
• T024 [US2] Implement metadata fetching (schema, columns) in backend/src/plugins/llm_analysis/plugin.py
• T025 [US2] Construct documentation prompt in backend/src/plugins/llm_analysis/service.py (handle schema changes)
• T026 [US2] Implement metadata update logic (write back to DB) in backend/src/services/mapping_service.py (handle partial failures)
• T027 [US2] Add "Generate Docs" button to frontend/src/components/tools/MapperTool.svelte (or Dataset view)
• T028 [US2] Create feedback/preview UI component frontend/src/components/llm/DocPreview.svelte (optional but recommended)

Phase 5: LLM Provider Configuration (US3)

Goal: Manage LLM providers via UI.

• T029 [US3] Implement CRUD endpoints for providers in backend/src/api/routes/llm.py
• T030 [US3] Create frontend/src/components/llm/ProviderConfig.svelte form
• T031 [US3] Create frontend/src/routes/admin/settings/llm/+page.svelte settings page
• T032 [US3] Implement "Test Connection" button logic in frontend/backend
• T033 [US3] Ensure API keys are masked in frontend responses

Phase 6: Git Commit Message Suggestion (US4)

Goal: AI-assisted commit messages.

• T034 [US4] Implement generate_commit_message logic in backend/src/plugins/git_plugin.py (or llm_analysis if preferred, but Git context lives in Git plugin)
• T035 [US4] Create endpoint POST /api/git/generate-message in backend/src/api/routes/git.py
• T036 [US4] Construct commit generation prompt (Diff + History) in backend/src/services/git_service.py
• T037 [US4] Add "Generate" button to frontend/src/components/git/CommitModal.svelte
• T038 [US4] Wire up frontend to call generation endpoint and populate textarea

Phase 7: Polish & Cross-Cutting

Goal: Finalize and verify.

• T039 Verify all permissions (plugin:llm:validate, etc.) are registered and enforceable
• T040 Verify i18n strings for all new UI components
• T041 Run full end-to-end test of Dashboard Validation flow (including fallback scenarios)
• T042 Run full end-to-end test of Documentation flow
• T043 Update README.md with new plugin capabilities
• T044 Verify API key masking in all UI responses and logs

Implementation Strategy

1. MVP: Focus on Phase 1, 2, 3, and 5. This delivers the core value (Dashboard Validation) and the necessary configuration.
2. Increment 1: Add Phase 4 (Documentation).
3. Increment 2: Add Phase 6 (Git Integration).

Parallel Execution Examples

• Backend/Frontend Split: T020 (Frontend Report) can be built while T014-T017 (Backend Logic) are in progress, using mock data.
• Story Parallelism: US2 (Documentation) and US4 (Git) are largely independent and can be worked on simultaneously by different developers (or sequentially without blocking).