Похоже работает

- Replace local backend.log reading with Superset API /log/ fetch
- Update DashboardValidationPlugin to use SupersetClient
- Filter logs by dashboard_id and last 24 hours
- Update spec FR-006 to reflect API usage

.gitignore

@@ -66,6 +66,6 @@ backend/mappings.db
 
 
 backend/tasks.db
-
+backend/logs
-# Git Integration repositories
+backend/auth.db
-backend/git_repos/
+semantics/reports

.kilocode/rules/specify-rules.md

@@ -27,6 +27,12 @@ Auto-generated from all feature plans. Last updated: 2025-12-19
 - SQLite (for config/history), Filesystem (local Git repositories) (011-git-integration-dashboard)
 - Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing) (013-unify-frontend-css)
 - LocalStorage (for language preference) (013-unify-frontend-css)
+- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend) (014-file-storage-ui)
+- Local Filesystem (for artifacts), Config (for storage path) (014-file-storage-ui)
+- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend) (015-frontend-nav-redesign)
+- N/A (UI reorganization and API integration) (015-frontend-nav-redesign)
+- SQLite (`auth.db`) for Users, Roles, Permissions, and Mappings. (016-multi-user-auth)
+- SQLite (existing `tasks.db` for results, `auth.db` for permissions, `mappings.db` or new `plugins.db` for provider config/metadata) (017-llm-analysis-plugin)
 
 - Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui)
 
@@ -47,9 +53,9 @@ cd src; pytest; ruff check .
 Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions
 
 ## Recent Changes
-- 013-unify-frontend-css: Added Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing)
+- 017-llm-analysis-plugin: Added Python 3.9+ (Backend), Node.js 18+ (Frontend)
-- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
+- 016-multi-user-auth: Added Python 3.9+ (Backend), Node.js 18+ (Frontend)
-- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
+- 015-frontend-nav-redesign: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
 
 
 <!-- MANUAL ADDITIONS START -->

.kilocode/workflows/read_semantic.md

@@ -0,0 +1,4 @@
+---
+description: USE SEMANTIC
+---
+Прочитай semantic_protocol.md. ОБЯЗАТЕЛЬНО используй его при разработке

.kilocode/workflows/speckit.analyze.md

@@ -63,6 +63,7 @@ Load only the minimal necessary context from each artifact:
 **From constitution:**
 
 - Load `.specify/memory/constitution.md` for principle validation
+- Load `semantic_protocol.md` for technical standard validation
 
 ### 3. Build Semantic Models
 

.kilocode/workflows/speckit.implement.md

@@ -1,5 +1,10 @@
 ---
 description: Execute the implementation plan by processing and executing all tasks defined in tasks.md
+handoffs: 
+  - label: Verify Changes
+    agent: speckit.test
+    prompt: Verify the implementation of...
+    send: true
 ---
 
 ## User Input
@@ -46,6 +51,7 @@ You **MUST** consider the user input before proceeding (if not empty).
      - Automatically proceed to step 3
 
 3. Load and analyze the implementation context:
+   - **REQUIRED**: Read `semantic_protocol.md` for strict coding standards and contract requirements
    - **REQUIRED**: Read tasks.md for the complete task list and execution plan
    - **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
    - **IF EXISTS**: Read data-model.md for entities and relationships
@@ -111,6 +117,7 @@ You **MUST** consider the user input before proceeding (if not empty).
    - **Validation checkpoints**: Verify each phase completion before proceeding
 
 7. Implementation execution rules:
+   - **Strict Adherence**: Apply `semantic_protocol.md` rules - every file must start with [DEF] header, include @TIER, and define contracts
    - **Setup first**: Initialize project structure, dependencies, configuration
    - **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
    - **Core development**: Implement models, services, CLI commands, endpoints

.kilocode/workflows/speckit.plan.md

@@ -22,7 +22,7 @@ You **MUST** consider the user input before proceeding (if not empty).
 
 1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
 
-2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
+2. **Load context**: Read FEATURE_SPEC, `ux_reference.md`, `semantic_protocol.md` and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
 
 3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
    - Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
@@ -64,12 +64,22 @@ You **MUST** consider the user input before proceeding (if not empty).
 
 **Prerequisites:** `research.md` complete
 
+0. **Validate Design against UX Reference**:
+   - Check if the proposed architecture supports the latency, interactivity, and flow defined in `ux_reference.md`.
+   - **CRITICAL**: If the technical plan requires compromising the UX defined in `ux_reference.md` (e.g. "We can't do real-time validation because X"), you **MUST STOP** and warn the user. Do not proceed until resolved.
+
 1. **Extract entities from feature spec** → `data-model.md`:
    - Entity name, fields, relationships
    - Validation rules from requirements
    - State transitions if applicable
 
-2. **Generate API contracts** from functional requirements:
+2. **Define Module & Function Contracts (Semantic Protocol)**:
+   - **MANDATORY**: For every new module, define the [DEF] Header and Module-level Contract (@TIER, @PURPOSE, @INVARIANT) as per `semantic_protocol.md`.
+   - **REQUIRED**: Define Function Contracts (@PRE, @POST) for critical logic.
+   - Output specific contract definitions to `contracts/modules.md` or append to `data-model.md` to guide implementation.
+   - Ensure strict adherence to `semantic_protocol.md` syntax.
+
+3. **Generate API contracts** from functional requirements:
    - For each user action → endpoint
    - Use standard REST/GraphQL patterns
    - Output OpenAPI/GraphQL schema to `/contracts/`

.kilocode/workflows/speckit.specify.md

@@ -70,7 +70,22 @@ Given that feature description, do this:
 
 3. Load `.specify/templates/spec-template.md` to understand required sections.
 
-4. Follow this execution flow:
+4. **Generate UX Reference**:
+   
+   a. Load `.specify/templates/ux-reference-template.md`.
+   
+   b. **Design the User Experience**:
+      - **Imagine you are the user**: Visualize the interface and interaction.
+      - **Persona**: Define who is using this.
+      - **Happy Path**: Write the story of the perfect interaction.
+      - **Mockups**: Create concrete CLI text blocks or UI descriptions.
+      - **Errors**: Define how the system guides the user out of failure.
+   
+   c. Write the `ux_reference.md` file in the feature directory.
+   
+   d. **CRITICAL**: This UX Reference is now the source of truth for the "feel" of the feature. The technical spec MUST support this experience.
+
+5. Follow this execution flow:
 
     1. Parse user description from Input
        If empty: ERROR "No feature description provided"
@@ -115,6 +130,12 @@ Given that feature description, do this:
       - [ ] Focused on user value and business needs
       - [ ] Written for non-technical stakeholders
       - [ ] All mandatory sections completed
+
+      ## UX Consistency
+      
+      - [ ] Functional requirements fully support the 'Happy Path' in ux_reference.md
+      - [ ] Error handling requirements match the 'Error Experience' in ux_reference.md
+      - [ ] No requirements contradict the defined User Persona or Context
       
       ## Requirement Completeness
       
@@ -190,7 +211,7 @@ Given that feature description, do this:
 
    d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
 
-7. Report completion with branch name, spec file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
+7. Report completion with branch name, spec file path, ux_reference file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
 
 **NOTE:** The script creates and checks out the new branch and initializes the spec file before writing.
 

.kilocode/workflows/speckit.tasks.md

@@ -24,7 +24,7 @@ You **MUST** consider the user input before proceeding (if not empty).
 1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
 
 2. **Load design documents**: Read from FEATURE_DIR:
-   - **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
+   - **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities), ux_reference.md (experience source of truth)
    - **Optional**: data-model.md (entities), contracts/ (API endpoints), research.md (decisions), quickstart.md (test scenarios)
    - Note: Not all projects have all documents. Generate tasks based on what's available.
 
@@ -70,6 +70,12 @@ The tasks.md should be immediately executable - each task must be specific enoug
 
 **Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
 
+### UX Preservation (CRITICAL)
+
+- **Source of Truth**: `ux_reference.md` is the absolute standard for the "feel" of the feature.
+- **Violation Warning**: If any task would inherently violate the UX (e.g. "Remove progress bar to simplify code"), you **MUST** flag this to the user immediately.
+- **Verification Task**: You **MUST** add a specific task at the end of each User Story phase: `- [ ] Txxx [USx] Verify implementation matches ux_reference.md (Happy Path & Errors)`
+
 ### Checklist Format (REQUIRED)
 
 Every task MUST strictly follow this format:

.kilocode/workflows/speckit.test.md

@@ -0,0 +1,66 @@
+---
+description: Run semantic validation and functional tests for a specific feature, module, or file.
+handoffs: 
+  - label: Fix Implementation
+    agent: speckit.implement
+    prompt: Fix the issues found during testing...
+    send: true
+---
+
+## User Input
+
+```text
+$ARGUMENTS
+```
+
+**Input format:** Can be a file path, a directory, or a feature name.
+
+## Outline
+
+1. **Context Analysis**:
+   - Determine the target scope (Backend vs Frontend vs Full Feature).
+   - Read `semantic_protocol.md` to load validation rules.
+
+2. **Phase 1: Semantic Static Analysis (The "Compiler" Check)**
+   - **Command:** Use `grep` or script to verify Protocol compliance before running code.
+   - **Check:**
+     - Does the file start with `[DEF:...]` header?
+     - Are `@TIER` and `@PURPOSE` defined?
+     - Are imports located *after* the contracts?
+     - Do functions marked "Critical" have `@PRE`/`@POST` tags?
+   - **Action:** If this phase fails, **STOP** and report "Semantic Compilation Failed". Do not run runtime tests.
+
+3. **Phase 2: Environment Prep**
+   - Detect project type:
+     - **Python**: Check if `.venv` is active.
+     - **Svelte**: Check if `node_modules` exists.
+   - **Command:** Run linter (e.g., `ruff check`, `eslint`) to catch syntax errors immediately.
+
+4. **Phase 3: Test Execution (Runtime)**
+   - Select the test runner based on the file path:
+     - **Backend (`*.py`)**:
+       - Command: `pytest <path_to_test_file> -v`
+       - If no specific test file exists, try to find it by convention: `tests/test_<module_name>.py`.
+     - **Frontend (`*.svelte`, `*.ts`)**:
+       - Command: `npm run test -- <path_to_component>`
+   
+   - **Verification**:
+     - Analyze output logs.
+     - If tests fail, summarize the failure (AssertionError, Timeout, etc.).
+
+5. **Phase 4: Contract Coverage Check (Manual/LLM verify)**
+   - Review the test cases executed.
+   - **Question**: Do the tests explicitly verify the `@POST` guarantees defined in the module header?
+   - **Report**: Mark as "Weak Coverage" if contracts exist but aren't tested.
+
+## Execution Rules
+
+- **Fail Fast**: If semantic headers are missing, don't waste time running pytest.
+- **No Silent Failures**: Always output the full error log if a command fails.
+- **Auto-Correction Hint**: If a test fails, suggest the specific `speckit.implement` command to fix it.
+
+## Example Commands
+
+- **Python**: `pytest backend/tests/test_auth.py`
+- **Svelte**: `npm run test:unit -- src/components/Button.svelte`
+- **Lint**: `ruff check backend/src/api/`

.kilocodemodes

@@ -13,20 +13,6 @@ customModes:
       - browser
       - mcp
     customInstructions: 1. Always begin by loading the relevant plan or task list from the `specs/` directory. 2. Do not assume a task is done just because it is checked; verify the code or functionality first if asked to audit. 3. When updating task lists, ensure you only mark items as complete if you have verified them.
-  - slug: product-manager
-    name: Product Manager
-    description: Executes SpecKit workflows for feature management
-    roleDefinition: |-
-      You are Kilo Code, acting as a Product Manager. Your purpose is to rigorously execute the workflows defined in `.kilocode/workflows/`.
-      You act as the orchestrator for: - Specification (`speckit.specify`, `speckit.clarify`) - Planning (`speckit.plan`) - Task Management (`speckit.tasks`, `speckit.taskstoissues`) - Quality Assurance (`speckit.analyze`, `speckit.checklist`) - Governance (`speckit.constitution`) - Implementation Oversight (`speckit.implement`)
-      For each task, you must read the relevant workflow file from `.kilocode/workflows/` and follow its Execution Steps precisely.
-    whenToUse: Use this mode when you need to run any /speckit.* command or when dealing with high-level feature planning, specification writing, or project management tasks.
-    groups:
-      - read
-      - edit
-      - command
-      - mcp
-    customInstructions: 1. Always read the specific workflow file in `.kilocode/workflows/` before executing a command. 2. Adhere strictly to the "Operating Constraints" and "Execution Steps" in the workflow files.
   - slug: semantic
     name: Semantic Agent
     roleDefinition: |-
@@ -43,3 +29,18 @@ customModes:
       - browser
       - mcp
     source: project
+  - slug: product-manager
+    name: Product Manager
+    roleDefinition: |-
+      Your purpose is to rigorously execute the workflows defined in `.kilocode/workflows/`.
+      You act as the orchestrator for: - Specification (`speckit.specify`, `speckit.clarify`) - Planning (`speckit.plan`) - Task Management (`speckit.tasks`, `speckit.taskstoissues`) - Quality Assurance (`speckit.analyze`, `speckit.checklist`) - Governance (`speckit.constitution`) - Implementation Oversight (`speckit.implement`)
+      For each task, you must read the relevant workflow file from `.kilocode/workflows/` and follow its Execution Steps precisely.
+    whenToUse: Use this mode when you need to run any /speckit.* command or when dealing with high-level feature planning, specification writing, or project management tasks.
+    description: Executes SpecKit workflows for feature management
+    customInstructions: 1. Always read the specific workflow file in `.kilocode/workflows/` before executing a command. 2. Adhere strictly to the "Operating Constraints" and "Execution Steps" in the workflow files.
+    groups:
+      - read
+      - edit
+      - command
+      - mcp
+    source: project

.specify/memory/constitution.md

@@ -1,11 +1,11 @@
 <!--
 SYNC IMPACT REPORT
-Version: 1.7.1 (Simplified Workflow)
+Version: 2.2.0 (ConfigManager Discipline)
 Changes:
-- Simplified Generation Workflow to a single phase: Code Generation from `tasks.md`.
+- Updated Principle II: Added mandatory requirement for using `ConfigManager` (via dependency injection) for all configuration access to ensure consistent environment handling and avoid hardcoded values.
-- Removed multi-phase Architecture/Implementation split to streamline development.
+- Updated Principle III: Refined `requestApi` requirement.
 Templates Status:
-- .specify/templates/plan-template.md: ✅ Aligned (Dynamic check).
+- .specify/templates/plan-template.md: ✅ Aligned.
 - .specify/templates/spec-template.md: ✅ Aligned.
 - .specify/templates/tasks-template.md: ✅ Aligned.
 -->
@@ -14,54 +14,42 @@ Templates Status:
 ## Core Principles
 
 ### I. Semantic Protocol Compliance
-The file `semantic_protocol.md` is the **authoritative technical standard** for this project. All code generation, refactoring, and architecture must strictly adhere to the standards, syntax, and workflows defined therein.
+The file `semantic_protocol.md` is the **sole and authoritative technical standard** for this project.
-- **Syntax**: `[DEF]` anchors, `@RELATION` tags, and metadata must match the Protocol specification.
+- **Law**: All code must adhere to the Axioms (Meaning First, Contract First, etc.) defined in the Protocol.
-- **Structure**: File layouts and headers must follow the "File Structure Standard".
+- **Syntax & Structure**: Anchors (`[DEF]`), Tags (`@KEY`), and File Structures must strictly match the Protocol.
-- **Workflow**: The technical steps for generating code must align with the Protocol.
+- **Compliance**: Any deviation from `semantic_protocol.md` constitutes a build failure.
 
-### II. Causal Validity (Contracts First)
+### II. Everything is a Plugin & Centralized Config
-As defined in the Protocol, Semantic definitions (Contracts) must ALWAYS precede implementation code. Logic is downstream of definition. We define the structure and constraints (`[DEF]`, `@PRE`, `@POST`) before writing the executable logic.
+All functional extensions, tools, or major features must be implemented as modular Plugins inheriting from `PluginBase`.
+- **Modularity**: Logic should not reside in standalone services or scripts unless strictly necessary for core infrastructure. This ensures a unified execution model via the `TaskManager`.
+- **Configuration Discipline**: All configuration access (environments, settings, paths) MUST use the `ConfigManager`. In the backend, the singleton instance MUST be obtained via dependency injection (`get_config_manager()`). Hardcoding environment IDs (e.g., "1") or paths is STRICTLY FORBIDDEN.
 
-### III. Immutability of Architecture
+### III. Unified Frontend Experience
-Architectural decisions in the Module Header (`@LAYER`, `@INVARIANT`, `@CONSTRAINT`) are treated as immutable constraints. Changes to these require an explicit refactoring step, not ad-hoc modification during implementation.
+To ensure a consistent and accessible user experience, all frontend implementations must strictly adhere to the unified design and localization standards.
+- **Component Reusability**: All UI elements MUST utilize the standardized Svelte component library (`src/lib/ui`) and centralized design tokens.
+- **Internationalization (i18n)**: All user-facing text MUST be extracted to the translation system (`src/lib/i18n`).
+- **Backend Communication**: All API requests MUST use the `requestApi` wrapper (or its derivatives like `fetchApi`, `postApi`) from `src/lib/api.js`. Direct use of the native `fetch` API for backend communication is FORBIDDEN to ensure consistent authentication (JWT) and error handling.
 
-### IV. Design by Contract (DbC)
+### IV. Security & Access Control
-Contracts are the Source of Truth. Functions and Classes must define their purpose, specifications, and constraints in the metadata block before implementation, strictly following the **Contracts (Section IV)** standard in `semantic_protocol.md`.
+To support the Role-Based Access Control (RBAC) system, all functional components must define explicit permissions.
+- **Granular Permissions**: Every Plugin MUST define a unique permission string (e.g., `plugin:name:execute`) required for its operation.
+- **Registration**: These permissions MUST be registered in the system database (`auth.db`) during initialization.
 
-### V. Belief State Logging
+### V. Independent Testability
-Agents must maintain belief state logs for debugging and coherence checks, strictly following the **Logging Standard (Section V)** defined in `semantic_protocol.md`.
+Every feature specification MUST define "Independent Tests" that allow the feature to be verified in isolation.
+- **Decoupling**: Features should be designed such that they can be tested without requiring the full application state or external dependencies where possible.
+- **Verification**: A feature is not complete until its Independent Test scenarios pass.
 
-### VI. Fractal Complexity Limit
+### VI. Asynchronous Execution
-To maintain semantic coherence, code must adhere to the complexity limits (Module/Function size) defined in the **Fractal Complexity Limit (Section VI)** of `semantic_protocol.md`.
+All long-running or resource-intensive operations (migrations, analysis, backups, external API calls) MUST be executed as asynchronous tasks via the `TaskManager`.
-
+- **Non-Blocking**: HTTP API endpoints MUST NOT block on these operations; they should spawn a task and return a Task ID.
-### VII. Everything is a Plugin
+- **Observability**: Tasks MUST emit real-time status updates via the WebSocket infrastructure.
-All functional extensions, tools, or major features must be implemented as modular Plugins inheriting from `PluginBase`. Logic should not reside in standalone services or scripts unless strictly necessary for core infrastructure. This ensures a unified execution model via the `TaskManager`, consistent logging, and modularity.
-
-## File Structure Standards
-Refer to **Section III (File Structure Standard)** in `semantic_protocol.md` for the authoritative definitions of:
-- Python Module Headers (`.py`)
-- Svelte Component Headers (`.svelte`)
-
-## Generation Workflow
-The development process follows a streamlined single-phase workflow:
-
-### 1. Code Generation Phase (Mode: `code`)
-**Input**: `tasks.md`
-**Responsibility**:
-- Select task from `tasks.md`.
-- Generate Scaffolding (`[DEF]` anchors, Headers, Contracts) AND Implementation in one pass.
-- Ensure strict adherence to Protocol Section IV (Contracts) and Section VII (Generation Workflow).
-- **Output**: Working code with passing tests.
-
-### 2. Validation
-If logic conflicts with Contract -> Stop -> Report Error.
 
 ## Governance
 This Constitution establishes the "Semantic Code Generation Protocol" as the supreme law of this repository.
 
-- **Authoritative Source**: `semantic_protocol.md` defines the specific implementation rules for these Principles.
+- **Authoritative Source**: `semantic_protocol.md` defines the specific implementation rules for Principle I.
-- **Automated Enforcement**: Tools must validate adherence to the `semantic_protocol.md` syntax.
 - **Amendments**: Changes to core principles require a Constitution amendment. Changes to technical syntax require a Protocol update.
 - **Compliance**: Failure to adhere to the Protocol constitutes a build failure.
 
-**Version**: 1.7.1 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-13
+**Version**: 2.2.0 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-29

.specify/templates/spec-template.md

@@ -1,6 +1,7 @@
 # Feature Specification: [FEATURE NAME]
 
 **Feature Branch**: `[###-feature-name]`  
+**Reference UX**: `[ux_reference.md]` (See specific folder)
 **Created**: [DATE]  
 **Status**: Draft  
 **Input**: User description: "$ARGUMENTS"

.specify/templates/tasks-arch-template.md

@@ -1,35 +0,0 @@
----
-
-description: "Architecture task list template (Contracts & Scaffolding)"
----
-
-# Architecture Tasks: [FEATURE NAME]
-
-**Role**: Architect Agent
-**Goal**: Define the "What" and "Why" (Contracts, Scaffolding, Models) before implementation.
-**Input**: Design documents from `/specs/[###-feature-name]/`
-**Output**: Files with `[DEF]` anchors, `@PRE`/`@POST` contracts, and `@RELATION` mappings. No business logic.
-
-## Phase 1: Setup & Models
-
-- [ ] A001 Create/Update data models in [path] with `[DEF]` and contracts
-- [ ] A002 Define API route structure/contracts in [path]
-- [ ] A003 Define shared utilities/interfaces
-
-## Phase 2: User Story 1 - [Title]
-
-- [ ] A004 [US1] Define contracts for [Component/Service] in [path]
-- [ ] A005 [US1] Define contracts for [Endpoint] in [path]
-- [ ] A006 [US1] Define contracts for [Frontend Component] in [path]
-
-## Phase 3: User Story 2 - [Title]
-
-- [ ] A007 [US2] Define contracts for [Component/Service] in [path]
-- [ ] A008 [US2] Define contracts for [Endpoint] in [path]
-
-## Handover Checklist
-
-- [ ] All new files created with `[DEF]` anchors
-- [ ] All functions/classes have `@PURPOSE`, `@PRE`, `@POST` tags
-- [ ] No "naked code" (logic outside of anchors)
-- [ ] `tasks-dev.md` is ready for the Developer Agent

.specify/templates/tasks-dev-template.md

@@ -1,35 +0,0 @@
----
-
-description: "Developer task list template (Implementation Logic)"
----
-
-# Developer Tasks: [FEATURE NAME]
-
-**Role**: Developer Agent
-**Goal**: Implement the "How" (Logic, State, Error Handling) inside the defined contracts.
-**Input**: `tasks-arch.md` (completed), Scaffolding files with `[DEF]` anchors.
-**Output**: Working code that satisfies `@PRE`/`@POST` conditions.
-
-## Phase 1: Setup & Models
-
-- [ ] D001 Implement logic for [Model] in [path]
-- [ ] D002 Implement logic for [API Route] in [path]
-- [ ] D003 Implement shared utilities
-
-## Phase 2: User Story 1 - [Title]
-
-- [ ] D004 [US1] Implement logic for [Component/Service] in [path]
-- [ ] D005 [US1] Implement logic for [Endpoint] in [path]
-- [ ] D006 [US1] Implement logic for [Frontend Component] in [path]
-- [ ] D007 [US1] Verify semantic compliance and belief state logging
-
-## Phase 3: User Story 2 - [Title]
-
-- [ ] D008 [US2] Implement logic for [Component/Service] in [path]
-- [ ] D009 [US2] Implement logic for [Endpoint] in [path]
-
-## Polish & Quality Assurance
-
-- [ ] DXXX Verify all tests pass
-- [ ] DXXX Check error handling and edge cases
-- [ ] DXXX Ensure code style compliance

.specify/templates/ux-reference-template.md

@@ -0,0 +1,67 @@
+# UX Reference: [FEATURE NAME]
+
+**Feature Branch**: `[###-feature-name]`
+**Created**: [DATE]
+**Status**: Draft
+
+## 1. User Persona & Context
+
+*   **Who is the user?**: [e.g. Junior Developer, System Administrator, End User]
+*   **What is their goal?**: [e.g. Quickly deploy a hotfix, Visualize complex data]
+*   **Context**: [e.g. Running a command in a terminal on a remote server, Browsing the dashboard on a mobile device]
+
+## 2. The "Happy Path" Narrative
+
+[Write a short story (3-5 sentences) describing the perfect interaction from the user's perspective. Focus on how it *feels* - is it instant? Does it guide them?]
+
+## 3. Interface Mockups
+
+### CLI Interaction (if applicable)
+
+```bash
+# User runs this command:
+$ command --flag value
+
+# System responds immediately with:
+[ spinner ] specific loading message...
+
+# Success output:
+✅ Operation completed successfully in 1.2s
+   - Created file: /path/to/file
+   - Updated config: /path/to/config
+```
+
+### UI Layout & Flow (if applicable)
+
+**Screen/Component**: [Name]
+
+*   **Layout**: [Description of structure, e.g., "Two-column layout, left sidebar navigation..."]
+*   **Key Elements**:
+    *   **[Button Name]**: Primary action. Color: Blue.
+    *   **[Input Field]**: Placeholder text: "Enter your name...". Validation: Real-time.
+*   **States**:
+    *   **Default**: Clean state, waiting for input.
+    *   **Loading**: Skeleton loader replaces content area.
+    *   **Success**: Toast notification appears top-right: "Saved!" (Green).
+
+## 4. The "Error" Experience
+
+**Philosophy**: Don't just report the error; guide the user to the fix.
+
+### Scenario A: [Common Error, e.g. Invalid Input]
+
+*   **User Action**: Enters "123" in a text-only field.
+*   **System Response**:
+    *   (UI) Input border turns Red. Message below input: "Please enter text only."
+    *   (CLI) `❌ Error: Invalid input '123'. Expected text format.`
+*   **Recovery**: User can immediately re-type without refreshing/re-running.
+
+### Scenario B: [System Failure, e.g. Network Timeout]
+
+*   **System Response**: "Unable to connect. Retrying in 3s... (Press C to cancel)"
+*   **Recovery**: Automatic retry or explicit "Retry Now" button.
+
+## 5. Tone & Voice
+
+*   **Style**: [e.g. Concise, Technical, Friendly, Verbose]
+*   **Terminology**: [e.g. Use "Repository" not "Repo", "Directory" not "Folder"]

README.md

@@ -1,119 +1,77 @@
-# Инструменты автоматизации Superset
+# Инструменты автоматизации Superset (ss-tools)
 
 ## Обзор
-Этот репозиторий содержит Python-скрипты и библиотеку (`superset_tool`) для автоматизации задач в Apache Superset, таких как:
+**ss-tools** — это современная платформа для автоматизации и управления экосистемой Apache Superset. Проект перешел от набора CLI-скриптов к полноценному веб-приложению с архитектурой Backend (FastAPI) + Frontend (SvelteKit), обеспечивая удобный интерфейс для сложных операций.
-- **Резервное копирование**: Экспорт всех дашбордов из экземпляра Superset в локальное хранилище.
+
-- **Миграция**: Перенос и преобразование дашбордов между разными средами Superset (например, Development, Sandbox, Production).
+## Основные возможности
+
+### 🚀 Миграция и управление дашбордами
+- **Dashboard Grid**: Удобный просмотр всех дашбордов во всех окружениях (Dev, Sandbox, Prod) в едином интерфейсе.
+- **Интеллектуальный маппинг**: Автоматическое и ручное сопоставление датасетов, таблиц и схем при переносе между окружениями.
+- **Проверка зависимостей**: Валидация наличия всех необходимых компонентов перед миграцией.
+
+### 📦 Резервное копирование
+- **Планировщик (Scheduler)**: Автоматическое создание резервных копий дашбордов и датасетов по расписанию.
+- **Хранилище**: Локальное хранение артефактов с возможностью управления через UI.
+
+### 🛠 Git Интеграция
+- **Version Control**: Возможность версионирования ассетов Superset.
+- **Git Dashboard**: Управление ветками, коммитами и деплоем изменений напрямую из интерфейса.
+- **Conflict Resolution**: Встроенные инструменты для разрешения конфликтов в YAML-конфигурациях.
+
+### 🤖 LLM Анализ (AI Plugin)
+- **Автоматический аудит**: Анализ состояния дашбордов на основе скриншотов и метаданных.
+- **Генерация документации**: Автоматическое описание датасетов и колонок с помощью LLM (OpenAI, OpenRouter и др.).
+- **Smart Validation**: Поиск аномалий и ошибок в визуализациях.
+
+### 🔐 Безопасность и администрирование
+- **Multi-user Auth**: Многопользовательский доступ с ролевой моделью (RBAC).
+- **Управление подключениями**: Централизованная настройка доступов к различным инстансам Superset.
+- **Логирование**: Подробная история выполнения всех фоновых задач.
+
+## Технологический стек
+- **Backend**: Python 3.9+, FastAPI, SQLAlchemy, APScheduler, Pydantic.
+- **Frontend**: Node.js 18+, SvelteKit, Tailwind CSS.
+- **Database**: SQLite (для хранения метаданных, задач и настроек доступа).
 
 ## Структура проекта
-- `backup_script.py`: Основной скрипт для выполнения запланированного резервного копирования дашбордов Superset.
+- `backend/` — Серверная часть, API и логика плагинов.
-- `migration_script.py`: Основной скрипт для переноса конкретных дашбордов между окружениями, включая переопределение соединений с базами данных.
+- `frontend/` — Клиентская часть (SvelteKit приложение).
-- `search_script.py`: Скрипт для поиска данных во всех доступных датасетах на сервере
+- `specs/` — Спецификации функций и планы реализации.
-- `run_mapper.py`: CLI-скрипт для маппинга метаданных датасетов.
+- `docs/` — Дополнительная документация по маппингу и разработке плагинов.
-- `superset_tool/`:
-    - `client.py`: Python-клиент для взаимодействия с API Superset.
-    - `exceptions.py`: Пользовательские классы исключений для структурированной обработки ошибок.
-    - `models.py`: Pydantic-модели для валидации конфигурационных данных.
-    - `utils/`:
-        - `fileio.py`: Утилиты для работы с файловой системой (работа с архивами, парсинг YAML).
-        - `logger.py`: Конфигурация логгера для единообразного логирования в проекте.
-        - `network.py`: HTTP-клиент для сетевых запросов с обработкой аутентификации и повторных попыток.
-        - `init_clients.py`: Утилита для инициализации клиентов Superset для разных окружений.
-        - `dataset_mapper.py`: Логика маппинга метаданных датасетов.
 
-## Настройка
+## Быстрый старт
 
 ### Требования
 - Python 3.9+
-- `pip` для управления пакетами.
+- Node.js 18+
-- `keyring` для безопасного хранения паролей.
+- Настроенный доступ к API Superset
 
-### Установка
+### Запуск
-1. **Клонируйте репозиторий:**
+Для автоматической настройки окружений и запуска обоих серверов (Backend & Frontend) используйте скрипт:
-    ```bash
-    git clone https://prod.gitlab.dwh.rusal.com/dwh_bi/superset-tools.git
-    cd superset-tools
-    ```
-2. **Установите зависимости:**
-    ```bash
-    pip install -r requirements.txt
-    ```
-    (Возможно, потребуется создать `requirements.txt` с `pydantic`, `requests`, `keyring`, `PyYAML`, `urllib3`)
-3. **Настройте пароли:**
-    Используйте `keyring` для хранения паролей API-пользователей Superset.
-    ```python
-    import keyring
-    keyring.set_password("system", "dev migrate", "пароль пользователя migrate_user")
-    keyring.set_password("system", "prod migrate", "пароль пользователя migrate_user")
-    keyring.set_password("system", "sandbox migrate", "пароль пользователя migrate_user")
-    ```
-
-## Использование
-
-### Запуск проекта (Web UI)
-Для запуска backend и frontend серверов одной командой:
 ```bash
 ./run.sh
 ```
+*Скрипт создаст виртуальное окружение Python, установит зависимости `pip` и `npm`, и запустит сервисы.*
+
 Опции:
-- `--skip-install`: Пропустить проверку и установку зависимостей.
+- `--skip-install`: Пропустить установку зависимостей.
 - `--help`: Показать справку.
 
 Переменные окружения:
-- `BACKEND_PORT`: Порт для backend (по умолчанию 8000).
+- `BACKEND_PORT`: Порт API (по умолчанию 8000).
-- `FRONTEND_PORT`: Порт для frontend (по умолчанию 5173).
+- `FRONTEND_PORT`: Порт UI (по умолчанию 5173).
 
-### Скрипт резервного копирования (`backup_script.py`)
+## Разработка
-Для создания резервных копий дашбордов из настроенных окружений Superset:
+Проект следует строгим правилам разработки:
-```bash
+1. **Semantic Code Generation**: Использование протокола `semantic_protocol.md` для обеспечения надежности кода.
-python backup_script.py
+2. **Design by Contract (DbC)**: Определение предусловий и постусловий для ключевых функций.
-```
+3. **Constitution**: Соблюдение правил, описанных в конституции проекта в папке `.specify/`.
-Резервные копии сохраняются в `P:\Superset\010 Бекапы\` по умолчанию. Логи хранятся в `P:\Superset\010 Бекапы\Logs`.
 
-### Скрипт миграции (`migration_script.py`)
+### Полезные команды
-Для переноса конкретного дашборда:
+- **Backend**: `cd backend && .venv/bin/python3 -m uvicorn src.app:app --reload`
-```bash
+- **Frontend**: `cd frontend && npm run dev`
-python migration_script.py
+- **Тесты**: `cd backend && .venv/bin/pytest`
-```
 
-### Скрипт поиска (`search_script.py`)
+## Контакты и вклад
-Для поиска по текстовым паттернам в метаданных датасетов Superset:
+Для добавления новых функций или исправления ошибок, пожалуйста, ознакомьтесь с `docs/plugin_dev.md` и создайте соответствующую спецификацию в `specs/`.
-```bash
-python search_script.py
-```
-Скрипт использует регулярные выражения для поиска в полях датасетов, таких как SQL-запросы. Результаты поиска выводятся в лог и в консоль.
-
-### Скрипт маппинга метаданных (`run_mapper.py`)
-Для обновления метаданных датасета (например, verbose names) в Superset:
-```bash
-python run_mapper.py --source <source_type> --dataset-id <dataset_id> [--table-name <table_name>] [--table-schema <table_schema>] [--excel-path <path_to_excel>] [--env <environment>]
-```
-Если вы используете XLSX - файл должен содержать два столбца - column_name | verbose_name
-
-
-Параметры:
-- `--source`: Источник данных ('postgres', 'excel' или 'both').
-- `--dataset-id`: ID датасета для обновления.
-- `--table-name`: Имя таблицы для PostgreSQL.
-- `--table-schema`: Схема таблицы для PostgreSQL.
-- `--excel-path`: Путь к Excel-файлу.
-- `--env`: Окружение Superset ('dev', 'prod' и т.д.).
-
-Пример использования:
-```bash
-python run_mapper.py --source postgres --dataset-id 123 --table-name account_debt --table-schema dm_view --env dev
-
-python run_mapper.py --source=excel --dataset-id=286 --excel-path=H:\dev\ss-tools\286_map.xlsx --env=dev
-```
-
-## Логирование
-Логи пишутся в файл в директории `Logs` (например, `P:\Superset\010 Бекапы\Logs` для резервных копий) и выводятся в консоль. Уровень логирования по умолчанию — `INFO`.
-
-## Разработка и вклад
-- Следуйте **Semantic Code Generation Protocol** (см. `semantic_protocol.md`):
-    - Все определения обернуты в `[DEF]...[/DEF]`.
-    - Контракты (`@PRE`, `@POST`) определяются ДО реализации.
-    - Строгая типизация и иммутабельность архитектурных решений.
-- Соблюдайте Конституцию проекта (`.specify/memory/constitution.md`).
-- Используйте `Pydantic`-модели для валидации данных.
-- Реализуйте всестороннюю обработку ошибок с помощью пользовательских исключений.

backend/backups/Logs/superset_tool_20251220.log

@@ -1,269 +0,0 @@
-2025-12-20 19:55:11,325 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 19:55:11,325 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 19:55:11,327 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 43, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 21:01:49,905 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 21:01:49,906 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 21:01:49,988 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 21:01:49,990 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 22:42:32,538 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 22:42:32,538 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 22:42:32,583 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 22:42:32,587 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 22:54:29,770 - INFO - [BackupPlugin][Entry] Starting backup for .
-2025-12-20 22:54:29,771 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 22:54:29,831 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 22:54:29,833 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 22:54:34,078 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 22:54:34,078 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 22:54:34,079 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 22:54:34,079 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 22:59:25,060 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 22:59:25,060 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 22:59:25,114 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 22:59:25,117 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:00:31,156 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:00:31,156 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:00:31,157 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:00:31,162 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:00:34,710 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:00:34,710 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:00:34,710 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:00:34,711 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:01:43,894 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:01:43,894 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:01:43,895 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:01:43,895 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:04:07,731 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:04:07,731 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:04:07,732 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:04:07,732 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:06:39,641 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:06:39,642 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:06:39,687 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:06:39,689 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-Traceback (most recent call last):
-  File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
-    config = SupersetConfig(
-             ^^^^^^^^^^^^^^^
-  File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
-    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
-                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
-base_url
-  Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
-    For further information visit https://errors.pydantic.dev/2.12/v/value_error
-2025-12-20 23:30:36,090 - INFO - [BackupPlugin][Entry] Starting backup for superset.
-2025-12-20 23:30:36,093 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
-2025-12-20 23:30:36,128 - INFO - [setup_clients][Action] Loading environments from ConfigManager
-2025-12-20 23:30:36,129 - INFO - [SupersetClient.__init__][Enter] Initializing SupersetClient.
-2025-12-20 23:30:36,129 - INFO - [APIClient.__init__][Entry] Initializing APIClient.
-2025-12-20 23:30:36,130 - WARNING - [_init_session][State] SSL verification disabled.
-2025-12-20 23:30:36,130 - INFO - [APIClient.__init__][Exit] APIClient initialized.
-2025-12-20 23:30:36,130 - INFO - [SupersetClient.__init__][Exit] SupersetClient initialized.
-2025-12-20 23:30:36,130 - INFO - [get_dashboards][Enter] Fetching dashboards.
-2025-12-20 23:30:36,131 - INFO - [authenticate][Enter] Authenticating to https://superset.bebesh.ru/api/v1
-2025-12-20 23:30:36,897 - INFO - [authenticate][Exit] Authenticated successfully.
-2025-12-20 23:30:37,527 - INFO - [get_dashboards][Exit] Found 11 dashboards.
-2025-12-20 23:30:37,527 - INFO - [BackupPlugin][Progress] Found 11 dashboards to export in superset.
-2025-12-20 23:30:37,529 - INFO - [export_dashboard][Enter] Exporting dashboard 11.
-2025-12-20 23:30:38,224 - INFO - [export_dashboard][Exit] Exported dashboard 11 to dashboard_export_20251220T203037.zip.
-2025-12-20 23:30:38,225 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:38,226 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/FCC New Coder Survey 2018/dashboard_export_20251220T203037.zip
-2025-12-20 23:30:38,227 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/FCC New Coder Survey 2018
-2025-12-20 23:30:38,230 - INFO - [export_dashboard][Enter] Exporting dashboard 10.
-2025-12-20 23:30:38,438 - INFO - [export_dashboard][Exit] Exported dashboard 10 to dashboard_export_20251220T203038.zip.
-2025-12-20 23:30:38,438 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:38,439 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/COVID Vaccine Dashboard/dashboard_export_20251220T203038.zip
-2025-12-20 23:30:38,439 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/COVID Vaccine Dashboard
-2025-12-20 23:30:38,440 - INFO - [export_dashboard][Enter] Exporting dashboard 9.
-2025-12-20 23:30:38,853 - INFO - [export_dashboard][Exit] Exported dashboard 9 to dashboard_export_20251220T203038.zip.
-2025-12-20 23:30:38,853 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:38,856 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Sales Dashboard/dashboard_export_20251220T203038.zip
-2025-12-20 23:30:38,856 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Sales Dashboard
-2025-12-20 23:30:38,858 - INFO - [export_dashboard][Enter] Exporting dashboard 8.
-2025-12-20 23:30:38,939 - INFO - [export_dashboard][Exit] Exported dashboard 8 to dashboard_export_20251220T203038.zip.
-2025-12-20 23:30:38,940 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:38,941 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Unicode Test/dashboard_export_20251220T203038.zip
-2025-12-20 23:30:38,941 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Unicode Test
-2025-12-20 23:30:38,942 - INFO - [export_dashboard][Enter] Exporting dashboard 7.
-2025-12-20 23:30:39,148 - INFO - [export_dashboard][Exit] Exported dashboard 7 to dashboard_export_20251220T203038.zip.
-2025-12-20 23:30:39,148 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:39,149 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Video Game Sales/dashboard_export_20251220T203038.zip
-2025-12-20 23:30:39,149 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Video Game Sales
-2025-12-20 23:30:39,150 - INFO - [export_dashboard][Enter] Exporting dashboard 6.
-2025-12-20 23:30:39,689 - INFO - [export_dashboard][Exit] Exported dashboard 6 to dashboard_export_20251220T203039.zip.
-2025-12-20 23:30:39,689 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:39,690 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Featured Charts/dashboard_export_20251220T203039.zip
-2025-12-20 23:30:39,691 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Featured Charts
-2025-12-20 23:30:39,692 - INFO - [export_dashboard][Enter] Exporting dashboard 5.
-2025-12-20 23:30:39,960 - INFO - [export_dashboard][Exit] Exported dashboard 5 to dashboard_export_20251220T203039.zip.
-2025-12-20 23:30:39,960 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:39,961 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Slack Dashboard/dashboard_export_20251220T203039.zip
-2025-12-20 23:30:39,961 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Slack Dashboard
-2025-12-20 23:30:39,962 - INFO - [export_dashboard][Enter] Exporting dashboard 4.
-2025-12-20 23:30:40,196 - INFO - [export_dashboard][Exit] Exported dashboard 4 to dashboard_export_20251220T203039.zip.
-2025-12-20 23:30:40,196 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:40,197 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/deck.gl Demo/dashboard_export_20251220T203039.zip
-2025-12-20 23:30:40,197 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/deck.gl Demo
-2025-12-20 23:30:40,198 - INFO - [export_dashboard][Enter] Exporting dashboard 3.
-2025-12-20 23:30:40,745 - INFO - [export_dashboard][Exit] Exported dashboard 3 to dashboard_export_20251220T203040.zip.
-2025-12-20 23:30:40,746 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:40,760 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Misc Charts/dashboard_export_20251220T203040.zip
-2025-12-20 23:30:40,761 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Misc Charts
-2025-12-20 23:30:40,762 - INFO - [export_dashboard][Enter] Exporting dashboard 2.
-2025-12-20 23:30:40,928 - INFO - [export_dashboard][Exit] Exported dashboard 2 to dashboard_export_20251220T203040.zip.
-2025-12-20 23:30:40,929 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:40,930 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/USA Births Names/dashboard_export_20251220T203040.zip
-2025-12-20 23:30:40,931 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/USA Births Names
-2025-12-20 23:30:40,932 - INFO - [export_dashboard][Enter] Exporting dashboard 1.
-2025-12-20 23:30:41,582 - INFO - [export_dashboard][Exit] Exported dashboard 1 to dashboard_export_20251220T203040.zip.
-2025-12-20 23:30:41,582 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
-2025-12-20 23:30:41,749 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/World Bank's Data/dashboard_export_20251220T203040.zip
-2025-12-20 23:30:41,750 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/World Bank's Data
-2025-12-20 23:30:41,752 - INFO - [consolidate_archive_folders][Enter] Consolidating archives in backups/SUPERSET
-2025-12-20 23:30:41,753 - INFO - [remove_empty_directories][Enter] Starting cleanup of empty directories in backups/SUPERSET
-2025-12-20 23:30:41,758 - INFO - [remove_empty_directories][Exit] Removed 0 empty directories.
-2025-12-20 23:30:41,758 - INFO - [BackupPlugin][CoherenceCheck:Passed] Backup logic completed for superset.

backend/get_full_key.py

@@ -0,0 +1 @@
+{"print(f'Length": {"else": "print('Provider not found')\ndb.close()"}}

backend/requirements.txt

@@ -25,9 +25,13 @@ keyring==25.7.0
 more-itertools==10.8.0
 pycparser==2.23
 pydantic==2.12.5
+pydantic-settings
 pydantic_core==2.41.5
 python-multipart==0.0.21
 PyYAML==6.0.3
+passlib[bcrypt]
+python-jose[cryptography]
+PyJWT
 RapidFuzz==3.14.3
 referencing==0.37.0
 requests==2.32.5
@@ -44,4 +48,9 @@ websockets==15.0.1
 pandas
 psycopg2-binary
 openpyxl
-GitPython==3.1.44
+GitPython==3.1.44
+itsdangerous
+email-validator
+openai
+playwright
+tenacity

backend/src/api/auth.py

@@ -1,59 +1,118 @@
-# [DEF:AuthModule:Module]
+# [DEF:backend.src.api.auth:Module]
-# @SEMANTICS: auth, authentication, adfs, oauth, middleware
+#
-# @PURPOSE: Implements ADFS authentication using Authlib for FastAPI. It provides a dependency to protect endpoints.
+# @SEMANTICS: api, auth, routes, login, logout
-# @LAYER: UI (API)
+# @PURPOSE:   Authentication API endpoints.
-# @RELATION: Used by API routers to protect endpoints that require authentication.
+# @LAYER:     API
+# @RELATION:  USES -> backend.src.services.auth_service.AuthService
+# @RELATION:  USES -> backend.src.core.database.get_auth_db
+#
+# @INVARIANT: All auth endpoints must return consistent error codes.
 
-from fastapi import Depends, HTTPException, status
+# [SECTION: IMPORTS]
-from fastapi.security import OAuth2AuthorizationCodeBearer
+from fastapi import APIRouter, Depends, HTTPException, status
-from authlib.integrations.starlette_client import OAuth
+from fastapi.security import OAuth2PasswordRequestForm
-from starlette.config import Config
+from sqlalchemy.orm import Session
+from ..core.database import get_auth_db
+from ..services.auth_service import AuthService
+from ..schemas.auth import Token, User as UserSchema
+from ..dependencies import get_current_user
+from ..core.auth.oauth import oauth, is_adfs_configured
+from ..core.auth.logger import log_security_event
+from ..core.logger import belief_scope
+import starlette.requests
+# [/SECTION]
 
-# Placeholder for ADFS configuration. In a real app, this would come from a secure source.
+# [DEF:router:Variable]
-# Create an in-memory .env file
+# @PURPOSE: APIRouter instance for authentication routes.
-from io import StringIO
+router = APIRouter(prefix="/api/auth", tags=["auth"])
-config_data = StringIO("""
+# [/DEF:router:Variable]
-ADFS_CLIENT_ID=your-client-id
-ADFS_CLIENT_SECRET=your-client-secret
-ADFS_SERVER_METADATA_URL=https://your-adfs-server/.well-known/openid-configuration
-""")
-config = Config(config_data)
-oauth = OAuth(config)
 
-oauth.register(
+# [DEF:login_for_access_token:Function]
-    name='adfs',
+# @PURPOSE: Authenticates a user and returns a JWT access token.
-    server_metadata_url=config('ADFS_SERVER_METADATA_URL'),
+# @PRE:     form_data contains username and password.
-    client_kwargs={'scope': 'openid profile email'}
+# @POST:    Returns a Token object on success.
-)
+# @THROW:   HTTPException 401 if authentication fails.
+# @PARAM:   form_data (OAuth2PasswordRequestForm) - Login credentials.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  Token - The generated JWT token.
+@router.post("/login", response_model=Token)
+async def login_for_access_token(
+    form_data: OAuth2PasswordRequestForm = Depends(),
+    db: Session = Depends(get_auth_db)
+):
+    with belief_scope("api.auth.login"):
+        auth_service = AuthService(db)
+        user = auth_service.authenticate_user(form_data.username, form_data.password)
+        if not user:
+            log_security_event("LOGIN_FAILED", form_data.username, {"reason": "Invalid credentials"})
+            raise HTTPException(
+                status_code=status.HTTP_401_UNAUTHORIZED,
+                detail="Incorrect username or password",
+                headers={"WWW-Authenticate": "Bearer"},
+            )
+        log_security_event("LOGIN_SUCCESS", user.username, {"source": "LOCAL"})
+        return auth_service.create_session(user)
+# [/DEF:login_for_access_token:Function]
 
-oauth2_scheme = OAuth2AuthorizationCodeBearer(
+# [DEF:read_users_me:Function]
-    authorizationUrl="https://your-adfs-server/adfs/oauth2/authorize",
+# @PURPOSE: Retrieves the profile of the currently authenticated user.
-    tokenUrl="https://your-adfs-server/adfs/oauth2/token",
+# @PRE:     Valid JWT token provided.
-)
+# @POST:    Returns the current user's data.
+# @PARAM:   current_user (UserSchema) - The user extracted from the token.
+# @RETURN:  UserSchema - The current user profile.
+@router.get("/me", response_model=UserSchema)
+async def read_users_me(current_user: UserSchema = Depends(get_current_user)):
+    with belief_scope("api.auth.me"):
+        return current_user
+# [/DEF:read_users_me:Function]
 
-# [DEF:get_current_user:Function]
+# [DEF:logout:Function]
-# @PURPOSE: Dependency to get the current user from the ADFS token.
+# @PURPOSE: Logs out the current user (placeholder for session revocation).
-# @PARAM: token (str) - The OAuth2 bearer token.
+# @PRE:     Valid JWT token provided.
-# @PRE: token should be provided via Authorization header.
+# @POST:    Returns success message.
-# @POST: Returns user details if authenticated, else raises 401.
+@router.post("/logout")
-# @RETURN: Dict[str, str] - User information.
+async def logout(current_user: UserSchema = Depends(get_current_user)):
-async def get_current_user(token: str = Depends(oauth2_scheme)):
+    with belief_scope("api.auth.logout"):
-    """
+        log_security_event("LOGOUT", current_user.username)
-    Dependency to get the current user from the ADFS token.
+        # In a stateless JWT setup, client-side token deletion is primary.
-    This is a placeholder and needs to be fully implemented.
+        # Server-side revocation (blacklisting) can be added here if needed.
-    """
+        return {"message": "Successfully logged out"}
-    # In a real implementation, you would:
+# [/DEF:logout:Function]
-    # 1. Validate the token with ADFS.
+
-    # 2. Fetch user information.
+# [DEF:login_adfs:Function]
-    # 3. Create a user object.
+# @PURPOSE: Initiates the ADFS OIDC login flow.
-    # For now, we'll just check if a token exists.
+# @POST:    Redirects the user to ADFS.
-    if not token:
+@router.get("/login/adfs")
-        raise HTTPException(
+async def login_adfs(request: starlette.requests.Request):
-            status_code=status.HTTP_401_UNAUTHORIZED,
+    with belief_scope("api.auth.login_adfs"):
-            detail="Not authenticated",
+        if not is_adfs_configured():
-            headers={"WWW-Authenticate": "Bearer"},
+            raise HTTPException(
-        )
+                status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
-    # A real implementation would return a user object.
+                detail="ADFS is not configured. Please set ADFS_CLIENT_ID, ADFS_CLIENT_SECRET, and ADFS_METADATA_URL environment variables."
-    return {"placeholder_user": "user@example.com"}
+            )
-# [/DEF:get_current_user:Function]
+        redirect_uri = request.url_for('auth_callback_adfs')
-# [/DEF:AuthModule:Module]
+        return await oauth.adfs.authorize_redirect(request, str(redirect_uri))
+# [/DEF:login_adfs:Function]
+
+# [DEF:auth_callback_adfs:Function]
+# @PURPOSE: Handles the callback from ADFS after successful authentication.
+# @POST:    Provisions user JIT and returns session token.
+@router.get("/callback/adfs", name="auth_callback_adfs")
+async def auth_callback_adfs(request: starlette.requests.Request, db: Session = Depends(get_auth_db)):
+    with belief_scope("api.auth.callback_adfs"):
+        if not is_adfs_configured():
+            raise HTTPException(
+                status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+                detail="ADFS is not configured. Please set ADFS_CLIENT_ID, ADFS_CLIENT_SECRET, and ADFS_METADATA_URL environment variables."
+            )
+        token = await oauth.adfs.authorize_access_token(request)
+        user_info = token.get('userinfo')
+        if not user_info:
+            raise HTTPException(status_code=400, detail="Failed to retrieve user info from ADFS")
+        
+        auth_service = AuthService(db)
+        user = auth_service.provision_adfs_user(user_info)
+        return auth_service.create_session(user)
+# [/DEF:auth_callback_adfs:Function]
+
+# [/DEF:backend.src.api.auth:Module]

backend/src/api/routes/__init__.py

@@ -1 +1 @@
-from . import plugins, tasks, settings, connections, environments, mappings, migration, git
+from . import plugins, tasks, settings, connections, environments, mappings, migration, git, storage, admin

backend/src/api/routes/admin.py

@@ -0,0 +1,310 @@
+# [DEF:backend.src.api.routes.admin:Module]
+#
+# @TIER:      STANDARD
+# @SEMANTICS: api, admin, users, roles, permissions
+# @PURPOSE:   Admin API endpoints for user and role management.
+# @LAYER:     API
+# @RELATION:  USES -> backend.src.core.auth.repository.AuthRepository
+# @RELATION:  USES -> backend.src.dependencies.has_permission
+#
+# @INVARIANT: All endpoints in this module require 'Admin' role or 'admin' scope.
+
+# [SECTION: IMPORTS]
+from typing import List
+from fastapi import APIRouter, Depends, HTTPException, status
+from sqlalchemy.orm import Session
+from ...core.database import get_auth_db
+from ...core.auth.repository import AuthRepository
+from ...core.auth.security import get_password_hash
+from ...schemas.auth import (
+    User as UserSchema, UserCreate, UserUpdate,
+    RoleSchema, RoleCreate, RoleUpdate, PermissionSchema,
+    ADGroupMappingSchema, ADGroupMappingCreate
+)
+from ...models.auth import User, Role, Permission, ADGroupMapping
+from ...dependencies import has_permission, get_current_user
+from ...core.logger import logger, belief_scope
+# [/SECTION]
+
+# [DEF:router:Variable]
+# @PURPOSE: APIRouter instance for admin routes.
+router = APIRouter(prefix="/api/admin", tags=["admin"])
+# [/DEF:router:Variable]
+
+# [DEF:list_users:Function]
+# @PURPOSE: Lists all registered users.
+# @PRE:     Current user has 'Admin' role.
+# @POST:    Returns a list of UserSchema objects.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  List[UserSchema] - List of users.
+@router.get("/users", response_model=List[UserSchema])
+async def list_users(
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:users", "READ"))
+):
+    with belief_scope("api.admin.list_users"):
+        users = db.query(User).all()
+        return users
+# [/DEF:list_users:Function]
+
+# [DEF:create_user:Function]
+# @PURPOSE: Creates a new local user.
+# @PRE:     Current user has 'Admin' role.
+# @POST:    New user is created in the database.
+# @PARAM:   user_in (UserCreate) - New user data.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  UserSchema - The created user.
+@router.post("/users", response_model=UserSchema, status_code=status.HTTP_201_CREATED)
+async def create_user(
+    user_in: UserCreate,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:users", "WRITE"))
+):
+    with belief_scope("api.admin.create_user"):
+        repo = AuthRepository(db)
+        if repo.get_user_by_username(user_in.username):
+            raise HTTPException(status_code=400, detail="Username already exists")
+        
+        new_user = User(
+            username=user_in.username,
+            email=user_in.email,
+            password_hash=get_password_hash(user_in.password),
+            auth_source="LOCAL",
+            is_active=user_in.is_active
+        )
+        
+        for role_name in user_in.roles:
+            role = repo.get_role_by_name(role_name)
+            if role:
+                new_user.roles.append(role)
+        
+        db.add(new_user)
+        db.commit()
+        db.refresh(new_user)
+        return new_user
+# [/DEF:create_user:Function]
+
+# [DEF:update_user:Function]
+# @PURPOSE: Updates an existing user.
+@router.put("/users/{user_id}", response_model=UserSchema)
+async def update_user(
+    user_id: str,
+    user_in: UserUpdate,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:users", "WRITE"))
+):
+    with belief_scope("api.admin.update_user"):
+        repo = AuthRepository(db)
+        user = repo.get_user_by_id(user_id)
+        if not user:
+            raise HTTPException(status_code=404, detail="User not found")
+        
+        if user_in.email is not None:
+            user.email = user_in.email
+        if user_in.is_active is not None:
+            user.is_active = user_in.is_active
+        if user_in.password is not None:
+            user.password_hash = get_password_hash(user_in.password)
+        
+        if user_in.roles is not None:
+            user.roles = []
+            for role_name in user_in.roles:
+                role = repo.get_role_by_name(role_name)
+                if role:
+                    user.roles.append(role)
+        
+        db.commit()
+        db.refresh(user)
+        return user
+# [/DEF:update_user:Function]
+
+# [DEF:delete_user:Function]
+# @PURPOSE: Deletes a user.
+@router.delete("/users/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def delete_user(
+    user_id: str,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:users", "WRITE"))
+):
+    with belief_scope("api.admin.delete_user"):
+        logger.info(f"[DEBUG] Attempting to delete user context={{'user_id': '{user_id}'}}")
+        repo = AuthRepository(db)
+        user = repo.get_user_by_id(user_id)
+        if not user:
+            logger.warning(f"[DEBUG] User not found for deletion context={{'user_id': '{user_id}'}}")
+            raise HTTPException(status_code=404, detail="User not found")
+        
+        logger.info(f"[DEBUG] Found user to delete context={{'username': '{user.username}'}}")
+        db.delete(user)
+        db.commit()
+        logger.info(f"[DEBUG] Successfully deleted user context={{'user_id': '{user_id}'}}")
+        return None
+# [/DEF:delete_user:Function]
+
+# [DEF:list_roles:Function]
+# @PURPOSE: Lists all available roles.
+# @RETURN:  List[RoleSchema] - List of roles.
+# @RELATION: CALLS -> backend.src.models.auth.Role
+@router.get("/roles", response_model=List[RoleSchema])
+async def list_roles(
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:roles", "READ"))
+):
+    with belief_scope("api.admin.list_roles"):
+        return db.query(Role).all()
+# [/DEF:list_roles:Function]
+
+# [DEF:create_role:Function]
+# @PURPOSE: Creates a new system role with associated permissions.
+# @PRE:     Role name must be unique.
+# @POST:    New Role record is created in auth.db.
+# @PARAM:   role_in (RoleCreate) - New role data.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  RoleSchema - The created role.
+# @SIDE_EFFECT: Commits new role and associations to auth.db.
+# @RELATION: CALLS -> backend.src.core.auth.repository.AuthRepository.get_permission_by_id
+@router.post("/roles", response_model=RoleSchema, status_code=status.HTTP_201_CREATED)
+async def create_role(
+    role_in: RoleCreate,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:roles", "WRITE"))
+):
+    with belief_scope("api.admin.create_role"):
+        if db.query(Role).filter(Role.name == role_in.name).first():
+            raise HTTPException(status_code=400, detail="Role already exists")
+        
+        new_role = Role(name=role_in.name, description=role_in.description)
+        repo = AuthRepository(db)
+        
+        for perm_id_or_str in role_in.permissions:
+            perm = repo.get_permission_by_id(perm_id_or_str)
+            if not perm and ":" in perm_id_or_str:
+                res, act = perm_id_or_str.split(":", 1)
+                perm = repo.get_permission_by_resource_action(res, act)
+            
+            if perm:
+                new_role.permissions.append(perm)
+        
+        db.add(new_role)
+        db.commit()
+        db.refresh(new_role)
+        return new_role
+# [/DEF:create_role:Function]
+
+# [DEF:update_role:Function]
+# @PURPOSE: Updates an existing role's metadata and permissions.
+# @PRE:     role_id must be a valid existing role UUID.
+# @POST:    Role record is updated in auth.db.
+# @PARAM:   role_id (str) - Target role identifier.
+# @PARAM:   role_in (RoleUpdate) - Updated role data.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  RoleSchema - The updated role.
+# @SIDE_EFFECT: Commits updates to auth.db.
+# @RELATION: CALLS -> backend.src.core.auth.repository.AuthRepository.get_role_by_id
+@router.put("/roles/{role_id}", response_model=RoleSchema)
+async def update_role(
+    role_id: str,
+    role_in: RoleUpdate,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:roles", "WRITE"))
+):
+    with belief_scope("api.admin.update_role"):
+        repo = AuthRepository(db)
+        role = repo.get_role_by_id(role_id)
+        if not role:
+            raise HTTPException(status_code=404, detail="Role not found")
+        
+        if role_in.name is not None:
+            role.name = role_in.name
+        if role_in.description is not None:
+            role.description = role_in.description
+        
+        if role_in.permissions is not None:
+            role.permissions = []
+            for perm_id_or_str in role_in.permissions:
+                perm = repo.get_permission_by_id(perm_id_or_str)
+                if not perm and ":" in perm_id_or_str:
+                    res, act = perm_id_or_str.split(":", 1)
+                    perm = repo.get_permission_by_resource_action(res, act)
+                
+                if perm:
+                    role.permissions.append(perm)
+        
+        db.commit()
+        db.refresh(role)
+        return role
+# [/DEF:update_role:Function]
+
+# [DEF:delete_role:Function]
+# @PURPOSE: Removes a role from the system.
+# @PRE:     role_id must be a valid existing role UUID.
+# @POST:    Role record is removed from auth.db.
+# @PARAM:   role_id (str) - Target role identifier.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  None
+# @SIDE_EFFECT: Deletes record from auth.db and commits.
+# @RELATION: CALLS -> backend.src.core.auth.repository.AuthRepository.get_role_by_id
+@router.delete("/roles/{role_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def delete_role(
+    role_id: str,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:roles", "WRITE"))
+):
+    with belief_scope("api.admin.delete_role"):
+        repo = AuthRepository(db)
+        role = repo.get_role_by_id(role_id)
+        if not role:
+            raise HTTPException(status_code=404, detail="Role not found")
+        
+        db.delete(role)
+        db.commit()
+        return None
+# [/DEF:delete_role:Function]
+
+# [DEF:list_permissions:Function]
+# @PURPOSE: Lists all available system permissions for assignment.
+# @POST:    Returns a list of all PermissionSchema objects.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  List[PermissionSchema] - List of permissions.
+# @RELATION: CALLS -> backend.src.core.auth.repository.AuthRepository.list_permissions
+@router.get("/permissions", response_model=List[PermissionSchema])
+async def list_permissions(
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:roles", "READ"))
+):
+    with belief_scope("api.admin.list_permissions"):
+        repo = AuthRepository(db)
+        return repo.list_permissions()
+# [/DEF:list_permissions:Function]
+
+# [DEF:list_ad_mappings:Function]
+# @PURPOSE: Lists all AD Group to Role mappings.
+@router.get("/ad-mappings", response_model=List[ADGroupMappingSchema])
+async def list_ad_mappings(
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
+    with belief_scope("api.admin.list_ad_mappings"):
+        return db.query(ADGroupMapping).all()
+# [/DEF:list_ad_mappings:Function]
+
+# [DEF:create_ad_mapping:Function]
+# @PURPOSE: Creates a new AD Group mapping.
+@router.post("/ad-mappings", response_model=ADGroupMappingSchema)
+async def create_ad_mapping(
+    mapping_in: ADGroupMappingCreate,
+    db: Session = Depends(get_auth_db),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
+):
+    with belief_scope("api.admin.create_ad_mapping"):
+        new_mapping = ADGroupMapping(
+            ad_group=mapping_in.ad_group,
+            role_id=mapping_in.role_id
+        )
+        db.add(new_mapping)
+        db.commit()
+        db.refresh(new_mapping)
+        return new_mapping
+# [/DEF:create_ad_mapping:Function]
+
+# [/DEF:backend.src.api.routes.admin:Module]

backend/src/api/routes/environments.py

@@ -11,7 +11,7 @@
 # [SECTION: IMPORTS]
 from fastapi import APIRouter, Depends, HTTPException
 from typing import List, Dict, Optional
-from ...dependencies import get_config_manager, get_scheduler_service
+from ...dependencies import get_config_manager, get_scheduler_service, has_permission
 from ...core.superset_client import SupersetClient
 from pydantic import BaseModel, Field
 from ...core.config_models import Environment as EnvModel
@@ -23,7 +23,7 @@ router = APIRouter()
 # [DEF:ScheduleSchema:DataClass]
 class ScheduleSchema(BaseModel):
     enabled: bool = False
-    cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){5,7})$')
+    cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){4,6})$')
 # [/DEF:ScheduleSchema:DataClass]
 
 # [DEF:EnvironmentResponse:DataClass]
@@ -47,7 +47,10 @@ class DatabaseResponse(BaseModel):
 # @POST:    Returns a list of EnvironmentResponse objects.
 # @RETURN:  List[EnvironmentResponse]
 @router.get("", response_model=List[EnvironmentResponse])
-async def get_environments(config_manager=Depends(get_config_manager)):
+async def get_environments(
+    config_manager=Depends(get_config_manager),
+    _ = Depends(has_permission("environments", "READ"))
+):
     with belief_scope("get_environments"):
         envs = config_manager.get_environments()
     # Ensure envs is a list
@@ -77,7 +80,8 @@ async def update_environment_schedule(
     id: str,
     schedule: ScheduleSchema,
     config_manager=Depends(get_config_manager),
-    scheduler_service=Depends(get_scheduler_service)
+    scheduler_service=Depends(get_scheduler_service),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
 ):
     with belief_scope("update_environment_schedule", f"id={id}"):
         envs = config_manager.get_environments()
@@ -104,7 +108,11 @@ async def update_environment_schedule(
 # @PARAM:   id (str) - The environment ID.
 # @RETURN:  List[Dict] - List of databases.
 @router.get("/{id}/databases")
-async def get_environment_databases(id: str, config_manager=Depends(get_config_manager)):
+async def get_environment_databases(
+    id: str,
+    config_manager=Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
     with belief_scope("get_environment_databases", f"id={id}"):
         envs = config_manager.get_environments()
     env = next((e for e in envs if e.id == id), None)

backend/src/api/routes/git.py

@@ -13,7 +13,7 @@ from fastapi import APIRouter, Depends, HTTPException
 from sqlalchemy.orm import Session
 from typing import List, Optional
 import typing
-from src.dependencies import get_config_manager
+from src.dependencies import get_config_manager, has_permission
 from src.core.database import get_db
 from src.models.git import GitServerConfig, GitStatus, DeploymentEnvironment, GitRepository
 from src.api.routes.git_schemas import (
@@ -34,7 +34,10 @@ git_service = GitService()
 # @POST: Returns a list of all GitServerConfig objects from the database.
 # @RETURN: List[GitServerConfigSchema]
 @router.get("/config", response_model=List[GitServerConfigSchema])
-async def get_git_configs(db: Session = Depends(get_db)):
+async def get_git_configs(
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
     with belief_scope("get_git_configs"):
         return db.query(GitServerConfig).all()
 # [/DEF:get_git_configs:Function]
@@ -46,7 +49,11 @@ async def get_git_configs(db: Session = Depends(get_db)):
 # @PARAM: config (GitServerConfigCreate)
 # @RETURN: GitServerConfigSchema
 @router.post("/config", response_model=GitServerConfigSchema)
-async def create_git_config(config: GitServerConfigCreate, db: Session = Depends(get_db)):
+async def create_git_config(
+    config: GitServerConfigCreate,
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
+):
     with belief_scope("create_git_config"):
         db_config = GitServerConfig(**config.dict())
         db.add(db_config)
@@ -61,7 +68,11 @@ async def create_git_config(config: GitServerConfigCreate, db: Session = Depends
 # @POST: The configuration record is removed from the database.
 # @PARAM: config_id (str)
 @router.delete("/config/{config_id}")
-async def delete_git_config(config_id: str, db: Session = Depends(get_db)):
+async def delete_git_config(
+    config_id: str,
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
+):
     with belief_scope("delete_git_config"):
         db_config = db.query(GitServerConfig).filter(GitServerConfig.id == config_id).first()
         if not db_config:
@@ -78,7 +89,10 @@ async def delete_git_config(config_id: str, db: Session = Depends(get_db)):
 # @POST: Returns success if the connection is validated via GitService.
 # @PARAM: config (GitServerConfigCreate)
 @router.post("/config/test")
-async def test_git_config(config: GitServerConfigCreate):
+async def test_git_config(
+    config: GitServerConfigCreate,
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
     with belief_scope("test_git_config"):
         success = await git_service.test_connection(config.provider, config.url, config.pat)
         if success:
@@ -94,7 +108,12 @@ async def test_git_config(config: GitServerConfigCreate):
 # @PARAM: dashboard_id (int)
 # @PARAM: init_data (RepoInitRequest)
 @router.post("/repositories/{dashboard_id}/init")
-async def init_repository(dashboard_id: int, init_data: RepoInitRequest, db: Session = Depends(get_db)):
+async def init_repository(
+    dashboard_id: int,
+    init_data: RepoInitRequest,
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("init_repository"):
         # 1. Get config
         config = db.query(GitServerConfig).filter(GitServerConfig.id == init_data.config_id).first()
@@ -138,7 +157,10 @@ async def init_repository(dashboard_id: int, init_data: RepoInitRequest, db: Ses
 # @PARAM: dashboard_id (int)
 # @RETURN: List[BranchSchema]
 @router.get("/repositories/{dashboard_id}/branches", response_model=List[BranchSchema])
-async def get_branches(dashboard_id: int):
+async def get_branches(
+    dashboard_id: int,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("get_branches"):
         try:
             return git_service.list_branches(dashboard_id)
@@ -153,7 +175,11 @@ async def get_branches(dashboard_id: int):
 # @PARAM: dashboard_id (int)
 # @PARAM: branch_data (BranchCreate)
 @router.post("/repositories/{dashboard_id}/branches")
-async def create_branch(dashboard_id: int, branch_data: BranchCreate):
+async def create_branch(
+    dashboard_id: int,
+    branch_data: BranchCreate,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("create_branch"):
         try:
             git_service.create_branch(dashboard_id, branch_data.name, branch_data.from_branch)
@@ -169,7 +195,11 @@ async def create_branch(dashboard_id: int, branch_data: BranchCreate):
 # @PARAM: dashboard_id (int)
 # @PARAM: checkout_data (BranchCheckout)
 @router.post("/repositories/{dashboard_id}/checkout")
-async def checkout_branch(dashboard_id: int, checkout_data: BranchCheckout):
+async def checkout_branch(
+    dashboard_id: int,
+    checkout_data: BranchCheckout,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("checkout_branch"):
         try:
             git_service.checkout_branch(dashboard_id, checkout_data.name)
@@ -185,7 +215,11 @@ async def checkout_branch(dashboard_id: int, checkout_data: BranchCheckout):
 # @PARAM: dashboard_id (int)
 # @PARAM: commit_data (CommitCreate)
 @router.post("/repositories/{dashboard_id}/commit")
-async def commit_changes(dashboard_id: int, commit_data: CommitCreate):
+async def commit_changes(
+    dashboard_id: int,
+    commit_data: CommitCreate,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("commit_changes"):
         try:
             git_service.commit_changes(dashboard_id, commit_data.message, commit_data.files)
@@ -200,7 +234,10 @@ async def commit_changes(dashboard_id: int, commit_data: CommitCreate):
 # @POST: Local commits are pushed to the remote repository.
 # @PARAM: dashboard_id (int)
 @router.post("/repositories/{dashboard_id}/push")
-async def push_changes(dashboard_id: int):
+async def push_changes(
+    dashboard_id: int,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("push_changes"):
         try:
             git_service.push_changes(dashboard_id)
@@ -215,7 +252,10 @@ async def push_changes(dashboard_id: int):
 # @POST: Remote changes are fetched and merged into the local branch.
 # @PARAM: dashboard_id (int)
 @router.post("/repositories/{dashboard_id}/pull")
-async def pull_changes(dashboard_id: int):
+async def pull_changes(
+    dashboard_id: int,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("pull_changes"):
         try:
             git_service.pull_changes(dashboard_id)
@@ -231,7 +271,11 @@ async def pull_changes(dashboard_id: int):
 # @PARAM: dashboard_id (int)
 # @PARAM: source_env_id (Optional[str])
 @router.post("/repositories/{dashboard_id}/sync")
-async def sync_dashboard(dashboard_id: int, source_env_id: typing.Optional[str] = None):
+async def sync_dashboard(
+    dashboard_id: int,
+    source_env_id: typing.Optional[str] = None,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("sync_dashboard"):
         try:
             from src.plugins.git_plugin import GitPlugin
@@ -251,7 +295,10 @@ async def sync_dashboard(dashboard_id: int, source_env_id: typing.Optional[str]
 # @POST: Returns a list of DeploymentEnvironmentSchema objects.
 # @RETURN: List[DeploymentEnvironmentSchema]
 @router.get("/environments", response_model=List[DeploymentEnvironmentSchema])
-async def get_environments(config_manager=Depends(get_config_manager)):
+async def get_environments(
+    config_manager=Depends(get_config_manager),
+    _ = Depends(has_permission("environments", "READ"))
+):
     with belief_scope("get_environments"):
         envs = config_manager.get_environments()
         return [
@@ -271,7 +318,11 @@ async def get_environments(config_manager=Depends(get_config_manager)):
 # @PARAM: dashboard_id (int)
 # @PARAM: deploy_data (DeployRequest)
 @router.post("/repositories/{dashboard_id}/deploy")
-async def deploy_dashboard(dashboard_id: int, deploy_data: DeployRequest):
+async def deploy_dashboard(
+    dashboard_id: int,
+    deploy_data: DeployRequest,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("deploy_dashboard"):
         try:
             from src.plugins.git_plugin import GitPlugin
@@ -293,7 +344,11 @@ async def deploy_dashboard(dashboard_id: int, deploy_data: DeployRequest):
 # @PARAM: limit (int)
 # @RETURN: List[CommitSchema]
 @router.get("/repositories/{dashboard_id}/history", response_model=List[CommitSchema])
-async def get_history(dashboard_id: int, limit: int = 50):
+async def get_history(
+    dashboard_id: int,
+    limit: int = 50,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("get_history"):
         try:
             return git_service.get_commit_history(dashboard_id, limit)
@@ -308,7 +363,10 @@ async def get_history(dashboard_id: int, limit: int = 50):
 # @PARAM: dashboard_id (int)
 # @RETURN: dict
 @router.get("/repositories/{dashboard_id}/status")
-async def get_repository_status(dashboard_id: int):
+async def get_repository_status(
+    dashboard_id: int,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("get_repository_status"):
         try:
             return git_service.get_status(dashboard_id)
@@ -325,7 +383,12 @@ async def get_repository_status(dashboard_id: int):
 # @PARAM: staged (bool)
 # @RETURN: str
 @router.get("/repositories/{dashboard_id}/diff")
-async def get_repository_diff(dashboard_id: int, file_path: Optional[str] = None, staged: bool = False):
+async def get_repository_diff(
+    dashboard_id: int,
+    file_path: Optional[str] = None,
+    staged: bool = False,
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
     with belief_scope("get_repository_diff"):
         try:
             diff_text = git_service.get_diff(dashboard_id, file_path, staged)
@@ -334,4 +397,59 @@ async def get_repository_diff(dashboard_id: int, file_path: Optional[str] = None
             raise HTTPException(status_code=400, detail=str(e))
 # [/DEF:get_repository_diff:Function]
 
+# [DEF:generate_commit_message:Function]
+# @PURPOSE: Generate a suggested commit message using LLM.
+# @PRE: Repository for `dashboard_id` is initialized.
+# @POST: Returns a suggested commit message string.
+@router.post("/repositories/{dashboard_id}/generate-message")
+async def generate_commit_message(
+    dashboard_id: int,
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("plugin:git", "EXECUTE"))
+):
+    with belief_scope("generate_commit_message"):
+        try:
+            # 1. Get Diff
+            diff = git_service.get_diff(dashboard_id, staged=True)
+            if not diff:
+                diff = git_service.get_diff(dashboard_id, staged=False)
+            
+            if not diff:
+                return {"message": "No changes detected"}
+
+            # 2. Get History
+            history_objs = git_service.get_commit_history(dashboard_id, limit=5)
+            history = [h.message for h in history_objs if hasattr(h, 'message')]
+
+            # 3. Get LLM Client
+            from ...services.llm_provider import LLMProviderService
+            from ...plugins.llm_analysis.service import LLMClient
+            from ...plugins.llm_analysis.models import LLMProviderType
+            
+            llm_service = LLMProviderService(db)
+            providers = llm_service.get_all_providers()
+            provider = next((p for p in providers if p.is_active), None)
+            
+            if not provider:
+                raise HTTPException(status_code=400, detail="No active LLM provider found")
+            
+            api_key = llm_service.get_decrypted_api_key(provider.id)
+            client = LLMClient(
+                provider_type=LLMProviderType(provider.provider_type),
+                api_key=api_key,
+                base_url=provider.base_url,
+                default_model=provider.default_model
+            )
+
+            # 4. Generate Message
+            from ...plugins.git.llm_extension import GitLLMExtension
+            extension = GitLLMExtension(client)
+            message = await extension.suggest_commit_message(diff, history)
+            
+            return {"message": message}
+        except Exception as e:
+            logger.error(f"Failed to generate commit message: {e}")
+            raise HTTPException(status_code=400, detail=str(e))
+# [/DEF:generate_commit_message:Function]
+
 # [/DEF:backend.src.api.routes.git:Module]

backend/src/api/routes/git_schemas.py

@@ -1,5 +1,6 @@
 # [DEF:backend.src.api.routes.git_schemas:Module]
 #
+# @TIER:      STANDARD
 # @SEMANTICS: git, schemas, pydantic, api, contracts
 # @PURPOSE:   Defines Pydantic models for the Git integration API layer.
 # @LAYER:     API
@@ -14,6 +15,7 @@ from uuid import UUID
 from src.models.git import GitProvider, GitStatus, SyncStatus
 
 # [DEF:GitServerConfigBase:Class]
+# @TIER: TRIVIAL
 # @PURPOSE: Base schema for Git server configuration attributes.
 class GitServerConfigBase(BaseModel):
     name: str = Field(..., description="Display name for the Git server")

backend/src/api/routes/llm.py

@@ -0,0 +1,207 @@
+# [DEF:backend/src/api/routes/llm.py:Module]
+# @TIER: STANDARD
+# @SEMANTICS: api, routes, llm
+# @PURPOSE: API routes for LLM provider configuration and management.
+# @LAYER: UI (API)
+
+from fastapi import APIRouter, Depends, HTTPException, status
+from typing import List
+from ...core.logger import logger
+from ...schemas.auth import User
+from ...dependencies import get_current_user as get_current_active_user
+from ...plugins.llm_analysis.models import LLMProviderConfig, LLMProviderType
+from ...services.llm_provider import LLMProviderService
+from ...core.database import get_db
+from sqlalchemy.orm import Session
+
+# [DEF:router:Global]
+# @PURPOSE: APIRouter instance for LLM routes.
+router = APIRouter(prefix="/api/llm", tags=["LLM"])
+# [/DEF:router:Global]
+
+# [DEF:get_providers:Function]
+# @PURPOSE: Retrieve all LLM provider configurations.
+# @PRE: User is authenticated.
+# @POST: Returns list of LLMProviderConfig.
+@router.get("/providers", response_model=List[LLMProviderConfig])
+async def get_providers(
+    current_user: User = Depends(get_current_active_user),
+    db: Session = Depends(get_db)
+):
+    """
+    Get all LLM provider configurations.
+    """
+    logger.info(f"[llm_routes][get_providers][Action] Fetching providers for user: {current_user.username}")
+    service = LLMProviderService(db)
+    providers = service.get_all_providers()
+    return [
+        LLMProviderConfig(
+            id=p.id,
+            provider_type=LLMProviderType(p.provider_type),
+            name=p.name,
+            base_url=p.base_url,
+            api_key="********",
+            default_model=p.default_model,
+            is_active=p.is_active
+        ) for p in providers
+    ]
+# [/DEF:get_providers:Function]
+
+# [DEF:create_provider:Function]
+# @PURPOSE: Create a new LLM provider configuration.
+# @PRE: User is authenticated and has admin permissions.
+# @POST: Returns the created LLMProviderConfig.
+@router.post("/providers", response_model=LLMProviderConfig, status_code=status.HTTP_201_CREATED)
+async def create_provider(
+    config: LLMProviderConfig,
+    current_user: User = Depends(get_current_active_user),
+    db: Session = Depends(get_db)
+):
+    """
+    Create a new LLM provider configuration.
+    """
+    service = LLMProviderService(db)
+    provider = service.create_provider(config)
+    return LLMProviderConfig(
+        id=provider.id,
+        provider_type=LLMProviderType(provider.provider_type),
+        name=provider.name,
+        base_url=provider.base_url,
+        api_key="********",
+        default_model=provider.default_model,
+        is_active=provider.is_active
+    )
+# [/DEF:create_provider:Function]
+
+# [DEF:update_provider:Function]
+# @PURPOSE: Update an existing LLM provider configuration.
+# @PRE: User is authenticated and has admin permissions.
+# @POST: Returns the updated LLMProviderConfig.
+@router.put("/providers/{provider_id}", response_model=LLMProviderConfig)
+async def update_provider(
+    provider_id: str,
+    config: LLMProviderConfig,
+    current_user: User = Depends(get_current_active_user),
+    db: Session = Depends(get_db)
+):
+    """
+    Update an existing LLM provider configuration.
+    """
+    service = LLMProviderService(db)
+    provider = service.update_provider(provider_id, config)
+    if not provider:
+        raise HTTPException(status_code=404, detail="Provider not found")
+    
+    return LLMProviderConfig(
+        id=provider.id,
+        provider_type=LLMProviderType(provider.provider_type),
+        name=provider.name,
+        base_url=provider.base_url,
+        api_key="********",
+        default_model=provider.default_model,
+        is_active=provider.is_active
+    )
+# [/DEF:update_provider:Function]
+
+# [DEF:delete_provider:Function]
+# @PURPOSE: Delete an LLM provider configuration.
+# @PRE: User is authenticated and has admin permissions.
+# @POST: Returns success status.
+@router.delete("/providers/{provider_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def delete_provider(
+    provider_id: str,
+    current_user: User = Depends(get_current_active_user),
+    db: Session = Depends(get_db)
+):
+    """
+    Delete an LLM provider configuration.
+    """
+    service = LLMProviderService(db)
+    if not service.delete_provider(provider_id):
+        raise HTTPException(status_code=404, detail="Provider not found")
+    return
+# [/DEF:delete_provider:Function]
+
+# [DEF:test_connection:Function]
+# @PURPOSE: Test connection to an LLM provider.
+# @PRE: User is authenticated.
+# @POST: Returns success status and message.
+@router.post("/providers/{provider_id}/test")
+async def test_connection(
+    provider_id: str,
+    current_user: User = Depends(get_current_active_user),
+    db: Session = Depends(get_db)
+):
+    logger.info(f"[llm_routes][test_connection][Action] Testing connection for provider_id: {provider_id}")
+    """
+    Test connection to an LLM provider.
+    """
+    from ...plugins.llm_analysis.service import LLMClient
+    service = LLMProviderService(db)
+    db_provider = service.get_provider(provider_id)
+    if not db_provider:
+        raise HTTPException(status_code=404, detail="Provider not found")
+    
+    api_key = service.get_decrypted_api_key(provider_id)
+    
+    # Check if API key was successfully decrypted
+    if not api_key:
+        logger.error(f"[llm_routes][test_connection] Failed to decrypt API key for provider {provider_id}")
+        raise HTTPException(
+            status_code=500,
+            detail="Failed to decrypt API key. The provider may have been encrypted with a different encryption key. Please update the provider with a new API key."
+        )
+    
+    client = LLMClient(
+        provider_type=LLMProviderType(db_provider.provider_type),
+        api_key=api_key,
+        base_url=db_provider.base_url,
+        default_model=db_provider.default_model
+    )
+    
+    try:
+        # Simple test call
+        await client.client.models.list()
+        return {"success": True, "message": "Connection successful"}
+    except Exception as e:
+        return {"success": False, "error": str(e)}
+# [/DEF:test_connection:Function]
+
+# [DEF:test_provider_config:Function]
+# @PURPOSE: Test connection with a provided configuration (not yet saved).
+# @PRE: User is authenticated.
+# @POST: Returns success status and message.
+@router.post("/providers/test")
+async def test_provider_config(
+    config: LLMProviderConfig,
+    current_user: User = Depends(get_current_active_user)
+):
+    """
+    Test connection with a provided configuration.
+    """
+    from ...plugins.llm_analysis.service import LLMClient
+    logger.info(f"[llm_routes][test_provider_config][Action] Testing config for {config.name}")
+    
+    # Check if API key is provided
+    if not config.api_key or config.api_key == "********":
+        raise HTTPException(
+            status_code=400,
+            detail="API key is required for testing connection"
+        )
+    
+    client = LLMClient(
+        provider_type=config.provider_type,
+        api_key=config.api_key,
+        base_url=config.base_url,
+        default_model=config.default_model
+    )
+    
+    try:
+        # Simple test call
+        await client.client.models.list()
+        return {"success": True, "message": "Connection successful"}
+    except Exception as e:
+        return {"success": False, "error": str(e)}
+# [/DEF:test_provider_config:Function]
+
+# [/DEF:backend/src/api/routes/llm.py]

backend/src/api/routes/mappings.py

@@ -14,7 +14,7 @@ from fastapi import APIRouter, Depends, HTTPException
 from sqlalchemy.orm import Session
 from typing import List, Optional
 from ...core.logger import belief_scope
-from ...dependencies import get_config_manager
+from ...dependencies import get_config_manager, has_permission
 from ...core.database import get_db
 from ...models.mapping import DatabaseMapping
 from pydantic import BaseModel
@@ -60,7 +60,8 @@ class SuggestRequest(BaseModel):
 async def get_mappings(
     source_env_id: Optional[str] = None,
     target_env_id: Optional[str] = None,
-    db: Session = Depends(get_db)
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("plugin:mapper", "EXECUTE"))
 ):
     with belief_scope("get_mappings"):
         query = db.query(DatabaseMapping)
@@ -76,7 +77,11 @@ async def get_mappings(
 # @PRE:     mapping is valid MappingCreate, db session is injected.
 # @POST:    DatabaseMapping created or updated in database.
 @router.post("", response_model=MappingResponse)
-async def create_mapping(mapping: MappingCreate, db: Session = Depends(get_db)):
+async def create_mapping(
+    mapping: MappingCreate,
+    db: Session = Depends(get_db),
+    _ = Depends(has_permission("plugin:mapper", "EXECUTE"))
+):
     with belief_scope("create_mapping"):
         # Check if mapping already exists
         existing = db.query(DatabaseMapping).filter(
@@ -106,10 +111,11 @@ async def create_mapping(mapping: MappingCreate, db: Session = Depends(get_db)):
 @router.post("/suggest")
 async def suggest_mappings_api(
     request: SuggestRequest,
-    config_manager=Depends(get_config_manager)
+    config_manager=Depends(get_config_manager),
+    _ = Depends(has_permission("plugin:mapper", "EXECUTE"))
 ):
     with belief_scope("suggest_mappings_api"):
-        from backend.src.services.mapping_service import MappingService
+        from ...services.mapping_service import MappingService
         service = MappingService(config_manager)
         try:
             return await service.get_suggestions(request.source_env_id, request.target_env_id)

backend/src/api/routes/migration.py

@@ -7,7 +7,7 @@
 
 from fastapi import APIRouter, Depends, HTTPException
 from typing import List, Dict
-from ...dependencies import get_config_manager, get_task_manager
+from ...dependencies import get_config_manager, get_task_manager, has_permission
 from ...models.dashboard import DashboardMetadata, DashboardSelection
 from ...core.superset_client import SupersetClient
 from ...core.logger import belief_scope
@@ -21,7 +21,11 @@ router = APIRouter(prefix="/api", tags=["migration"])
 # @PARAM:   env_id (str) - The ID of the environment to fetch from.
 # @RETURN:  List[DashboardMetadata]
 @router.get("/environments/{env_id}/dashboards", response_model=List[DashboardMetadata])
-async def get_dashboards(env_id: str, config_manager=Depends(get_config_manager)):
+async def get_dashboards(
+    env_id: str,
+    config_manager=Depends(get_config_manager),
+    _ = Depends(has_permission("plugin:migration", "EXECUTE"))
+):
     with belief_scope("get_dashboards", f"env_id={env_id}"):
         environments = config_manager.get_environments()
     env = next((e for e in environments if e.id == env_id), None)
@@ -40,7 +44,12 @@ async def get_dashboards(env_id: str, config_manager=Depends(get_config_manager)
 # @PARAM:   selection (DashboardSelection) - The dashboards to migrate.
 # @RETURN:  Dict - {"task_id": str, "message": str}
 @router.post("/migration/execute")
-async def execute_migration(selection: DashboardSelection, config_manager=Depends(get_config_manager), task_manager=Depends(get_task_manager)):
+async def execute_migration(
+    selection: DashboardSelection,
+    config_manager=Depends(get_config_manager),
+    task_manager=Depends(get_task_manager),
+    _ = Depends(has_permission("plugin:migration", "EXECUTE"))
+):
     with belief_scope("execute_migration"):
         # Validate environments exist
         environments = config_manager.get_environments()

backend/src/api/routes/plugins.py

@@ -7,7 +7,7 @@ from typing import List
 from fastapi import APIRouter, Depends
 
 from ...core.plugin_base import PluginConfig
-from ...dependencies import get_plugin_loader
+from ...dependencies import get_plugin_loader, has_permission
 from ...core.logger import belief_scope
 
 router = APIRouter()
@@ -19,7 +19,8 @@ router = APIRouter()
 # @RETURN: List[PluginConfig] - List of registered plugins.
 @router.get("", response_model=List[PluginConfig])
 async def list_plugins(
-    plugin_loader = Depends(get_plugin_loader)
+    plugin_loader = Depends(get_plugin_loader),
+    _ = Depends(has_permission("plugins", "READ"))
 ):
     with belief_scope("list_plugins"):
         """

backend/src/api/routes/settings.py

@@ -13,7 +13,8 @@
 from fastapi import APIRouter, Depends, HTTPException
 from typing import List
 from ...core.config_models import AppConfig, Environment, GlobalSettings
-from ...dependencies import get_config_manager
+from ...models.storage import StorageConfig
+from ...dependencies import get_config_manager, has_permission
 from ...core.config_manager import ConfigManager
 from ...core.logger import logger, belief_scope
 from ...core.superset_client import SupersetClient
@@ -28,7 +29,10 @@ router = APIRouter()
 # @POST:    Returns masked AppConfig.
 # @RETURN: AppConfig - The current configuration.
 @router.get("", response_model=AppConfig)
-async def get_settings(config_manager: ConfigManager = Depends(get_config_manager)):
+async def get_settings(
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
     with belief_scope("get_settings"):
         logger.info("[get_settings][Entry] Fetching all settings")
     config = config_manager.get_config().copy(deep=True)
@@ -48,21 +52,60 @@ async def get_settings(config_manager: ConfigManager = Depends(get_config_manage
 @router.patch("/global", response_model=GlobalSettings)
 async def update_global_settings(
     settings: GlobalSettings,
-    config_manager: ConfigManager = Depends(get_config_manager)
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
 ):
     with belief_scope("update_global_settings"):
         logger.info("[update_global_settings][Entry] Updating global settings")
+    
     config_manager.update_global_settings(settings)
     return settings
 # [/DEF:update_global_settings:Function]
 
+# [DEF:get_storage_settings:Function]
+# @PURPOSE: Retrieves storage-specific settings.
+# @RETURN: StorageConfig - The storage configuration.
+@router.get("/storage", response_model=StorageConfig)
+async def get_storage_settings(
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
+    with belief_scope("get_storage_settings"):
+        return config_manager.get_config().settings.storage
+# [/DEF:get_storage_settings:Function]
+
+# [DEF:update_storage_settings:Function]
+# @PURPOSE: Updates storage-specific settings.
+# @PARAM: storage (StorageConfig) - The new storage settings.
+# @POST: Storage settings are updated and saved.
+# @RETURN: StorageConfig - The updated storage settings.
+@router.put("/storage", response_model=StorageConfig)
+async def update_storage_settings(
+    storage: StorageConfig,
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
+):
+    with belief_scope("update_storage_settings"):
+        is_valid, message = config_manager.validate_path(storage.root_path)
+        if not is_valid:
+            raise HTTPException(status_code=400, detail=message)
+        
+        settings = config_manager.get_config().settings
+        settings.storage = storage
+        config_manager.update_global_settings(settings)
+        return config_manager.get_config().settings.storage
+# [/DEF:update_storage_settings:Function]
+
 # [DEF:get_environments:Function]
 # @PURPOSE: Lists all configured Superset environments.
 # @PRE:     Config manager is available.
 # @POST:    Returns list of environments.
 # @RETURN: List[Environment] - List of environments.
 @router.get("/environments", response_model=List[Environment])
-async def get_environments(config_manager: ConfigManager = Depends(get_config_manager)):
+async def get_environments(
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "READ"))
+):
     with belief_scope("get_environments"):
         logger.info("[get_environments][Entry] Fetching environments")
     return config_manager.get_environments()
@@ -77,7 +120,8 @@ async def get_environments(config_manager: ConfigManager = Depends(get_config_ma
 @router.post("/environments", response_model=Environment)
 async def add_environment(
     env: Environment,
-    config_manager: ConfigManager = Depends(get_config_manager)
+    config_manager: ConfigManager = Depends(get_config_manager),
+    _ = Depends(has_permission("admin:settings", "WRITE"))
 ):
     with belief_scope("add_environment"):
         logger.info(f"[add_environment][Entry] Adding environment {env.id}")
@@ -179,30 +223,5 @@ async def test_environment_connection(
         return {"status": "error", "message": str(e)}
 # [/DEF:test_environment_connection:Function]
 
-# [DEF:validate_backup_path:Function]
-# @PURPOSE: Validates if a backup path exists and is writable.
-# @PRE:     Path is provided in path_data.
-# @POST:    Returns success or error status.
-# @PARAM: path (str) - The path to validate.
-# @RETURN: dict - Validation result.
-@router.post("/validate-path")
-async def validate_backup_path(
-    path_data: dict,
-    config_manager: ConfigManager = Depends(get_config_manager)
-):
-    with belief_scope("validate_backup_path"):
-        path = path_data.get("path")
-        if not path:
-            raise HTTPException(status_code=400, detail="Path is required")
-        
-        logger.info(f"[validate_backup_path][Entry] Validating path: {path}")
-    
-    valid, message = config_manager.validate_path(path)
-    
-    if not valid:
-        return {"status": "error", "message": message}
-    
-    return {"status": "success", "message": message}
-# [/DEF:validate_backup_path:Function]
 
 # [/DEF:SettingsRouter:Module]

backend/src/api/routes/storage.py

@@ -0,0 +1,145 @@
+# [DEF:storage_routes:Module]
+#
+# @SEMANTICS: storage, files, upload, download, backup, repository
+# @PURPOSE:   API endpoints for file storage management (backups and repositories).
+# @LAYER:     API
+# @RELATION:  DEPENDS_ON -> backend.src.models.storage
+#
+# @INVARIANT: All paths must be validated against path traversal.
+
+# [SECTION: IMPORTS]
+from pathlib import Path
+from fastapi import APIRouter, Depends, UploadFile, File, Form, HTTPException
+from fastapi.responses import FileResponse
+from typing import List, Optional
+from ...models.storage import StoredFile, FileCategory
+from ...dependencies import get_plugin_loader, has_permission
+from ...plugins.storage.plugin import StoragePlugin
+from ...core.logger import belief_scope
+# [/SECTION]
+
+router = APIRouter(tags=["storage"])
+
+# [DEF:list_files:Function]
+# @PURPOSE:     List all files and directories in the storage system.
+#
+# @PRE:         None.
+# @POST:        Returns a list of StoredFile objects.
+#
+# @PARAM:       category (Optional[FileCategory]) - Filter by category.
+# @PARAM:       path (Optional[str]) - Subpath within the category.
+# @RETURN:      List[StoredFile] - List of files/directories.
+#
+# @RELATION:    CALLS -> StoragePlugin.list_files
+@router.get("/files", response_model=List[StoredFile])
+async def list_files(
+    category: Optional[FileCategory] = None,
+    path: Optional[str] = None,
+    plugin_loader=Depends(get_plugin_loader),
+    _ = Depends(has_permission("plugin:storage", "READ"))
+):
+    with belief_scope("list_files"):
+        storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
+        if not storage_plugin:
+            raise HTTPException(status_code=500, detail="Storage plugin not loaded")
+        return storage_plugin.list_files(category, path)
+# [/DEF:list_files:Function]
+
+# [DEF:upload_file:Function]
+# @PURPOSE:     Upload a file to the storage system.
+#
+# @PRE:         category must be a valid FileCategory.
+# @PRE:         file must be a valid UploadFile.
+# @POST:        Returns the StoredFile object of the uploaded file.
+#
+# @PARAM:       category (FileCategory) - Target category.
+# @PARAM:       path (Optional[str]) - Target subpath.
+# @PARAM:       file (UploadFile) - The file content.
+# @RETURN:      StoredFile - Metadata of the uploaded file.
+#
+# @SIDE_EFFECT: Writes file to the filesystem.
+#
+# @RELATION:    CALLS -> StoragePlugin.save_file
+@router.post("/upload", response_model=StoredFile, status_code=201)
+async def upload_file(
+    category: FileCategory = Form(...),
+    path: Optional[str] = Form(None),
+    file: UploadFile = File(...),
+    plugin_loader=Depends(get_plugin_loader),
+    _ = Depends(has_permission("plugin:storage", "WRITE"))
+):
+    with belief_scope("upload_file"):
+        storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
+        if not storage_plugin:
+            raise HTTPException(status_code=500, detail="Storage plugin not loaded")
+        try:
+            return await storage_plugin.save_file(file, category, path)
+        except ValueError as e:
+            raise HTTPException(status_code=400, detail=str(e))
+# [/DEF:upload_file:Function]
+
+# [DEF:delete_file:Function]
+# @PURPOSE:     Delete a specific file or directory.
+#
+# @PRE:         category must be a valid FileCategory.
+# @POST:        Item is removed from storage.
+#
+# @PARAM:       category (FileCategory) - File category.
+# @PARAM:       path (str) - Relative path of the item.
+# @RETURN:      None
+#
+# @SIDE_EFFECT: Deletes item from the filesystem.
+#
+# @RELATION:    CALLS -> StoragePlugin.delete_file
+@router.delete("/files/{category}/{path:path}", status_code=204)
+async def delete_file(
+    category: FileCategory,
+    path: str,
+    plugin_loader=Depends(get_plugin_loader),
+    _ = Depends(has_permission("plugin:storage", "WRITE"))
+):
+    with belief_scope("delete_file"):
+        storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
+        if not storage_plugin:
+            raise HTTPException(status_code=500, detail="Storage plugin not loaded")
+        try:
+            storage_plugin.delete_file(category, path)
+        except FileNotFoundError:
+            raise HTTPException(status_code=404, detail="File not found")
+        except ValueError as e:
+            raise HTTPException(status_code=400, detail=str(e))
+# [/DEF:delete_file:Function]
+
+# [DEF:download_file:Function]
+# @PURPOSE:     Retrieve a file for download.
+#
+# @PRE:         category must be a valid FileCategory.
+# @POST:        Returns a FileResponse.
+#
+# @PARAM:       category (FileCategory) - File category.
+# @PARAM:       path (str) - Relative path of the file.
+# @RETURN:      FileResponse - The file content.
+#
+# @RELATION:    CALLS -> StoragePlugin.get_file_path
+@router.get("/download/{category}/{path:path}")
+async def download_file(
+    category: FileCategory,
+    path: str,
+    plugin_loader=Depends(get_plugin_loader),
+    _ = Depends(has_permission("plugin:storage", "READ"))
+):
+    with belief_scope("download_file"):
+        storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
+        if not storage_plugin:
+            raise HTTPException(status_code=500, detail="Storage plugin not loaded")
+        try:
+            abs_path = storage_plugin.get_file_path(category, path)
+            filename = Path(path).name
+            return FileResponse(path=abs_path, filename=filename)
+        except FileNotFoundError:
+            raise HTTPException(status_code=404, detail="File not found")
+        except ValueError as e:
+            raise HTTPException(status_code=400, detail=str(e))
+# [/DEF:download_file:Function]
+
+# [/DEF:storage_routes:Module]

backend/src/api/routes/tasks.py

@@ -9,7 +9,7 @@ from pydantic import BaseModel
 from ...core.logger import belief_scope
 
 from ...core.task_manager import TaskManager, Task, TaskStatus, LogEntry
-from ...dependencies import get_task_manager
+from ...dependencies import get_task_manager, has_permission, get_current_user
 
 router = APIRouter()
 
@@ -33,13 +33,31 @@ class ResumeTaskRequest(BaseModel):
 # @RETURN: Task - The created task instance.
 async def create_task(
     request: CreateTaskRequest,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    current_user = Depends(get_current_user)
 ):
+    # Dynamic permission check based on plugin_id
+    has_permission(f"plugin:{request.plugin_id}", "EXECUTE")(current_user)
     """
     Create and start a new task for a given plugin.
     """
     with belief_scope("create_task"):
         try:
+            # Special handling for validation task to include provider config
+            if request.plugin_id == "llm_dashboard_validation":
+                from ...core.database import SessionLocal
+                from ...services.llm_provider import LLMProviderService
+                db = SessionLocal()
+                try:
+                    llm_service = LLMProviderService(db)
+                    provider_id = request.params.get("provider_id")
+                    if provider_id:
+                        db_provider = llm_service.get_provider(provider_id)
+                        if not db_provider:
+                            raise ValueError(f"LLM Provider {provider_id} not found")
+                finally:
+                    db.close()
+
             task = await task_manager.create_task(
                 plugin_id=request.plugin_id,
                 params=request.params
@@ -63,7 +81,8 @@ async def list_tasks(
     limit: int = 10,
     offset: int = 0,
     status: Optional[TaskStatus] = None,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "READ"))
 ):
     """
     Retrieve a list of tasks with pagination and optional status filter.
@@ -82,7 +101,8 @@ async def list_tasks(
 # @RETURN: Task - The task details.
 async def get_task(
     task_id: str,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "READ"))
 ):
     """
     Retrieve the details of a specific task.
@@ -104,7 +124,8 @@ async def get_task(
 # @RETURN: List[LogEntry] - List of log entries.
 async def get_task_logs(
     task_id: str,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "READ"))
 ):
     """
     Retrieve logs for a specific task.
@@ -128,7 +149,8 @@ async def get_task_logs(
 async def resolve_task(
     task_id: str,
     request: ResolveTaskRequest,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "WRITE"))
 ):
     """
     Resolve a task that is awaiting mapping.
@@ -153,7 +175,8 @@ async def resolve_task(
 async def resume_task(
     task_id: str,
     request: ResumeTaskRequest,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "WRITE"))
 ):
     """
     Resume a task that is awaiting input (e.g., passwords).
@@ -175,7 +198,8 @@ async def resume_task(
 # @POST: Tasks are removed from memory/persistence.
 async def clear_tasks(
     status: Optional[TaskStatus] = None,
-    task_manager: TaskManager = Depends(get_task_manager)
+    task_manager: TaskManager = Depends(get_task_manager),
+    _ = Depends(has_permission("tasks", "WRITE"))
 ):
     """
     Clear tasks matching the status filter. If no filter, clears all non-running tasks.

backend/src/app.py

@@ -10,6 +10,7 @@ from pathlib import Path
 project_root = Path(__file__).resolve().parent.parent.parent
 
 from fastapi import FastAPI, WebSocket, WebSocketDisconnect, Depends, Request, HTTPException
+from starlette.middleware.sessions import SessionMiddleware
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.staticfiles import StaticFiles
 from fastapi.responses import FileResponse
@@ -17,8 +18,10 @@ import asyncio
 import os
 
 from .dependencies import get_task_manager, get_scheduler_service
+from .core.utils.network import NetworkError
 from .core.logger import logger, belief_scope
-from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git
+from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git, storage, admin, llm
+from .api import auth
 from .core.database import init_db
 
 # [DEF:App:Global]
@@ -55,6 +58,10 @@ async def shutdown_event():
     scheduler.stop()
 # [/DEF:shutdown_event:Function]
 
+# Configure Session Middleware (required by Authlib for OAuth2 flow)
+from .core.auth.config import auth_config
+app.add_middleware(SessionMiddleware, secret_key=auth_config.SECRET_KEY)
+
 # Configure CORS
 app.add_middleware(
     CORSMiddleware,
@@ -71,16 +78,39 @@ app.add_middleware(
 # @POST: Logs request and response details.
 # @PARAM: request (Request) - The incoming request object.
 # @PARAM: call_next (Callable) - The next middleware or route handler.
+@app.exception_handler(NetworkError)
+async def network_error_handler(request: Request, exc: NetworkError):
+    with belief_scope("network_error_handler"):
+        logger.error(f"Network error: {exc}")
+        return HTTPException(
+            status_code=503,
+            detail="Environment unavailable. Please check if the Superset instance is running."
+        )
+
 @app.middleware("http")
 async def log_requests(request: Request, call_next):
-    with belief_scope("log_requests", f"{request.method} {request.url.path}"):
+    # Avoid spamming logs for polling endpoints
-        logger.info(f"[DEBUG] Incoming request: {request.method} {request.url.path}")
+    is_polling = request.url.path.endswith("/api/tasks") and request.method == "GET"
+    
+    if not is_polling:
+        logger.info(f"Incoming request: {request.method} {request.url.path}")
+    
+    try:
         response = await call_next(request)
-        logger.info(f"[DEBUG] Response status: {response.status_code} for {request.url.path}")
+        if not is_polling:
+            logger.info(f"Response status: {response.status_code} for {request.url.path}")
         return response
+    except NetworkError as e:
+        logger.error(f"Network error caught in middleware: {e}")
+        raise HTTPException(
+            status_code=503,
+            detail="Environment unavailable. Please check if the Superset instance is running."
+        )
 # [/DEF:log_requests:Function]
 
 # Include API routes
+app.include_router(auth.router)
+app.include_router(admin.router)
 app.include_router(plugins.router, prefix="/api/plugins", tags=["Plugins"])
 app.include_router(tasks.router, prefix="/api/tasks", tags=["Tasks"])
 app.include_router(settings.router, prefix="/api/settings", tags=["Settings"])
@@ -89,6 +119,8 @@ app.include_router(environments.router, prefix="/api/environments", tags=["Envir
 app.include_router(mappings.router)
 app.include_router(migration.router)
 app.include_router(git.router)
+app.include_router(llm.router)
+app.include_router(storage.router, prefix="/api/storage", tags=["Storage"])
 
 # [DEF:websocket_endpoint:Function]
 # @PURPOSE: Provides a WebSocket endpoint for real-time log streaming of a task.

backend/src/core/auth/config.py

@@ -0,0 +1,45 @@
+# [DEF:backend.src.core.auth.config:Module]
+#
+# @SEMANTICS: auth, config, settings, jwt, adfs
+# @PURPOSE:   Centralized configuration for authentication and authorization.
+# @LAYER:     Core
+# @RELATION:  DEPENDS_ON -> pydantic
+#
+# @INVARIANT: All sensitive configuration must have defaults or be loaded from environment.
+
+# [SECTION: IMPORTS]
+from pydantic import Field
+from pydantic_settings import BaseSettings
+import os
+# [/SECTION]
+
+# [DEF:AuthConfig:Class]
+# @PURPOSE: Holds authentication-related settings.
+# @PRE:     Environment variables may be provided via .env file.
+# @POST:    Returns a configuration object with validated settings.
+class AuthConfig(BaseSettings):
+    # JWT Settings
+    SECRET_KEY: str = Field(default="super-secret-key-change-in-production", env="AUTH_SECRET_KEY")
+    ALGORITHM: str = "HS256"
+    ACCESS_TOKEN_EXPIRE_MINUTES: int = 480
+    REFRESH_TOKEN_EXPIRE_DAYS: int = 7
+
+    # Database Settings
+    AUTH_DATABASE_URL: str = Field(default="sqlite:///./backend/auth.db", env="AUTH_DATABASE_URL")
+
+    # ADFS Settings
+    ADFS_CLIENT_ID: str = Field(default="", env="ADFS_CLIENT_ID")
+    ADFS_CLIENT_SECRET: str = Field(default="", env="ADFS_CLIENT_SECRET")
+    ADFS_METADATA_URL: str = Field(default="", env="ADFS_METADATA_URL")
+    
+    class Config:
+        env_file = ".env"
+        extra = "ignore"
+# [/DEF:AuthConfig:Class]
+
+# [DEF:auth_config:Variable]
+# @PURPOSE: Singleton instance of AuthConfig.
+auth_config = AuthConfig()
+# [/DEF:auth_config:Variable]
+
+# [/DEF:backend.src.core.auth.config:Module]

backend/src/core/auth/jwt.py

@@ -0,0 +1,54 @@
+# [DEF:backend.src.core.auth.jwt:Module]
+#
+# @SEMANTICS: jwt, token, session, auth
+# @PURPOSE:   JWT token generation and validation logic.
+# @LAYER:     Core
+# @RELATION:  DEPENDS_ON -> jose
+# @RELATION:  USES -> backend.src.core.auth.config.auth_config
+#
+# @INVARIANT: Tokens must include expiration time and user identifier.
+
+# [SECTION: IMPORTS]
+from datetime import datetime, timedelta
+from typing import Optional, List
+from jose import JWTError, jwt
+from .config import auth_config
+from ..logger import belief_scope
+# [/SECTION]
+
+# [DEF:create_access_token:Function]
+# @PURPOSE: Generates a new JWT access token.
+# @PRE:     data dict contains 'sub' (user_id) and optional 'scopes' (roles).
+# @POST:    Returns a signed JWT string.
+#
+# @PARAM:   data (dict) - Payload data for the token.
+# @PARAM:   expires_delta (Optional[timedelta]) - Custom expiration time.
+# @RETURN:  str - The encoded JWT.
+def create_access_token(data: dict, expires_delta: Optional[timedelta] = None) -> str:
+    with belief_scope("create_access_token"):
+        to_encode = data.copy()
+        if expires_delta:
+            expire = datetime.utcnow() + expires_delta
+        else:
+            expire = datetime.utcnow() + timedelta(minutes=auth_config.ACCESS_TOKEN_EXPIRE_MINUTES)
+        
+        to_encode.update({"exp": expire})
+        encoded_jwt = jwt.encode(to_encode, auth_config.SECRET_KEY, algorithm=auth_config.ALGORITHM)
+        return encoded_jwt
+# [/DEF:create_access_token:Function]
+
+# [DEF:decode_token:Function]
+# @PURPOSE: Decodes and validates a JWT token.
+# @PRE:     token is a signed JWT string.
+# @POST:    Returns the decoded payload if valid.
+#
+# @PARAM:   token (str) - The JWT to decode.
+# @RETURN:  dict - The decoded payload.
+# @THROW:   jose.JWTError - If token is invalid or expired.
+def decode_token(token: str) -> dict:
+    with belief_scope("decode_token"):
+        payload = jwt.decode(token, auth_config.SECRET_KEY, algorithms=[auth_config.ALGORITHM])
+        return payload
+# [/DEF:decode_token:Function]
+
+# [/DEF:backend.src.core.auth.jwt:Module]

backend/src/core/auth/logger.py

@@ -0,0 +1,31 @@
+# [DEF:backend.src.core.auth.logger:Module]
+#
+# @SEMANTICS: auth, logger, audit, security
+# @PURPOSE:   Audit logging for security-related events.
+# @LAYER:     Core
+# @RELATION:  USES -> backend.src.core.logger.belief_scope
+#
+# @INVARIANT: Must not log sensitive data like passwords or full tokens.
+
+# [SECTION: IMPORTS]
+from ..logger import logger, belief_scope
+from datetime import datetime
+# [/SECTION]
+
+# [DEF:log_security_event:Function]
+# @PURPOSE: Logs a security-related event for audit trails.
+# @PRE:     event_type and username are strings.
+# @POST:    Security event is written to the application log.
+# @PARAM:   event_type (str) - Type of event (e.g., LOGIN_SUCCESS, PERMISSION_DENIED).
+# @PARAM:   username (str) - The user involved in the event.
+# @PARAM:   details (dict) - Additional non-sensitive metadata.
+def log_security_event(event_type: str, username: str, details: dict = None):
+    with belief_scope("log_security_event", f"{event_type}:{username}"):
+        timestamp = datetime.utcnow().isoformat()
+        msg = f"[AUDIT][{timestamp}][{event_type}] User: {username}"
+        if details:
+            msg += f" Details: {details}"
+        logger.info(msg)
+# [/DEF:log_security_event:Function]
+
+# [/DEF:backend.src.core.auth.logger:Module]

backend/src/core/auth/oauth.py

@@ -0,0 +1,51 @@
+# [DEF:backend.src.core.auth.oauth:Module]
+#
+# @SEMANTICS: auth, oauth, oidc, adfs
+# @PURPOSE:   ADFS OIDC configuration and client using Authlib.
+# @LAYER:     Core
+# @RELATION:  DEPENDS_ON -> authlib
+# @RELATION:  USES -> backend.src.core.auth.config.auth_config
+#
+# @INVARIANT: Must use secure OIDC flows.
+
+# [SECTION: IMPORTS]
+from authlib.integrations.starlette_client import OAuth
+from .config import auth_config
+# [/SECTION]
+
+# [DEF:oauth:Variable]
+# @PURPOSE: Global Authlib OAuth registry.
+oauth = OAuth()
+# [/DEF:oauth:Variable]
+
+# [DEF:register_adfs:Function]
+# @PURPOSE: Registers the ADFS OIDC client.
+# @PRE:     ADFS configuration is provided in auth_config.
+# @POST:    ADFS client is registered in oauth registry.
+def register_adfs():
+    if auth_config.ADFS_CLIENT_ID:
+        oauth.register(
+            name='adfs',
+            client_id=auth_config.ADFS_CLIENT_ID,
+            client_secret=auth_config.ADFS_CLIENT_SECRET,
+            server_metadata_url=auth_config.ADFS_METADATA_URL,
+            client_kwargs={
+                'scope': 'openid email profile groups'
+            }
+        )
+# [/DEF:register_adfs:Function]
+
+# [DEF:is_adfs_configured:Function]
+# @PURPOSE: Checks if ADFS is properly configured.
+# @PRE:     None.
+# @POST:    Returns True if ADFS client is registered, False otherwise.
+# @RETURN:  bool - Configuration status.
+def is_adfs_configured() -> bool:
+    """Check if ADFS OAuth client is registered."""
+    return 'adfs' in oauth._registry
+# [/DEF:is_adfs_configured:Function]
+
+# Initial registration
+register_adfs()
+
+# [/DEF:backend.src.core.auth.oauth:Module]

backend/src/core/auth/repository.py

@@ -0,0 +1,123 @@
+# [DEF:backend.src.core.auth.repository:Module]
+#
+# @SEMANTICS: auth, repository, database, user, role
+# @PURPOSE:   Data access layer for authentication-related entities.
+# @LAYER:     Core
+# @RELATION:  DEPENDS_ON -> sqlalchemy
+# @RELATION:  USES -> backend.src.models.auth
+#
+# @INVARIANT: All database operations must be performed within a session.
+
+# [SECTION: IMPORTS]
+from typing import Optional, List
+from sqlalchemy.orm import Session
+from ...models.auth import User, Role, Permission, ADGroupMapping
+from ..logger import belief_scope
+# [/SECTION]
+
+# [DEF:AuthRepository:Class]
+# @PURPOSE: Encapsulates database operations for authentication.
+class AuthRepository:
+    # [DEF:__init__:Function]
+    # @PURPOSE: Initializes the repository with a database session.
+    # @PARAM:   db (Session) - SQLAlchemy session.
+    def __init__(self, db: Session):
+        self.db = db
+    # [/DEF:__init__:Function]
+
+    # [DEF:get_user_by_username:Function]
+    # @PURPOSE: Retrieves a user by their username.
+    # @PRE:     username is a string.
+    # @POST:    Returns User object if found, else None.
+    # @PARAM:   username (str) - The username to search for.
+    # @RETURN:  Optional[User] - The found user or None.
+    def get_user_by_username(self, username: str) -> Optional[User]:
+        with belief_scope("AuthRepository.get_user_by_username"):
+            return self.db.query(User).filter(User.username == username).first()
+    # [/DEF:get_user_by_username:Function]
+
+    # [DEF:get_user_by_id:Function]
+    # @PURPOSE: Retrieves a user by their unique ID.
+    # @PRE:     user_id is a valid UUID string.
+    # @POST:    Returns User object if found, else None.
+    # @PARAM:   user_id (str) - The user's unique identifier.
+    # @RETURN:  Optional[User] - The found user or None.
+    def get_user_by_id(self, user_id: str) -> Optional[User]:
+        with belief_scope("AuthRepository.get_user_by_id"):
+            return self.db.query(User).filter(User.id == user_id).first()
+    # [/DEF:get_user_by_id:Function]
+
+    # [DEF:get_role_by_name:Function]
+    # @PURPOSE: Retrieves a role by its name.
+    # @PRE:     name is a string.
+    # @POST:    Returns Role object if found, else None.
+    # @PARAM:   name (str) - The role name to search for.
+    # @RETURN:  Optional[Role] - The found role or None.
+    def get_role_by_name(self, name: str) -> Optional[Role]:
+        with belief_scope("AuthRepository.get_role_by_name"):
+            return self.db.query(Role).filter(Role.name == name).first()
+    # [/DEF:get_role_by_name:Function]
+
+    # [DEF:update_last_login:Function]
+    # @PURPOSE: Updates the last_login timestamp for a user.
+    # @PRE:     user object is a valid User instance.
+    # @POST:    User's last_login is updated in the database.
+    # @SIDE_EFFECT: Commits the transaction.
+    # @PARAM:   user (User) - The user to update.
+    def update_last_login(self, user: User):
+        with belief_scope("AuthRepository.update_last_login"):
+            from datetime import datetime
+            user.last_login = datetime.utcnow()
+            self.db.add(user)
+            self.db.commit()
+    # [/DEF:update_last_login:Function]
+
+    # [DEF:get_role_by_id:Function]
+    # @PURPOSE: Retrieves a role by its unique ID.
+    # @PRE:     role_id is a string.
+    # @POST:    Returns Role object if found, else None.
+    # @PARAM:   role_id (str) - The role's unique identifier.
+    # @RETURN:  Optional[Role] - The found role or None.
+    def get_role_by_id(self, role_id: str) -> Optional[Role]:
+        with belief_scope("AuthRepository.get_role_by_id"):
+            return self.db.query(Role).filter(Role.id == role_id).first()
+    # [/DEF:get_role_by_id:Function]
+
+    # [DEF:get_permission_by_id:Function]
+    # @PURPOSE: Retrieves a permission by its unique ID.
+    # @PRE:     perm_id is a string.
+    # @POST:    Returns Permission object if found, else None.
+    # @PARAM:   perm_id (str) - The permission's unique identifier.
+    # @RETURN:  Optional[Permission] - The found permission or None.
+    def get_permission_by_id(self, perm_id: str) -> Optional[Permission]:
+        with belief_scope("AuthRepository.get_permission_by_id"):
+            return self.db.query(Permission).filter(Permission.id == perm_id).first()
+    # [/DEF:get_permission_by_id:Function]
+
+    # [DEF:get_permission_by_resource_action:Function]
+    # @PURPOSE: Retrieves a permission by resource and action.
+    # @PRE:     resource and action are strings.
+    # @POST:    Returns Permission object if found, else None.
+    # @PARAM:   resource (str) - The resource name.
+    # @PARAM:   action (str) - The action name.
+    # @RETURN:  Optional[Permission] - The found permission or None.
+    def get_permission_by_resource_action(self, resource: str, action: str) -> Optional[Permission]:
+        with belief_scope("AuthRepository.get_permission_by_resource_action"):
+            return self.db.query(Permission).filter(
+                Permission.resource == resource,
+                Permission.action == action
+            ).first()
+    # [/DEF:get_permission_by_resource_action:Function]
+
+    # [DEF:list_permissions:Function]
+    # @PURPOSE: Lists all available permissions.
+    # @POST:    Returns a list of all Permission objects.
+    # @RETURN:  List[Permission] - List of permissions.
+    def list_permissions(self) -> List[Permission]:
+        with belief_scope("AuthRepository.list_permissions"):
+            return self.db.query(Permission).all()
+    # [/DEF:list_permissions:Function]
+
+# [/DEF:AuthRepository:Class]
+
+# [/DEF:backend.src.core.auth.repository:Module]

backend/src/core/auth/security.py

@@ -0,0 +1,42 @@
+# [DEF:backend.src.core.auth.security:Module]
+#
+# @SEMANTICS: security, password, hashing, bcrypt
+# @PURPOSE:   Utility for password hashing and verification using Passlib.
+# @LAYER:     Core
+# @RELATION:  DEPENDS_ON -> passlib
+#
+# @INVARIANT: Uses bcrypt for hashing with standard work factor.
+
+# [SECTION: IMPORTS]
+from passlib.context import CryptContext
+# [/SECTION]
+
+# [DEF:pwd_context:Variable]
+# @PURPOSE: Passlib CryptContext for password management.
+pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
+# [/DEF:pwd_context:Variable]
+
+# [DEF:verify_password:Function]
+# @PURPOSE: Verifies a plain password against a hashed password.
+# @PRE:     plain_password is a string, hashed_password is a bcrypt hash.
+# @POST:    Returns True if password matches, False otherwise.
+#
+# @PARAM:   plain_password (str) - The unhashed password.
+# @PARAM:   hashed_password (str) - The stored hash.
+# @RETURN:  bool - Verification result.
+def verify_password(plain_password: str, hashed_password: str) -> bool:
+    return pwd_context.verify(plain_password, hashed_password)
+# [/DEF:verify_password:Function]
+
+# [DEF:get_password_hash:Function]
+# @PURPOSE: Generates a bcrypt hash for a plain password.
+# @PRE:     password is a string.
+# @POST:    Returns a secure bcrypt hash string.
+#
+# @PARAM:   password (str) - The password to hash.
+# @RETURN:  str - The generated hash.
+def get_password_hash(password: str) -> str:
+    return pwd_context.hash(password)
+# [/DEF:get_password_hash:Function]
+
+# [/DEF:backend.src.core.auth.security:Module]

backend/src/core/config_manager.py

@@ -62,14 +62,18 @@ class ConfigManager:
             logger.info(f"[_load_config][Action] Config file not found. Creating default.")
             default_config = AppConfig(
                 environments=[],
-                settings=GlobalSettings(backup_path="backups")
+                settings=GlobalSettings()
             )
             self._save_config_to_disk(default_config)
             return default_config
-        
         try:
             with open(self.config_path, "r") as f:
                 data = json.load(f)
+                
+                # Check for deprecated field
+                if "settings" in data and "backup_path" in data["settings"]:
+                    del data["settings"]["backup_path"]
+
                 config = AppConfig(**data)
                 logger.info(f"[_load_config][Coherence:OK] Configuration loaded")
                 return config
@@ -79,7 +83,7 @@ class ConfigManager:
             # For now, return default to be safe, but log the error prominently.
             return AppConfig(
                 environments=[],
-                settings=GlobalSettings(backup_path="backups")
+                settings=GlobalSettings(storage=StorageConfig())
             )
     # [/DEF:_load_config:Function]
 

backend/src/core/config_models.py

@@ -7,6 +7,7 @@
 
 from pydantic import BaseModel, Field
 from typing import List, Optional
+from ..models.storage import StorageConfig
 
 # [DEF:Schedule:DataClass]
 # @PURPOSE: Represents a backup schedule configuration.
@@ -42,7 +43,7 @@ class LoggingConfig(BaseModel):
 # [DEF:GlobalSettings:DataClass]
 # @PURPOSE: Represents global application settings.
 class GlobalSettings(BaseModel):
-    backup_path: str
+    storage: StorageConfig = Field(default_factory=StorageConfig)
     default_environment_id: Optional[str] = None
     logging: LoggingConfig = Field(default_factory=LoggingConfig)
     

backend/src/core/database.py

@@ -5,6 +5,7 @@
 # @LAYER:     Core
 # @RELATION:  DEPENDS_ON -> sqlalchemy
 # @RELATION:  USES -> backend.src.models.mapping
+# @RELATION:  USES -> backend.src.core.auth.config
 #
 # @INVARIANT: A single engine instance is used for the entire application.
 
@@ -16,44 +17,83 @@ from ..models.mapping import Base
 from ..models.task import TaskRecord
 from ..models.connection import ConnectionConfig
 from ..models.git import GitServerConfig, GitRepository, DeploymentEnvironment
+from ..models.auth import User, Role, Permission, ADGroupMapping
+from ..models.llm import LLMProvider, ValidationRecord
 from .logger import belief_scope
+from .auth.config import auth_config
 import os
+from pathlib import Path
 # [/SECTION]
 
+# [DEF:BASE_DIR:Variable]
+# @PURPOSE: Base directory for the backend (where .db files should reside).
+BASE_DIR = Path(__file__).resolve().parent.parent.parent
+# [/DEF:BASE_DIR:Variable]
+
 # [DEF:DATABASE_URL:Constant]
-DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./mappings.db")
+# @PURPOSE: URL for the main mappings database.
+DATABASE_URL = os.getenv("DATABASE_URL", f"sqlite:///{BASE_DIR}/mappings.db")
 # [/DEF:DATABASE_URL:Constant]
 
 # [DEF:TASKS_DATABASE_URL:Constant]
-TASKS_DATABASE_URL = os.getenv("TASKS_DATABASE_URL", "sqlite:///./tasks.db")
+# @PURPOSE: URL for the tasks execution database.
+TASKS_DATABASE_URL = os.getenv("TASKS_DATABASE_URL", f"sqlite:///{BASE_DIR}/tasks.db")
 # [/DEF:TASKS_DATABASE_URL:Constant]
 
+# [DEF:AUTH_DATABASE_URL:Constant]
+# @PURPOSE: URL for the authentication database.
+AUTH_DATABASE_URL = os.getenv("AUTH_DATABASE_URL", auth_config.AUTH_DATABASE_URL)
+# If it's a relative sqlite path starting with ./backend/, fix it to be absolute or relative to BASE_DIR
+if AUTH_DATABASE_URL.startswith("sqlite:///./backend/"):
+    AUTH_DATABASE_URL = AUTH_DATABASE_URL.replace("sqlite:///./backend/", f"sqlite:///{BASE_DIR}/")
+elif AUTH_DATABASE_URL.startswith("sqlite:///./") and not AUTH_DATABASE_URL.startswith("sqlite:///./backend/"):
+    # If it's just ./ but we are in backend, it's fine, but let's make it absolute for robustness
+    AUTH_DATABASE_URL = AUTH_DATABASE_URL.replace("sqlite:///./", f"sqlite:///{BASE_DIR}/")
+# [/DEF:AUTH_DATABASE_URL:Constant]
+
 # [DEF:engine:Variable]
+# @PURPOSE: SQLAlchemy engine for mappings database.
 engine = create_engine(DATABASE_URL, connect_args={"check_same_thread": False})
 # [/DEF:engine:Variable]
 
 # [DEF:tasks_engine:Variable]
+# @PURPOSE: SQLAlchemy engine for tasks database.
 tasks_engine = create_engine(TASKS_DATABASE_URL, connect_args={"check_same_thread": False})
 # [/DEF:tasks_engine:Variable]
 
+# [DEF:auth_engine:Variable]
+# @PURPOSE: SQLAlchemy engine for authentication database.
+auth_engine = create_engine(AUTH_DATABASE_URL, connect_args={"check_same_thread": False})
+# [/DEF:auth_engine:Variable]
+
 # [DEF:SessionLocal:Class]
 # @PURPOSE: A session factory for the main mappings database.
+# @PRE: engine is initialized.
 SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
 # [/DEF:SessionLocal:Class]
 
 # [DEF:TasksSessionLocal:Class]
 # @PURPOSE: A session factory for the tasks execution database.
+# @PRE: tasks_engine is initialized.
 TasksSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=tasks_engine)
 # [/DEF:TasksSessionLocal:Class]
 
+# [DEF:AuthSessionLocal:Class]
+# @PURPOSE: A session factory for the authentication database.
+# @PRE: auth_engine is initialized.
+AuthSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=auth_engine)
+# [/DEF:AuthSessionLocal:Class]
+
 # [DEF:init_db:Function]
 # @PURPOSE: Initializes the database by creating all tables.
-# @PRE: engine and tasks_engine are initialized.
+# @PRE: engine, tasks_engine and auth_engine are initialized.
-# @POST: Database tables created.
+# @POST: Database tables created in all databases.
+# @SIDE_EFFECT: Creates physical database files if they don't exist.
 def init_db():
     with belief_scope("init_db"):
         Base.metadata.create_all(bind=engine)
         Base.metadata.create_all(bind=tasks_engine)
+        Base.metadata.create_all(bind=auth_engine)
 # [/DEF:init_db:Function]
 
 # [DEF:get_db:Function]
@@ -84,4 +124,18 @@ def get_tasks_db():
             db.close()
 # [/DEF:get_tasks_db:Function]
 
+# [DEF:get_auth_db:Function]
+# @PURPOSE: Dependency for getting an authentication database session.
+# @PRE: AuthSessionLocal is initialized.
+# @POST:    Session is closed after use.
+# @RETURN:  Generator[Session, None, None]
+def get_auth_db():
+    with belief_scope("get_auth_db"):
+        db = AuthSessionLocal()
+        try:
+            yield db
+        finally:
+            db.close()
+# [/DEF:get_auth_db:Function]
+
 # [/DEF:backend.src.core.database:Module]

backend/src/core/logger.py

@@ -28,6 +28,7 @@ class BeliefFormatter(logging.Formatter):
     # @POST: Returns formatted string.
     # @PARAM: record (logging.LogRecord) - The log record to format.
     # @RETURN: str - The formatted log message.
+    # @SEMANTICS: logging, formatter, context
     def format(self, record):
         anchor_id = getattr(_belief_state, 'anchor_id', None)
         if anchor_id:
@@ -54,6 +55,7 @@ class LogEntry(BaseModel):
 # @PARAM: message (str) - Optional entry message.
 # @PRE: anchor_id must be provided.
 # @POST: Thread-local belief state is updated and entry/exit logs are generated.
+# @SEMANTICS: logging, context, belief_state
 @contextmanager
 def belief_scope(anchor_id: str, message: str = ""):
     # Log Entry if enabled
@@ -88,6 +90,7 @@ def belief_scope(anchor_id: str, message: str = ""):
 # @PRE: config is a valid LoggingConfig instance.
 # @POST: Logger level, handlers, and belief state flag are updated.
 # @PARAM: config (LoggingConfig) - The logging configuration.
+# @SEMANTICS: logging, configuration, initialization
 def configure_logger(config):
     global _enable_belief_state
     _enable_belief_state = config.enable_belief_state
@@ -140,6 +143,7 @@ class WebSocketLogHandler(logging.Handler):
     # @PRE: capacity is an integer.
     # @POST: Instance initialized with empty deque.
     # @PARAM: capacity (int) - Maximum number of logs to keep in memory.
+    # @SEMANTICS: logging, initialization, buffer
     def __init__(self, capacity: int = 1000):
         super().__init__()
         self.log_buffer: deque[LogEntry] = deque(maxlen=capacity)
@@ -152,6 +156,7 @@ class WebSocketLogHandler(logging.Handler):
     # @PRE: record is a logging.LogRecord.
     # @POST: Log is added to the log_buffer.
     # @PARAM: record (logging.LogRecord) - The log record to emit.
+    # @SEMANTICS: logging, handler, buffer
     def emit(self, record: logging.LogRecord):
         try:
             log_entry = LogEntry(
@@ -179,6 +184,7 @@ class WebSocketLogHandler(logging.Handler):
     # @PRE: None.
     # @POST: Returns list of LogEntry objects.
     # @RETURN: List[LogEntry] - List of buffered log entries.
+    # @SEMANTICS: logging, buffer, retrieval
     def get_recent_logs(self) -> List[LogEntry]:
         """
         Returns a list of recent log entries from the buffer.
@@ -196,12 +202,18 @@ logger = logging.getLogger("superset_tools_app")
 # [DEF:believed:Function]
 # @PURPOSE: A decorator that wraps a function in a belief scope.
 # @PARAM: anchor_id (str) - The identifier for the semantic block.
+# @PRE: anchor_id must be a string.
+# @POST: Returns a decorator function.
 def believed(anchor_id: str):
     # [DEF:decorator:Function]
     # @PURPOSE: Internal decorator for belief scope.
+    # @PRE: func must be a callable.
+    # @POST: Returns the wrapped function.
     def decorator(func):
         # [DEF:wrapper:Function]
         # @PURPOSE: Internal wrapper that enters belief scope.
+        # @PRE: None.
+        # @POST: Executes the function within a belief scope.
         def wrapper(*args, **kwargs):
             with belief_scope(anchor_id):
                 return func(*args, **kwargs)

backend/src/core/plugin_base.py

@@ -1,5 +1,5 @@
 from abc import ABC, abstractmethod
-from typing import Dict, Any
+from typing import Dict, Any, Optional
 from .logger import belief_scope
 
 from pydantic import BaseModel, Field
@@ -68,6 +68,33 @@ class PluginBase(ABC):
             pass
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:required_permission:Function]
+    # @PURPOSE: Returns the required permission string to execute this plugin.
+    # @PRE: Plugin instance exists.
+    # @POST: Returns string permission.
+    # @RETURN: str - Required permission (e.g., "plugin:backup:execute").
+    def required_permission(self) -> str:
+        """The permission string required to execute this plugin."""
+        with belief_scope("required_permission"):
+            return f"plugin:{self.id}:execute"
+    # [/DEF:required_permission:Function]
+
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the plugin's UI, if applicable.
+    # @PRE: Plugin instance exists.
+    # @POST: Returns string route or None.
+    # @RETURN: Optional[str] - Frontend route.
+    def ui_route(self) -> Optional[str]:
+        """
+        The frontend route for the plugin's UI.
+        Returns None if the plugin does not have a dedicated UI page.
+        """
+        with belief_scope("ui_route"):
+            return None
+    # [/DEF:ui_route:Function]
+
     @abstractmethod
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for the plugin's input parameters.
@@ -111,5 +138,6 @@ class PluginConfig(BaseModel):
     name: str = Field(..., description="Human-readable name for the plugin")
     description: str = Field(..., description="Brief description of what the plugin does")
     version: str = Field(..., description="Version of the plugin")
+    ui_route: Optional[str] = Field(None, description="Frontend route for the plugin UI")
     input_schema: Dict[str, Any] = Field(..., description="JSON schema for input parameters", alias="schema")
 # [/DEF:PluginConfig:Class]

backend/src/core/plugin_loader.py

@@ -50,9 +50,18 @@ class PluginLoader:
             sys.path.insert(0, plugin_parent_dir)
 
         for filename in os.listdir(self.plugin_dir):
+            file_path = os.path.join(self.plugin_dir, filename)
+            
+            # Handle directory-based plugins (packages)
+            if os.path.isdir(file_path):
+                init_file = os.path.join(file_path, "__init__.py")
+                if os.path.exists(init_file):
+                    self._load_module(filename, init_file)
+                continue
+
+            # Handle single-file plugins
             if filename.endswith(".py") and filename != "__init__.py":
                 module_name = filename[:-3]
-                file_path = os.path.join(self.plugin_dir, filename)
                 self._load_module(module_name, file_path)
     # [/DEF:_load_plugins:Function]
 
@@ -132,6 +141,7 @@ class PluginLoader:
                 name=plugin_instance.name,
                 description=plugin_instance.description,
                 version=plugin_instance.version,
+                ui_route=plugin_instance.ui_route,
                 schema=schema,
             )
             # The following line is commented out because it requires a schema to be passed to validate against.

backend/src/core/task_manager/persistence.py

@@ -36,6 +36,7 @@ class TaskPersistenceService:
     # @PRE:     isinstance(task, Task)
     # @POST:    Task record created or updated in database.
     # @PARAM:   task (Task) - The task object to persist.
+    # @SIDE_EFFECT: Writes to task_records table in tasks.db
     def persist_task(self, task: Task) -> None:
         with belief_scope("TaskPersistenceService.persist_task", f"task_id={task.id}"):
             session: Session = TasksSessionLocal()
@@ -50,8 +51,19 @@ class TaskPersistenceService:
                 record.environment_id = task.params.get("environment_id") or task.params.get("source_env_id")
                 record.started_at = task.started_at
                 record.finished_at = task.finished_at
-                record.params = task.params
+                
-                record.result = task.result
+                # Ensure params and result are JSON serializable
+                def json_serializable(obj):
+                    if isinstance(obj, dict):
+                        return {k: json_serializable(v) for k, v in obj.items()}
+                    elif isinstance(obj, list):
+                        return [json_serializable(v) for v in obj]
+                    elif isinstance(obj, datetime):
+                        return obj.isoformat()
+                    return obj
+
+                record.params = json_serializable(task.params)
+                record.result = json_serializable(task.result)
                 
                 # Store logs as JSON, converting datetime to string
                 record.logs = []
@@ -59,6 +71,9 @@ class TaskPersistenceService:
                     log_dict = log.dict()
                     if isinstance(log_dict.get('timestamp'), datetime):
                         log_dict['timestamp'] = log_dict['timestamp'].isoformat()
+                    # Also clean up any datetimes in context
+                    if log_dict.get('context'):
+                        log_dict['context'] = json_serializable(log_dict['context'])
                     record.logs.append(log_dict)
                 
                 # Extract error if failed

backend/src/core/utils/network.py

@@ -140,7 +140,16 @@ class APIClient:
             app_logger.info("[authenticate][Enter] Authenticating to %s", self.base_url)
             try:
                 login_url = f"{self.base_url}/security/login"
+                # Log the payload keys and values (masking password)
+                masked_auth = {k: ("******" if k == "password" else v) for k, v in self.auth.items()}
+                app_logger.info(f"[authenticate][Debug] Login URL: {login_url}")
+                app_logger.info(f"[authenticate][Debug] Auth payload: {masked_auth}")
+                
                 response = self.session.post(login_url, json=self.auth, timeout=self.request_settings["timeout"])
+                
+                if response.status_code != 200:
+                    app_logger.error(f"[authenticate][Error] Status: {response.status_code}, Response: {response.text}")
+                
                 response.raise_for_status()
                 access_token = response.json()["access_token"]
                 
@@ -153,6 +162,9 @@ class APIClient:
                 app_logger.info("[authenticate][Exit] Authenticated successfully.")
                 return self._tokens
             except requests.exceptions.HTTPError as e:
+                status_code = e.response.status_code if e.response is not None else None
+                if status_code in [502, 503, 504]:
+                    raise NetworkError(f"Environment unavailable during authentication (Status {status_code})", status_code=status_code) from e
                 raise AuthenticationError(f"Authentication failed: {e}") from e
             except (requests.exceptions.RequestException, KeyError) as e:
                 raise NetworkError(f"Network or parsing error during authentication: {e}") from e
@@ -209,6 +221,8 @@ class APIClient:
     def _handle_http_error(self, e: requests.exceptions.HTTPError, endpoint: str):
         with belief_scope("_handle_http_error"):
             status_code = e.response.status_code
+        if status_code == 502 or status_code == 503 or status_code == 504:
+            raise NetworkError(f"Environment unavailable (Status {status_code})", status_code=status_code) from e
         if status_code == 404: raise DashboardNotFoundError(endpoint) from e
         if status_code == 403: raise PermissionDeniedError() from e
         if status_code == 401: raise AuthenticationError() from e

backend/src/dependencies.py

@@ -1,16 +1,23 @@
 # [DEF:Dependencies:Module]
-# @SEMANTICS: dependency, injection, singleton, factory
+# @SEMANTICS: dependency, injection, singleton, factory, auth, jwt
 # @PURPOSE: Manages the creation and provision of shared application dependencies, such as the PluginLoader and TaskManager, to avoid circular imports.
 # @LAYER: Core
 # @RELATION: Used by the main app and API routers to get access to shared instances.
 
 from pathlib import Path
+from typing import Optional
+from fastapi import Depends, HTTPException, status
+from fastapi.security import OAuth2PasswordBearer
+from jose import JWTError
 from .core.plugin_loader import PluginLoader
 from .core.task_manager import TaskManager
 from .core.config_manager import ConfigManager
 from .core.scheduler import SchedulerService
-from .core.database import init_db
+from .core.database import init_db, get_auth_db
 from .core.logger import logger, belief_scope
+from .core.auth.jwt import decode_token
+from .core.auth.repository import AuthRepository
+from .models.auth import User
 
 # Initialize singletons
 # Use absolute path relative to this file to ensure plugins are found regardless of CWD
@@ -28,8 +35,7 @@ init_db()
 # @RETURN: ConfigManager - The shared config manager instance.
 def get_config_manager() -> ConfigManager:
     """Dependency injector for the ConfigManager."""
-    with belief_scope("get_config_manager"):
+    return config_manager
-        return config_manager
 # [/DEF:get_config_manager:Function]
 
 plugin_dir = Path(__file__).parent / "plugins"
@@ -51,8 +57,7 @@ logger.info("SchedulerService initialized")
 # @RETURN: PluginLoader - The shared plugin loader instance.
 def get_plugin_loader() -> PluginLoader:
     """Dependency injector for the PluginLoader."""
-    with belief_scope("get_plugin_loader"):
+    return plugin_loader
-        return plugin_loader
 # [/DEF:get_plugin_loader:Function]
 
 # [DEF:get_task_manager:Function]
@@ -62,8 +67,7 @@ def get_plugin_loader() -> PluginLoader:
 # @RETURN: TaskManager - The shared task manager instance.
 def get_task_manager() -> TaskManager:
     """Dependency injector for the TaskManager."""
-    with belief_scope("get_task_manager"):
+    return task_manager
-        return task_manager
 # [/DEF:get_task_manager:Function]
 
 # [DEF:get_scheduler_service:Function]
@@ -73,8 +77,71 @@ def get_task_manager() -> TaskManager:
 # @RETURN: SchedulerService - The shared scheduler service instance.
 def get_scheduler_service() -> SchedulerService:
     """Dependency injector for the SchedulerService."""
-    with belief_scope("get_scheduler_service"):
+    return scheduler_service
-        return scheduler_service
 # [/DEF:get_scheduler_service:Function]
 
+# [DEF:oauth2_scheme:Variable]
+# @PURPOSE: OAuth2 password bearer scheme for token extraction.
+oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/auth/login")
+# [/DEF:oauth2_scheme:Variable]
+
+# [DEF:get_current_user:Function]
+# @PURPOSE: Dependency for retrieving the currently authenticated user from a JWT.
+# @PRE:     JWT token provided in Authorization header.
+# @POST:    Returns the User object if token is valid.
+# @THROW:   HTTPException 401 if token is invalid or user not found.
+# @PARAM:   token (str) - Extracted JWT token.
+# @PARAM:   db (Session) - Auth database session.
+# @RETURN:  User - The authenticated user.
+def get_current_user(token: str = Depends(oauth2_scheme), db = Depends(get_auth_db)):
+    credentials_exception = HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Could not validate credentials",
+        headers={"WWW-Authenticate": "Bearer"},
+    )
+    try:
+        payload = decode_token(token)
+        username: str = payload.get("sub")
+        if username is None:
+            raise credentials_exception
+    except JWTError:
+        raise credentials_exception
+        
+    repo = AuthRepository(db)
+    user = repo.get_user_by_username(username)
+    if user is None:
+        raise credentials_exception
+    return user
+# [/DEF:get_current_user:Function]
+
+# [DEF:has_permission:Function]
+# @PURPOSE: Dependency for checking if the current user has a specific permission.
+# @PRE:     User is authenticated.
+# @POST:    Returns True if user has permission.
+# @THROW:   HTTPException 403 if permission is denied.
+# @PARAM:   resource (str) - The resource identifier.
+# @PARAM:   action (str) - The action identifier (READ, EXECUTE, WRITE).
+# @RETURN:  User - The authenticated user if permission granted.
+def has_permission(resource: str, action: str):
+    def permission_checker(current_user: User = Depends(get_current_user)):
+        # Union of all permissions across all roles
+        for role in current_user.roles:
+            for perm in role.permissions:
+                if perm.resource == resource and perm.action == action:
+                    return current_user
+        
+        # Special case for Admin role (full access)
+        if any(role.name == "Admin" for role in current_user.roles):
+            return current_user
+        
+        from .core.auth.logger import log_security_event
+        log_security_event("PERMISSION_DENIED", current_user.username, {"resource": resource, "action": action})
+            
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN,
+            detail=f"Permission denied for {resource}:{action}"
+        )
+    return permission_checker
+# [/DEF:has_permission:Function]
+
 # [/DEF:Dependencies:Module]

backend/src/models/auth.py

@@ -0,0 +1,105 @@
+# [DEF:backend.src.models.auth:Module]
+#
+# @TIER:      STANDARD
+# @SEMANTICS: auth, models, user, role, permission, sqlalchemy
+# @PURPOSE:   SQLAlchemy models for multi-user authentication and authorization.
+# @LAYER:     Domain
+# @RELATION:  INHERITS_FROM -> backend.src.models.mapping.Base
+#
+# @INVARIANT: Usernames and emails must be unique.
+
+# [SECTION: IMPORTS]
+import uuid
+from datetime import datetime
+from sqlalchemy import Column, String, Boolean, DateTime, ForeignKey, Table, Enum
+from sqlalchemy.orm import relationship
+from .mapping import Base
+# [/SECTION]
+
+# [DEF:generate_uuid:Function]
+# @PURPOSE: Generates a unique UUID string.
+# @POST:    Returns a string representation of a new UUID.
+def generate_uuid():
+    return str(uuid.uuid4())
+# [/DEF:generate_uuid:Function]
+
+# [DEF:user_roles:Table]
+# @PURPOSE: Association table for many-to-many relationship between Users and Roles.
+user_roles = Table(
+    "user_roles",
+    Base.metadata,
+    Column("user_id", String, ForeignKey("users.id"), primary_key=True),
+    Column("role_id", String, ForeignKey("roles.id"), primary_key=True),
+)
+# [/DEF:user_roles:Table]
+
+# [DEF:role_permissions:Table]
+# @PURPOSE: Association table for many-to-many relationship between Roles and Permissions.
+role_permissions = Table(
+    "role_permissions",
+    Base.metadata,
+    Column("role_id", String, ForeignKey("roles.id"), primary_key=True),
+    Column("permission_id", String, ForeignKey("permissions.id"), primary_key=True),
+)
+# [/DEF:role_permissions:Table]
+
+# [DEF:User:Class]
+# @PURPOSE: Represents an identity that can authenticate to the system.
+# @RELATION: HAS_MANY -> Role (via user_roles)
+class User(Base):
+    __tablename__ = "users"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    username = Column(String, unique=True, index=True, nullable=False)
+    email = Column(String, unique=True, index=True, nullable=True)
+    password_hash = Column(String, nullable=True)
+    auth_source = Column(String, default="LOCAL") # LOCAL or ADFS
+    is_active = Column(Boolean, default=True)
+    created_at = Column(DateTime, default=datetime.utcnow)
+    last_login = Column(DateTime, nullable=True)
+    
+    roles = relationship("Role", secondary=user_roles, back_populates="users")
+# [/DEF:User:Class]
+
+# [DEF:Role:Class]
+# @PURPOSE: Represents a collection of permissions.
+# @RELATION: HAS_MANY -> User (via user_roles)
+# @RELATION: HAS_MANY -> Permission (via role_permissions)
+class Role(Base):
+    __tablename__ = "roles"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    name = Column(String, unique=True, index=True, nullable=False)
+    description = Column(String, nullable=True)
+    
+    users = relationship("User", secondary=user_roles, back_populates="roles")
+    permissions = relationship("Permission", secondary=role_permissions, back_populates="roles")
+# [/DEF:Role:Class]
+
+# [DEF:Permission:Class]
+# @PURPOSE: Represents a specific capability within the system.
+# @RELATION: HAS_MANY -> Role (via role_permissions)
+class Permission(Base):
+    __tablename__ = "permissions"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    resource = Column(String, nullable=False) # e.g. "plugin:backup"
+    action = Column(String, nullable=False)   # e.g. "READ", "EXECUTE", "WRITE"
+    
+    roles = relationship("Role", secondary=role_permissions, back_populates="permissions")
+# [/DEF:Permission:Class]
+
+# [DEF:ADGroupMapping:Class]
+# @PURPOSE: Maps an Active Directory group to a local System Role.
+# @RELATION: DEPENDS_ON -> Role
+class ADGroupMapping(Base):
+    __tablename__ = "ad_group_mappings"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    ad_group = Column(String, unique=True, index=True, nullable=False)
+    role_id = Column(String, ForeignKey("roles.id"), nullable=False)
+    
+    role = relationship("Role")
+# [/DEF:ADGroupMapping:Class]
+
+# [/DEF:backend.src.models.auth:Module]

backend/src/models/dashboard.py

@@ -1,4 +1,5 @@
 # [DEF:backend.src.models.dashboard:Module]
+# @TIER:      STANDARD
 # @SEMANTICS: dashboard, model, metadata, migration
 # @PURPOSE:   Defines data models for dashboard metadata and selection.
 # @LAYER:     Model
@@ -8,6 +9,7 @@ from pydantic import BaseModel
 from typing import List
 
 # [DEF:DashboardMetadata:Class]
+# @TIER: TRIVIAL
 # @PURPOSE: Represents a dashboard available for migration.
 class DashboardMetadata(BaseModel):
     id: int
@@ -17,6 +19,7 @@ class DashboardMetadata(BaseModel):
 # [/DEF:DashboardMetadata:Class]
 
 # [DEF:DashboardSelection:Class]
+# @TIER: TRIVIAL
 # @PURPOSE: Represents the user's selection of dashboards to migrate.
 class DashboardSelection(BaseModel):
     selected_ids: List[int]

backend/src/models/llm.py

@@ -0,0 +1,46 @@
+# [DEF:backend.src.models.llm:Module]
+# @TIER: STANDARD
+# @SEMANTICS: llm, models, sqlalchemy, persistence
+# @PURPOSE: SQLAlchemy models for LLM provider configuration and validation results.
+# @LAYER: Domain
+# @RELATION: INHERITS_FROM -> backend.src.models.mapping.Base
+
+from sqlalchemy import Column, String, Boolean, DateTime, JSON, Enum, Text
+from datetime import datetime
+import uuid
+from .mapping import Base
+
+def generate_uuid():
+    return str(uuid.uuid4())
+
+# [DEF:LLMProvider:Class]
+# @PURPOSE: SQLAlchemy model for LLM provider configuration.
+class LLMProvider(Base):
+    __tablename__ = "llm_providers"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    provider_type = Column(String, nullable=False) # openai, openrouter, kilo
+    name = Column(String, nullable=False)
+    base_url = Column(String, nullable=False)
+    api_key = Column(String, nullable=False) # Should be encrypted
+    default_model = Column(String, nullable=False)
+    is_active = Column(Boolean, default=True)
+    created_at = Column(DateTime, default=datetime.utcnow)
+# [/DEF:LLMProvider:Class]
+
+# [DEF:ValidationRecord:Class]
+# @PURPOSE: SQLAlchemy model for dashboard validation history.
+class ValidationRecord(Base):
+    __tablename__ = "llm_validation_results"
+    
+    id = Column(String, primary_key=True, default=generate_uuid)
+    dashboard_id = Column(String, nullable=False, index=True)
+    timestamp = Column(DateTime, default=datetime.utcnow)
+    status = Column(String, nullable=False) # PASS, WARN, FAIL
+    screenshot_path = Column(String, nullable=True)
+    issues = Column(JSON, nullable=False)
+    summary = Column(Text, nullable=False)
+    raw_response = Column(Text, nullable=True)
+# [/DEF:ValidationRecord:Class]
+
+# [/DEF:backend.src.models.llm:Module]

backend/src/models/mapping.py

@@ -1,5 +1,6 @@
 # [DEF:backend.src.models.mapping:Module]
 #
+# @TIER:      STANDARD
 # @SEMANTICS: database, mapping, environment, migration, sqlalchemy, sqlite
 # @PURPOSE:   Defines the database schema for environment metadata and database mappings using SQLAlchemy.
 # @LAYER:     Domain
@@ -19,6 +20,7 @@ import enum
 Base = declarative_base()
 
 # [DEF:MigrationStatus:Class]
+# @TIER: TRIVIAL
 # @PURPOSE: Enumeration of possible migration job statuses.
 class MigrationStatus(enum.Enum):
     PENDING = "PENDING"
@@ -29,6 +31,7 @@ class MigrationStatus(enum.Enum):
 # [/DEF:MigrationStatus:Class]
 
 # [DEF:Environment:Class]
+# @TIER: STANDARD
 # @PURPOSE: Represents a Superset instance environment.
 class Environment(Base):
     __tablename__ = "environments"
@@ -40,6 +43,7 @@ class Environment(Base):
 # [/DEF:Environment:Class]
 
 # [DEF:DatabaseMapping:Class]
+# @TIER: STANDARD
 # @PURPOSE: Represents a mapping between source and target databases.
 class DatabaseMapping(Base):
     __tablename__ = "database_mappings"

backend/src/models/storage.py

@@ -0,0 +1,31 @@
+from datetime import datetime
+from enum import Enum
+from typing import Optional
+from pydantic import BaseModel, Field
+
+# [DEF:FileCategory:Class]
+# @PURPOSE: Enumeration of supported file categories in the storage system.
+class FileCategory(str, Enum):
+    BACKUP = "backups"
+    REPOSITORY = "repositorys"
+# [/DEF:FileCategory:Class]
+
+# [DEF:StorageConfig:Class]
+# @PURPOSE: Configuration model for the storage system, defining paths and naming patterns.
+class StorageConfig(BaseModel):
+    root_path: str = Field(default="backups", description="Absolute path to the storage root directory.")
+    backup_structure_pattern: str = Field(default="{category}/", description="Pattern for backup directory structure.")
+    repo_structure_pattern: str = Field(default="{category}/", description="Pattern for repository directory structure.")
+    filename_pattern: str = Field(default="{name}_{timestamp}", description="Pattern for filenames.")
+# [/DEF:StorageConfig:Class]
+
+# [DEF:StoredFile:Class]
+# @PURPOSE: Data model representing metadata for a file stored in the system.
+class StoredFile(BaseModel):
+    name: str = Field(..., description="Name of the file (including extension).")
+    path: str = Field(..., description="Relative path from storage root.")
+    size: int = Field(..., ge=0, description="Size of the file in bytes.")
+    created_at: datetime = Field(..., description="Creation timestamp.")
+    category: FileCategory = Field(..., description="Category of the file.")
+    mime_type: Optional[str] = Field(None, description="MIME type of the file.")
+# [/DEF:StoredFile:Class]

backend/src/plugins/backup.py

@@ -75,6 +75,15 @@ class BackupPlugin(PluginBase):
             return "1.0.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the backup plugin.
+    # @RETURN: str - "/tools/backups"
+    def ui_route(self) -> str:
+        with belief_scope("ui_route"):
+            return "/tools/backups"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for backup plugin parameters.
     # @PRE: Plugin instance exists.
@@ -84,7 +93,7 @@ class BackupPlugin(PluginBase):
         with belief_scope("get_schema"):
             config_manager = get_config_manager()
             envs = [e.name for e in config_manager.get_environments()]
-        default_path = config_manager.get_config().settings.backup_path
+        default_path = config_manager.get_config().settings.storage.root_path
         
         return {
             "type": "object",
@@ -95,14 +104,8 @@ class BackupPlugin(PluginBase):
                     "description": "The Superset environment to back up.",
                     "enum": envs if envs else [],
                 },
-                "backup_path": {
-                    "type": "string",
-                    "title": "Backup Path",
-                    "description": "The root directory to save backups to.",
-                    "default": default_path
-                }
             },
-            "required": ["env", "backup_path"],
+            "required": ["env"],
         }
     # [/DEF:get_schema:Function]
 
@@ -126,8 +129,9 @@ class BackupPlugin(PluginBase):
             if not env:
                 raise KeyError("env")
 
-            backup_path_str = params.get("backup_path") or config_manager.get_config().settings.backup_path
+            storage_settings = config_manager.get_config().settings.storage
-            backup_path = Path(backup_path_str)
+            # Use 'backups' subfolder within the storage root
+            backup_path = Path(storage_settings.root_path) / "backups"
             
             from ..core.logger import logger as app_logger
             app_logger.info(f"[BackupPlugin][Entry] Starting backup for {env}.")

backend/src/plugins/debug.py

@@ -63,6 +63,15 @@ class DebugPlugin(PluginBase):
             return "1.0.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the debug plugin.
+    # @RETURN: str - "/tools/debug"
+    def ui_route(self) -> str:
+        with belief_scope("ui_route"):
+            return "/tools/debug"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for the debug plugin parameters.
     # @PRE: Plugin instance exists.

backend/src/plugins/git/llm_extension.py

@@ -0,0 +1,66 @@
+# [DEF:backend/src/plugins/git/llm_extension:Module]
+# @TIER: STANDARD
+# @SEMANTICS: git, llm, commit
+# @PURPOSE: LLM-based extensions for the Git plugin, specifically for commit message generation.
+# @LAYER: Domain
+# @RELATION: DEPENDS_ON -> backend.src.plugins.llm_analysis.service.LLMClient
+
+from typing import List, Optional
+from tenacity import retry, stop_after_attempt, wait_exponential
+from ..llm_analysis.service import LLMClient
+from ..llm_analysis.models import LLMProviderType
+from ...core.logger import belief_scope, logger
+
+# [DEF:GitLLMExtension:Class]
+# @PURPOSE: Provides LLM capabilities to the Git plugin.
+class GitLLMExtension:
+    def __init__(self, client: LLMClient):
+        self.client = client
+
+    # [DEF:suggest_commit_message:Function]
+    # @PURPOSE: Generates a suggested commit message based on a diff and history.
+    # @PARAM: diff (str) - The git diff of staged changes.
+    # @PARAM: history (List[str]) - Recent commit messages for context.
+    # @RETURN: str - The suggested commit message.
+    @retry(
+        stop=stop_after_attempt(2),
+        wait=wait_exponential(multiplier=1, min=2, max=10),
+        reraise=True
+    )
+    async def suggest_commit_message(self, diff: str, history: List[str]) -> str:
+        with belief_scope("suggest_commit_message"):
+            history_text = "\n".join(history)
+            prompt = f"""
+            Generate a concise and professional git commit message based on the following diff and recent history.
+            Use Conventional Commits format (e.g., feat: ..., fix: ..., docs: ...).
+            
+            Recent History:
+            {history_text}
+            
+            Diff:
+            {diff}
+            
+            Commit Message:
+            """
+            
+            logger.debug(f"[suggest_commit_message] Calling LLM with model: {self.client.default_model}")
+            response = await self.client.client.chat.completions.create(
+                model=self.client.default_model,
+                messages=[{"role": "user", "content": prompt}],
+                temperature=0.7
+            )
+            
+            logger.debug(f"[suggest_commit_message] LLM Response: {response}")
+            
+            if not response or not hasattr(response, 'choices') or not response.choices:
+                error_info = getattr(response, 'error', 'No choices in response')
+                logger.error(f"[suggest_commit_message] Invalid LLM response. Error info: {error_info}")
+                
+                # If it's a timeout/provider error, we might want to throw to trigger retry if decorated
+                # but for now we return a safe fallback to avoid UI crash
+                return "Update dashboard configurations (LLM generation failed)"
+
+            return response.choices[0].message.content.strip()
+# [/DEF:GitLLMExtension:Class]
+
+# [/DEF:backend/src/plugins/git/llm_extension:Module]

backend/src/plugins/git_plugin.py

@@ -99,6 +99,15 @@ class GitPlugin(PluginBase):
             return "0.1.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the git plugin.
+    # @RETURN: str - "/git"
+    def ui_route(self) -> str:
+        with belief_scope("GitPlugin.ui_route"):
+            return "/git"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Возвращает JSON-схему параметров для выполнения задач плагина.
     # @PRE: GitPlugin is initialized.

backend/src/plugins/llm_analysis/__init__.py

@@ -0,0 +1,12 @@
+# [DEF:backend/src/plugins/llm_analysis/__init__.py:Module]
+# @TIER: TRIVIAL
+# @PURPOSE: Initialize the LLM Analysis plugin package.
+# @LAYER: Domain
+
+"""
+LLM Analysis Plugin for automated dashboard validation and dataset documentation.
+"""
+
+from .plugin import DashboardValidationPlugin, DocumentationPlugin
+
+# [/DEF:backend/src/plugins/llm_analysis/__init__.py:Module]

backend/src/plugins/llm_analysis/models.py

@@ -0,0 +1,61 @@
+# [DEF:backend/src/plugins/llm_analysis/models.py:Module]
+# @TIER: STANDARD
+# @SEMANTICS: pydantic, models, llm
+# @PURPOSE: Define Pydantic models for LLM Analysis plugin.
+# @LAYER: Domain
+
+from typing import List, Optional
+from pydantic import BaseModel, Field
+from datetime import datetime
+from enum import Enum
+
+# [DEF:LLMProviderType:Class]
+# @PURPOSE: Enum for supported LLM providers.
+class LLMProviderType(str, Enum):
+    OPENAI = "openai"
+    OPENROUTER = "openrouter"
+    KILO = "kilo"
+# [/DEF:LLMProviderType:Class]
+
+# [DEF:LLMProviderConfig:Class]
+# @PURPOSE: Configuration for an LLM provider.
+class LLMProviderConfig(BaseModel):
+    id: Optional[str] = None
+    provider_type: LLMProviderType
+    name: str
+    base_url: str
+    api_key: Optional[str] = None
+    default_model: str
+    is_active: bool = True
+# [/DEF:LLMProviderConfig:Class]
+
+# [DEF:ValidationStatus:Class]
+# @PURPOSE: Enum for dashboard validation status.
+class ValidationStatus(str, Enum):
+    PASS = "PASS"
+    WARN = "WARN"
+    FAIL = "FAIL"
+# [/DEF:ValidationStatus:Class]
+
+# [DEF:DetectedIssue:Class]
+# @PURPOSE: Model for a single issue detected during validation.
+class DetectedIssue(BaseModel):
+    severity: ValidationStatus
+    message: str
+    location: Optional[str] = None
+# [/DEF:DetectedIssue:Class]
+
+# [DEF:ValidationResult:Class]
+# @PURPOSE: Model for dashboard validation result.
+class ValidationResult(BaseModel):
+    id: Optional[str] = None
+    dashboard_id: str
+    timestamp: datetime = Field(default_factory=datetime.utcnow)
+    status: ValidationStatus
+    screenshot_path: Optional[str] = None
+    issues: List[DetectedIssue]
+    summary: str
+    raw_response: Optional[str] = None
+# [/DEF:ValidationResult:Class]
+
+# [/DEF:backend/src/plugins/llm_analysis/models.py:Module]

backend/src/plugins/llm_analysis/plugin.py

@@ -0,0 +1,377 @@
+# [DEF:backend/src/plugins/llm_analysis/plugin.py:Module]
+# @TIER: STANDARD
+# @SEMANTICS: plugin, llm, analysis, documentation
+# @PURPOSE: Implements DashboardValidationPlugin and DocumentationPlugin.
+# @LAYER: Domain
+# @RELATION: INHERITS -> backend.src.core.plugin_base.PluginBase
+# @RELATION: CALLS -> backend.src.plugins.llm_analysis.service.ScreenshotService
+# @RELATION: CALLS -> backend.src.plugins.llm_analysis.service.LLMClient
+# @RELATION: CALLS -> backend.src.services.llm_provider.LLMProviderService
+# @INVARIANT: All LLM interactions must be executed as asynchronous tasks.
+
+from typing import Dict, Any, Optional, List
+import os
+import json
+import logging
+from datetime import datetime, timedelta
+from ...core.plugin_base import PluginBase
+from ...core.logger import belief_scope, logger
+from ...core.database import SessionLocal
+from ...core.config_manager import ConfigManager
+from ...services.llm_provider import LLMProviderService
+from ...core.superset_client import SupersetClient
+from .service import ScreenshotService, LLMClient
+from .models import LLMProviderType, ValidationStatus, ValidationResult, DetectedIssue
+from ...models.llm import ValidationRecord
+
+# [DEF:DashboardValidationPlugin:Class]
+# @PURPOSE: Plugin for automated dashboard health analysis using LLMs.
+# @RELATION: IMPLEMENTS -> backend.src.core.plugin_base.PluginBase
+class DashboardValidationPlugin(PluginBase):
+    @property
+    def id(self) -> str:
+        return "llm_dashboard_validation"
+
+    @property
+    def name(self) -> str:
+        return "Dashboard LLM Validation"
+
+    @property
+    def description(self) -> str:
+        return "Automated dashboard health analysis using multimodal LLMs."
+
+    @property
+    def version(self) -> str:
+        return "1.0.0"
+
+    def get_schema(self) -> Dict[str, Any]:
+        return {
+            "type": "object",
+            "properties": {
+                "dashboard_id": {"type": "string", "title": "Dashboard ID"},
+                "environment_id": {"type": "string", "title": "Environment ID"},
+                "provider_id": {"type": "string", "title": "LLM Provider ID"}
+            },
+            "required": ["dashboard_id", "environment_id", "provider_id"]
+        }
+
+    # [DEF:DashboardValidationPlugin.execute:Function]
+    # @PURPOSE: Executes the dashboard validation task.
+    # @PRE: params contains dashboard_id, environment_id, and provider_id.
+    # @POST: Returns a dictionary with validation results and persists them to the database.
+    # @SIDE_EFFECT: Captures a screenshot, calls LLM API, and writes to the database.
+    async def execute(self, params: Dict[str, Any]):
+        with belief_scope("execute", f"plugin_id={self.id}"):
+            logger.info(f"Executing {self.name} with params: {params}")
+            
+            dashboard_id = params.get("dashboard_id")
+            env_id = params.get("environment_id")
+            provider_id = params.get("provider_id")
+            task_id = params.get("_task_id")
+
+            # Helper to log to both app logger and task manager logs
+            def task_log(level: str, message: str, context: Optional[Dict] = None):
+                logger.log(getattr(logging, level.upper()), message)
+                if task_id:
+                    from ...dependencies import get_task_manager
+                    try:
+                        tm = get_task_manager()
+                        tm._add_log(task_id, level.upper(), message, context)
+                    except: pass
+
+            db = SessionLocal()
+            try:
+                # 1. Get Environment
+                from ...dependencies import get_config_manager
+                config_mgr = get_config_manager()
+                env = config_mgr.get_environment(env_id)
+                if not env:
+                    raise ValueError(f"Environment {env_id} not found")
+
+                # 2. Get LLM Provider
+                llm_service = LLMProviderService(db)
+                db_provider = llm_service.get_provider(provider_id)
+                if not db_provider:
+                    raise ValueError(f"LLM Provider {provider_id} not found")
+                
+                logger.info(f"[DashboardValidationPlugin.execute] Retrieved provider config:")
+                logger.info(f"[DashboardValidationPlugin.execute]   Provider ID: {db_provider.id}")
+                logger.info(f"[DashboardValidationPlugin.execute]   Provider Name: {db_provider.name}")
+                logger.info(f"[DashboardValidationPlugin.execute]   Provider Type: {db_provider.provider_type}")
+                logger.info(f"[DashboardValidationPlugin.execute]   Base URL: {db_provider.base_url}")
+                logger.info(f"[DashboardValidationPlugin.execute]   Default Model: {db_provider.default_model}")
+                logger.info(f"[DashboardValidationPlugin.execute]   Is Active: {db_provider.is_active}")
+                
+                api_key = llm_service.get_decrypted_api_key(provider_id)
+                logger.info(f"[DashboardValidationPlugin.execute]   API Key decrypted (first 8 chars): {api_key[:8] if api_key and len(api_key) > 8 else 'EMPTY_OR_NONE'}...")
+                logger.info(f"[DashboardValidationPlugin.execute]   API Key Length: {len(api_key) if api_key else 0}")
+                
+                # Check if API key was successfully decrypted
+                if not api_key:
+                    raise ValueError(
+                        f"Failed to decrypt API key for provider {provider_id}. "
+                        f"The provider may have been encrypted with a different encryption key. "
+                        f"Please update the provider with a new API key through the UI."
+                    )
+
+                # 3. Capture Screenshot
+                screenshot_service = ScreenshotService(env)
+                
+                storage_root = config_mgr.get_config().settings.storage.root_path
+                screenshots_dir = os.path.join(storage_root, "screenshots")
+                os.makedirs(screenshots_dir, exist_ok=True)
+                
+                filename = f"{dashboard_id}_{datetime.now().strftime('%Y%m%d_%H%M%S')}.png"
+                screenshot_path = os.path.join(screenshots_dir, filename)
+                
+                await screenshot_service.capture_dashboard(dashboard_id, screenshot_path)
+
+                # 4. Fetch Logs (from Environment /api/v1/log/)
+                logs = []
+                try:
+                    client = SupersetClient(env)
+                    
+                    # Calculate time window (last 24 hours)
+                    start_time = (datetime.now() - timedelta(hours=24)).isoformat()
+                    
+                    # Construct filter for logs
+                    # Note: We filter by dashboard_id matching the object
+                    query_params = {
+                        "filters": [
+                            {"col": "dashboard_id", "opr": "eq", "value": dashboard_id},
+                            {"col": "dttm", "opr": "gt", "value": start_time}
+                        ],
+                        "order_column": "dttm",
+                        "order_direction": "desc",
+                        "page": 0,
+                        "page_size": 100
+                    }
+                    
+                    response = client.network.request(
+                        method="GET",
+                        endpoint="/log/",
+                        params={"q": json.dumps(query_params)}
+                    )
+                    
+                    if isinstance(response, dict) and "result" in response:
+                        for item in response["result"]:
+                            action = item.get("action", "unknown")
+                            dttm = item.get("dttm", "")
+                            details = item.get("json", "")
+                            logs.append(f"[{dttm}] {action}: {details}")
+                    
+                    if not logs:
+                        logs = ["No recent logs found for this dashboard."]
+
+                except Exception as e:
+                    logger.warning(f"Failed to fetch logs from environment: {e}")
+                    logs = [f"Error fetching remote logs: {str(e)}"]
+
+                # 5. Analyze with LLM
+                llm_client = LLMClient(
+                    provider_type=LLMProviderType(db_provider.provider_type),
+                    api_key=api_key,
+                    base_url=db_provider.base_url,
+                    default_model=db_provider.default_model
+                )
+                
+                analysis = await llm_client.analyze_dashboard(screenshot_path, logs)
+                
+                # Log analysis summary to task logs for better visibility
+                task_log("INFO", f"[ANALYSIS_SUMMARY] Status: {analysis['status']}")
+                task_log("INFO", f"[ANALYSIS_SUMMARY] Summary: {analysis['summary']}")
+                if analysis.get("issues"):
+                    for i, issue in enumerate(analysis["issues"]):
+                        task_log("INFO", f"[ANALYSIS_ISSUE][{i+1}] {issue.get('severity')}: {issue.get('message')} (Location: {issue.get('location', 'N/A')})")
+
+                # 6. Persist Result
+                validation_result = ValidationResult(
+                    dashboard_id=dashboard_id,
+                    status=ValidationStatus(analysis["status"]),
+                    summary=analysis["summary"],
+                    issues=[DetectedIssue(**issue) for issue in analysis["issues"]],
+                    screenshot_path=screenshot_path,
+                    raw_response=str(analysis)
+                )
+
+                db_record = ValidationRecord(
+                    dashboard_id=validation_result.dashboard_id,
+                    status=validation_result.status.value,
+                    summary=validation_result.summary,
+                    issues=[issue.dict() for issue in validation_result.issues],
+                    screenshot_path=validation_result.screenshot_path,
+                    raw_response=validation_result.raw_response
+                )
+                db.add(db_record)
+                db.commit()
+
+                # 7. Notification on failure (US1 / FR-015)
+                if validation_result.status == ValidationStatus.FAIL:
+                    task_log("WARNING", f"Dashboard {dashboard_id} validation FAILED. Summary: {validation_result.summary}")
+                    # Placeholder for Email/Pulse notification dispatch
+                    # In a real implementation, we would call a NotificationService here
+                    # with a payload containing the summary and a link to the report.
+
+                # Final log to ensure all analysis is visible in task logs
+                task_log("INFO", f"Validation completed for dashboard {dashboard_id}. Status: {validation_result.status.value}")
+                
+                return validation_result.dict()
+
+            finally:
+                db.close()
+    # [/DEF:DashboardValidationPlugin.execute:Function]
+# [/DEF:DashboardValidationPlugin:Class]
+
+# [DEF:DocumentationPlugin:Class]
+# @PURPOSE: Plugin for automated dataset documentation using LLMs.
+# @RELATION: IMPLEMENTS -> backend.src.core.plugin_base.PluginBase
+class DocumentationPlugin(PluginBase):
+    @property
+    def id(self) -> str:
+        return "llm_documentation"
+
+    @property
+    def name(self) -> str:
+        return "Dataset LLM Documentation"
+
+    @property
+    def description(self) -> str:
+        return "Automated dataset and column documentation using LLMs."
+
+    @property
+    def version(self) -> str:
+        return "1.0.0"
+
+    def get_schema(self) -> Dict[str, Any]:
+        return {
+            "type": "object",
+            "properties": {
+                "dataset_id": {"type": "string", "title": "Dataset ID"},
+                "environment_id": {"type": "string", "title": "Environment ID"},
+                "provider_id": {"type": "string", "title": "LLM Provider ID"}
+            },
+            "required": ["dataset_id", "environment_id", "provider_id"]
+        }
+
+    # [DEF:DocumentationPlugin.execute:Function]
+    # @PURPOSE: Executes the dataset documentation task.
+    # @PRE: params contains dataset_id, environment_id, and provider_id.
+    # @POST: Returns generated documentation and updates the dataset in Superset.
+    # @SIDE_EFFECT: Calls LLM API and updates dataset metadata in Superset.
+    async def execute(self, params: Dict[str, Any]):
+        with belief_scope("execute", f"plugin_id={self.id}"):
+            logger.info(f"Executing {self.name} with params: {params}")
+            
+            dataset_id = params.get("dataset_id")
+            env_id = params.get("environment_id")
+            provider_id = params.get("provider_id")
+
+            db = SessionLocal()
+            try:
+                # 1. Get Environment
+                from ...dependencies import get_config_manager
+                config_mgr = get_config_manager()
+                env = config_mgr.get_environment(env_id)
+                if not env:
+                    raise ValueError(f"Environment {env_id} not found")
+
+                # 2. Get LLM Provider
+                llm_service = LLMProviderService(db)
+                db_provider = llm_service.get_provider(provider_id)
+                if not db_provider:
+                    raise ValueError(f"LLM Provider {provider_id} not found")
+                
+                logger.info(f"[DocumentationPlugin.execute] Retrieved provider config:")
+                logger.info(f"[DocumentationPlugin.execute]   Provider ID: {db_provider.id}")
+                logger.info(f"[DocumentationPlugin.execute]   Provider Name: {db_provider.name}")
+                logger.info(f"[DocumentationPlugin.execute]   Provider Type: {db_provider.provider_type}")
+                logger.info(f"[DocumentationPlugin.execute]   Base URL: {db_provider.base_url}")
+                logger.info(f"[DocumentationPlugin.execute]   Default Model: {db_provider.default_model}")
+                logger.info(f"[DocumentationPlugin.execute]   Is Active: {db_provider.is_active}")
+                
+                api_key = llm_service.get_decrypted_api_key(provider_id)
+                logger.info(f"[DocumentationPlugin.execute]   API Key decrypted (first 8 chars): {api_key[:8] if api_key and len(api_key) > 8 else 'EMPTY_OR_NONE'}...")
+                logger.info(f"[DocumentationPlugin.execute]   API Key Length: {len(api_key) if api_key else 0}")
+                
+                # Check if API key was successfully decrypted
+                if not api_key:
+                    raise ValueError(
+                        f"Failed to decrypt API key for provider {provider_id}. "
+                        f"The provider may have been encrypted with a different encryption key. "
+                        f"Please update the provider with a new API key through the UI."
+                    )
+
+                # 3. Fetch Metadata (US2 / T024)
+                from ...core.superset_client import SupersetClient
+                client = SupersetClient(env)
+                
+                # Optimistic locking check (T045)
+                dataset = client.get_dataset(int(dataset_id))
+                # dataset structure might vary, ensure we get the right field
+                original_changed_on = dataset.get("changed_on_utc") or dataset.get("result", {}).get("changed_on_utc")
+                
+                # Extract columns and existing descriptions
+                columns_data = []
+                for col in dataset.get("columns", []):
+                    columns_data.append({
+                        "name": col.get("column_name"),
+                        "type": col.get("type"),
+                        "description": col.get("description")
+                    })
+
+                # 4. Construct Prompt & Analyze (US2 / T025)
+                llm_client = LLMClient(
+                    provider_type=LLMProviderType(db_provider.provider_type),
+                    api_key=api_key,
+                    base_url=db_provider.base_url,
+                    default_model=db_provider.default_model
+                )
+                
+                prompt = f"""
+                Generate professional documentation for the following dataset and its columns.
+                Dataset: {dataset.get('table_name')}
+                Columns: {columns_data}
+                
+                Provide the documentation in JSON format:
+                {{
+                    "dataset_description": "General description of the dataset",
+                    "column_descriptions": [
+                        {{
+                            "name": "column_name",
+                            "description": "Generated description"
+                        }}
+                    ]
+                }}
+                """
+                
+                # Using a generic chat completion for text-only US2
+                # We use the shared get_json_completion method from LLMClient
+                doc_result = await llm_client.get_json_completion([{"role": "user", "content": prompt}])
+
+                # 5. Update Metadata (US2 / T026)
+                # This part normally goes to mapping_service, but we implement the logic here for the plugin flow
+                # We'll update the dataset in Superset
+                update_payload = {
+                    "description": doc_result["dataset_description"],
+                    "columns": []
+                }
+                
+                # Map generated descriptions back to column IDs
+                for col_doc in doc_result["column_descriptions"]:
+                    for col in dataset.get("columns", []):
+                        if col.get("column_name") == col_doc["name"]:
+                            update_payload["columns"].append({
+                                "id": col.get("id"),
+                                "description": col_doc["description"]
+                            })
+
+                client.update_dataset(int(dataset_id), update_payload)
+                
+                return doc_result
+
+            finally:
+                db.close()
+    # [/DEF:DocumentationPlugin.execute:Function]
+# [/DEF:DocumentationPlugin:Class]
+
+# [/DEF:backend/src/plugins/llm_analysis/plugin.py:Module]

backend/src/plugins/llm_analysis/scheduler.py

@@ -0,0 +1,60 @@
+# [DEF:backend/src/plugins/llm_analysis/scheduler.py:Module]
+# @TIER: STANDARD
+# @SEMANTICS: scheduler, task, automation
+# @PURPOSE: Provides helper functions to schedule LLM-based validation tasks.
+# @LAYER: Domain
+# @RELATION: DEPENDS_ON -> backend.src.core.scheduler
+
+from typing import Dict, Any
+from ...dependencies import get_task_manager, get_scheduler_service
+from ...core.logger import belief_scope, logger
+
+# [DEF:schedule_dashboard_validation:Function]
+# @PURPOSE: Schedules a recurring dashboard validation task.
+# @PARAM: dashboard_id (str) - ID of the dashboard to validate.
+# @PARAM: cron_expression (str) - Standard cron expression for scheduling.
+# @PARAM: params (Dict[str, Any]) - Task parameters (environment_id, provider_id).
+# @SIDE_EFFECT: Adds a job to the scheduler service.
+def schedule_dashboard_validation(dashboard_id: str, cron_expression: str, params: Dict[str, Any]):
+    with belief_scope("schedule_dashboard_validation", f"dashboard_id={dashboard_id}"):
+        scheduler = get_scheduler_service()
+        task_manager = get_task_manager()
+        
+        job_id = f"llm_val_{dashboard_id}"
+        
+        async def job_func():
+            await task_manager.create_task(
+                plugin_id="llm_dashboard_validation",
+                params={
+                    "dashboard_id": dashboard_id,
+                    **params
+                }
+            )
+
+        scheduler.add_job(
+            job_func,
+            "cron",
+            id=job_id,
+            replace_existing=True,
+            **_parse_cron(cron_expression)
+        )
+        logger.info(f"Scheduled validation for dashboard {dashboard_id} with cron {cron_expression}")
+
+# [DEF:_parse_cron:Function]
+# @PURPOSE: Basic cron parser placeholder.
+# @PARAM: cron (str) - Cron expression.
+# @RETURN: Dict[str, str] - Parsed cron parts.
+def _parse_cron(cron: str) -> Dict[str, str]:
+    # Basic cron parser placeholder
+    parts = cron.split()
+    if len(parts) != 5:
+        return {}
+    return {
+        "minute": parts[0],
+        "hour": parts[1],
+        "day": parts[2],
+        "month": parts[3],
+        "day_of_week": parts[4]
+    }
+
+# [/DEF:backend/src/plugins/llm_analysis/scheduler.py:Module]

backend/src/plugins/llm_analysis/service.py

@@ -0,0 +1,629 @@
+# [DEF:backend/src/plugins/llm_analysis/service.py:Module]
+# @TIER: STANDARD
+# @SEMANTICS: service, llm, screenshot, playwright, openai
+# @PURPOSE: Services for LLM interaction and dashboard screenshots.
+# @LAYER: Domain
+# @RELATION: DEPENDS_ON -> playwright
+# @RELATION: DEPENDS_ON -> openai
+# @RELATION: DEPENDS_ON -> tenacity
+# @INVARIANT: Screenshots must be 1920px width and capture full page height.
+
+import asyncio
+import base64
+import json
+import io
+from typing import List, Optional, Dict, Any
+from PIL import Image
+from playwright.async_api import async_playwright
+from openai import AsyncOpenAI, RateLimitError, AuthenticationError as OpenAIAuthenticationError
+from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception
+from .models import LLMProviderType, ValidationResult, ValidationStatus, DetectedIssue
+from ...core.logger import belief_scope, logger
+from ...core.config_models import Environment
+
+# [DEF:ScreenshotService:Class]
+# @PURPOSE: Handles capturing screenshots of Superset dashboards.
+class ScreenshotService:
+    # [DEF:ScreenshotService.__init__:Function]
+    # @PURPOSE: Initializes the ScreenshotService with environment configuration.
+    # @PRE: env is a valid Environment object.
+    def __init__(self, env: Environment):
+        self.env = env
+    # [/DEF:ScreenshotService.__init__:Function]
+
+    # [DEF:ScreenshotService.capture_dashboard:Function]
+    # @PURPOSE: Captures a full-page screenshot of a dashboard using Playwright and CDP.
+    # @PRE: dashboard_id is a valid string, output_path is a writable path.
+    # @POST: Returns True if screenshot is saved successfully.
+    # @SIDE_EFFECT: Launches a browser, performs UI login, switches tabs, and writes a PNG file.
+    # @UX_STATE: [Navigating] -> Loading dashboard UI
+    # @UX_STATE: [TabSwitching] -> Iterating through dashboard tabs to trigger lazy loading
+    # @UX_STATE: [CalculatingHeight] -> Determining dashboard dimensions
+    # @UX_STATE: [Capturing] -> Executing CDP screenshot
+    async def capture_dashboard(self, dashboard_id: str, output_path: str) -> bool:
+        with belief_scope("capture_dashboard", f"dashboard_id={dashboard_id}"):
+            logger.info(f"Capturing screenshot for dashboard {dashboard_id}")
+            async with async_playwright() as p:
+                browser = await p.chromium.launch(
+                    headless=True,
+                    args=[
+                        "--disable-blink-features=AutomationControlled",
+                        "--disable-infobars",
+                        "--no-sandbox"
+                    ]
+                )
+                # Set a realistic user agent to avoid 403 Forbidden from OpenResty/WAF
+                user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
+                # Construct base UI URL from environment (strip /api/v1 suffix)
+                base_ui_url = self.env.url.rstrip("/")
+                if base_ui_url.endswith("/api/v1"):
+                    base_ui_url = base_ui_url[:-len("/api/v1")]
+                
+                # Create browser context with realistic headers
+                context = await browser.new_context(
+                    viewport={'width': 1280, 'height': 720},
+                    user_agent=user_agent,
+                    extra_http_headers={
+                        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
+                        "Accept-Language": "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7",
+                        "Upgrade-Insecure-Requests": "1",
+                        "Sec-Fetch-Dest": "document",
+                        "Sec-Fetch-Mode": "navigate",
+                        "Sec-Fetch-Site": "none",
+                        "Sec-Fetch-User": "?1"
+                    }
+                )
+                logger.info("Browser context created successfully")
+
+                page = await context.new_page()
+                # Bypass navigator.webdriver detection
+                await page.add_init_script("delete Object.getPrototypeOf(navigator).webdriver")
+
+                # 1. Navigate to login page and authenticate
+                login_url = f"{base_ui_url.rstrip('/')}/login/"
+                logger.info(f"[DEBUG] Navigating to login page: {login_url}")
+                
+                response = await page.goto(login_url, wait_until="networkidle", timeout=60000)
+                if response:
+                    logger.info(f"[DEBUG] Login page response status: {response.status}")
+                
+                # Wait for login form to be ready
+                await page.wait_for_load_state("domcontentloaded")
+                
+                # More exhaustive list of selectors for various Superset versions/themes
+                selectors = {
+                    "username": ['input[name="username"]', 'input#username', 'input[placeholder*="Username"]', 'input[type="text"]'],
+                    "password": ['input[name="password"]', 'input#password', 'input[placeholder*="Password"]', 'input[type="password"]'],
+                    "submit": ['button[type="submit"]', 'button#submit', '.btn-primary', 'input[type="submit"]']
+                }
+                logger.info(f"[DEBUG] Attempting to find login form elements...")
+                
+                try:
+                    # Find and fill username
+                    u_selector = None
+                    for s in selectors["username"]:
+                        count = await page.locator(s).count()
+                        logger.info(f"[DEBUG] Selector '{s}': {count} elements found")
+                        if count > 0:
+                            u_selector = s
+                            break
+                    
+                    if not u_selector:
+                        # Log all input fields on the page for debugging
+                        all_inputs = await page.locator('input').all()
+                        logger.info(f"[DEBUG] Found {len(all_inputs)} input fields on page")
+                        for i, inp in enumerate(all_inputs[:5]):  # Log first 5
+                            inp_type = await inp.get_attribute('type')
+                            inp_name = await inp.get_attribute('name')
+                            inp_id = await inp.get_attribute('id')
+                            logger.info(f"[DEBUG] Input {i}: type={inp_type}, name={inp_name}, id={inp_id}")
+                        raise RuntimeError("Could not find username input field on login page")
+                    
+                    logger.info(f"[DEBUG] Filling username field with selector: {u_selector}")
+                    await page.fill(u_selector, self.env.username)
+                    
+                    # Find and fill password
+                    p_selector = None
+                    for s in selectors["password"]:
+                        if await page.locator(s).count() > 0:
+                            p_selector = s
+                            break
+                    
+                    if not p_selector:
+                        raise RuntimeError("Could not find password input field on login page")
+                    
+                    logger.info(f"[DEBUG] Filling password field with selector: {p_selector}")
+                    await page.fill(p_selector, self.env.password)
+                    
+                    # Click submit
+                    s_selector = selectors["submit"][0]
+                    for s in selectors["submit"]:
+                        if await page.locator(s).count() > 0:
+                            s_selector = s
+                            break
+                    
+                    logger.info(f"[DEBUG] Clicking submit button with selector: {s_selector}")
+                    await page.click(s_selector)
+                    
+                    # Wait for navigation after login
+                    await page.wait_for_load_state("networkidle", timeout=30000)
+                    
+                    # Check if login was successful
+                    if "/login" in page.url:
+                        # Check for error messages on page
+                        error_msg = await page.locator(".alert-danger, .error-message").text_content() if await page.locator(".alert-danger, .error-message").count() > 0 else "Unknown error"
+                        logger.error(f"[DEBUG] Login failed. Still on login page. Error: {error_msg}")
+                        debug_path = output_path.replace(".png", "_debug_failed_login.png")
+                        await page.screenshot(path=debug_path)
+                        raise RuntimeError(f"Login failed: {error_msg}. Debug screenshot saved to {debug_path}")
+                    
+                    logger.info(f"[DEBUG] Login successful. Current URL: {page.url}")
+                    
+                    # Check cookies after successful login
+                    page_cookies = await context.cookies()
+                    logger.info(f"[DEBUG] Cookies after login: {len(page_cookies)}")
+                    for c in page_cookies:
+                        logger.info(f"[DEBUG] Cookie: name={c['name']}, domain={c['domain']}, value={c.get('value', '')[:20]}...")
+                    
+                except Exception as e:
+                    page_title = await page.title()
+                    logger.error(f"UI Login failed. Page title: {page_title}, URL: {page.url}, Error: {str(e)}")
+                    debug_path = output_path.replace(".png", "_debug_failed_login.png")
+                    await page.screenshot(path=debug_path)
+                    raise RuntimeError(f"Login failed: {str(e)}. Debug screenshot saved to {debug_path}")
+
+                # 2. Navigate to dashboard
+                # @UX_STATE: [Navigating] -> Loading dashboard UI
+                dashboard_url = f"{base_ui_url.rstrip('/')}/superset/dashboard/{dashboard_id}/?standalone=true"
+                
+                if base_ui_url.startswith("https://") and dashboard_url.startswith("http://"):
+                    dashboard_url = dashboard_url.replace("http://", "https://")
+
+                logger.info(f"[DEBUG] Navigating to dashboard: {dashboard_url}")
+                
+                # Use networkidle to ensure all initial assets are loaded
+                response = await page.goto(dashboard_url, wait_until="networkidle", timeout=60000)
+                
+                if response:
+                    logger.info(f"[DEBUG] Dashboard navigation response status: {response.status}, URL: {response.url}")
+                
+                try:
+                    # Wait for the dashboard grid to be present
+                    await page.wait_for_selector('.dashboard-component, .dashboard-header, [data-test="dashboard-grid"]', timeout=30000)
+                    logger.info(f"[DEBUG] Dashboard container loaded")
+                    
+                    # Wait for charts to finish loading (Superset uses loading spinners/skeletons)
+                    # We wait until loading indicators disappear or a timeout occurs
+                    try:
+                        # Wait for loading indicators to disappear
+                        await page.wait_for_selector('.loading, .ant-skeleton, .spinner', state="hidden", timeout=60000)
+                        logger.info(f"[DEBUG] Loading indicators hidden")
+                    except:
+                        logger.warning(f"[DEBUG] Timeout waiting for loading indicators to hide")
+
+                    # Wait for charts to actually render their content (e.g., ECharts, NVD3)
+                    # We look for common chart containers that should have content
+                    try:
+                        await page.wait_for_selector('.chart-container canvas, .slice_container svg, .superset-chart-canvas, .grid-content .chart-container', timeout=60000)
+                        logger.info(f"[DEBUG] Chart content detected")
+                    except:
+                        logger.warning(f"[DEBUG] Timeout waiting for chart content")
+
+                    # Additional check: wait for all chart containers to have non-empty content
+                    logger.info(f"[DEBUG] Waiting for all charts to have rendered content...")
+                    await page.wait_for_function("""() => {
+                        const charts = document.querySelectorAll('.chart-container, .slice_container');
+                        if (charts.length === 0) return true; // No charts to wait for
+                        
+                        // Check if all charts have rendered content (canvas, svg, or non-empty div)
+                        return Array.from(charts).every(chart => {
+                            const hasCanvas = chart.querySelector('canvas') !== null;
+                            const hasSvg = chart.querySelector('svg') !== null;
+                            const hasContent = chart.innerText.trim().length > 0 || chart.children.length > 0;
+                            return hasCanvas || hasSvg || hasContent;
+                        });
+                    }""", timeout=60000)
+                    logger.info(f"[DEBUG] All charts have rendered content")
+
+                    # Scroll to bottom and back to top to trigger lazy loading of all charts
+                    logger.info(f"[DEBUG] Scrolling to trigger lazy loading...")
+                    await page.evaluate("""async () => {
+                        const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
+                        for (let i = 0; i < document.body.scrollHeight; i += 500) {
+                            window.scrollTo(0, i);
+                            await delay(100);
+                        }
+                        window.scrollTo(0, 0);
+                        await delay(500);
+                    }""")
+
+                except Exception as e:
+                    logger.warning(f"[DEBUG] Dashboard content wait failed: {e}, proceeding anyway after delay")
+                
+                # Final stabilization delay - increased for complex dashboards
+                logger.info(f"[DEBUG] Final stabilization delay...")
+                await asyncio.sleep(15)
+
+                # Logic to handle tabs and full-page capture
+                try:
+                    # 1. Handle Tabs (Recursive switching)
+                    # @UX_STATE: [TabSwitching] -> Iterating through dashboard tabs to trigger lazy loading
+                    processed_tabs = set()
+
+                    async def switch_tabs(depth=0):
+                        if depth > 3: return # Limit recursion depth
+                        
+                        tab_selectors = [
+                            '.ant-tabs-nav-list .ant-tabs-tab',
+                            '.dashboard-component-tabs .ant-tabs-tab',
+                            '[data-test="dashboard-component-tabs"] .ant-tabs-tab'
+                        ]
+                        
+                        found_tabs = []
+                        for selector in tab_selectors:
+                            found_tabs = await page.locator(selector).all()
+                            if found_tabs: break
+                            
+                        if found_tabs:
+                            logger.info(f"[DEBUG][TabSwitching] Found {len(found_tabs)} tabs at depth {depth}")
+                            for i, tab in enumerate(found_tabs):
+                                try:
+                                    tab_text = (await tab.inner_text()).strip()
+                                    tab_id = f"{depth}_{i}_{tab_text}"
+                                    
+                                    if tab_id in processed_tabs:
+                                        continue
+                                    
+                                    if await tab.is_visible():
+                                        logger.info(f"[DEBUG][TabSwitching] Switching to tab: {tab_text}")
+                                        processed_tabs.add(tab_id)
+                                        
+                                        is_active = "ant-tabs-tab-active" in (await tab.get_attribute("class") or "")
+                                        if not is_active:
+                                            await tab.click()
+                                            await asyncio.sleep(2) # Wait for content to render
+                                        
+                                        await switch_tabs(depth + 1)
+                                except Exception as tab_e:
+                                    logger.warning(f"[DEBUG][TabSwitching] Failed to process tab {i}: {tab_e}")
+                            
+                            try:
+                                first_tab = found_tabs[0]
+                                if "ant-tabs-tab-active" not in (await first_tab.get_attribute("class") or ""):
+                                    await first_tab.click()
+                                    await asyncio.sleep(1)
+                            except: pass
+
+                    await switch_tabs()
+
+                    # 2. Calculate full height for screenshot
+                    # @UX_STATE: [CalculatingHeight] -> Determining dashboard dimensions
+                    full_height = await page.evaluate("""() => {
+                        const body = document.body;
+                        const html = document.documentElement;
+                        const dashboardContent = document.querySelector('.dashboard-content');
+                        
+                        return Math.max(
+                            body.scrollHeight, body.offsetHeight,
+                            html.clientHeight, html.scrollHeight, html.offsetHeight,
+                            dashboardContent ? dashboardContent.scrollHeight + 100 : 0
+                        );
+                    }""")
+                    logger.info(f"[DEBUG] Calculated full height: {full_height}")
+                    
+                    # DIAGNOSTIC: Count chart elements before resize
+                    chart_count_before = await page.evaluate("""() => {
+                        return {
+                            chartContainers: document.querySelectorAll('.chart-container, .slice_container').length,
+                            canvasElements: document.querySelectorAll('canvas').length,
+                            svgElements: document.querySelectorAll('.chart-container svg, .slice_container svg').length,
+                            visibleCharts: document.querySelectorAll('.chart-container:visible, .slice_container:visible').length
+                        };
+                    }""")
+                    logger.info(f"[DIAGNOSTIC] Chart elements BEFORE viewport resize: {chart_count_before}")
+                    
+                    # DIAGNOSTIC: Capture pre-resize screenshot for comparison
+                    pre_resize_path = output_path.replace(".png", "_preresize.png")
+                    try:
+                        await page.screenshot(path=pre_resize_path, full_page=False, timeout=10000)
+                        import os
+                        pre_resize_size = os.path.getsize(pre_resize_path) if os.path.exists(pre_resize_path) else 0
+                        logger.info(f"[DIAGNOSTIC] Pre-resize screenshot saved: {pre_resize_path} ({pre_resize_size} bytes)")
+                    except Exception as pre_e:
+                        logger.warning(f"[DIAGNOSTIC] Failed to capture pre-resize screenshot: {pre_e}")
+                    
+                    logger.info(f"[DIAGNOSTIC] Resizing viewport from current to 1920x{int(full_height)}")
+                    await page.set_viewport_size({"width": 1920, "height": int(full_height)})
+                    
+                    # DIAGNOSTIC: Increased wait time and log timing
+                    logger.info("[DIAGNOSTIC] Waiting 10 seconds after viewport resize for re-render...")
+                    await asyncio.sleep(10)
+                    logger.info("[DIAGNOSTIC] Wait completed")
+                    
+                    # DIAGNOSTIC: Count chart elements after resize and wait
+                    chart_count_after = await page.evaluate("""() => {
+                        return {
+                            chartContainers: document.querySelectorAll('.chart-container, .slice_container').length,
+                            canvasElements: document.querySelectorAll('canvas').length,
+                            svgElements: document.querySelectorAll('.chart-container svg, .slice_container svg').length,
+                            visibleCharts: document.querySelectorAll('.chart-container:visible, .slice_container:visible').length
+                        };
+                    }""")
+                    logger.info(f"[DIAGNOSTIC] Chart elements AFTER viewport resize + wait: {chart_count_after}")
+                    
+                    # DIAGNOSTIC: Check if any charts have error states
+                    chart_errors = await page.evaluate("""() => {
+                        const errors = [];
+                        document.querySelectorAll('.chart-container, .slice_container').forEach((chart, i) => {
+                            const errorEl = chart.querySelector('.error, .alert-danger, .ant-alert-error');
+                            if (errorEl) {
+                                errors.push({index: i, text: errorEl.innerText.substring(0, 100)});
+                            }
+                        });
+                        return errors;
+                    }""")
+                    if chart_errors:
+                        logger.warning(f"[DIAGNOSTIC] Charts with error states detected: {chart_errors}")
+                    else:
+                        logger.info("[DIAGNOSTIC] No chart error states detected")
+
+                    # 3. Take screenshot using CDP to bypass Playwright's font loading wait
+                    # @UX_STATE: [Capturing] -> Executing CDP screenshot
+                    logger.info("[DEBUG] Attempting full-page screenshot via CDP...")
+                    cdp = await page.context.new_cdp_session(page)
+                    
+                    screenshot_data = await cdp.send("Page.captureScreenshot", {
+                        "format": "png",
+                        "fromSurface": True,
+                        "captureBeyondViewport": True
+                    })
+                    
+                    image_data = base64.b64decode(screenshot_data["data"])
+                    
+                    with open(output_path, 'wb') as f:
+                        f.write(image_data)
+                    
+                    # DIAGNOSTIC: Verify screenshot file
+                    import os
+                    final_size = os.path.getsize(output_path) if os.path.exists(output_path) else 0
+                    logger.info(f"[DIAGNOSTIC] Final screenshot saved: {output_path}")
+                    logger.info(f"[DIAGNOSTIC] Final screenshot size: {final_size} bytes ({final_size / 1024:.2f} KB)")
+                    
+                    # DIAGNOSTIC: Get image dimensions
+                    try:
+                        with Image.open(output_path) as final_img:
+                            logger.info(f"[DIAGNOSTIC] Final screenshot dimensions: {final_img.width}x{final_img.height}")
+                    except Exception as img_err:
+                        logger.warning(f"[DIAGNOSTIC] Could not read final image dimensions: {img_err}")
+                        
+                    logger.info(f"Full-page screenshot saved to {output_path} (via CDP)")
+                except Exception as e:
+                    logger.error(f"[DEBUG] Full-page/Tab capture failed: {e}")
+                    try:
+                        await page.screenshot(path=output_path, full_page=True, timeout=10000)
+                    except Exception as e2:
+                        logger.error(f"[DEBUG] Fallback screenshot also failed: {e2}")
+                        await page.screenshot(path=output_path, timeout=5000)
+                
+                await browser.close()
+                return True
+    # [/DEF:ScreenshotService.capture_dashboard:Function]
+# [/DEF:ScreenshotService:Class]
+
+# [DEF:LLMClient:Class]
+# @PURPOSE: Wrapper for LLM provider APIs.
+class LLMClient:
+    # [DEF:LLMClient.__init__:Function]
+    # @PURPOSE: Initializes the LLMClient with provider settings.
+    # @PRE: api_key, base_url, and default_model are non-empty strings.
+    def __init__(self, provider_type: LLMProviderType, api_key: str, base_url: str, default_model: str):
+        self.provider_type = provider_type
+        self.api_key = api_key
+        self.base_url = base_url
+        self.default_model = default_model
+        
+        # DEBUG: Log initialization parameters (without exposing full API key)
+        logger.info(f"[LLMClient.__init__] Initializing LLM client:")
+        logger.info(f"[LLMClient.__init__]   Provider Type: {provider_type}")
+        logger.info(f"[LLMClient.__init__]   Base URL: {base_url}")
+        logger.info(f"[LLMClient.__init__]   Default Model: {default_model}")
+        logger.info(f"[LLMClient.__init__]   API Key (first 8 chars): {api_key[:8] if api_key and len(api_key) > 8 else 'EMPTY_OR_NONE'}...")
+        logger.info(f"[LLMClient.__init__]   API Key Length: {len(api_key) if api_key else 0}")
+        
+        self.client = AsyncOpenAI(api_key=api_key, base_url=base_url)
+    # [/DEF:LLMClient.__init__:Function]
+
+    # [DEF:LLMClient.get_json_completion:Function]
+    # @PURPOSE: Helper to handle LLM calls with JSON mode and fallback parsing.
+    # @PRE: messages is a list of valid message dictionaries.
+    # @POST: Returns a parsed JSON dictionary.
+    # @SIDE_EFFECT: Calls external LLM API.
+    def _should_retry(exception: Exception) -> bool:
+        """Custom retry predicate that excludes authentication errors."""
+        # Don't retry on authentication errors
+        if isinstance(exception, OpenAIAuthenticationError):
+            return False
+        # Retry on rate limit errors and other exceptions
+        return isinstance(exception, (RateLimitError, Exception))
+    
+    @retry(
+        stop=stop_after_attempt(5),
+        wait=wait_exponential(multiplier=2, min=5, max=60),
+        retry=retry_if_exception(_should_retry),
+        reraise=True
+    )
+    async def get_json_completion(self, messages: List[Dict[str, Any]]) -> Dict[str, Any]:
+        with belief_scope("get_json_completion"):
+            response = None
+            try:
+                try:
+                    logger.info(f"[get_json_completion] Attempting LLM call with JSON mode for model: {self.default_model}")
+                    logger.info(f"[get_json_completion] Base URL being used: {self.base_url}")
+                    logger.info(f"[get_json_completion] Number of messages: {len(messages)}")
+                    logger.info(f"[get_json_completion] API Key present: {bool(self.api_key and len(self.api_key) > 0)}")
+                    
+                    response = await self.client.chat.completions.create(
+                        model=self.default_model,
+                        messages=messages,
+                        response_format={"type": "json_object"}
+                    )
+                except Exception as e:
+                    if "JSON mode is not enabled" in str(e) or "400" in str(e):
+                        logger.warning(f"[get_json_completion] JSON mode failed or not supported: {str(e)}. Falling back to plain text response.")
+                        response = await self.client.chat.completions.create(
+                            model=self.default_model,
+                            messages=messages
+                        )
+                    else:
+                        raise e
+                
+                logger.debug(f"[get_json_completion] LLM Response: {response}")
+            except OpenAIAuthenticationError as e:
+                logger.error(f"[get_json_completion] Authentication error: {str(e)}")
+                # Do not retry on auth errors - re-raise to stop retry
+                raise
+            except RateLimitError as e:
+                logger.warning(f"[get_json_completion] Rate limit hit: {str(e)}")
+                
+                # Extract retry_delay from error metadata if available
+                retry_delay = 5.0 # Default fallback
+                try:
+                    # Based on logs, the raw response is in e.body or e.response.json()
+                    # The logs show 'metadata': {'raw': '...'} which suggests a proxy or specific client wrapper
+                    # Let's try to find the 'retryDelay' in the error message or response
+                    import re
+                    
+                    # Try to find "retryDelay": "XXs" in the string representation of the error
+                    error_str = str(e)
+                    match = re.search(r'"retryDelay":\s*"(\d+)s"', error_str)
+                    if match:
+                        retry_delay = float(match.group(1))
+                    else:
+                        # Try to parse from response if it's a standard OpenAI-like error with body
+                        if hasattr(e, 'body') and isinstance(e.body, dict):
+                            # Some providers put it in details
+                            details = e.body.get('error', {}).get('details', [])
+                            for detail in details:
+                                if detail.get('@type') == 'type.googleapis.com/google.rpc.RetryInfo':
+                                    delay_str = detail.get('retryDelay', '5s')
+                                    retry_delay = float(delay_str.rstrip('s'))
+                                    break
+                except Exception as parse_e:
+                    logger.debug(f"[get_json_completion] Failed to parse retry delay: {parse_e}")
+                
+                # Add a small safety margin (0.5s) as requested
+                wait_time = retry_delay + 0.5
+                logger.info(f"[get_json_completion] Waiting for {wait_time}s before retry...")
+                await asyncio.sleep(wait_time)
+                raise
+            except Exception as e:
+                logger.error(f"[get_json_completion] LLM call failed: {str(e)}")
+                raise
+
+            if not response or not hasattr(response, 'choices') or not response.choices:
+                raise RuntimeError(f"Invalid LLM response: {response}")
+
+            content = response.choices[0].message.content
+            logger.debug(f"[get_json_completion] Raw content to parse: {content}")
+            
+            try:
+                return json.loads(content)
+            except json.JSONDecodeError:
+                logger.warning("[get_json_completion] Failed to parse JSON directly, attempting to extract from code blocks")
+                if "```json" in content:
+                    json_str = content.split("```json")[1].split("```")[0].strip()
+                    return json.loads(json_str)
+                elif "```" in content:
+                    json_str = content.split("```")[1].split("```")[0].strip()
+                    return json.loads(json_str)
+                else:
+                    raise
+    # [/DEF:LLMClient.get_json_completion:Function]
+
+    # [DEF:LLMClient.analyze_dashboard:Function]
+    # @PURPOSE: Sends dashboard data (screenshot + logs) to LLM for health analysis.
+    # @PRE: screenshot_path exists, logs is a list of strings.
+    # @POST: Returns a structured analysis dictionary (status, summary, issues).
+    # @SIDE_EFFECT: Reads screenshot file and calls external LLM API.
+    async def analyze_dashboard(self, screenshot_path: str, logs: List[str]) -> Dict[str, Any]:
+        with belief_scope("analyze_dashboard"):
+            # Optimize image to reduce token count (US1 / T023)
+            # Gemini/Gemma models have limits on input tokens, and large images contribute significantly.
+            try:
+                with Image.open(screenshot_path) as img:
+                    # Convert to RGB if necessary
+                    if img.mode in ("RGBA", "P"):
+                        img = img.convert("RGB")
+                    
+                    # Resize if too large (max 1024px width while maintaining aspect ratio)
+                    # We reduce width further to 1024px to stay within token limits for long dashboards
+                    max_width = 1024
+                    if img.width > max_width or img.height > 2048:
+                        # Calculate scaling factor to fit within 1024x2048
+                        scale = min(max_width / img.width, 2048 / img.height)
+                        if scale < 1.0:
+                            new_width = int(img.width * scale)
+                            new_height = int(img.height * scale)
+                            img = img.resize((new_width, new_height), Image.Resampling.LANCZOS)
+                            logger.info(f"[analyze_dashboard] Resized image from {img.width}x{img.height} to {new_width}x{new_height}")
+                    
+                    # Compress and convert to base64
+                    buffer = io.BytesIO()
+                    # Lower quality to 60% to further reduce payload size
+                    img.save(buffer, format="JPEG", quality=60, optimize=True)
+                    base_64_image = base64.b64encode(buffer.getvalue()).decode('utf-8')
+                    logger.info(f"[analyze_dashboard] Optimized image size: {len(buffer.getvalue()) / 1024:.2f} KB")
+            except Exception as img_e:
+                logger.warning(f"[analyze_dashboard] Image optimization failed: {img_e}. Using raw image.")
+                with open(screenshot_path, "rb") as image_file:
+                    base_64_image = base64.b64encode(image_file.read()).decode('utf-8')
+
+            log_text = "\n".join(logs)
+            prompt = f"""
+            Analyze the attached dashboard screenshot and the following execution logs for health and visual issues.
+            
+            Logs:
+            {log_text}
+            
+            Provide the analysis in JSON format with the following structure:
+            {{
+                "status": "PASS" | "WARN" | "FAIL",
+                "summary": "Short summary of findings",
+                "issues": [
+                    {{
+                        "severity": "WARN" | "FAIL",
+                        "message": "Description of the issue",
+                        "location": "Optional location info (e.g. chart name)"
+                    }}
+                ]
+            }}
+            """
+            
+            messages = [
+                {
+                    "role": "user",
+                    "content": [
+                        {"type": "text", "text": prompt},
+                        {
+                            "type": "image_url",
+                            "image_url": {
+                                "url": f"data:image/jpeg;base64,{base_64_image}"
+                            }
+                        }
+                    ]
+                }
+            ]
+            
+            try:
+                return await self.get_json_completion(messages)
+            except Exception as e:
+                logger.error(f"[analyze_dashboard] Failed to get analysis: {str(e)}")
+                return {
+                    "status": "FAIL",
+                    "summary": f"Failed to get response from LLM: {str(e)}",
+                    "issues": [{"severity": "FAIL", "message": "LLM provider returned empty or invalid response"}]
+                }
+    # [/DEF:LLMClient.analyze_dashboard:Function]
+# [/DEF:LLMClient:Class]
+
+# [/DEF:backend/src/plugins/llm_analysis/service.py:Module]

backend/src/plugins/mapper.py

@@ -66,6 +66,15 @@ class MapperPlugin(PluginBase):
             return "1.0.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the mapper plugin.
+    # @RETURN: str - "/tools/mapper"
+    def ui_route(self) -> str:
+        with belief_scope("ui_route"):
+            return "/tools/mapper"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for the mapper plugin parameters.
     # @PRE: Plugin instance exists.

backend/src/plugins/migration.py

@@ -71,6 +71,15 @@ class MigrationPlugin(PluginBase):
             return "1.0.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the migration plugin.
+    # @RETURN: str - "/migration"
+    def ui_route(self) -> str:
+        with belief_scope("ui_route"):
+            return "/migration"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for migration plugin parameters.
     # @PRE:     Config manager is available.
@@ -294,9 +303,9 @@ class MigrationPlugin(PluginBase):
                 
                 try:
                     exported_content, _ = from_c.export_dashboard(dash_id)
-                    with create_temp_file(content=exported_content, dry_run=True, suffix=".zip", logger=logger) as tmp_zip_path:
+                    with create_temp_file(content=exported_content, dry_run=True, suffix=".zip") as tmp_zip_path:
                         # Always transform to strip databases to avoid password errors
-                        with create_temp_file(suffix=".zip", dry_run=True, logger=logger) as tmp_new_zip:
+                        with create_temp_file(suffix=".zip", dry_run=True) as tmp_new_zip:
                             success = engine.transform_zip(str(tmp_zip_path), str(tmp_new_zip), db_mapping, strip_databases=False)
                             
                             if not success and replace_db_config:

backend/src/plugins/search.py

@@ -64,6 +64,15 @@ class SearchPlugin(PluginBase):
             return "1.0.0"
     # [/DEF:version:Function]
 
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the search plugin.
+    # @RETURN: str - "/tools/search"
+    def ui_route(self) -> str:
+        with belief_scope("ui_route"):
+            return "/tools/search"
+    # [/DEF:ui_route:Function]
+
     # [DEF:get_schema:Function]
     # @PURPOSE: Returns the JSON schema for the search plugin parameters.
     # @PRE: Plugin instance exists.

backend/src/plugins/storage/__init__.py

@@ -0,0 +1,3 @@
+from .plugin import StoragePlugin
+
+__all__ = ["StoragePlugin"]

backend/src/plugins/storage/plugin.py

@@ -0,0 +1,333 @@
+# [DEF:StoragePlugin:Module]
+#
+# @SEMANTICS: storage, files, filesystem, plugin
+# @PURPOSE:   Provides core filesystem operations for managing backups and repositories.
+# @LAYER:     App
+# @RELATION:  IMPLEMENTS -> PluginBase
+# @RELATION:  DEPENDS_ON -> backend.src.models.storage
+#
+# @INVARIANT: All file operations must be restricted to the configured storage root.
+
+# [SECTION: IMPORTS]
+import os
+import shutil
+from pathlib import Path
+from datetime import datetime
+from typing import Dict, Any, List, Optional
+from fastapi import UploadFile
+
+from ...core.plugin_base import PluginBase
+from ...core.logger import belief_scope, logger
+from ...models.storage import StoredFile, FileCategory, StorageConfig
+from ...dependencies import get_config_manager
+# [/SECTION]
+
+# [DEF:StoragePlugin:Class]
+# @PURPOSE: Implementation of the storage management plugin.
+class StoragePlugin(PluginBase):
+    """
+    Plugin for managing local file storage for backups and repositories.
+    """
+
+    # [DEF:__init__:Function]
+    # @PURPOSE: Initializes the StoragePlugin and ensures required directories exist.
+    # @PRE: Configuration manager must be accessible.
+    # @POST: Storage root and category directories are created on disk.
+    def __init__(self):
+        with belief_scope("StoragePlugin:init"):
+            self.ensure_directories()
+    # [/DEF:__init__:Function]
+
+    @property
+    # [DEF:id:Function]
+    # @PURPOSE: Returns the unique identifier for the storage plugin.
+    # @PRE: None.
+    # @POST: Returns the plugin ID string.
+    # @RETURN: str - "storage-manager"
+    def id(self) -> str:
+        with belief_scope("StoragePlugin:id"):
+            return "storage-manager"
+    # [/DEF:id:Function]
+
+    @property
+    # [DEF:name:Function]
+    # @PURPOSE: Returns the human-readable name of the storage plugin.
+    # @PRE: None.
+    # @POST: Returns the plugin name string.
+    # @RETURN: str - "Storage Manager"
+    def name(self) -> str:
+        with belief_scope("StoragePlugin:name"):
+            return "Storage Manager"
+    # [/DEF:name:Function]
+
+    @property
+    # [DEF:description:Function]
+    # @PURPOSE: Returns a description of the storage plugin.
+    # @PRE: None.
+    # @POST: Returns the plugin description string.
+    # @RETURN: str - Plugin description.
+    def description(self) -> str:
+        with belief_scope("StoragePlugin:description"):
+            return "Manages local file storage for backups and repositories."
+    # [/DEF:description:Function]
+
+    @property
+    # [DEF:version:Function]
+    # @PURPOSE: Returns the version of the storage plugin.
+    # @PRE: None.
+    # @POST: Returns the version string.
+    # @RETURN: str - "1.0.0"
+    def version(self) -> str:
+        with belief_scope("StoragePlugin:version"):
+            return "1.0.0"
+    # [/DEF:version:Function]
+
+    @property
+    # [DEF:ui_route:Function]
+    # @PURPOSE: Returns the frontend route for the storage plugin.
+    # @RETURN: str - "/tools/storage"
+    def ui_route(self) -> str:
+        with belief_scope("StoragePlugin:ui_route"):
+            return "/tools/storage"
+    # [/DEF:ui_route:Function]
+
+    # [DEF:get_schema:Function]
+    # @PURPOSE: Returns the JSON schema for storage plugin parameters.
+    # @PRE: None.
+    # @POST: Returns a dictionary representing the JSON schema.
+    # @RETURN: Dict[str, Any] - JSON schema.
+    def get_schema(self) -> Dict[str, Any]:
+        with belief_scope("StoragePlugin:get_schema"):
+            return {
+            "type": "object",
+            "properties": {
+                "category": {
+                    "type": "string",
+                    "enum": [c.value for c in FileCategory],
+                    "title": "Category"
+                }
+            },
+            "required": ["category"]
+        }
+    # [/DEF:get_schema:Function]
+
+    # [DEF:execute:Function]
+    # @PURPOSE: Executes storage-related tasks (placeholder for PluginBase compliance).
+    # @PRE: params must match the plugin schema.
+    # @POST: Task is executed and logged.
+    async def execute(self, params: Dict[str, Any]):
+        with belief_scope("StoragePlugin:execute"):
+            logger.info(f"[StoragePlugin][Action] Executing with params: {params}")
+    # [/DEF:execute:Function]
+
+    # [DEF:get_storage_root:Function]
+    # @PURPOSE: Resolves the absolute path to the storage root.
+    # @PRE: Settings must define a storage root path.
+    # @POST: Returns a Path object representing the storage root.
+    def get_storage_root(self) -> Path:
+        with belief_scope("StoragePlugin:get_storage_root"):
+            config_manager = get_config_manager()
+            global_settings = config_manager.get_config().settings
+            
+            # Use storage.root_path as the source of truth for storage UI
+            root = Path(global_settings.storage.root_path)
+                
+            if not root.is_absolute():
+                # Resolve relative to the backend directory
+                # Path(__file__) is backend/src/plugins/storage/plugin.py
+                # parents[3] is the project root (ss-tools)
+                # We need to ensure it's relative to where backend/ is
+                project_root = Path(__file__).parents[3]
+                root = (project_root / root).resolve()
+            return root
+    # [/DEF:get_storage_root:Function]
+
+    # [DEF:resolve_path:Function]
+    # @PURPOSE: Resolves a dynamic path pattern using provided variables.
+    # @PARAM: pattern (str) - The path pattern to resolve.
+    # @PARAM: variables (Dict[str, str]) - Variables to substitute in the pattern.
+    # @PRE: pattern must be a valid format string.
+    # @POST: Returns the resolved path string.
+    # @RETURN: str - The resolved path.
+    def resolve_path(self, pattern: str, variables: Dict[str, str]) -> str:
+        with belief_scope("StoragePlugin:resolve_path"):
+            # Add common variables
+            vars_with_defaults = {
+                "timestamp": datetime.now().strftime("%Y%m%dT%H%M%S"),
+                **variables
+            }
+            try:
+                resolved = pattern.format(**vars_with_defaults)
+                # Clean up any double slashes or leading/trailing slashes for relative path
+                return os.path.normpath(resolved).strip("/")
+            except KeyError as e:
+                logger.warning(f"[StoragePlugin][Coherence:Failed] Missing variable for path resolution: {e}")
+                # Fallback to literal pattern if formatting fails partially (or handle as needed)
+                return pattern.replace("{", "").replace("}", "")
+    # [/DEF:resolve_path:Function]
+
+    # [DEF:ensure_directories:Function]
+    # @PURPOSE: Creates the storage root and category subdirectories if they don't exist.
+    # @PRE: Storage root must be resolvable.
+    # @POST: Directories are created on the filesystem.
+    # @SIDE_EFFECT: Creates directories on the filesystem.
+    def ensure_directories(self):
+        with belief_scope("StoragePlugin:ensure_directories"):
+            root = self.get_storage_root()
+            for category in FileCategory:
+                # Use singular name for consistency with BackupPlugin and GitService
+                path = root / category.value
+                path.mkdir(parents=True, exist_ok=True)
+                logger.debug(f"[StoragePlugin][Action] Ensured directory: {path}")
+    # [/DEF:ensure_directories:Function]
+
+    # [DEF:validate_path:Function]
+    # @PURPOSE: Prevents path traversal attacks by ensuring the path is within the storage root.
+    # @PRE: path must be a Path object.
+    # @POST: Returns the resolved absolute path if valid, otherwise raises ValueError.
+    def validate_path(self, path: Path) -> Path:
+        with belief_scope("StoragePlugin:validate_path"):
+            root = self.get_storage_root().resolve()
+            resolved = path.resolve()
+            try:
+                resolved.relative_to(root)
+            except ValueError:
+                logger.error(f"[StoragePlugin][Coherence:Failed] Path traversal detected: {resolved} is not under {root}")
+                raise ValueError("Access denied: Path is outside of storage root.")
+            return resolved
+    # [/DEF:validate_path:Function]
+
+    # [DEF:list_files:Function]
+    # @PURPOSE: Lists all files and directories in a specific category and subpath.
+    # @PARAM: category (Optional[FileCategory]) - The category to list.
+    # @PARAM: subpath (Optional[str]) - Nested path within the category.
+    # @PRE: Storage root must exist.
+    # @POST: Returns a list of StoredFile objects.
+    # @RETURN: List[StoredFile] - List of file and directory metadata objects.
+    def list_files(self, category: Optional[FileCategory] = None, subpath: Optional[str] = None) -> List[StoredFile]:
+        with belief_scope("StoragePlugin:list_files"):
+            root = self.get_storage_root()
+            logger.info(f"[StoragePlugin][Action] Listing files in root: {root}, category: {category}, subpath: {subpath}")
+            files = []
+            
+            categories = [category] if category else list(FileCategory)
+            
+            for cat in categories:
+                # Scan the category subfolder + optional subpath
+                base_dir = root / cat.value
+                if subpath:
+                    target_dir = self.validate_path(base_dir / subpath)
+                else:
+                    target_dir = base_dir
+
+                if not target_dir.exists():
+                    continue
+                
+                logger.debug(f"[StoragePlugin][Action] Scanning directory: {target_dir}")
+                
+                # Use os.scandir for better performance and to distinguish files vs dirs
+                with os.scandir(target_dir) as it:
+                    for entry in it:
+                        # Skip logs
+                        if "Logs" in entry.path:
+                            continue
+                        
+                        stat = entry.stat()
+                        is_dir = entry.is_dir()
+                        
+                        files.append(StoredFile(
+                            name=entry.name,
+                            path=str(Path(entry.path).relative_to(root)),
+                            size=stat.st_size if not is_dir else 0,
+                            created_at=datetime.fromtimestamp(stat.st_ctime),
+                            category=cat,
+                            mime_type="directory" if is_dir else None
+                        ))
+            
+            # Sort: directories first, then by name
+            return sorted(files, key=lambda x: (x.mime_type != "directory", x.name))
+    # [/DEF:list_files:Function]
+
+    # [DEF:save_file:Function]
+    # @PURPOSE: Saves an uploaded file to the specified category and optional subpath.
+    # @PARAM: file (UploadFile) - The uploaded file.
+    # @PARAM: category (FileCategory) - The target category.
+    # @PARAM: subpath (Optional[str]) - The target subpath.
+    # @PRE: file must be a valid UploadFile; category must be valid.
+    # @POST: File is written to disk and metadata is returned.
+    # @RETURN: StoredFile - Metadata of the saved file.
+    # @SIDE_EFFECT: Writes file to disk.
+    async def save_file(self, file: UploadFile, category: FileCategory, subpath: Optional[str] = None) -> StoredFile:
+        with belief_scope("StoragePlugin:save_file"):
+            root = self.get_storage_root()
+            dest_dir = root / category.value
+            if subpath:
+                dest_dir = dest_dir / subpath
+            
+            dest_dir.mkdir(parents=True, exist_ok=True)
+            
+            dest_path = self.validate_path(dest_dir / file.filename)
+            
+            with dest_path.open("wb") as buffer:
+                shutil.copyfileobj(file.file, buffer)
+            
+            stat = dest_path.stat()
+            return StoredFile(
+                name=dest_path.name,
+                path=str(dest_path.relative_to(root)),
+                size=stat.st_size,
+                created_at=datetime.fromtimestamp(stat.st_ctime),
+                category=category,
+                mime_type=file.content_type
+            )
+    # [/DEF:save_file:Function]
+
+    # [DEF:delete_file:Function]
+    # @PURPOSE: Deletes a file or directory from the specified category and path.
+    # @PARAM: category (FileCategory) - The category.
+    # @PARAM: path (str) - The relative path of the file or directory.
+    # @PRE: path must belong to the specified category and exist on disk.
+    # @POST: The file or directory is removed from disk.
+    # @SIDE_EFFECT: Removes item from disk.
+    def delete_file(self, category: FileCategory, path: str):
+        with belief_scope("StoragePlugin:delete_file"):
+            root = self.get_storage_root()
+            # path is relative to root, but we ensure it starts with category
+            full_path = self.validate_path(root / path)
+            
+            if not str(Path(path)).startswith(category.value):
+                raise ValueError(f"Path {path} does not belong to category {category}")
+
+            if full_path.exists():
+                if full_path.is_dir():
+                    shutil.rmtree(full_path)
+                else:
+                    full_path.unlink()
+                logger.info(f"[StoragePlugin][Action] Deleted: {full_path}")
+            else:
+                raise FileNotFoundError(f"Item {path} not found")
+    # [/DEF:delete_file:Function]
+
+    # [DEF:get_file_path:Function]
+    # @PURPOSE: Returns the absolute path of a file for download.
+    # @PARAM: category (FileCategory) - The category.
+    # @PARAM: path (str) - The relative path of the file.
+    # @PRE: path must belong to the specified category and be a file.
+    # @POST: Returns the absolute Path to the file.
+    # @RETURN: Path - Absolute path to the file.
+    def get_file_path(self, category: FileCategory, path: str) -> Path:
+        with belief_scope("StoragePlugin:get_file_path"):
+            root = self.get_storage_root()
+            file_path = self.validate_path(root / path)
+            
+            if not str(Path(path)).startswith(category.value):
+                raise ValueError(f"Path {path} does not belong to category {category}")
+
+            if not file_path.exists() or file_path.is_dir():
+                raise FileNotFoundError(f"File {path} not found")
+            
+            return file_path
+    # [/DEF:get_file_path:Function]
+
+# [/DEF:StoragePlugin:Class]
+# [/DEF:StoragePlugin:Module]

backend/src/schemas/auth.py

@@ -0,0 +1,128 @@
+# [DEF:backend.src.schemas.auth:Module]
+#
+# @TIER:      STANDARD
+# @SEMANTICS: auth, schemas, pydantic, user, token
+# @PURPOSE:   Pydantic schemas for authentication requests and responses.
+# @LAYER:     API
+# @RELATION:  DEPENDS_ON -> pydantic
+#
+# @INVARIANT: Sensitive fields like password must not be included in response schemas.
+
+# [SECTION: IMPORTS]
+from typing import List, Optional
+from pydantic import BaseModel, EmailStr, Field
+from datetime import datetime
+# [/SECTION]
+
+# [DEF:Token:Class]
+# @TIER: TRIVIAL
+# @PURPOSE: Represents a JWT access token response.
+class Token(BaseModel):
+    access_token: str
+    token_type: str
+# [/DEF:Token:Class]
+
+# [DEF:TokenData:Class]
+# @TIER: TRIVIAL
+# @PURPOSE: Represents the data encoded in a JWT token.
+class TokenData(BaseModel):
+    username: Optional[str] = None
+    scopes: List[str] = []
+# [/DEF:TokenData:Class]
+
+# [DEF:PermissionSchema:Class]
+# @TIER: TRIVIAL
+# @PURPOSE: Represents a permission in API responses.
+class PermissionSchema(BaseModel):
+    id: Optional[str] = None
+    resource: str
+    action: str
+
+    class Config:
+        from_attributes = True
+# [/DEF:PermissionSchema:Class]
+
+# [DEF:RoleSchema:Class]
+# @PURPOSE: Represents a role in API responses.
+class RoleSchema(BaseModel):
+    id: str
+    name: str
+    description: Optional[str] = None
+    permissions: List[PermissionSchema] = []
+
+    class Config:
+        from_attributes = True
+# [/DEF:RoleSchema:Class]
+
+# [DEF:RoleCreate:Class]
+# @PURPOSE: Schema for creating a new role.
+class RoleCreate(BaseModel):
+    name: str
+    description: Optional[str] = None
+    permissions: List[str] = [] # List of permission IDs or "resource:action" strings
+# [/DEF:RoleCreate:Class]
+
+# [DEF:RoleUpdate:Class]
+# @PURPOSE: Schema for updating an existing role.
+class RoleUpdate(BaseModel):
+    name: Optional[str] = None
+    description: Optional[str] = None
+    permissions: Optional[List[str]] = None
+# [/DEF:RoleUpdate:Class]
+
+# [DEF:ADGroupMappingSchema:Class]
+# @PURPOSE: Represents an AD Group to Role mapping in API responses.
+class ADGroupMappingSchema(BaseModel):
+    id: str
+    ad_group: str
+    role_id: str
+
+    class Config:
+        from_attributes = True
+# [/DEF:ADGroupMappingSchema:Class]
+
+# [DEF:ADGroupMappingCreate:Class]
+# @PURPOSE: Schema for creating an AD Group mapping.
+class ADGroupMappingCreate(BaseModel):
+    ad_group: str
+    role_id: str
+# [/DEF:ADGroupMappingCreate:Class]
+
+# [DEF:UserBase:Class]
+# @PURPOSE: Base schema for user data.
+class UserBase(BaseModel):
+    username: str
+    email: Optional[EmailStr] = None
+    is_active: bool = True
+# [/DEF:UserBase:Class]
+
+# [DEF:UserCreate:Class]
+# @PURPOSE: Schema for creating a new user.
+class UserCreate(UserBase):
+    password: str
+    roles: List[str] = []
+# [/DEF:UserCreate:Class]
+
+# [DEF:UserUpdate:Class]
+# @PURPOSE: Schema for updating an existing user.
+class UserUpdate(BaseModel):
+    email: Optional[EmailStr] = None
+    password: Optional[str] = None
+    is_active: Optional[bool] = None
+    roles: Optional[List[str]] = None
+# [/DEF:UserUpdate:Class]
+
+# [DEF:User:Class]
+# @PURPOSE: Schema for user data in API responses.
+class User(UserBase):
+    id: str
+    auth_source: str
+    created_at: datetime
+    last_login: Optional[datetime] = None
+    roles: List[RoleSchema] = []
+
+    class Config:
+        from_attributes = True
+# [/DEF:User:Class]
+
+# [/DEF:backend.src.schemas.auth:Module]

backend/src/scripts/create_admin.py

@@ -0,0 +1,82 @@
+# [DEF:backend.src.scripts.create_admin:Module]
+#
+# @SEMANTICS: admin, setup, user, auth, cli
+# @PURPOSE:   CLI tool for creating the initial admin user.
+# @LAYER:     Scripts
+# @RELATION:  USES -> backend.src.core.auth.security
+# @RELATION:  USES -> backend.src.core.database
+# @RELATION:  USES -> backend.src.models.auth
+#
+# @INVARIANT: Admin user must have the "Admin" role.
+
+# [SECTION: IMPORTS]
+import sys
+import argparse
+from pathlib import Path
+
+# Add src to path
+sys.path.append(str(Path(__file__).parent.parent.parent))
+
+from src.core.database import AuthSessionLocal, init_db
+from src.core.auth.security import get_password_hash
+from src.models.auth import User, Role, Permission
+from src.core.logger import logger, belief_scope
+# [/SECTION]
+
+# [DEF:create_admin:Function]
+# @PURPOSE: Creates an admin user and necessary roles/permissions.
+# @PRE:     username and password provided via CLI.
+# @POST:    Admin user exists in auth.db.
+#
+# @PARAM:   username (str) - Admin username.
+# @PARAM:   password (str) - Admin password.
+def create_admin(username, password):
+    with belief_scope("create_admin"):
+        db = AuthSessionLocal()
+        try:
+            # 1. Ensure Admin role exists
+            admin_role = db.query(Role).filter(Role.name == "Admin").first()
+            if not admin_role:
+                logger.info("Creating Admin role...")
+                admin_role = Role(name="Admin", description="System Administrator")
+                db.add(admin_role)
+                db.commit()
+                db.refresh(admin_role)
+
+            # 2. Check if user already exists
+            existing_user = db.query(User).filter(User.username == username).first()
+            if existing_user:
+                logger.warning(f"User {username} already exists.")
+                return
+
+            # 3. Create Admin user
+            logger.info(f"Creating admin user: {username}")
+            new_user = User(
+                username=username,
+                password_hash=get_password_hash(password),
+                auth_source="LOCAL",
+                is_active=True
+            )
+            new_user.roles.append(admin_role)
+            db.add(new_user)
+            db.commit()
+            logger.info(f"Admin user {username} created successfully.")
+            
+        except Exception as e:
+            logger.error(f"Failed to create admin user: {e}")
+            db.rollback()
+        finally:
+            db.close()
+# [/DEF:create_admin:Function]
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Create initial admin user")
+    parser.add_argument("--username", required=True, help="Admin username")
+    parser.add_argument("--password", required=True, help="Admin password")
+    args = parser.parse_args()
+    
+    # Ensure DB is initialized before creating admin
+    init_db()
+    create_admin(args.username, args.password)
+
+# [/DEF:backend.src.scripts.create_admin:Module]

backend/src/scripts/init_auth_db.py

@@ -0,0 +1,44 @@
+# [DEF:backend.src.scripts.init_auth_db:Module]
+#
+# @SEMANTICS: setup, database, auth, migration
+# @PURPOSE:   Initializes the auth database and creates the necessary tables.
+# @LAYER:     Scripts
+# @RELATION:  CALLS -> backend.src.core.database.init_db
+#
+# @INVARIANT: Safe to run multiple times (idempotent).
+
+# [SECTION: IMPORTS]
+import sys
+import os
+from pathlib import Path
+
+# Add src to path
+sys.path.append(str(Path(__file__).parent.parent.parent))
+
+from src.core.database import init_db, auth_engine
+from src.core.logger import logger, belief_scope
+from src.scripts.seed_permissions import seed_permissions
+# [/SECTION]
+
+# [DEF:run_init:Function]
+# @PURPOSE: Main entry point for the initialization script.
+# @POST:    auth.db is initialized with the correct schema and seeded permissions.
+def run_init():
+    with belief_scope("init_auth_db"):
+        logger.info("Initializing authentication database...")
+        try:
+            init_db()
+            logger.info("Authentication database initialized successfully.")
+            
+            # Seed permissions
+            seed_permissions()
+            
+        except Exception as e:
+            logger.error(f"Failed to initialize authentication database: {e}")
+            sys.exit(1)
+# [/DEF:run_init:Function]
+
+if __name__ == "__main__":
+    run_init()
+
+# [/DEF:backend.src.scripts.init_auth_db:Module]

backend/src/scripts/seed_permissions.py

@@ -0,0 +1,116 @@
+# [DEF:backend.src.scripts.seed_permissions:Module]
+#
+# @SEMANTICS: setup, database, auth, permissions, seeding
+# @PURPOSE:   Populates the auth database with initial system permissions.
+# @LAYER:     Scripts
+# @RELATION:  USES -> backend.src.core.database.get_auth_db
+# @RELATION:  USES -> backend.src.models.auth.Permission
+#
+# @INVARIANT: Safe to run multiple times (idempotent).
+
+# [SECTION: IMPORTS]
+import sys
+from pathlib import Path
+
+# Add src to path
+sys.path.append(str(Path(__file__).parent.parent.parent))
+
+from src.core.database import AuthSessionLocal
+from src.models.auth import Permission, Role
+from src.core.auth.repository import AuthRepository
+from src.core.logger import logger, belief_scope
+# [/SECTION]
+
+# [DEF:INITIAL_PERMISSIONS:Constant]
+INITIAL_PERMISSIONS = [
+    # Admin Permissions
+    {"resource": "admin:users", "action": "READ"},
+    {"resource": "admin:users", "action": "WRITE"},
+    {"resource": "admin:roles", "action": "READ"},
+    {"resource": "admin:roles", "action": "WRITE"},
+    {"resource": "admin:settings", "action": "READ"},
+    {"resource": "admin:settings", "action": "WRITE"},
+    {"resource": "environments", "action": "READ"},
+    {"resource": "plugins", "action": "READ"},
+    {"resource": "tasks", "action": "READ"},
+    {"resource": "tasks", "action": "WRITE"},
+    
+    # Plugin Permissions
+    {"resource": "plugin:backup", "action": "EXECUTE"},
+    {"resource": "plugin:migration", "action": "EXECUTE"},
+    {"resource": "plugin:mapper", "action": "EXECUTE"},
+    {"resource": "plugin:search", "action": "EXECUTE"},
+    {"resource": "plugin:git", "action": "EXECUTE"},
+    {"resource": "plugin:storage", "action": "EXECUTE"},
+    {"resource": "plugin:storage", "action": "READ"},
+    {"resource": "plugin:storage", "action": "WRITE"},
+    {"resource": "plugin:debug", "action": "EXECUTE"},
+]
+# [/DEF:INITIAL_PERMISSIONS:Constant]
+
+# [DEF:seed_permissions:Function]
+# @PURPOSE: Inserts missing permissions into the database.
+# @POST:    All INITIAL_PERMISSIONS exist in the DB.
+def seed_permissions():
+    with belief_scope("seed_permissions"):
+        db = AuthSessionLocal()
+        try:
+            logger.info("Seeding permissions...")
+            count = 0
+            for perm_data in INITIAL_PERMISSIONS:
+                exists = db.query(Permission).filter(
+                    Permission.resource == perm_data["resource"],
+                    Permission.action == perm_data["action"]
+                ).first()
+                
+                if not exists:
+                    new_perm = Permission(
+                        resource=perm_data["resource"],
+                        action=perm_data["action"]
+                    )
+                    db.add(new_perm)
+                    count += 1
+            
+            db.commit()
+            logger.info(f"Seeding completed. Added {count} new permissions.")
+
+            # Assign permissions to User role
+            repo = AuthRepository(db)
+            user_role = repo.get_role_by_name("User")
+            if not user_role:
+                user_role = Role(name="User", description="Standard user with plugin access")
+                db.add(user_role)
+                db.flush()
+            
+            user_permissions = [
+                ("plugin:mapper", "EXECUTE"),
+                ("plugin:migration", "EXECUTE"),
+                ("plugin:backup", "EXECUTE"),
+                ("plugin:git", "EXECUTE"),
+                ("plugin:storage", "READ"),
+                ("plugin:storage", "WRITE"),
+                ("environments", "READ"),
+                ("plugins", "READ"),
+                ("tasks", "READ"),
+                ("tasks", "WRITE"),
+            ]
+
+            for res, act in user_permissions:
+                perm = repo.get_permission_by_resource_action(res, act)
+                if perm and perm not in user_role.permissions:
+                    user_role.permissions.append(perm)
+            
+            db.commit()
+            logger.info("User role permissions updated.")
+
+        except Exception as e:
+            logger.error(f"Failed to seed permissions: {e}")
+            db.rollback()
+        finally:
+            db.close()
+# [/DEF:seed_permissions:Function]
+
+if __name__ == "__main__":
+    seed_permissions()
+
+# [/DEF:backend.src.scripts.seed_permissions:Module]

backend/src/services/auth_service.py

@@ -0,0 +1,115 @@
+# [DEF:backend.src.services.auth_service:Module]
+#
+# @SEMANTICS: auth, service, business-logic, login, jwt
+# @PURPOSE:   Orchestrates authentication business logic.
+# @LAYER:     Service
+# @RELATION:  USES -> backend.src.core.auth.repository.AuthRepository
+# @RELATION:  USES -> backend.src.core.auth.security
+# @RELATION:  USES -> backend.src.core.auth.jwt
+#
+# @INVARIANT: Authentication must verify both credentials and account status.
+
+# [SECTION: IMPORTS]
+from typing import Optional, Dict, Any, List
+from sqlalchemy.orm import Session
+from ..models.auth import User, Role
+from ..core.auth.repository import AuthRepository
+from ..core.auth.security import verify_password, get_password_hash
+from ..core.auth.jwt import create_access_token
+from ..core.logger import belief_scope
+# [/SECTION]
+
+# [DEF:AuthService:Class]
+# @PURPOSE: Provides high-level authentication services.
+class AuthService:
+    # [DEF:__init__:Function]
+    # @PURPOSE: Initializes the service with a database session.
+    # @PARAM:   db (Session) - SQLAlchemy session.
+    def __init__(self, db: Session):
+        self.repo = AuthRepository(db)
+    # [/DEF:__init__:Function]
+
+    # [DEF:authenticate_user:Function]
+    # @PURPOSE: Authenticates a user with username and password.
+    # @PRE:     username and password are provided.
+    # @POST:    Returns User object if authentication succeeds, else None.
+    # @SIDE_EFFECT: Updates last_login timestamp on success.
+    # @PARAM:   username (str) - The username.
+    # @PARAM:   password (str) - The plain password.
+    # @RETURN:  Optional[User] - The authenticated user or None.
+    def authenticate_user(self, username: str, password: str):
+        with belief_scope("AuthService.authenticate_user"):
+            user = self.repo.get_user_by_username(username)
+            if not user:
+                return None
+            
+            if not user.is_active:
+                return None
+                
+            if not user.password_hash or not verify_password(password, user.password_hash):
+                return None
+            
+            self.repo.update_last_login(user)
+            return user
+    # [/DEF:authenticate_user:Function]
+
+    # [DEF:create_session:Function]
+    # @PURPOSE: Creates a JWT session for an authenticated user.
+    # @PRE:     user is a valid User object.
+    # @POST:    Returns a dictionary with access_token and token_type.
+    # @PARAM:   user (User) - The authenticated user.
+    # @RETURN:  Dict[str, str] - Session data.
+    def create_session(self, user) -> Dict[str, str]:
+        with belief_scope("AuthService.create_session"):
+            # Collect role names for scopes
+            scopes = [role.name for role in user.roles]
+            
+            token_data = {
+                "sub": user.username,
+                "scopes": scopes
+            }
+            
+            access_token = create_access_token(data=token_data)
+            return {
+                "access_token": access_token,
+                "token_type": "bearer"
+            }
+    # [/DEF:create_session:Function]
+
+    # [DEF:provision_adfs_user:Function]
+    # @PURPOSE: Just-In-Time (JIT) provisioning for ADFS users based on group mappings.
+    # @PRE:     user_info contains 'upn' (username), 'email', and 'groups'.
+    # @POST:    User is created/updated and assigned roles based on groups.
+    # @PARAM:   user_info (Dict[str, Any]) - Claims from ADFS token.
+    # @RETURN:  User - The provisioned user.
+    def provision_adfs_user(self, user_info: Dict[str, Any]) -> User:
+        with belief_scope("AuthService.provision_adfs_user"):
+            username = user_info.get("upn") or user_info.get("email")
+            email = user_info.get("email")
+            ad_groups = user_info.get("groups", [])
+            
+            user = self.repo.get_user_by_username(username)
+            if not user:
+                user = User(
+                    username=username,
+                    email=email,
+                    auth_source="ADFS",
+                    is_active=True
+                )
+                self.repo.db.add(user)
+            
+            # Update roles based on group mappings
+            from ..models.auth import ADGroupMapping
+            mapped_roles = self.repo.db.query(Role).join(ADGroupMapping).filter(
+                ADGroupMapping.ad_group.in_(ad_groups)
+            ).all()
+            
+            user.roles = mapped_roles
+            self.repo.db.commit()
+            self.repo.db.refresh(user)
+            return user
+    # [/DEF:provision_adfs_user:Function]
+
+# [/DEF:AuthService:Class]
+
+# [/DEF:backend.src.services.auth_service:Module]

backend/src/services/git_service.py

@@ -31,9 +31,15 @@ class GitService:
     # @PARAM: base_path (str) - Root directory for all Git clones.
     # @PRE: base_path is a valid string path.
     # @POST: GitService is initialized; base_path directory exists.
-    def __init__(self, base_path: str = "backend/git_repos"):
+    def __init__(self, base_path: str = "git_repos"):
         with belief_scope("GitService.__init__"):
-            self.base_path = base_path
+            # Resolve relative to the backend directory
+            # Path(__file__) is backend/src/services/git_service.py
+            # parents[2] is backend/
+            from pathlib import Path
+            backend_root = Path(__file__).parents[2]
+            
+            self.base_path = str((backend_root / base_path).resolve())
             if not os.path.exists(self.base_path):
                 os.makedirs(self.base_path)
     # [/DEF:__init__:Function]

backend/src/services/llm_provider.py

@@ -0,0 +1,129 @@
+# [DEF:backend.src.services.llm_provider:Module]
+# @TIER: STANDARD
+# @SEMANTICS: service, llm, provider, encryption
+# @PURPOSE: Service for managing LLM provider configurations with encrypted API keys.
+# @LAYER: Domain
+# @RELATION: DEPENDS_ON -> backend.src.core.database
+# @RELATION: DEPENDS_ON -> backend.src.models.llm
+
+from typing import List, Optional
+from sqlalchemy.orm import Session
+from ..models.llm import LLMProvider
+from ..plugins.llm_analysis.models import LLMProviderConfig, LLMProviderType
+from ..core.logger import belief_scope, logger
+from cryptography.fernet import Fernet
+import os
+
+# [DEF:EncryptionManager:Class]
+# @PURPOSE: Handles encryption and decryption of sensitive data like API keys.
+class EncryptionManager:
+    # @INVARIANT: Uses a secret key from environment or a default one (fallback only for dev).
+    def __init__(self):
+        self.key = os.getenv("ENCRYPTION_KEY", "ZcytYzi0iHIl4Ttr-GdAEk117aGRogkGvN3wiTxrPpE=").encode()
+        self.fernet = Fernet(self.key)
+
+    def encrypt(self, data: str) -> str:
+        return self.fernet.encrypt(data.encode()).decode()
+
+    def decrypt(self, encrypted_data: str) -> str:
+        return self.fernet.decrypt(encrypted_data.encode()).decode()
+# [/DEF:EncryptionManager:Class]
+
+# [DEF:LLMProviderService:Class]
+# @PURPOSE: Service to manage LLM provider lifecycle.
+class LLMProviderService:
+    def __init__(self, db: Session):
+        self.db = db
+        self.encryption = EncryptionManager()
+
+    # [DEF:get_all_providers:Function]
+    # @PURPOSE: Returns all configured LLM providers.
+    def get_all_providers(self) -> List[LLMProvider]:
+        with belief_scope("get_all_providers"):
+            return self.db.query(LLMProvider).all()
+    # [/DEF:get_all_providers:Function]
+
+    # [DEF:get_provider:Function]
+    # @PURPOSE: Returns a single LLM provider by ID.
+    def get_provider(self, provider_id: str) -> Optional[LLMProvider]:
+        with belief_scope("get_provider"):
+            return self.db.query(LLMProvider).filter(LLMProvider.id == provider_id).first()
+    # [/DEF:get_provider:Function]
+
+    # [DEF:create_provider:Function]
+    # @PURPOSE: Creates a new LLM provider with encrypted API key.
+    def create_provider(self, config: LLMProviderConfig) -> LLMProvider:
+        with belief_scope("create_provider"):
+            encrypted_key = self.encryption.encrypt(config.api_key)
+            db_provider = LLMProvider(
+                provider_type=config.provider_type.value,
+                name=config.name,
+                base_url=config.base_url,
+                api_key=encrypted_key,
+                default_model=config.default_model,
+                is_active=config.is_active
+            )
+            self.db.add(db_provider)
+            self.db.commit()
+            self.db.refresh(db_provider)
+            return db_provider
+    # [/DEF:create_provider:Function]
+
+    # [DEF:update_provider:Function]
+    # @PURPOSE: Updates an existing LLM provider.
+    def update_provider(self, provider_id: str, config: LLMProviderConfig) -> Optional[LLMProvider]:
+        with belief_scope("update_provider"):
+            db_provider = self.get_provider(provider_id)
+            if not db_provider:
+                return None
+            
+            db_provider.provider_type = config.provider_type.value
+            db_provider.name = config.name
+            db_provider.base_url = config.base_url
+            # Only update API key if provided (not None and not empty)
+            if config.api_key is not None and config.api_key != "":
+                db_provider.api_key = self.encryption.encrypt(config.api_key)
+            db_provider.default_model = config.default_model
+            db_provider.is_active = config.is_active
+            
+            self.db.commit()
+            self.db.refresh(db_provider)
+            return db_provider
+    # [/DEF:update_provider:Function]
+
+    # [DEF:delete_provider:Function]
+    # @PURPOSE: Deletes an LLM provider.
+    def delete_provider(self, provider_id: str) -> bool:
+        with belief_scope("delete_provider"):
+            db_provider = self.get_provider(provider_id)
+            if not db_provider:
+                return False
+            self.db.delete(db_provider)
+            self.db.commit()
+            return True
+    # [/DEF:delete_provider:Function]
+
+    # [DEF:get_decrypted_api_key:Function]
+    # @PURPOSE: Returns the decrypted API key for a provider.
+    def get_decrypted_api_key(self, provider_id: str) -> Optional[str]:
+        with belief_scope("get_decrypted_api_key"):
+            db_provider = self.get_provider(provider_id)
+            if not db_provider:
+                logger.warning(f"[get_decrypted_api_key] Provider {provider_id} not found in database")
+                return None
+            
+            logger.info(f"[get_decrypted_api_key] Provider found: {db_provider.id}")
+            logger.info(f"[get_decrypted_api_key] Encrypted API key length: {len(db_provider.api_key) if db_provider.api_key else 0}")
+            
+            try:
+                decrypted_key = self.encryption.decrypt(db_provider.api_key)
+                logger.info(f"[get_decrypted_api_key] Decryption successful, key length: {len(decrypted_key) if decrypted_key else 0}")
+                return decrypted_key
+            except Exception as e:
+                logger.error(f"[get_decrypted_api_key] Decryption failed: {str(e)}")
+                return None
+    # [/DEF:get_decrypted_api_key:Function]
+
+# [/DEF:LLMProviderService:Class]
+
+# [/DEF:backend.src.services.llm_provider:Module]

backend/src/services/mapping_service.py

@@ -10,9 +10,9 @@
 
 # [SECTION: IMPORTS]
 from typing import List, Dict
-from backend.src.core.logger import belief_scope
+from ..core.logger import belief_scope
-from backend.src.core.superset_client import SupersetClient
+from ..core.superset_client import SupersetClient
-from backend.src.core.utils.matching import suggest_mappings
+from ..core.utils.matching import suggest_mappings
 # [/SECTION]
 
 # [DEF:MappingService:Class]

backend/test_auth_debug.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+"""Debug script to test Superset API authentication"""
+
+import json
+import requests
+from pprint import pprint
+from src.core.superset_client import SupersetClient
+from src.core.config_manager import ConfigManager
+
+
+def main():
+    print("Debugging Superset API authentication...")
+    
+    config = ConfigManager()
+    
+    # Select first available environment
+    environments = config.get_environments()
+    
+    if not environments:
+        print("No environments configured")
+        return
+        
+    env = environments[0]
+    print(f"\nTesting environment: {env.name}")
+    print(f"URL: {env.url}")
+    
+    try:
+        # Test API client authentication
+        print("\n--- Testing API Authentication ---")
+        client = SupersetClient(env)
+        tokens = client.authenticate()
+        
+        print("\nAPI Auth Success!")
+        print(f"Access Token: {tokens.get('access_token', 'N/A')}")
+        print(f"CSRF Token: {tokens.get('csrf_token', 'N/A')}")
+        
+        # Debug cookies from session
+        print("\n--- Session Cookies ---")
+        for cookie in client.network.session.cookies:
+            print(f"{cookie.name}={cookie.value}")
+        
+        # Test accessing UI via requests
+        print("\n--- Testing UI Access ---")
+        ui_url = env.url.rstrip('/').replace('/api/v1', '')
+        print(f"UI URL: {ui_url}")
+        
+        # Try to access UI home page
+        ui_response = client.network.session.get(ui_url, timeout=30, allow_redirects=True)
+        print(f"Status Code: {ui_response.status_code}")
+        print(f"URL: {ui_response.url}")
+        
+        # Check response headers
+        print("\n--- Response Headers ---")
+        pprint(dict(ui_response.headers))
+        
+        print(f"\n--- Response Content Preview (200 chars) ---")
+        print(repr(ui_response.text[:200]))
+        
+        if ui_response.status_code == 200:
+            print("\nUI Access: Success")
+            
+            # Try to access a dashboard
+            # For testing, just use the home page
+            print("\n--- Checking if login is required ---")
+            if "login" in ui_response.url.lower() or "login" in ui_response.text.lower():
+                print("❌ Not logged in to UI")
+            else:
+                print("✅ Logged in to UI")
+        
+    except Exception as e:
+        print(f"\n❌ Error: {type(e).__name__}: {e}")
+        import traceback
+        print("\nStack Trace:")
+        print(traceback.format_exc())
+        
+
+if __name__ == "__main__":
+    main()

backend/test_decryption.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+"""Test script to debug API key decryption issue."""
+
+from src.core.database import SessionLocal
+from src.models.llm import LLMProvider
+from cryptography.fernet import Fernet
+import os
+
+# Get the encryption key
+key = os.getenv("ENCRYPTION_KEY", "ZcytYzi0iHIl4Ttr-GdAEk117aGRogkGvN3wiTxrPpE=").encode()
+print(f"Encryption key (first 20 chars): {key[:20]}")
+print(f"Encryption key length: {len(key)}")
+
+# Create Fernet instance
+fernet = Fernet(key)
+
+# Get provider from database
+db = SessionLocal()
+provider = db.query(LLMProvider).filter(LLMProvider.id == '6c899741-4108-4196-aea4-f38ad2f0150e').first()
+
+if provider:
+    print(f"\nProvider found:")
+    print(f"  ID: {provider.id}")
+    print(f"  Name: {provider.name}")
+    print(f"  Encrypted API Key (first 50 chars): {provider.api_key[:50]}")
+    print(f"  Encrypted API Key Length: {len(provider.api_key)}")
+    
+    # Test decryption
+    print(f"\nAttempting decryption...")
+    try:
+        decrypted = fernet.decrypt(provider.api_key.encode()).decode()
+        print(f"Decryption successful!")
+        print(f"  Decrypted key length: {len(decrypted)}")
+        print(f"  Decrypted key (first 8 chars): {decrypted[:8]}")
+        print(f"  Decrypted key is empty: {len(decrypted) == 0}")
+    except Exception as e:
+        print(f"Decryption failed with error: {e}")
+        print(f"Error type: {type(e).__name__}")
+        import traceback
+        traceback.print_exc()
+else:
+    print("Provider not found")
+
+db.close()

backend/test_encryption.py

@@ -0,0 +1 @@
+[{"key[": 20, ")\n\n# Create Fernet instance\nfernet = Fernet(key)\n\n# Test encrypting an empty string\nempty_encrypted = fernet.encrypt(b\"": ".", "print(f": "nEncrypted empty string: {empty_encrypted"}, {"test-api-key-12345\"\ntest_encrypted = fernet.encrypt(test_key.encode()).decode()\nprint(f": "nEncrypted test key: {test_encrypted"}, {"gAAAAABphhwSZie0OwXjJ78Fk-c4Uo6doNJXipX49AX7Bypzp4ohiRX3hXPXKb45R1vhNUOqbm6Ke3-eRwu_KdWMZ9chFBKmqw==\"\nprint(f": "nStored encrypted key: {stored_key"}, {"len(stored_key)}": "Check if stored key matches empty string encryption\nif stored_key == empty_encrypted:\n    print(", "string!": "else:\n    print(", "print(f": "mpty string encryption: {empty_encrypted"}, {"stored_key}": "Try to decrypt the stored key\ntry:\n    decrypted = fernet.decrypt(stored_key.encode()).decode()\n    print(f", "print(f": "ecrypted key length: {len(decrypted)"}, {")\nexcept Exception as e:\n    print(f": "nDecryption failed with error: {e"}]

backend/tests/test_auth.py

@@ -0,0 +1,162 @@
+import sys
+import os
+from pathlib import Path
+
+# Add src to path
+sys.path.append(str(Path(__file__).parent.parent / "src"))
+
+import pytest
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+from src.core.database import Base, get_auth_db
+from src.models.auth import User, Role, Permission, ADGroupMapping
+from src.services.auth_service import AuthService
+from src.core.auth.repository import AuthRepository
+from src.core.auth.security import verify_password, get_password_hash
+
+# Create in-memory SQLite database for testing
+SQLALCHEMY_DATABASE_URL = "sqlite:///:memory:"
+
+engine = create_engine(SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False})
+TestingSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
+
+# Create all tables
+Base.metadata.create_all(bind=engine)
+
+@pytest.fixture
+def db_session():
+    """Create a new database session with a transaction, rollback after test"""
+    connection = engine.connect()
+    transaction = connection.begin()
+    session = TestingSessionLocal(bind=connection)
+    
+    yield session
+    
+    session.close()
+    transaction.rollback()
+    connection.close()
+
+@pytest.fixture
+def auth_service(db_session):
+    return AuthService(db_session)
+
+@pytest.fixture
+def auth_repo(db_session):
+    return AuthRepository(db_session)
+
+def test_create_user(auth_repo):
+    """Test user creation"""
+    user = User(
+        username="testuser",
+        email="test@example.com",
+        password_hash=get_password_hash("testpassword123"),
+        auth_source="LOCAL"
+    )
+    
+    auth_repo.db.add(user)
+    auth_repo.db.commit()
+    
+    retrieved_user = auth_repo.get_user_by_username("testuser")
+    assert retrieved_user is not None
+    assert retrieved_user.username == "testuser"
+    assert retrieved_user.email == "test@example.com"
+    assert verify_password("testpassword123", retrieved_user.password_hash)
+
+def test_authenticate_user(auth_service, auth_repo):
+    """Test user authentication with valid and invalid credentials"""
+    user = User(
+        username="testuser",
+        email="test@example.com",
+        password_hash=get_password_hash("testpassword123"),
+        auth_source="LOCAL"
+    )
+    
+    auth_repo.db.add(user)
+    auth_repo.db.commit()
+    
+    # Test valid credentials
+    authenticated_user = auth_service.authenticate_user("testuser", "testpassword123")
+    assert authenticated_user is not None
+    assert authenticated_user.username == "testuser"
+    
+    # Test invalid password
+    invalid_user = auth_service.authenticate_user("testuser", "wrongpassword")
+    assert invalid_user is None
+    
+    # Test invalid username
+    invalid_user = auth_service.authenticate_user("nonexistent", "testpassword123")
+    assert invalid_user is None
+
+def test_create_session(auth_service, auth_repo):
+    """Test session token creation"""
+    user = User(
+        username="testuser",
+        email="test@example.com",
+        password_hash=get_password_hash("testpassword123"),
+        auth_source="LOCAL"
+    )
+    
+    auth_repo.db.add(user)
+    auth_repo.db.commit()
+    
+    session = auth_service.create_session(user)
+    assert "access_token" in session
+    assert "token_type" in session
+    assert session["token_type"] == "bearer"
+    assert len(session["access_token"]) > 0
+
+def test_role_permission_association(auth_repo):
+    """Test role and permission association"""
+    role = Role(name="Admin", description="System administrator")
+    perm1 = Permission(resource="admin:users", action="READ")
+    perm2 = Permission(resource="admin:users", action="WRITE")
+    
+    role.permissions.extend([perm1, perm2])
+    
+    auth_repo.db.add(role)
+    auth_repo.db.commit()
+    
+    retrieved_role = auth_repo.get_role_by_name("Admin")
+    assert retrieved_role is not None
+    assert len(retrieved_role.permissions) == 2
+    
+    permissions = [f"{p.resource}:{p.action}" for p in retrieved_role.permissions]
+    assert "admin:users:READ" in permissions
+    assert "admin:users:WRITE" in permissions
+
+def test_user_role_association(auth_repo):
+    """Test user and role association"""
+    role = Role(name="Admin", description="System administrator")
+    user = User(
+        username="adminuser",
+        email="admin@example.com",
+        password_hash=get_password_hash("adminpass123"),
+        auth_source="LOCAL"
+    )
+    
+    user.roles.append(role)
+    
+    auth_repo.db.add(role)
+    auth_repo.db.add(user)
+    auth_repo.db.commit()
+    
+    retrieved_user = auth_repo.get_user_by_username("adminuser")
+    assert retrieved_user is not None
+    assert len(retrieved_user.roles) == 1
+    assert retrieved_user.roles[0].name == "Admin"
+
+def test_ad_group_mapping(auth_repo):
+    """Test AD group mapping"""
+    role = Role(name="ADFS_Admin", description="ADFS administrators")
+    
+    auth_repo.db.add(role)
+    auth_repo.db.commit()
+    
+    mapping = ADGroupMapping(ad_group="DOMAIN\\ADFS_Admins", role_id=role.id)
+    
+    auth_repo.db.add(mapping)
+    auth_repo.db.commit()
+    
+    retrieved_mapping = auth_repo.db.query(ADGroupMapping).filter_by(ad_group="DOMAIN\\ADFS_Admins").first()
+    assert retrieved_mapping is not None
+    assert retrieved_mapping.role_id == role.id

docs/architecture_decision_superset_migration.md

@@ -0,0 +1,38 @@
+# Архитектурное решение: Standalone сервис vs Интеграция в Superset
+
+## Рекомендация: Оставить отдельным сервисом
+
+Хотя интеграция непосредственно в Superset может показаться привлекательной для создания "единого интерфейса", **настоятельно рекомендуется оставить `ss-tools` отдельным сервисом** по следующим архитектурным и техническим причинам.
+
+### 1. Архитектурный паттерн: Оркестратор против Инстанса
+*   **Проблема:** `ss-tools` действует как **"Менеджер менеджеров"**. Его основная ценность заключается в подключении и перемещении данных *между* различными окружениями Superset (Dev → Stage → Prod).
+*   **Почему разделение лучше:** Если встроить эту логику в Superset, придется сделать один конкретный инстанс (например, "Dev") "главным контроллером" для остальных. Если этот инстанс упадет, вы потеряете возможности управления. Отдельный инструмент находится *над* окружениями, рассматривая их все как равные endpoints.
+
+### 2. Несовместимость технологического стека
+*   **Superset:** Бэкенд на **Flask** (Python) с фронтендом на **React**.
+*   **ss-tools:** Бэкенд на **FastAPI** (Python) с фронтендом на **SvelteKit**.
+*   **Цена миграции:** Перенос в Superset потребует **полного переписывания** всего фронтенда с Svelte на React, чтобы он мог работать как плагин или расширение Superset. Также потребуется рефакторинг внедрения зависимостей (Dependency Injection) из FastAPI, чтобы соответствовать структуре Flask-AppBuilder в Superset.
+
+### 3. Цикл релизов и гибкость
+*   **Superset:** Имеет сложный и медленный цикл релизов. Внедрение кастомных фич в основной репозиторий (upstream) затруднительно, а поддержка собственного "форка" Superset — это кошмар сопровождения (конфликты слияния при каждом обновлении).
+*   **ss-tools:** Как отдельный микросервис, вы можете развертывать новые функции (например, новые LLM плагины или Git-процессы) мгновенно, не беспокоясь о том, что сломаете основную BI-платформу.
+
+### 4. Безопасность и изоляция
+*   **Права доступа:** `ss-tools` имеет свои собственные страницы администрирования (`AdminRolesPage` и `AdminUsersPage`), управляющие правами специально для задач *деплоя* и *резервного копирования*. Это задачи DevOps, а не бизнес-аналитики (BI). Смешивание привилегий развертывания с привилегиями просмотра данных в одной системе увеличивает "радиус поражения" в случае проблем с безопасностью.
+
+### Техническая реализуемость (если все же решиться)
+Это технически возможно, но процесс будет выглядеть так:
+1.  **Frontend:** Придется переписать все компоненты Svelte (`DashboardGrid`, `MappingTable` и т.д.) как компоненты React, чтобы они могли жить внутри `superset-frontend`.
+2.  **Backend:** Вероятно, придется реализовать это как серверное **расширение Superset** (используя Flask Blueprints), создав новые API endpoints, которые будет вызывать фронтенд на React.
+3.  **Database:** Придется добавить таблицы проекта (`task_records`, `connection_configs`) в базу данных метаданных Superset через миграции Alembic.
+
+### Итоговая таблица
+
+| Характеристика | Отдельный сервис (Текущий) | Интеграция в Superset |
+| :--- | :--- | :--- |
+| **Скорость разработки** | 🚀 Высокая (Независимая) | 🐢 Низкая (Связана с монолитом) |
+| **Технологический стек** | Современный (FastAPI + SvelteKit) | Legacy/Строгий (Flask + React) |
+| **Управление мульти-окружением** | ✅ Нативно (Спроектирован для этого) | ⚠️ Неудобно (Один инстанс управляет всеми) |
+| **Поддержка (Maintenance)** | ✅ Низкая (Обновление в любое время) | ❌ Высокая (Rebase при каждом обновлении Superset) |
+
+**Вывод:** Проект уже хорошо спроектирован как специализированный инструмент DevOps (слой `DevOps/Tooling` в вашей семантической карте). Слияние его с Superset, скорее всего, приведет к техническому долгу и регрессии функциональности.

docs/settings.md

@@ -13,7 +13,7 @@ The settings mechanism allows users to configure multiple Superset environments
 Configuration is structured using Pydantic models in `backend/src/core/config_models.py`:
 
 - `Environment`: Represents a Superset instance (URL, credentials). The `base_url` is automatically normalized to include the `/api/v1` suffix if missing.
-- `GlobalSettings`: Global application parameters (e.g., `backup_path`).
+- `GlobalSettings`: Global application parameters (e.g., `storage.root_path`).
 - `AppConfig`: The root configuration object.
 
 ### Configuration Manager
@@ -43,4 +43,4 @@ The settings page is located at `frontend/src/pages/Settings.svelte`. It provide
 
 Existing plugins and utilities use the `ConfigManager` to fetch configuration:
 - `superset_tool/utils/init_clients.py`: Dynamically initializes Superset clients from the configured environments.
-- `BackupPlugin`: Uses the configured `backup_path` as the default storage location.
+- `BackupPlugin`: Uses the configured `storage.root_path` as the default storage location.

frontend/src/components/DashboardGrid.svelte

@@ -15,11 +15,14 @@
   import { t } from '../lib/i18n';
   import { Button, Input } from '../lib/ui';
   import GitManager from './git/GitManager.svelte';
+  import { api } from '../lib/api';
+  import { addToast as toast } from '../lib/toasts.js';
   // [/SECTION]
 
   // [SECTION: PROPS]
   export let dashboards: DashboardMetadata[] = [];
   export let selectedIds: number[] = [];
+  export let environmentId: string = "ss1";
   // [/SECTION]
 
   // [SECTION: STATE]
@@ -34,8 +37,54 @@
   let showGitManager = false;
   let gitDashboardId: number | null = null;
   let gitDashboardTitle = "";
+  let validatingIds: Set<number> = new Set();
   // [/SECTION]
 
+  // [DEF:handleValidate:Function]
+  /**
+   * @purpose Triggers dashboard validation task.
+   */
+  async function handleValidate(dashboard: DashboardMetadata) {
+    if (validatingIds.has(dashboard.id)) return;
+    
+    validatingIds.add(dashboard.id);
+    validatingIds = validatingIds; // Trigger reactivity
+    
+    try {
+        // TODO: Get provider_id from settings or prompt user
+        // For now, we assume a default provider or let the backend handle it if possible,
+        // but the plugin requires provider_id.
+        // In a real implementation, we might open a modal to select provider if not configured globally.
+        // Or we pick the first active one.
+        
+        // Fetch active provider first
+        const providers = await api.fetchApi('/llm/providers');
+        const activeProvider = providers.find((p: any) => p.is_active);
+        
+        if (!activeProvider) {
+            toast('No active LLM provider found. Please configure one in settings.', 'error');
+            return;
+        }
+
+        await api.postApi('/tasks', {
+            plugin_id: 'llm_dashboard_validation',
+            params: {
+                dashboard_id: dashboard.id.toString(),
+                environment_id: environmentId,
+                provider_id: activeProvider.id
+            }
+        });
+        
+        toast('Validation task started', 'success');
+    } catch (e: any) {
+        toast(e.message || 'Validation failed to start', 'error');
+    } finally {
+        validatingIds.delete(dashboard.id);
+        validatingIds = validatingIds;
+    }
+  }
+  // [/DEF:handleValidate:Function]
+
   // [SECTION: DERIVED]
   $: filteredDashboards = dashboards.filter(d =>
     d.title.toLowerCase().includes(filterText.toLowerCase())
@@ -175,6 +224,7 @@
           <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider cursor-pointer hover:text-gray-700 transition-colors" on:click={() => handleSort('status')}>
             {$t.dashboard.status} {sortColumn === 'status' ? (sortDirection === 'asc' ? '↑' : '↓') : ''}
           </th>
+          <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.dashboard.validation}</th>
           <th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.dashboard.git}</th>
         </tr>
       </thead>
@@ -196,6 +246,17 @@
                 {dashboard.status}
               </span>
             </td>
+            <td class="px-6 py-4 whitespace-nowrap text-sm font-medium">
+              <Button
+                variant="secondary"
+                size="sm"
+                on:click={() => handleValidate(dashboard)}
+                disabled={validatingIds.has(dashboard.id)}
+                class="text-purple-600 hover:text-purple-900"
+              >
+                {validatingIds.has(dashboard.id) ? 'Validating...' : 'Validate'}
+              </Button>
+            </td>
             <td class="px-6 py-4 whitespace-nowrap text-sm font-medium">
               <Button
                 variant="ghost"

frontend/src/components/EnvSelector.svelte

@@ -57,4 +57,4 @@
   /* Component specific styles */
 </style>
 
-<!-- [/DEF:EnvSelector:Component] -->
+<!-- [/DEF:EnvSelector:Component] -->

frontend/src/components/Navbar.svelte

@@ -9,6 +9,13 @@
     import { page } from '$app/stores';
     import { t } from '$lib/i18n';
     import { LanguageSwitcher } from '$lib/ui';
+    import { auth } from '$lib/auth/store';
+    import { goto } from '$app/navigation';
+
+    function handleLogout() {
+        auth.logout();
+        goto('/login');
+    }
 </script>
 
 <header class="bg-white shadow-md p-4 flex justify-between items-center">
@@ -25,34 +32,12 @@
         >
             {$t.nav.dashboard}
         </a>
-        <a
-            href="/migration"
-            class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/migration') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
-        >
-            {$t.nav.migration}
-        </a>
-        <a
-            href="/git"
-            class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/git') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
-        >
-            {$t.nav.git}
-        </a>
         <a
             href="/tasks"
             class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/tasks') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
         >
             {$t.nav.tasks}
         </a>
-        <div class="relative inline-block group">
-            <button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/tools') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
-                {$t.nav.tools}
-            </button>
-            <div class="absolute hidden group-hover:block bg-white shadow-lg rounded-md mt-1 py-2 w-48 z-10 border border-gray-100 before:absolute before:-top-2 before:left-0 before:right-0 before:h-2 before:content-[''] right-0">
-                <a href="/tools/search" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_search}</a>
-                <a href="/tools/mapper" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_mapper}</a>
-                <a href="/tools/debug" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_debug}</a>
-            </div>
-        </div>
         <div class="relative inline-block group">
             <button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/settings') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
                 {$t.nav.settings}
@@ -61,10 +46,35 @@
                 <a href="/settings" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_general}</a>
                 <a href="/settings/connections" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_connections}</a>
                 <a href="/settings/git" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_git}</a>
-                <a href="/settings/environments" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_environments}</a>
             </div>
         </div>
+
+        {#if $auth.isAuthenticated && $auth.user?.roles?.some(r => r.name === 'Admin')}
+            <div class="relative inline-block group">
+                <button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/admin') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
+                    {$t.nav.admin}
+                </button>
+                <div class="absolute hidden group-hover:block bg-white shadow-lg rounded-md mt-1 py-2 w-48 z-10 border border-gray-100 before:absolute before:-top-2 before:left-0 before:right-0 before:h-2 before:content-[''] right-0">
+                    <a href="/admin/users" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.admin_users}</a>
+                    <a href="/admin/roles" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.admin_roles}</a>
+                    <a href="/admin/settings" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.admin_settings}</a>
+                    <a href="/admin/settings/llm" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.admin_llm}</a>
+                </div>
+            </div>
+        {/if}
         <LanguageSwitcher />
+        
+        {#if $auth.isAuthenticated}
+            <div class="flex items-center space-x-2 border-l pl-4 ml-4">
+                <span class="text-sm text-gray-600">{$auth.user?.username}</span>
+                <button
+                    on:click={handleLogout}
+                    class="text-sm text-red-600 hover:text-red-800 font-medium"
+                >
+                    Logout
+                </button>
+            </div>
+        {/if}
     </nav>
 </header>
 <!-- [/DEF:Navbar:Component] -->