10 Commits

82 changed files with 11300 additions and 7869 deletions

4
.gitignore vendored
View File

@@ -66,6 +66,4 @@ backend/mappings.db
backend/tasks.db
# Git Integration repositories
backend/git_repos/
backend/logs

View File

@@ -27,6 +27,10 @@ Auto-generated from all feature plans. Last updated: 2025-12-19
- SQLite (for config/history), Filesystem (local Git repositories) (011-git-integration-dashboard)
- Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing) (013-unify-frontend-css)
- LocalStorage (for language preference) (013-unify-frontend-css)
- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend) (014-file-storage-ui)
- Local Filesystem (for artifacts), Config (for storage path) (014-file-storage-ui)
- Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend) (015-frontend-nav-redesign)
- N/A (UI reorganization and API integration) (015-frontend-nav-redesign)
- Python 3.9+ (Backend), Node.js 18+ (Frontend Build) (001-plugin-arch-svelte-ui)
@@ -47,9 +51,9 @@ cd src; pytest; ruff check .
Python 3.9+ (Backend), Node.js 18+ (Frontend Build): Follow standard conventions
## Recent Changes
- 015-frontend-nav-redesign: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
- 014-file-storage-ui: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI (Backend), SvelteKit (Frontend)
- 013-unify-frontend-css: Added Node.js 18+ (Frontend Build), Svelte 5.x + SvelteKit, Tailwind CSS, `date-fns` (existing)
- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
- 011-git-integration-dashboard: Added Python 3.9+ (Backend), Node.js 18+ (Frontend) + FastAPI, SvelteKit, GitPython (or CLI git), Pydantic, SQLAlchemy, Superset API
<!-- MANUAL ADDITIONS START -->

View File

@@ -1,11 +1,10 @@
<!--
SYNC IMPACT REPORT
Version: 1.7.1 (Simplified Workflow)
Version: 1.8.0 (Frontend Unification)
Changes:
- Simplified Generation Workflow to a single phase: Code Generation from `tasks.md`.
- Removed multi-phase Architecture/Implementation split to streamline development.
- Added Principle VIII: Unified Frontend Experience (Mandating Design System & i18n).
Templates Status:
- .specify/templates/plan-template.md: ✅ Aligned (Dynamic check).
- .specify/templates/plan-template.md: ✅ Aligned.
- .specify/templates/spec-template.md: ✅ Aligned.
- .specify/templates/tasks-template.md: ✅ Aligned.
-->
@@ -37,6 +36,11 @@ To maintain semantic coherence, code must adhere to the complexity limits (Modul
### VII. Everything is a Plugin
All functional extensions, tools, or major features must be implemented as modular Plugins inheriting from `PluginBase`. Logic should not reside in standalone services or scripts unless strictly necessary for core infrastructure. This ensures a unified execution model via the `TaskManager`, consistent logging, and modularity.
### VIII. Unified Frontend Experience
To ensure a consistent and accessible user experience, all frontend implementations must strictly adhere to the unified design and localization standards.
- **Component Reusability**: All UI elements MUST utilize the standardized Svelte component library (`src/lib/ui`) and centralized design tokens. Ad-hoc styling and hardcoded values are prohibited.
- **Internationalization (i18n)**: All user-facing text MUST be extracted to the translation system (`src/lib/i18n`). Hardcoded strings in the UI are prohibited.
## File Structure Standards
Refer to **Section III (File Structure Standard)** in `semantic_protocol.md` for the authoritative definitions of:
- Python Module Headers (`.py`)
@@ -64,4 +68,4 @@ This Constitution establishes the "Semantic Code Generation Protocol" as the sup
- **Amendments**: Changes to core principles require a Constitution amendment. Changes to technical syntax require a Protocol update.
- **Compliance**: Failure to adhere to the Protocol constitutes a build failure.
**Version**: 1.7.1 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-13
**Version**: 1.8.0 | **Ratified**: 2025-12-19 | **Last Amended**: 2026-01-26

Submodule backend/backend/git_repos/12 updated: d592fa7ed5...f46772443a

View File

@@ -1,269 +0,0 @@
2025-12-20 19:55:11,325 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 19:55:11,325 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 19:55:11,327 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 43, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 21:01:49,905 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 21:01:49,906 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 21:01:49,988 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 21:01:49,990 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 22:42:32,538 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 22:42:32,538 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 22:42:32,583 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 22:42:32,587 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 22:54:29,770 - INFO - [BackupPlugin][Entry] Starting backup for .
2025-12-20 22:54:29,771 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 22:54:29,831 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 22:54:29,833 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 22:54:34,078 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 22:54:34,078 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 22:54:34,079 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 22:54:34,079 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 22:59:25,060 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 22:59:25,060 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 22:59:25,114 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 22:59:25,117 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:00:31,156 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:00:31,156 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:00:31,157 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:00:31,162 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:00:34,710 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:00:34,710 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:00:34,710 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:00:34,711 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:01:43,894 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:01:43,894 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:01:43,895 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:01:43,895 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:04:07,731 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:04:07,731 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:04:07,732 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:04:07,732 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:06:39,641 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:06:39,642 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:06:39,687 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:06:39,689 - CRITICAL - [setup_clients][Failure] Critical error during client initialization: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
Traceback (most recent call last):
File "/home/user/ss-tools/superset_tool/utils/init_clients.py", line 66, in setup_clients
config = SupersetConfig(
^^^^^^^^^^^^^^^
File "/home/user/ss-tools/backend/.venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for SupersetConfig
base_url
Value error, Invalid URL format: https://superset.bebesh.ru. Must include '/api/v1'. [type=value_error, input_value='https://superset.bebesh.ru', input_type=str]
For further information visit https://errors.pydantic.dev/2.12/v/value_error
2025-12-20 23:30:36,090 - INFO - [BackupPlugin][Entry] Starting backup for superset.
2025-12-20 23:30:36,093 - INFO - [setup_clients][Enter] Starting Superset clients initialization.
2025-12-20 23:30:36,128 - INFO - [setup_clients][Action] Loading environments from ConfigManager
2025-12-20 23:30:36,129 - INFO - [SupersetClient.__init__][Enter] Initializing SupersetClient.
2025-12-20 23:30:36,129 - INFO - [APIClient.__init__][Entry] Initializing APIClient.
2025-12-20 23:30:36,130 - WARNING - [_init_session][State] SSL verification disabled.
2025-12-20 23:30:36,130 - INFO - [APIClient.__init__][Exit] APIClient initialized.
2025-12-20 23:30:36,130 - INFO - [SupersetClient.__init__][Exit] SupersetClient initialized.
2025-12-20 23:30:36,130 - INFO - [get_dashboards][Enter] Fetching dashboards.
2025-12-20 23:30:36,131 - INFO - [authenticate][Enter] Authenticating to https://superset.bebesh.ru/api/v1
2025-12-20 23:30:36,897 - INFO - [authenticate][Exit] Authenticated successfully.
2025-12-20 23:30:37,527 - INFO - [get_dashboards][Exit] Found 11 dashboards.
2025-12-20 23:30:37,527 - INFO - [BackupPlugin][Progress] Found 11 dashboards to export in superset.
2025-12-20 23:30:37,529 - INFO - [export_dashboard][Enter] Exporting dashboard 11.
2025-12-20 23:30:38,224 - INFO - [export_dashboard][Exit] Exported dashboard 11 to dashboard_export_20251220T203037.zip.
2025-12-20 23:30:38,225 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:38,226 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/FCC New Coder Survey 2018/dashboard_export_20251220T203037.zip
2025-12-20 23:30:38,227 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/FCC New Coder Survey 2018
2025-12-20 23:30:38,230 - INFO - [export_dashboard][Enter] Exporting dashboard 10.
2025-12-20 23:30:38,438 - INFO - [export_dashboard][Exit] Exported dashboard 10 to dashboard_export_20251220T203038.zip.
2025-12-20 23:30:38,438 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:38,439 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/COVID Vaccine Dashboard/dashboard_export_20251220T203038.zip
2025-12-20 23:30:38,439 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/COVID Vaccine Dashboard
2025-12-20 23:30:38,440 - INFO - [export_dashboard][Enter] Exporting dashboard 9.
2025-12-20 23:30:38,853 - INFO - [export_dashboard][Exit] Exported dashboard 9 to dashboard_export_20251220T203038.zip.
2025-12-20 23:30:38,853 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:38,856 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Sales Dashboard/dashboard_export_20251220T203038.zip
2025-12-20 23:30:38,856 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Sales Dashboard
2025-12-20 23:30:38,858 - INFO - [export_dashboard][Enter] Exporting dashboard 8.
2025-12-20 23:30:38,939 - INFO - [export_dashboard][Exit] Exported dashboard 8 to dashboard_export_20251220T203038.zip.
2025-12-20 23:30:38,940 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:38,941 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Unicode Test/dashboard_export_20251220T203038.zip
2025-12-20 23:30:38,941 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Unicode Test
2025-12-20 23:30:38,942 - INFO - [export_dashboard][Enter] Exporting dashboard 7.
2025-12-20 23:30:39,148 - INFO - [export_dashboard][Exit] Exported dashboard 7 to dashboard_export_20251220T203038.zip.
2025-12-20 23:30:39,148 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:39,149 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Video Game Sales/dashboard_export_20251220T203038.zip
2025-12-20 23:30:39,149 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Video Game Sales
2025-12-20 23:30:39,150 - INFO - [export_dashboard][Enter] Exporting dashboard 6.
2025-12-20 23:30:39,689 - INFO - [export_dashboard][Exit] Exported dashboard 6 to dashboard_export_20251220T203039.zip.
2025-12-20 23:30:39,689 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:39,690 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Featured Charts/dashboard_export_20251220T203039.zip
2025-12-20 23:30:39,691 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Featured Charts
2025-12-20 23:30:39,692 - INFO - [export_dashboard][Enter] Exporting dashboard 5.
2025-12-20 23:30:39,960 - INFO - [export_dashboard][Exit] Exported dashboard 5 to dashboard_export_20251220T203039.zip.
2025-12-20 23:30:39,960 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:39,961 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Slack Dashboard/dashboard_export_20251220T203039.zip
2025-12-20 23:30:39,961 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Slack Dashboard
2025-12-20 23:30:39,962 - INFO - [export_dashboard][Enter] Exporting dashboard 4.
2025-12-20 23:30:40,196 - INFO - [export_dashboard][Exit] Exported dashboard 4 to dashboard_export_20251220T203039.zip.
2025-12-20 23:30:40,196 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:40,197 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/deck.gl Demo/dashboard_export_20251220T203039.zip
2025-12-20 23:30:40,197 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/deck.gl Demo
2025-12-20 23:30:40,198 - INFO - [export_dashboard][Enter] Exporting dashboard 3.
2025-12-20 23:30:40,745 - INFO - [export_dashboard][Exit] Exported dashboard 3 to dashboard_export_20251220T203040.zip.
2025-12-20 23:30:40,746 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:40,760 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/Misc Charts/dashboard_export_20251220T203040.zip
2025-12-20 23:30:40,761 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/Misc Charts
2025-12-20 23:30:40,762 - INFO - [export_dashboard][Enter] Exporting dashboard 2.
2025-12-20 23:30:40,928 - INFO - [export_dashboard][Exit] Exported dashboard 2 to dashboard_export_20251220T203040.zip.
2025-12-20 23:30:40,929 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:40,930 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/USA Births Names/dashboard_export_20251220T203040.zip
2025-12-20 23:30:40,931 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/USA Births Names
2025-12-20 23:30:40,932 - INFO - [export_dashboard][Enter] Exporting dashboard 1.
2025-12-20 23:30:41,582 - INFO - [export_dashboard][Exit] Exported dashboard 1 to dashboard_export_20251220T203040.zip.
2025-12-20 23:30:41,582 - INFO - [save_and_unpack_dashboard][Enter] Processing dashboard. Unpack: False
2025-12-20 23:30:41,749 - INFO - [save_and_unpack_dashboard][State] Dashboard saved to: backups/SUPERSET/World Bank's Data/dashboard_export_20251220T203040.zip
2025-12-20 23:30:41,750 - INFO - [archive_exports][Enter] Managing archive in backups/SUPERSET/World Bank's Data
2025-12-20 23:30:41,752 - INFO - [consolidate_archive_folders][Enter] Consolidating archives in backups/SUPERSET
2025-12-20 23:30:41,753 - INFO - [remove_empty_directories][Enter] Starting cleanup of empty directories in backups/SUPERSET
2025-12-20 23:30:41,758 - INFO - [remove_empty_directories][Exit] Removed 0 empty directories.
2025-12-20 23:30:41,758 - INFO - [BackupPlugin][CoherenceCheck:Passed] Backup logic completed for superset.

Binary file not shown.

View File

@@ -1 +1 @@
from . import plugins, tasks, settings, connections, environments, mappings, migration, git
from . import plugins, tasks, settings, connections, environments, mappings, migration, git, storage

View File

@@ -23,7 +23,7 @@ router = APIRouter()
# [DEF:ScheduleSchema:DataClass]
class ScheduleSchema(BaseModel):
enabled: bool = False
cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){5,7})$')
cron_expression: str = Field(..., pattern=r'^(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|((((\d+,)*\d+|(\d+(\/|-)\d+)|\d+|\*) ?){4,6})$')
# [/DEF:ScheduleSchema:DataClass]
# [DEF:EnvironmentResponse:DataClass]

View File

@@ -13,6 +13,7 @@
from fastapi import APIRouter, Depends, HTTPException
from typing import List
from ...core.config_models import AppConfig, Environment, GlobalSettings
from ...models.storage import StorageConfig
from ...dependencies import get_config_manager
from ...core.config_manager import ConfigManager
from ...core.logger import logger, belief_scope
@@ -52,10 +53,38 @@ async def update_global_settings(
):
with belief_scope("update_global_settings"):
logger.info("[update_global_settings][Entry] Updating global settings")
config_manager.update_global_settings(settings)
return settings
# [/DEF:update_global_settings:Function]
# [DEF:get_storage_settings:Function]
# @PURPOSE: Retrieves storage-specific settings.
# @RETURN: StorageConfig - The storage configuration.
@router.get("/storage", response_model=StorageConfig)
async def get_storage_settings(config_manager: ConfigManager = Depends(get_config_manager)):
with belief_scope("get_storage_settings"):
return config_manager.get_config().settings.storage
# [/DEF:get_storage_settings:Function]
# [DEF:update_storage_settings:Function]
# @PURPOSE: Updates storage-specific settings.
# @PARAM: storage (StorageConfig) - The new storage settings.
# @POST: Storage settings are updated and saved.
# @RETURN: StorageConfig - The updated storage settings.
@router.put("/storage", response_model=StorageConfig)
async def update_storage_settings(storage: StorageConfig, config_manager: ConfigManager = Depends(get_config_manager)):
with belief_scope("update_storage_settings"):
is_valid, message = config_manager.validate_path(storage.root_path)
if not is_valid:
raise HTTPException(status_code=400, detail=message)
settings = config_manager.get_config().settings
settings.storage = storage
config_manager.update_global_settings(settings)
return config_manager.get_config().settings.storage
# [/DEF:update_storage_settings:Function]
# [DEF:get_environments:Function]
# @PURPOSE: Lists all configured Superset environments.
# @PRE: Config manager is available.
@@ -179,30 +208,5 @@ async def test_environment_connection(
return {"status": "error", "message": str(e)}
# [/DEF:test_environment_connection:Function]
# [DEF:validate_backup_path:Function]
# @PURPOSE: Validates if a backup path exists and is writable.
# @PRE: Path is provided in path_data.
# @POST: Returns success or error status.
# @PARAM: path (str) - The path to validate.
# @RETURN: dict - Validation result.
@router.post("/validate-path")
async def validate_backup_path(
path_data: dict,
config_manager: ConfigManager = Depends(get_config_manager)
):
with belief_scope("validate_backup_path"):
path = path_data.get("path")
if not path:
raise HTTPException(status_code=400, detail="Path is required")
logger.info(f"[validate_backup_path][Entry] Validating path: {path}")
valid, message = config_manager.validate_path(path)
if not valid:
return {"status": "error", "message": message}
return {"status": "success", "message": message}
# [/DEF:validate_backup_path:Function]
# [/DEF:SettingsRouter:Module]

View File

@@ -0,0 +1,132 @@
# [DEF:storage_routes:Module]
#
# @SEMANTICS: storage, files, upload, download, backup, repository
# @PURPOSE: API endpoints for file storage management (backups and repositories).
# @LAYER: API
# @RELATION: DEPENDS_ON -> backend.src.models.storage
#
# @INVARIANT: All paths must be validated against path traversal.
# [SECTION: IMPORTS]
from fastapi import APIRouter, Depends, UploadFile, File, Form, HTTPException
from fastapi.responses import FileResponse
from typing import List, Optional
from ...models.storage import StoredFile, FileCategory
from ...dependencies import get_plugin_loader
from ...plugins.storage.plugin import StoragePlugin
from ...core.logger import belief_scope
# [/SECTION]
router = APIRouter(tags=["storage"])
# [DEF:list_files:Function]
# @PURPOSE: List all files and directories in the storage system.
#
# @PRE: None.
# @POST: Returns a list of StoredFile objects.
#
# @PARAM: category (Optional[FileCategory]) - Filter by category.
# @PARAM: path (Optional[str]) - Subpath within the category.
# @RETURN: List[StoredFile] - List of files/directories.
#
# @RELATION: CALLS -> StoragePlugin.list_files
@router.get("/files", response_model=List[StoredFile])
async def list_files(
category: Optional[FileCategory] = None,
path: Optional[str] = None,
plugin_loader=Depends(get_plugin_loader)
):
with belief_scope("list_files"):
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
if not storage_plugin:
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
return storage_plugin.list_files(category, path)
# [/DEF:list_files:Function]
# [DEF:upload_file:Function]
# @PURPOSE: Upload a file to the storage system.
#
# @PRE: category must be a valid FileCategory.
# @PRE: file must be a valid UploadFile.
# @POST: Returns the StoredFile object of the uploaded file.
#
# @PARAM: category (FileCategory) - Target category.
# @PARAM: path (Optional[str]) - Target subpath.
# @PARAM: file (UploadFile) - The file content.
# @RETURN: StoredFile - Metadata of the uploaded file.
#
# @SIDE_EFFECT: Writes file to the filesystem.
#
# @RELATION: CALLS -> StoragePlugin.save_file
@router.post("/upload", response_model=StoredFile, status_code=201)
async def upload_file(
category: FileCategory = Form(...),
path: Optional[str] = Form(None),
file: UploadFile = File(...),
plugin_loader=Depends(get_plugin_loader)
):
with belief_scope("upload_file"):
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
if not storage_plugin:
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
try:
return await storage_plugin.save_file(file, category, path)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# [/DEF:upload_file:Function]
# [DEF:delete_file:Function]
# @PURPOSE: Delete a specific file or directory.
#
# @PRE: category must be a valid FileCategory.
# @POST: Item is removed from storage.
#
# @PARAM: category (FileCategory) - File category.
# @PARAM: path (str) - Relative path of the item.
# @RETURN: None
#
# @SIDE_EFFECT: Deletes item from the filesystem.
#
# @RELATION: CALLS -> StoragePlugin.delete_file
@router.delete("/files/{category}/{path:path}", status_code=204)
async def delete_file(category: FileCategory, path: str, plugin_loader=Depends(get_plugin_loader)):
with belief_scope("delete_file"):
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
if not storage_plugin:
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
try:
storage_plugin.delete_file(category, path)
except FileNotFoundError:
raise HTTPException(status_code=404, detail="File not found")
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# [/DEF:delete_file:Function]
# [DEF:download_file:Function]
# @PURPOSE: Retrieve a file for download.
#
# @PRE: category must be a valid FileCategory.
# @POST: Returns a FileResponse.
#
# @PARAM: category (FileCategory) - File category.
# @PARAM: path (str) - Relative path of the file.
# @RETURN: FileResponse - The file content.
#
# @RELATION: CALLS -> StoragePlugin.get_file_path
@router.get("/download/{category}/{path:path}")
async def download_file(category: FileCategory, path: str, plugin_loader=Depends(get_plugin_loader)):
with belief_scope("download_file"):
storage_plugin: StoragePlugin = plugin_loader.get_plugin("storage-manager")
if not storage_plugin:
raise HTTPException(status_code=500, detail="Storage plugin not loaded")
try:
abs_path = storage_plugin.get_file_path(category, path)
filename = Path(path).name
return FileResponse(path=abs_path, filename=filename)
except FileNotFoundError:
raise HTTPException(status_code=404, detail="File not found")
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# [/DEF:download_file:Function]
# [/DEF:storage_routes:Module]

View File

@@ -18,7 +18,7 @@ import os
from .dependencies import get_task_manager, get_scheduler_service
from .core.logger import logger, belief_scope
from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git
from .api.routes import plugins, tasks, settings, environments, mappings, migration, connections, git, storage
from .core.database import init_db
# [DEF:App:Global]
@@ -89,6 +89,7 @@ app.include_router(environments.router, prefix="/api/environments", tags=["Envir
app.include_router(mappings.router)
app.include_router(migration.router)
app.include_router(git.router)
app.include_router(storage.router, prefix="/api/storage", tags=["Storage"])
# [DEF:websocket_endpoint:Function]
# @PURPOSE: Provides a WebSocket endpoint for real-time log streaming of a task.

View File

@@ -62,14 +62,18 @@ class ConfigManager:
logger.info(f"[_load_config][Action] Config file not found. Creating default.")
default_config = AppConfig(
environments=[],
settings=GlobalSettings(backup_path="backups")
settings=GlobalSettings()
)
self._save_config_to_disk(default_config)
return default_config
try:
with open(self.config_path, "r") as f:
data = json.load(f)
# Check for deprecated field
if "settings" in data and "backup_path" in data["settings"]:
del data["settings"]["backup_path"]
config = AppConfig(**data)
logger.info(f"[_load_config][Coherence:OK] Configuration loaded")
return config
@@ -79,7 +83,7 @@ class ConfigManager:
# For now, return default to be safe, but log the error prominently.
return AppConfig(
environments=[],
settings=GlobalSettings(backup_path="backups")
settings=GlobalSettings(storage=StorageConfig())
)
# [/DEF:_load_config:Function]

View File

@@ -7,6 +7,7 @@
from pydantic import BaseModel, Field
from typing import List, Optional
from ..models.storage import StorageConfig
# [DEF:Schedule:DataClass]
# @PURPOSE: Represents a backup schedule configuration.
@@ -42,7 +43,7 @@ class LoggingConfig(BaseModel):
# [DEF:GlobalSettings:DataClass]
# @PURPOSE: Represents global application settings.
class GlobalSettings(BaseModel):
backup_path: str
storage: StorageConfig = Field(default_factory=StorageConfig)
default_environment_id: Optional[str] = None
logging: LoggingConfig = Field(default_factory=LoggingConfig)

View File

@@ -28,6 +28,7 @@ class BeliefFormatter(logging.Formatter):
# @POST: Returns formatted string.
# @PARAM: record (logging.LogRecord) - The log record to format.
# @RETURN: str - The formatted log message.
# @SEMANTICS: logging, formatter, context
def format(self, record):
anchor_id = getattr(_belief_state, 'anchor_id', None)
if anchor_id:
@@ -54,6 +55,7 @@ class LogEntry(BaseModel):
# @PARAM: message (str) - Optional entry message.
# @PRE: anchor_id must be provided.
# @POST: Thread-local belief state is updated and entry/exit logs are generated.
# @SEMANTICS: logging, context, belief_state
@contextmanager
def belief_scope(anchor_id: str, message: str = ""):
# Log Entry if enabled
@@ -88,6 +90,7 @@ def belief_scope(anchor_id: str, message: str = ""):
# @PRE: config is a valid LoggingConfig instance.
# @POST: Logger level, handlers, and belief state flag are updated.
# @PARAM: config (LoggingConfig) - The logging configuration.
# @SEMANTICS: logging, configuration, initialization
def configure_logger(config):
global _enable_belief_state
_enable_belief_state = config.enable_belief_state
@@ -140,6 +143,7 @@ class WebSocketLogHandler(logging.Handler):
# @PRE: capacity is an integer.
# @POST: Instance initialized with empty deque.
# @PARAM: capacity (int) - Maximum number of logs to keep in memory.
# @SEMANTICS: logging, initialization, buffer
def __init__(self, capacity: int = 1000):
super().__init__()
self.log_buffer: deque[LogEntry] = deque(maxlen=capacity)
@@ -152,6 +156,7 @@ class WebSocketLogHandler(logging.Handler):
# @PRE: record is a logging.LogRecord.
# @POST: Log is added to the log_buffer.
# @PARAM: record (logging.LogRecord) - The log record to emit.
# @SEMANTICS: logging, handler, buffer
def emit(self, record: logging.LogRecord):
try:
log_entry = LogEntry(
@@ -179,6 +184,7 @@ class WebSocketLogHandler(logging.Handler):
# @PRE: None.
# @POST: Returns list of LogEntry objects.
# @RETURN: List[LogEntry] - List of buffered log entries.
# @SEMANTICS: logging, buffer, retrieval
def get_recent_logs(self) -> List[LogEntry]:
"""
Returns a list of recent log entries from the buffer.
@@ -196,12 +202,18 @@ logger = logging.getLogger("superset_tools_app")
# [DEF:believed:Function]
# @PURPOSE: A decorator that wraps a function in a belief scope.
# @PARAM: anchor_id (str) - The identifier for the semantic block.
# @PRE: anchor_id must be a string.
# @POST: Returns a decorator function.
def believed(anchor_id: str):
# [DEF:decorator:Function]
# @PURPOSE: Internal decorator for belief scope.
# @PRE: func must be a callable.
# @POST: Returns the wrapped function.
def decorator(func):
# [DEF:wrapper:Function]
# @PURPOSE: Internal wrapper that enters belief scope.
# @PRE: None.
# @POST: Executes the function within a belief scope.
def wrapper(*args, **kwargs):
with belief_scope(anchor_id):
return func(*args, **kwargs)

View File

@@ -1,5 +1,5 @@
from abc import ABC, abstractmethod
from typing import Dict, Any
from typing import Dict, Any, Optional
from .logger import belief_scope
from pydantic import BaseModel, Field
@@ -68,6 +68,21 @@ class PluginBase(ABC):
pass
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the plugin's UI, if applicable.
# @PRE: Plugin instance exists.
# @POST: Returns string route or None.
# @RETURN: Optional[str] - Frontend route.
def ui_route(self) -> Optional[str]:
"""
The frontend route for the plugin's UI.
Returns None if the plugin does not have a dedicated UI page.
"""
with belief_scope("ui_route"):
return None
# [/DEF:ui_route:Function]
@abstractmethod
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for the plugin's input parameters.
@@ -111,5 +126,6 @@ class PluginConfig(BaseModel):
name: str = Field(..., description="Human-readable name for the plugin")
description: str = Field(..., description="Brief description of what the plugin does")
version: str = Field(..., description="Version of the plugin")
ui_route: Optional[str] = Field(None, description="Frontend route for the plugin UI")
input_schema: Dict[str, Any] = Field(..., description="JSON schema for input parameters", alias="schema")
# [/DEF:PluginConfig:Class]

View File

@@ -50,9 +50,18 @@ class PluginLoader:
sys.path.insert(0, plugin_parent_dir)
for filename in os.listdir(self.plugin_dir):
file_path = os.path.join(self.plugin_dir, filename)
# Handle directory-based plugins (packages)
if os.path.isdir(file_path):
init_file = os.path.join(file_path, "__init__.py")
if os.path.exists(init_file):
self._load_module(filename, init_file)
continue
# Handle single-file plugins
if filename.endswith(".py") and filename != "__init__.py":
module_name = filename[:-3]
file_path = os.path.join(self.plugin_dir, filename)
self._load_module(module_name, file_path)
# [/DEF:_load_plugins:Function]
@@ -132,6 +141,7 @@ class PluginLoader:
name=plugin_instance.name,
description=plugin_instance.description,
version=plugin_instance.version,
ui_route=plugin_instance.ui_route,
schema=schema,
)
# The following line is commented out because it requires a schema to be passed to validate against.

View File

@@ -0,0 +1,31 @@
from datetime import datetime
from enum import Enum
from typing import Optional
from pydantic import BaseModel, Field
# [DEF:FileCategory:Class]
# @PURPOSE: Enumeration of supported file categories in the storage system.
class FileCategory(str, Enum):
BACKUP = "backups"
REPOSITORY = "repositorys"
# [/DEF:FileCategory:Class]
# [DEF:StorageConfig:Class]
# @PURPOSE: Configuration model for the storage system, defining paths and naming patterns.
class StorageConfig(BaseModel):
root_path: str = Field(default="backups", description="Absolute path to the storage root directory.")
backup_structure_pattern: str = Field(default="{category}/", description="Pattern for backup directory structure.")
repo_structure_pattern: str = Field(default="{category}/", description="Pattern for repository directory structure.")
filename_pattern: str = Field(default="{name}_{timestamp}", description="Pattern for filenames.")
# [/DEF:StorageConfig:Class]
# [DEF:StoredFile:Class]
# @PURPOSE: Data model representing metadata for a file stored in the system.
class StoredFile(BaseModel):
name: str = Field(..., description="Name of the file (including extension).")
path: str = Field(..., description="Relative path from storage root.")
size: int = Field(..., ge=0, description="Size of the file in bytes.")
created_at: datetime = Field(..., description="Creation timestamp.")
category: FileCategory = Field(..., description="Category of the file.")
mime_type: Optional[str] = Field(None, description="MIME type of the file.")
# [/DEF:StoredFile:Class]

View File

@@ -75,6 +75,15 @@ class BackupPlugin(PluginBase):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the backup plugin.
# @RETURN: str - "/tools/backups"
def ui_route(self) -> str:
with belief_scope("ui_route"):
return "/tools/backups"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for backup plugin parameters.
# @PRE: Plugin instance exists.
@@ -84,7 +93,7 @@ class BackupPlugin(PluginBase):
with belief_scope("get_schema"):
config_manager = get_config_manager()
envs = [e.name for e in config_manager.get_environments()]
default_path = config_manager.get_config().settings.backup_path
default_path = config_manager.get_config().settings.storage.root_path
return {
"type": "object",
@@ -95,14 +104,8 @@ class BackupPlugin(PluginBase):
"description": "The Superset environment to back up.",
"enum": envs if envs else [],
},
"backup_path": {
"type": "string",
"title": "Backup Path",
"description": "The root directory to save backups to.",
"default": default_path
}
},
"required": ["env", "backup_path"],
"required": ["env"],
}
# [/DEF:get_schema:Function]
@@ -126,8 +129,9 @@ class BackupPlugin(PluginBase):
if not env:
raise KeyError("env")
backup_path_str = params.get("backup_path") or config_manager.get_config().settings.backup_path
backup_path = Path(backup_path_str)
storage_settings = config_manager.get_config().settings.storage
# Use 'backups' subfolder within the storage root
backup_path = Path(storage_settings.root_path) / "backups"
from ..core.logger import logger as app_logger
app_logger.info(f"[BackupPlugin][Entry] Starting backup for {env}.")

View File

@@ -63,6 +63,15 @@ class DebugPlugin(PluginBase):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the debug plugin.
# @RETURN: str - "/tools/debug"
def ui_route(self) -> str:
with belief_scope("ui_route"):
return "/tools/debug"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for the debug plugin parameters.
# @PRE: Plugin instance exists.

View File

@@ -99,6 +99,15 @@ class GitPlugin(PluginBase):
return "0.1.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the git plugin.
# @RETURN: str - "/git"
def ui_route(self) -> str:
with belief_scope("GitPlugin.ui_route"):
return "/git"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Возвращает JSON-схему параметров для выполнения задач плагина.
# @PRE: GitPlugin is initialized.

View File

@@ -66,6 +66,15 @@ class MapperPlugin(PluginBase):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the mapper plugin.
# @RETURN: str - "/tools/mapper"
def ui_route(self) -> str:
with belief_scope("ui_route"):
return "/tools/mapper"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for the mapper plugin parameters.
# @PRE: Plugin instance exists.

View File

@@ -71,6 +71,15 @@ class MigrationPlugin(PluginBase):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the migration plugin.
# @RETURN: str - "/migration"
def ui_route(self) -> str:
with belief_scope("ui_route"):
return "/migration"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for migration plugin parameters.
# @PRE: Config manager is available.

View File

@@ -64,6 +64,15 @@ class SearchPlugin(PluginBase):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the search plugin.
# @RETURN: str - "/tools/search"
def ui_route(self) -> str:
with belief_scope("ui_route"):
return "/tools/search"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for the search plugin parameters.
# @PRE: Plugin instance exists.

View File

@@ -0,0 +1,3 @@
from .plugin import StoragePlugin
__all__ = ["StoragePlugin"]

View File

@@ -0,0 +1,333 @@
# [DEF:StoragePlugin:Module]
#
# @SEMANTICS: storage, files, filesystem, plugin
# @PURPOSE: Provides core filesystem operations for managing backups and repositories.
# @LAYER: App
# @RELATION: IMPLEMENTS -> PluginBase
# @RELATION: DEPENDS_ON -> backend.src.models.storage
#
# @INVARIANT: All file operations must be restricted to the configured storage root.
# [SECTION: IMPORTS]
import os
import shutil
from pathlib import Path
from datetime import datetime
from typing import Dict, Any, List, Optional
from fastapi import UploadFile
from ...core.plugin_base import PluginBase
from ...core.logger import belief_scope, logger
from ...models.storage import StoredFile, FileCategory, StorageConfig
from ...dependencies import get_config_manager
# [/SECTION]
# [DEF:StoragePlugin:Class]
# @PURPOSE: Implementation of the storage management plugin.
class StoragePlugin(PluginBase):
"""
Plugin for managing local file storage for backups and repositories.
"""
# [DEF:__init__:Function]
# @PURPOSE: Initializes the StoragePlugin and ensures required directories exist.
# @PRE: Configuration manager must be accessible.
# @POST: Storage root and category directories are created on disk.
def __init__(self):
with belief_scope("StoragePlugin:init"):
self.ensure_directories()
# [/DEF:__init__:Function]
@property
# [DEF:id:Function]
# @PURPOSE: Returns the unique identifier for the storage plugin.
# @PRE: None.
# @POST: Returns the plugin ID string.
# @RETURN: str - "storage-manager"
def id(self) -> str:
with belief_scope("StoragePlugin:id"):
return "storage-manager"
# [/DEF:id:Function]
@property
# [DEF:name:Function]
# @PURPOSE: Returns the human-readable name of the storage plugin.
# @PRE: None.
# @POST: Returns the plugin name string.
# @RETURN: str - "Storage Manager"
def name(self) -> str:
with belief_scope("StoragePlugin:name"):
return "Storage Manager"
# [/DEF:name:Function]
@property
# [DEF:description:Function]
# @PURPOSE: Returns a description of the storage plugin.
# @PRE: None.
# @POST: Returns the plugin description string.
# @RETURN: str - Plugin description.
def description(self) -> str:
with belief_scope("StoragePlugin:description"):
return "Manages local file storage for backups and repositories."
# [/DEF:description:Function]
@property
# [DEF:version:Function]
# @PURPOSE: Returns the version of the storage plugin.
# @PRE: None.
# @POST: Returns the version string.
# @RETURN: str - "1.0.0"
def version(self) -> str:
with belief_scope("StoragePlugin:version"):
return "1.0.0"
# [/DEF:version:Function]
@property
# [DEF:ui_route:Function]
# @PURPOSE: Returns the frontend route for the storage plugin.
# @RETURN: str - "/tools/storage"
def ui_route(self) -> str:
with belief_scope("StoragePlugin:ui_route"):
return "/tools/storage"
# [/DEF:ui_route:Function]
# [DEF:get_schema:Function]
# @PURPOSE: Returns the JSON schema for storage plugin parameters.
# @PRE: None.
# @POST: Returns a dictionary representing the JSON schema.
# @RETURN: Dict[str, Any] - JSON schema.
def get_schema(self) -> Dict[str, Any]:
with belief_scope("StoragePlugin:get_schema"):
return {
"type": "object",
"properties": {
"category": {
"type": "string",
"enum": [c.value for c in FileCategory],
"title": "Category"
}
},
"required": ["category"]
}
# [/DEF:get_schema:Function]
# [DEF:execute:Function]
# @PURPOSE: Executes storage-related tasks (placeholder for PluginBase compliance).
# @PRE: params must match the plugin schema.
# @POST: Task is executed and logged.
async def execute(self, params: Dict[str, Any]):
with belief_scope("StoragePlugin:execute"):
logger.info(f"[StoragePlugin][Action] Executing with params: {params}")
# [/DEF:execute:Function]
# [DEF:get_storage_root:Function]
# @PURPOSE: Resolves the absolute path to the storage root.
# @PRE: Settings must define a storage root path.
# @POST: Returns a Path object representing the storage root.
def get_storage_root(self) -> Path:
with belief_scope("StoragePlugin:get_storage_root"):
config_manager = get_config_manager()
global_settings = config_manager.get_config().settings
# Use storage.root_path as the source of truth for storage UI
root = Path(global_settings.storage.root_path)
if not root.is_absolute():
# Resolve relative to the backend directory
# Path(__file__) is backend/src/plugins/storage/plugin.py
# parents[3] is the project root (ss-tools)
# We need to ensure it's relative to where backend/ is
project_root = Path(__file__).parents[3]
root = (project_root / root).resolve()
return root
# [/DEF:get_storage_root:Function]
# [DEF:resolve_path:Function]
# @PURPOSE: Resolves a dynamic path pattern using provided variables.
# @PARAM: pattern (str) - The path pattern to resolve.
# @PARAM: variables (Dict[str, str]) - Variables to substitute in the pattern.
# @PRE: pattern must be a valid format string.
# @POST: Returns the resolved path string.
# @RETURN: str - The resolved path.
def resolve_path(self, pattern: str, variables: Dict[str, str]) -> str:
with belief_scope("StoragePlugin:resolve_path"):
# Add common variables
vars_with_defaults = {
"timestamp": datetime.now().strftime("%Y%m%dT%H%M%S"),
**variables
}
try:
resolved = pattern.format(**vars_with_defaults)
# Clean up any double slashes or leading/trailing slashes for relative path
return os.path.normpath(resolved).strip("/")
except KeyError as e:
logger.warning(f"[StoragePlugin][Coherence:Failed] Missing variable for path resolution: {e}")
# Fallback to literal pattern if formatting fails partially (or handle as needed)
return pattern.replace("{", "").replace("}", "")
# [/DEF:resolve_path:Function]
# [DEF:ensure_directories:Function]
# @PURPOSE: Creates the storage root and category subdirectories if they don't exist.
# @PRE: Storage root must be resolvable.
# @POST: Directories are created on the filesystem.
# @SIDE_EFFECT: Creates directories on the filesystem.
def ensure_directories(self):
with belief_scope("StoragePlugin:ensure_directories"):
root = self.get_storage_root()
for category in FileCategory:
# Use singular name for consistency with BackupPlugin and GitService
path = root / category.value
path.mkdir(parents=True, exist_ok=True)
logger.debug(f"[StoragePlugin][Action] Ensured directory: {path}")
# [/DEF:ensure_directories:Function]
# [DEF:validate_path:Function]
# @PURPOSE: Prevents path traversal attacks by ensuring the path is within the storage root.
# @PRE: path must be a Path object.
# @POST: Returns the resolved absolute path if valid, otherwise raises ValueError.
def validate_path(self, path: Path) -> Path:
with belief_scope("StoragePlugin:validate_path"):
root = self.get_storage_root().resolve()
resolved = path.resolve()
try:
resolved.relative_to(root)
except ValueError:
logger.error(f"[StoragePlugin][Coherence:Failed] Path traversal detected: {resolved} is not under {root}")
raise ValueError("Access denied: Path is outside of storage root.")
return resolved
# [/DEF:validate_path:Function]
# [DEF:list_files:Function]
# @PURPOSE: Lists all files and directories in a specific category and subpath.
# @PARAM: category (Optional[FileCategory]) - The category to list.
# @PARAM: subpath (Optional[str]) - Nested path within the category.
# @PRE: Storage root must exist.
# @POST: Returns a list of StoredFile objects.
# @RETURN: List[StoredFile] - List of file and directory metadata objects.
def list_files(self, category: Optional[FileCategory] = None, subpath: Optional[str] = None) -> List[StoredFile]:
with belief_scope("StoragePlugin:list_files"):
root = self.get_storage_root()
logger.info(f"[StoragePlugin][Action] Listing files in root: {root}, category: {category}, subpath: {subpath}")
files = []
categories = [category] if category else list(FileCategory)
for cat in categories:
# Scan the category subfolder + optional subpath
base_dir = root / cat.value
if subpath:
target_dir = self.validate_path(base_dir / subpath)
else:
target_dir = base_dir
if not target_dir.exists():
continue
logger.debug(f"[StoragePlugin][Action] Scanning directory: {target_dir}")
# Use os.scandir for better performance and to distinguish files vs dirs
with os.scandir(target_dir) as it:
for entry in it:
# Skip logs
if "Logs" in entry.path:
continue
stat = entry.stat()
is_dir = entry.is_dir()
files.append(StoredFile(
name=entry.name,
path=str(Path(entry.path).relative_to(root)),
size=stat.st_size if not is_dir else 0,
created_at=datetime.fromtimestamp(stat.st_ctime),
category=cat,
mime_type="directory" if is_dir else None
))
# Sort: directories first, then by name
return sorted(files, key=lambda x: (x.mime_type != "directory", x.name))
# [/DEF:list_files:Function]
# [DEF:save_file:Function]
# @PURPOSE: Saves an uploaded file to the specified category and optional subpath.
# @PARAM: file (UploadFile) - The uploaded file.
# @PARAM: category (FileCategory) - The target category.
# @PARAM: subpath (Optional[str]) - The target subpath.
# @PRE: file must be a valid UploadFile; category must be valid.
# @POST: File is written to disk and metadata is returned.
# @RETURN: StoredFile - Metadata of the saved file.
# @SIDE_EFFECT: Writes file to disk.
async def save_file(self, file: UploadFile, category: FileCategory, subpath: Optional[str] = None) -> StoredFile:
with belief_scope("StoragePlugin:save_file"):
root = self.get_storage_root()
dest_dir = root / category.value
if subpath:
dest_dir = dest_dir / subpath
dest_dir.mkdir(parents=True, exist_ok=True)
dest_path = self.validate_path(dest_dir / file.filename)
with dest_path.open("wb") as buffer:
shutil.copyfileobj(file.file, buffer)
stat = dest_path.stat()
return StoredFile(
name=dest_path.name,
path=str(dest_path.relative_to(root)),
size=stat.st_size,
created_at=datetime.fromtimestamp(stat.st_ctime),
category=category,
mime_type=file.content_type
)
# [/DEF:save_file:Function]
# [DEF:delete_file:Function]
# @PURPOSE: Deletes a file or directory from the specified category and path.
# @PARAM: category (FileCategory) - The category.
# @PARAM: path (str) - The relative path of the file or directory.
# @PRE: path must belong to the specified category and exist on disk.
# @POST: The file or directory is removed from disk.
# @SIDE_EFFECT: Removes item from disk.
def delete_file(self, category: FileCategory, path: str):
with belief_scope("StoragePlugin:delete_file"):
root = self.get_storage_root()
# path is relative to root, but we ensure it starts with category
full_path = self.validate_path(root / path)
if not str(Path(path)).startswith(category.value):
raise ValueError(f"Path {path} does not belong to category {category}")
if full_path.exists():
if full_path.is_dir():
shutil.rmtree(full_path)
else:
full_path.unlink()
logger.info(f"[StoragePlugin][Action] Deleted: {full_path}")
else:
raise FileNotFoundError(f"Item {path} not found")
# [/DEF:delete_file:Function]
# [DEF:get_file_path:Function]
# @PURPOSE: Returns the absolute path of a file for download.
# @PARAM: category (FileCategory) - The category.
# @PARAM: path (str) - The relative path of the file.
# @PRE: path must belong to the specified category and be a file.
# @POST: Returns the absolute Path to the file.
# @RETURN: Path - Absolute path to the file.
def get_file_path(self, category: FileCategory, path: str) -> Path:
with belief_scope("StoragePlugin:get_file_path"):
root = self.get_storage_root()
file_path = self.validate_path(root / path)
if not str(Path(path)).startswith(category.value):
raise ValueError(f"Path {path} does not belong to category {category}")
if not file_path.exists() or file_path.is_dir():
raise FileNotFoundError(f"File {path} not found")
return file_path
# [/DEF:get_file_path:Function]
# [/DEF:StoragePlugin:Class]
# [/DEF:StoragePlugin:Module]

View File

@@ -31,9 +31,15 @@ class GitService:
# @PARAM: base_path (str) - Root directory for all Git clones.
# @PRE: base_path is a valid string path.
# @POST: GitService is initialized; base_path directory exists.
def __init__(self, base_path: str = "backend/git_repos"):
def __init__(self, base_path: str = "git_repos"):
with belief_scope("GitService.__init__"):
self.base_path = base_path
# Resolve relative to the backend directory
# Path(__file__) is backend/src/services/git_service.py
# parents[2] is backend/
from pathlib import Path
backend_root = Path(__file__).parents[2]
self.base_path = str((backend_root / base_path).resolve())
if not os.path.exists(self.base_path):
os.makedirs(self.base_path)
# [/DEF:__init__:Function]

Binary file not shown.

View File

@@ -13,7 +13,7 @@ The settings mechanism allows users to configure multiple Superset environments
Configuration is structured using Pydantic models in `backend/src/core/config_models.py`:
- `Environment`: Represents a Superset instance (URL, credentials). The `base_url` is automatically normalized to include the `/api/v1` suffix if missing.
- `GlobalSettings`: Global application parameters (e.g., `backup_path`).
- `GlobalSettings`: Global application parameters (e.g., `storage.root_path`).
- `AppConfig`: The root configuration object.
### Configuration Manager
@@ -43,4 +43,4 @@ The settings page is located at `frontend/src/pages/Settings.svelte`. It provide
Existing plugins and utilities use the `ConfigManager` to fetch configuration:
- `superset_tool/utils/init_clients.py`: Dynamically initializes Superset clients from the configured environments.
- `BackupPlugin`: Uses the configured `backup_path` as the default storage location.
- `BackupPlugin`: Uses the configured `storage.root_path` as the default storage location.

View File

@@ -57,4 +57,4 @@
/* Component specific styles */
</style>
<!-- [/DEF:EnvSelector:Component] -->
<!-- [/DEF:EnvSelector:Component] -->

View File

@@ -25,34 +25,12 @@
>
{$t.nav.dashboard}
</a>
<a
href="/migration"
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/migration') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
>
{$t.nav.migration}
</a>
<a
href="/git"
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/git') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
>
{$t.nav.git}
</a>
<a
href="/tasks"
class="text-gray-600 hover:text-blue-600 font-medium {$page.url.pathname.startsWith('/tasks') ? 'text-blue-600 border-b-2 border-blue-600' : ''}"
>
{$t.nav.tasks}
</a>
<div class="relative inline-block group">
<button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/tools') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
{$t.nav.tools}
</button>
<div class="absolute hidden group-hover:block bg-white shadow-lg rounded-md mt-1 py-2 w-48 z-10 border border-gray-100 before:absolute before:-top-2 before:left-0 before:right-0 before:h-2 before:content-[''] right-0">
<a href="/tools/search" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_search}</a>
<a href="/tools/mapper" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_mapper}</a>
<a href="/tools/debug" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.tools_debug}</a>
</div>
</div>
<div class="relative inline-block group">
<button class="text-gray-600 hover:text-blue-600 font-medium pb-1 {$page.url.pathname.startsWith('/settings') ? 'text-blue-600 border-b-2 border-blue-600' : ''}">
{$t.nav.settings}
@@ -61,7 +39,6 @@
<a href="/settings" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_general}</a>
<a href="/settings/connections" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_connections}</a>
<a href="/settings/git" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_git}</a>
<a href="/settings/environments" class="block px-4 py-2 text-sm text-gray-700 hover:bg-blue-50 hover:text-blue-600">{$t.nav.settings_environments}</a>
</div>
</div>
<LanguageSwitcher />

View File

@@ -0,0 +1,84 @@
<!-- [DEF:BackupList:Component] -->
<!--
@SEMANTICS: backup, list, table
@PURPOSE: Displays a list of existing backups.
@LAYER: Component
@RELATION: USED_BY -> frontend/src/components/backups/BackupManager.svelte
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { t } from '../../lib/i18n';
import { Button } from '../../lib/ui';
import type { Backup } from '../../types/backup';
import { goto } from '$app/navigation';
// [/SECTION]
// [SECTION: PROPS]
/**
* @type {Backup[]}
* @description Array of backup objects to display.
*/
export let backups: Backup[] = [];
// [/SECTION]
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="overflow-x-auto rounded-lg border border-gray-200">
<table class="min-w-full divide-y divide-gray-200">
<thead class="bg-gray-50">
<tr>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
{$t.storage.table.name}
</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
{$t.tasks.target_env}
</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
{$t.storage.table.created_at}
</th>
<th class="px-6 py-3 text-right text-xs font-medium text-gray-500 uppercase tracking-wider">
{$t.storage.table.actions}
</th>
</tr>
</thead>
<tbody class="bg-white divide-y divide-gray-200">
{#each backups as backup}
<tr class="hover:bg-gray-50 transition-colors">
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">
{backup.name}
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{backup.environment}
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{new Date(backup.created_at).toLocaleString()}
</td>
<td class="px-6 py-4 whitespace-nowrap text-right text-sm font-medium">
<Button
variant="ghost"
size="sm"
class="text-blue-600 hover:text-blue-900"
on:click={() => goto(`/tools/storage?path=backups/${backup.name}`)}
>
{$t.storage.table.go_to_storage}
</Button>
</td>
</tr>
{:else}
<tr>
<td colspan="4" class="px-6 py-10 text-center text-gray-500">
{$t.storage.no_files}
</td>
</tr>
{/each}
</tbody>
</table>
</div>
<!-- [/SECTION] -->
<style>
</style>
<!-- [/DEF:BackupList:Component] -->

View File

@@ -0,0 +1,241 @@
<!-- [DEF:BackupManager:Component] -->
<!--
@SEMANTICS: backup, manager, orchestrator
@PURPOSE: Main container for backup management, handling creation and listing.
@LAYER: Feature
@RELATION: USES -> BackupList
@RELATION: USES -> api
@INVARIANT: Only one backup task can be triggered at a time from the UI.
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { onMount } from 'svelte';
import { t } from '../../lib/i18n';
import { api, requestApi } from '../../lib/api';
import { addToast } from '../../lib/toasts';
import { Button, Card, Select, Input } from '../../lib/ui';
import BackupList from './BackupList.svelte';
import type { Backup } from '../../types/backup';
// [/SECTION]
// [SECTION: STATE]
let backups: Backup[] = [];
let environments: any[] = [];
let selectedEnvId = '';
let loading = true;
let creating = false;
let savingSchedule = false;
// Schedule state for selected environment
let scheduleEnabled = false;
let cronExpression = '0 0 * * *';
$: selectedEnv = environments.find(e => e.id === selectedEnvId);
$: if (selectedEnv) {
scheduleEnabled = selectedEnv.backup_schedule?.enabled ?? false;
cronExpression = selectedEnv.backup_schedule?.cron_expression ?? '0 0 * * *';
}
// [/SECTION]
// [DEF:loadData:Function]
/**
* @purpose Loads backups and environments from the backend.
*
* @pre API must be reachable.
* @post environments and backups stores are populated.
*
* @returns {Promise<void>}
* @side_effect Updates local state variables.
*/
// @RELATION: CALLS -> api.getEnvironmentsList
// @RELATION: CALLS -> api.requestApi
async function loadData() {
console.log("[BackupManager][Entry] Loading data.");
loading = true;
try {
const [envsData, storageData] = await Promise.all([
api.getEnvironmentsList(),
requestApi('/storage/files?category=backups')
]);
environments = envsData;
// Map storage files to Backup type
backups = (storageData || []).map((file: any) => ({
id: file.name,
name: file.name,
environment: file.path.split('/')[0] || 'Unknown',
created_at: file.created_at,
size_bytes: file.size,
status: 'success'
}));
console.log("[BackupManager][Action] Data loaded successfully.");
} catch (error) {
console.error("[BackupManager][Coherence:Failed] Load failed", error);
} finally {
loading = false;
}
}
// [/DEF:loadData:Function]
// [DEF:handleCreateBackup:Function]
/**
* @purpose Triggers a new backup task for the selected environment.
*
* @pre selectedEnvId must be a valid environment ID.
* @post A new task is created on the backend.
*
* @returns {Promise<void>}
* @side_effect Dispatches a toast notification.
*/
// @RELATION: CALLS -> api.createTask
// [DEF:handleUpdateSchedule:Function]
/**
* @purpose Updates the backup schedule for the selected environment.
* @pre selectedEnvId must be set.
* @post Environment config is updated on the backend.
*/
async function handleUpdateSchedule() {
if (!selectedEnvId) return;
console.log(`[BackupManager][Action] Updating schedule for env: ${selectedEnvId}`);
savingSchedule = true;
try {
await api.updateEnvironmentSchedule(selectedEnvId, {
enabled: scheduleEnabled,
cron_expression: cronExpression
});
addToast($t.common.success, 'success');
// Update local state
environments = environments.map(e =>
e.id === selectedEnvId
? { ...e, backup_schedule: { enabled: scheduleEnabled, cron_expression: cronExpression } }
: e
);
} catch (error) {
console.error("[BackupManager][Coherence:Failed] Schedule update failed", error);
} finally {
savingSchedule = false;
}
}
// [/DEF:handleUpdateSchedule:Function]
async function handleCreateBackup() {
if (!selectedEnvId) {
addToast($t.tasks.select_env, 'error');
return;
}
console.log(`[BackupManager][Action] Triggering backup for env: ${selectedEnvId}`);
creating = true;
try {
await api.createTask('superset-backup', { environment_id: selectedEnvId });
addToast($t.common.success, 'success');
console.log("[BackupManager][Coherence:OK] Backup task triggered.");
} catch (error) {
console.error("[BackupManager][Coherence:Failed] Create failed", error);
} finally {
creating = false;
}
}
// [/DEF:handleCreateBackup:Function]
onMount(loadData);
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="space-y-6">
<Card title={$t.tasks.manual_backup}>
<div class="space-y-4">
<div class="flex items-end gap-4">
<div class="flex-1">
<Select
label={$t.tasks.target_env}
bind:value={selectedEnvId}
options={[
{ value: '', label: $t.tasks.select_env },
...environments.map(e => ({ value: e.id, label: e.name }))
]}
/>
</div>
<Button
variant="primary"
on:click={handleCreateBackup}
disabled={creating || !selectedEnvId}
>
{creating ? $t.common.loading : $t.tasks.start_backup}
</Button>
</div>
{#if selectedEnvId}
<div class="pt-6 border-t border-gray-100 mt-4">
<h3 class="text-sm font-semibold text-gray-800 mb-4 flex items-center gap-2">
<svg xmlns="http://www.w3.org/2000/svg" class="h-4 w-4 text-blue-500" fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
{$t.tasks.backup_schedule}
</h3>
<div class="bg-gray-50 rounded-lg p-4 border border-gray-200">
<div class="flex flex-col md:flex-row md:items-start gap-6">
<div class="pt-8">
<label class="flex items-center gap-3 cursor-pointer group">
<div class="relative inline-flex items-center">
<input type="checkbox" bind:checked={scheduleEnabled} class="sr-only peer" />
<div class="w-11 h-6 bg-gray-200 peer-focus:outline-none peer-focus:ring-4 peer-focus:ring-blue-300 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:border-gray-300 after:border after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-blue-600"></div>
<span class="ml-3 text-sm font-medium text-gray-700 group-hover:text-gray-900 transition-colors">{$t.tasks.schedule_enabled}</span>
</div>
</label>
</div>
<div class="flex-1 space-y-2">
<Input
label={$t.tasks.cron_label}
placeholder="0 0 * * *"
bind:value={cronExpression}
disabled={!scheduleEnabled}
/>
<p class="text-xs text-gray-500 italic">{$t.tasks.cron_hint}</p>
</div>
<div class="pt-8">
<Button
variant="secondary"
on:click={handleUpdateSchedule}
disabled={savingSchedule}
class="min-w-[100px]"
>
{#if savingSchedule}
<span class="flex items-center gap-2">
<svg class="animate-spin h-4 w-4" viewBox="0 0 24 24">
<circle class="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4" fill="none"></circle>
<path class="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
{$t.common.loading}
</span>
{:else}
{$t.common.save}
{/if}
</Button>
</div>
</div>
</div>
</div>
{/if}
</div>
</Card>
<div class="space-y-3">
<h2 class="text-lg font-semibold text-gray-700">{$t.storage.backups}</h2>
{#if loading}
<div class="py-10 text-center text-gray-500">{$t.common.loading}</div>
{:else}
<BackupList {backups} />
{/if}
</div>
</div>
<!-- [/SECTION] -->
<!-- [/DEF:BackupManager:Component] -->

View File

@@ -0,0 +1,134 @@
<!-- [DEF:FileList:Component] -->
<!--
@SEMANTICS: storage, files, list, table
@PURPOSE: Displays a table of files with metadata and actions.
@LAYER: Component
@RELATION: DEPENDS_ON -> storageService
@PROPS: files (Array) - List of StoredFile objects.
@EVENTS: delete (filename) - Dispatched when a file is deleted.
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { createEventDispatcher } from 'svelte';
import { downloadFileUrl } from '../../services/storageService';
import { t } from '../../lib/i18n';
// [/SECTION: IMPORTS]
export let files = [];
const dispatch = createEventDispatcher();
// [DEF:isDirectory:Function]
/**
* @purpose Checks if a file object represents a directory.
* @param {Object} file - The file object to check.
* @return {boolean} True if it's a directory, false otherwise.
*/
function isDirectory(file) {
return file.mime_type === 'directory';
}
// [/DEF:isDirectory:Function]
// [DEF:formatSize:Function]
/**
* @purpose Formats file size in bytes into a human-readable string.
* @param {number} bytes - The size in bytes.
* @return {string} Formatted size (e.g., "1.2 MB").
*/
function formatSize(bytes) {
if (bytes === 0) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i];
}
// [/DEF:formatSize:Function]
// [DEF:formatDate:Function]
/**
* @purpose Formats an ISO date string into a localized readable format.
* @param {string} dateStr - The date string to format.
* @return {string} Localized date and time.
*/
function formatDate(dateStr) {
return new Date(dateStr).toLocaleString();
}
// [/DEF:formatDate:Function]
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="overflow-x-auto">
<table class="min-w-full bg-white border border-gray-200">
<thead class="bg-gray-50">
<tr>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.name}</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.category}</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.size}</th>
<th class="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.created_at}</th>
<th class="px-6 py-3 text-right text-xs font-medium text-gray-500 uppercase tracking-wider">{$t.storage.table.actions}</th>
</tr>
</thead>
<tbody class="divide-y divide-gray-200">
{#each files as file}
<tr class="hover:bg-gray-50">
<td class="px-6 py-4 whitespace-nowrap text-sm font-medium text-gray-900">
{#if isDirectory(file)}
<button
on:click={() => dispatch('navigate', file.path)}
class="flex items-center text-indigo-600 hover:text-indigo-900"
>
<svg class="h-5 w-5 mr-2 text-yellow-400" fill="currentColor" viewBox="0 0 20 20">
<path d="M2 6a2 2 0 012-2h5l2 2h5a2 2 0 012 2v6a2 2 0 01-2 2H4a2 2 0 01-2-2V6z" />
</svg>
{file.name}
</button>
{:else}
<div class="flex items-center">
<svg class="h-5 w-5 mr-2 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M7 21h10a2 2 0 002-2V9.414a1 1 0 00-.293-.707l-5.414-5.414A1 1 0 0012.586 3H7a2 2 0 00-2 2v14a2 2 0 002 2z" />
</svg>
{file.name}
</div>
{/if}
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500 capitalize">{file.category}</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{isDirectory(file) ? '--' : formatSize(file.size)}
</td>
<td class="px-6 py-4 whitespace-nowrap text-sm text-gray-500">{formatDate(file.created_at)}</td>
<td class="px-6 py-4 whitespace-nowrap text-right text-sm font-medium">
{#if !isDirectory(file)}
<a
href={downloadFileUrl(file.category, file.path)}
download={file.name}
class="text-indigo-600 hover:text-indigo-900 mr-4"
>
{$t.storage.table.download}
</a>
{/if}
<button
on:click={() => dispatch('delete', { category: file.category, path: file.path, name: file.name })}
class="text-red-600 hover:text-red-900"
>
{$t.storage.table.delete}
</button>
</td>
</tr>
{:else}
<tr>
<td colspan="5" class="px-6 py-10 text-center text-sm text-gray-500">
{$t.storage.no_files}
</td>
</tr>
{/each}
</tbody>
</table>
</div>
<!-- [/SECTION: TEMPLATE] -->
<style>
/* ... */
</style>
<!-- [/DEF:FileList:Component] -->

View File

@@ -0,0 +1,134 @@
<!-- [DEF:FileUpload:Component] -->
<!--
@SEMANTICS: storage, upload, files
@PURPOSE: Provides a form for uploading files to a specific category.
@LAYER: Component
@RELATION: DEPENDS_ON -> storageService
@PROPS: None
@EVENTS: uploaded - Dispatched when a file is successfully uploaded.
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { createEventDispatcher } from 'svelte';
import { uploadFile } from '../../services/storageService';
import { addToast } from '../../lib/toasts';
import { t } from '../../lib/i18n';
// [/SECTION: IMPORTS]
// [DEF:handleUpload:Function]
/**
* @purpose Handles the file upload process.
* @pre A file must be selected in the file input.
* @post The file is uploaded to the server and a success toast is shown.
*/
const dispatch = createEventDispatcher();
let fileInput;
export let category = 'backups';
export let path = '';
let isUploading = false;
let dragOver = false;
async function handleUpload() {
const file = fileInput.files[0];
if (!file) return;
isUploading = true;
try {
// path is relative to root, but upload endpoint expects path within category
// FileList.path is like "backup/folder", we need just "folder"
const subpath = path.startsWith(category)
? path.substring(category.length).replace(/^\/+/, '')
: path;
await uploadFile(file, category, subpath);
addToast($t.storage.messages.upload_success.replace('{name}', file.name), 'success');
fileInput.value = '';
dispatch('uploaded');
} catch (error) {
addToast($t.storage.messages.upload_failed.replace('{error}', error.message), 'error');
} finally {
isUploading = false;
}
}
// [/DEF:handleUpload:Function]
// [DEF:handleDrop:Function]
/**
* @purpose Handles the file drop event for drag-and-drop.
* @param {DragEvent} event - The drop event.
*/
function handleDrop(event) {
event.preventDefault();
dragOver = false;
const files = event.dataTransfer.files;
if (files.length > 0) {
fileInput.files = files;
handleUpload();
}
}
// [/DEF:handleDrop:Function]
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="bg-white p-6 rounded-lg border border-gray-200 shadow-sm">
<h2 class="text-lg font-semibold mb-4">{$t.storage.upload_title}</h2>
<div class="space-y-4">
<div>
<label class="block text-sm font-medium text-gray-700 mb-1">{$t.storage.target_category}</label>
<select
bind:value={category}
class="block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
>
<option value="backups">{$t.storage.backups}</option>
<option value="repositorys">{$t.storage.repositories}</option>
</select>
</div>
<div
class="mt-1 flex justify-center px-6 pt-5 pb-6 border-2 border-dashed rounded-md transition-colors
{dragOver ? 'border-indigo-500 bg-indigo-50' : 'border-gray-300'}"
on:dragover|preventDefault={() => dragOver = true}
on:dragleave|preventDefault={() => dragOver = false}
on:drop|preventDefault={handleDrop}
>
<div class="space-y-1 text-center">
<svg class="mx-auto h-12 w-12 text-gray-400" stroke="currentColor" fill="none" viewBox="0 0 48 48" aria-hidden="true">
<path d="M28 8H12a4 4 0 00-4 4v20m32-12v8m0 0v8a4 4 0 01-4 4H12a4 4 0 01-4-4v-4m32-4l-3.172-3.172a4 4 0 00-5.656 0L28 28M8 32l9.172-9.172a4 4 0 015.656 0L28 28m0 0l4 4m4-24h8m-4-4v8m-12 4h.02" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" />
</svg>
<div class="flex text-sm text-gray-600">
<label for="file-upload" class="relative cursor-pointer bg-white rounded-md font-medium text-indigo-600 hover:text-indigo-500 focus-within:outline-none focus-within:ring-2 focus-within:ring-offset-2 focus-within:ring-indigo-500">
<span>{$t.storage.upload_button}</span>
<input
id="file-upload"
name="file-upload"
type="file"
class="sr-only"
bind:this={fileInput}
on:change={handleUpload}
disabled={isUploading}
>
</label>
<p class="pl-1">{$t.storage.drag_drop}</p>
</div>
<p class="text-xs text-gray-500">{$t.storage.supported_formats}</p>
</div>
</div>
{#if isUploading}
<div class="flex items-center justify-center space-x-2 text-indigo-600">
<div class="animate-spin rounded-full h-4 w-4 border-b-2 border-indigo-600"></div>
<span class="text-sm font-medium">{$t.storage.uploading}</span>
</div>
{/if}
</div>
</div>
<!-- [/SECTION: TEMPLATE] -->
<style>
/* ... */
</style>
<!-- [/DEF:FileUpload:Component] -->

View File

@@ -13,6 +13,8 @@
import { getConnections } from '../../services/connectionService.js';
import { selectedTask } from '../../lib/stores.js';
import { addToast } from '../../lib/toasts.js';
import { t } from '../../lib/i18n';
import { Button, Card, Select, Input } from '../../lib/ui';
// [/SECTION]
let envs = [];
@@ -36,7 +38,7 @@
envs = await envsRes.json();
connections = await getConnections();
} catch (e) {
addToast('Failed to fetch data', 'error');
addToast($t.mapper.errors.fetch_failed, 'error');
}
}
// [/DEF:fetchData:Function]
@@ -47,17 +49,17 @@
// @POST: Mapper task is started and selectedTask is updated.
async function handleRunMapper() {
if (!selectedEnv || !datasetId) {
addToast('Please fill in required fields', 'warning');
addToast($t.mapper.errors.required_fields, 'warning');
return;
}
if (source === 'postgres' && (!selectedConnection || !tableName)) {
addToast('Connection and Table Name are required for postgres source', 'warning');
addToast($t.mapper.errors.postgres_required, 'warning');
return;
}
if (source === 'excel' && !excelPath) {
addToast('Excel path is required for excel source', 'warning');
addToast($t.mapper.errors.excel_required, 'warning');
return;
}
@@ -75,7 +77,7 @@
});
selectedTask.set(task);
addToast('Mapper task started', 'success');
addToast($t.mapper.success.started, 'success');
} catch (e) {
addToast(e.message, 'error');
} finally {
@@ -88,78 +90,94 @@
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="bg-white p-6 rounded-lg shadow-sm border border-gray-200">
<h3 class="text-lg font-medium text-gray-900 mb-4">Dataset Column Mapper</h3>
<div class="space-y-4">
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<div>
<label for="mapper-env" class="block text-sm font-medium text-gray-700">Environment</label>
<select id="mapper-env" bind:value={selectedEnv} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
<option value="" disabled>-- Select Environment --</option>
{#each envs as env}
<option value={env.id}>{env.name}</option>
{/each}
</select>
</div>
<div>
<label for="mapper-ds-id" class="block text-sm font-medium text-gray-700">Dataset ID</label>
<input type="number" id="mapper-ds-id" bind:value={datasetId} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
</div>
</div>
<div>
<label class="block text-sm font-medium text-gray-700">Mapping Source</label>
<div class="mt-2 flex space-x-4">
<label class="inline-flex items-center">
<input type="radio" bind:group={source} value="postgres" class="focus:ring-indigo-500 h-4 w-4 text-indigo-600 border-gray-300" />
<span class="ml-2 text-sm text-gray-700">PostgreSQL</span>
</label>
<label class="inline-flex items-center">
<input type="radio" bind:group={source} value="excel" class="focus:ring-indigo-500 h-4 w-4 text-indigo-600 border-gray-300" />
<span class="ml-2 text-sm text-gray-700">Excel</span>
</label>
</div>
</div>
{#if source === 'postgres'}
<div class="space-y-4 p-4 bg-gray-50 rounded-md border border-gray-100">
<div class="space-y-6">
<Card title={$t.mapper.title}>
<div class="space-y-4">
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<div>
<label for="mapper-conn" class="block text-sm font-medium text-gray-700">Saved Connection</label>
<select id="mapper-conn" bind:value={selectedConnection} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm">
<option value="" disabled>-- Select Connection --</option>
{#each connections as conn}
<option value={conn.id}>{conn.name}</option>
{/each}
</select>
<Select
label={$t.mapper.environment}
bind:value={selectedEnv}
options={[
{ value: '', label: $t.mapper.select_env },
...envs.map(e => ({ value: e.id, label: e.name }))
]}
/>
</div>
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<div>
<label for="mapper-table" class="block text-sm font-medium text-gray-700">Table Name</label>
<input type="text" id="mapper-table" bind:value={tableName} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
</div>
<div>
<label for="mapper-schema" class="block text-sm font-medium text-gray-700">Table Schema</label>
<input type="text" id="mapper-schema" bind:value={tableSchema} class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
</div>
<div>
<Input
label={$t.mapper.dataset_id}
type="number"
bind:value={datasetId}
/>
</div>
</div>
{:else}
<div class="p-4 bg-gray-50 rounded-md border border-gray-100">
<label for="mapper-excel" class="block text-sm font-medium text-gray-700">Excel File Path</label>
<input type="text" id="mapper-excel" bind:value={excelPath} placeholder="/path/to/mapping.xlsx" class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm" />
</div>
{/if}
<div class="flex justify-end">
<button
on:click={handleRunMapper}
disabled={isRunning}
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500 disabled:opacity-50"
>
{isRunning ? 'Starting...' : 'Run Mapper'}
</button>
<div>
<label class="block text-sm font-medium text-gray-700 mb-2">{$t.mapper.source}</label>
<div class="flex space-x-4">
<label class="inline-flex items-center">
<input type="radio" bind:group={source} value="postgres" class="focus:ring-blue-500 h-4 w-4 text-blue-600 border-gray-300" />
<span class="ml-2 text-sm text-gray-700">{$t.mapper.source_postgres}</span>
</label>
<label class="inline-flex items-center">
<input type="radio" bind:group={source} value="excel" class="focus:ring-blue-500 h-4 w-4 text-blue-600 border-gray-300" />
<span class="ml-2 text-sm text-gray-700">{$t.mapper.source_excel}</span>
</label>
</div>
</div>
{#if source === 'postgres'}
<div class="space-y-4 p-4 bg-gray-50 rounded-md border border-gray-100">
<div>
<Select
label={$t.mapper.connection}
bind:value={selectedConnection}
options={[
{ value: '', label: $t.mapper.select_connection },
...connections.map(c => ({ value: c.id, label: c.name }))
]}
/>
</div>
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">
<div>
<Input
label={$t.mapper.table_name}
type="text"
bind:value={tableName}
/>
</div>
<div>
<Input
label={$t.mapper.table_schema}
type="text"
bind:value={tableSchema}
/>
</div>
</div>
</div>
{:else}
<div class="p-4 bg-gray-50 rounded-md border border-gray-100">
<Input
label={$t.mapper.excel_path}
type="text"
bind:value={excelPath}
placeholder="/path/to/mapping.xlsx"
/>
</div>
{/if}
<div class="flex justify-end pt-2">
<Button
variant="primary"
on:click={handleRunMapper}
disabled={isRunning}
>
{isRunning ? $t.mapper.starting : $t.mapper.run}
</Button>
</div>
</div>
</div>
</Card>
</div>
<!-- [/SECTION] -->
<!-- [/DEF:MapperTool:Component] -->

View File

@@ -1,186 +0,0 @@
<!-- [DEF:SearchTool:Component] -->
<!--
@SEMANTICS: search, tool, dataset, regex
@PURPOSE: UI component for searching datasets using the SearchPlugin.
@LAYER: UI
@RELATION: USES -> frontend/src/services/toolsService.js
-->
<script>
// [SECTION: IMPORTS]
import { onMount } from 'svelte';
import { runTask, getTaskStatus } from '../../services/toolsService.js';
import { selectedTask } from '../../lib/stores.js';
import { addToast } from '../../lib/toasts.js';
// [/SECTION]
let envs = [];
let selectedEnv = '';
let searchQuery = '';
let isRunning = false;
let results = null;
let pollInterval;
// [DEF:fetchEnvironments:Function]
// @PURPOSE: Fetches the list of available environments.
// @PRE: None.
// @POST: envs array is populated.
async function fetchEnvironments() {
try {
const res = await fetch('/api/environments');
envs = await res.json();
} catch (e) {
addToast('Failed to fetch environments', 'error');
}
}
// [/DEF:fetchEnvironments:Function]
// [DEF:handleSearch:Function]
// @PURPOSE: Triggers the SearchPlugin task.
// @PRE: selectedEnv and searchQuery must be set.
// @POST: Task is started and polling begins.
async function handleSearch() {
if (!selectedEnv || !searchQuery) {
addToast('Please select environment and enter query', 'warning');
return;
}
isRunning = true;
results = null;
try {
// Find the environment name from ID
const env = envs.find(e => e.id === selectedEnv);
const task = await runTask('search-datasets', {
env: env.name,
query: searchQuery
});
selectedTask.set(task);
startPolling(task.id);
} catch (e) {
isRunning = false;
addToast(e.message, 'error');
}
}
// [/DEF:handleSearch:Function]
// [DEF:startPolling:Function]
// @PURPOSE: Polls for task completion and results.
// @PRE: taskId is provided.
// @POST: pollInterval is set and results are updated on success.
function startPolling(taskId) {
if (pollInterval) clearInterval(pollInterval);
pollInterval = setInterval(async () => {
try {
const task = await getTaskStatus(taskId);
selectedTask.set(task);
if (task.status === 'SUCCESS') {
clearInterval(pollInterval);
isRunning = false;
results = task.result;
addToast('Search completed', 'success');
} else if (task.status === 'FAILED') {
clearInterval(pollInterval);
isRunning = false;
addToast('Search failed', 'error');
}
} catch (e) {
clearInterval(pollInterval);
isRunning = false;
addToast('Error polling task status', 'error');
}
}, 2000);
}
// [/DEF:startPolling:Function]
onMount(fetchEnvironments);
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="space-y-6">
<div class="bg-white p-6 rounded-lg shadow-sm border border-gray-200">
<h3 class="text-lg font-medium text-gray-900 mb-4">Search Dataset Metadata</h3>
<div class="grid grid-cols-1 md:grid-cols-2 gap-4 items-end">
<div>
<label for="env-select" class="block text-sm font-medium text-gray-700">Environment</label>
<select
id="env-select"
bind:value={selectedEnv}
class="mt-1 block w-full pl-3 pr-10 py-2 text-base border-gray-300 focus:outline-none focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm rounded-md"
>
<option value="" disabled>-- Select Environment --</option>
{#each envs as env}
<option value={env.id}>{env.name}</option>
{/each}
</select>
</div>
<div>
<label for="search-query" class="block text-sm font-medium text-gray-700">Regex Pattern</label>
<input
type="text"
id="search-query"
bind:value={searchQuery}
placeholder="e.g. from dm.*\.account"
class="mt-1 block w-full border-gray-300 rounded-md shadow-sm focus:ring-indigo-500 focus:border-indigo-500 sm:text-sm"
/>
</div>
</div>
<div class="mt-4 flex justify-end">
<button
on:click={handleSearch}
disabled={isRunning}
class="inline-flex items-center px-4 py-2 border border-transparent text-sm font-medium rounded-md shadow-sm text-white bg-indigo-600 hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500 disabled:opacity-50"
>
{#if isRunning}
<svg class="animate-spin -ml-1 mr-3 h-5 w-5 text-white" xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24">
<circle class="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" stroke-width="4"></circle>
<path class="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
</svg>
Searching...
{:else}
Search
{/if}
</button>
</div>
</div>
{#if results}
<div class="bg-white shadow overflow-hidden sm:rounded-md border border-gray-200">
<div class="px-4 py-5 sm:px-6 flex justify-between items-center bg-gray-50 border-b border-gray-200">
<h3 class="text-lg leading-6 font-medium text-gray-900">
Search Results
</h3>
<span class="inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium bg-blue-100 text-blue-800">
{results.count} matches
</span>
</div>
<ul class="divide-y divide-gray-200">
{#each results.results as item}
<li class="p-4 hover:bg-gray-50">
<div class="flex items-center justify-between">
<div class="text-sm font-medium text-indigo-600 truncate">
{item.dataset_name} (ID: {item.dataset_id})
</div>
<div class="ml-2 flex-shrink-0 flex">
<p class="px-2 inline-flex text-xs leading-5 font-semibold rounded-full bg-green-100 text-green-800">
Field: {item.field}
</p>
</div>
</div>
<div class="mt-2">
<pre class="text-xs text-gray-500 bg-gray-50 p-2 rounded border border-gray-100 overflow-x-auto">{item.match_context}</pre>
</div>
</li>
{/each}
{#if results.count === 0}
<li class="p-8 text-center text-gray-500 italic">
No matches found for the given pattern.
</li>
{/if}
</ul>
</div>
{/if}
</div>
<!-- [/SECTION] -->
<!-- [/DEF:SearchTool:Component] -->

View File

@@ -95,7 +95,10 @@ async function requestApi(endpoint, method = 'GET', body = null) {
const response = await fetch(`${API_BASE_URL}${endpoint}`, options);
if (!response.ok) {
const errorData = await response.json().catch(() => ({}));
throw new Error(errorData.detail || `API request failed with status ${response.status}`);
const message = errorData.detail
? (typeof errorData.detail === 'string' ? errorData.detail : JSON.stringify(errorData.detail))
: `API request failed with status ${response.status}`;
throw new Error(message);
}
return await response.json();
} catch (error) {
@@ -123,6 +126,8 @@ export const api = {
deleteEnvironment: (id) => requestApi(`/settings/environments/${id}`, 'DELETE'),
testEnvironmentConnection: (id) => postApi(`/settings/environments/${id}/test`, {}),
updateEnvironmentSchedule: (id, schedule) => requestApi(`/environments/${id}/schedule`, 'PUT', schedule),
getStorageSettings: () => fetchApi('/settings/storage'),
updateStorageSettings: (storage) => requestApi('/settings/storage', 'PUT', storage),
getEnvironmentsList: () => fetchApi('/environments'),
};
// [/DEF:api:Data]
@@ -130,6 +135,7 @@ export const api = {
// [/DEF:api_module:Module]
// Export individual functions for easier use in components
export { requestApi };
export const getPlugins = api.getPlugins;
export const getTasks = api.getTasks;
export const getTask = api.getTask;
@@ -143,3 +149,5 @@ export const deleteEnvironment = api.deleteEnvironment;
export const testEnvironmentConnection = api.testEnvironmentConnection;
export const updateEnvironmentSchedule = api.updateEnvironmentSchedule;
export const getEnvironmentsList = api.getEnvironmentsList;
export const getStorageSettings = api.getStorageSettings;
export const updateStorageSettings = api.updateStorageSettings;

View File

@@ -20,7 +20,6 @@
let settings = {
environments: [],
settings: {
backup_path: '',
default_environment_id: null,
logging: {
level: 'INFO',
@@ -204,12 +203,6 @@
<section class="mb-8 bg-white p-6 rounded shadow">
<h2 class="text-xl font-semibold mb-4">Global Settings</h2>
<div class="grid grid-cols-1 gap-4">
<div>
<label for="backup_path" class="block text-sm font-medium text-gray-700">Backup Storage Path</label>
<input type="text" id="backup_path" bind:value={settings.settings.backup_path} class="mt-1 block w-full border border-gray-300 rounded-md shadow-sm p-2" />
</div>
</div>
<h3 class="text-lg font-medium mb-4 mt-6">Logging Configuration</h3>
<div class="grid grid-cols-1 md:grid-cols-2 gap-4">

View File

@@ -23,10 +23,8 @@
*/
function selectPlugin(plugin) {
console.log(`[Dashboard][Action] Selecting plugin: ${plugin.id}`);
if (plugin.id === 'superset-migration') {
goto('/migration');
} else if (plugin.id === 'git-integration') {
goto('/git');
if (plugin.ui_route) {
goto(plugin.ui_route);
} else {
selectedPlugin.set(plugin);
}
@@ -82,7 +80,7 @@
{/if}
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
{#each data.plugins as plugin}
{#each data.plugins.filter(p => p.id !== 'superset-search') as plugin}
<div
on:click={() => selectPlugin(plugin)}
role="button"

View File

@@ -1,6 +1,6 @@
<script>
import { onMount } from 'svelte';
import { updateGlobalSettings, addEnvironment, updateEnvironment, deleteEnvironment, testEnvironmentConnection } from '../../lib/api';
import { updateGlobalSettings, addEnvironment, updateEnvironment, deleteEnvironment, testEnvironmentConnection, updateStorageSettings } from '../../lib/api';
import { addToast } from '../../lib/toasts';
import { t } from '$lib/i18n';
import { Button, Input, Card, PageHeader } from '$lib/ui';
@@ -41,6 +41,24 @@
}
// [/DEF:handleSaveGlobal:Function]
// [DEF:handleSaveStorage:Function]
/* @PURPOSE: Saves storage-specific settings.
@PRE: settings.settings.storage must contain valid configuration.
@POST: Storage settings are updated via API.
*/
async function handleSaveStorage() {
try {
console.log("[Settings.handleSaveStorage][Action] Saving storage settings.");
await updateStorageSettings(settings.settings.storage);
addToast('Storage settings saved', 'success');
console.log("[Settings.handleSaveStorage][Coherence:OK] Storage settings saved.");
} catch (error) {
console.error("[Settings.handleSaveStorage][Coherence:Failed] Failed to save storage settings:", error);
addToast(error.message || 'Failed to save storage settings', 'error');
}
}
// [/DEF:handleSaveStorage:Function]
// [DEF:handleAddOrUpdateEnv:Function]
/* @PURPOSE: Adds a new environment or updates an existing one.
@PRE: newEnv must contain valid environment details.
@@ -152,14 +170,37 @@
</div>
{/if}
<div class="mb-8">
<Card title={$t.settings?.global_title || "Global Settings"}>
<div class="grid grid-cols-1 gap-6">
<Card title={$t.settings?.storage_title || "File Storage Configuration"}>
<div class="grid grid-cols-1 md:grid-cols-2 gap-6">
<div class="md:col-span-2">
<Input
label={$t.settings?.storage_root || "Storage Root Path"}
bind:value={settings.settings.storage.root_path}
/>
</div>
<Input
label={$t.settings?.backup_path || "Backup Storage Path"}
bind:value={settings.settings.backup_path}
label={$t.settings?.storage_backup_pattern || "Backup Directory Pattern"}
bind:value={settings.settings.storage.backup_structure_pattern}
/>
<Button on:click={handleSaveGlobal}>
<Input
label={$t.settings?.storage_repo_pattern || "Repository Directory Pattern"}
bind:value={settings.settings.storage.repo_structure_pattern}
/>
<Input
label={$t.settings?.storage_filename_pattern || "Filename Pattern"}
bind:value={settings.settings.storage.filename_pattern}
/>
<div class="bg-gray-50 p-4 rounded border border-gray-200">
<span class="block text-xs font-semibold text-gray-500 uppercase mb-2">{$t.settings?.storage_preview || "Path Preview"}</span>
<code class="text-sm text-indigo-600">
{settings.settings.storage.root_path}/backups/sample_backup.zip
</code>
</div>
</div>
<div class="mt-6">
<Button on:click={handleSaveStorage}>
{$t.common.save}
</Button>
</div>

View File

@@ -18,7 +18,6 @@ export async function load() {
settings: {
environments: [],
settings: {
backup_path: '',
default_environment_id: null
}
},

View File

@@ -1,40 +0,0 @@
<script>
import { onMount } from 'svelte';
import { gitService } from '../../../services/gitService';
import { addToast as toast } from '../../../lib/toasts.js';
let environments = [];
onMount(async () => {
try {
environments = await gitService.getEnvironments();
} catch (e) {
toast(e.message, 'error');
}
});
</script>
<div class="p-6">
<h1 class="text-2xl font-bold mb-6">Deployment Environments</h1>
<div class="bg-white p-6 rounded shadow">
<h2 class="text-xl font-semibold mb-4">Target Environments</h2>
{#if environments.length === 0}
<p class="text-gray-500">No deployment environments configured.</p>
{:else}
<ul class="divide-y">
{#each environments as env}
<li class="py-3 flex justify-between items-center">
<div>
<span class="font-medium">{env.name}</span>
<div class="text-xs text-gray-400">{env.superset_url}</div>
</div>
<span class="px-2 py-1 text-xs rounded {env.is_active ? 'bg-green-100 text-green-800' : 'bg-gray-100 text-gray-800'}">
{env.is_active ? 'Active' : 'Inactive'}
</span>
</li>
{/each}
</ul>
{/if}
</div>
</div>

View File

@@ -116,13 +116,7 @@
</script>
<div class="container mx-auto p-4 max-w-6xl">
<PageHeader title={$t.tasks.management}>
<div slot="actions">
<Button on:click={() => showBackupModal = true}>
{$t.tasks.run_backup}
</Button>
</div>
</PageHeader>
<PageHeader title={$t.tasks.management} />
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
<div class="lg:col-span-1">

View File

@@ -0,0 +1,27 @@
<!-- [DEF:BackupPage:Component] -->
<!--
@SEMANTICS: backup, page, tools
@PURPOSE: Entry point for the Backup Management interface.
@LAYER: Page
@RELATION: USES -> BackupManager
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { t } from '../../../lib/i18n';
import { PageHeader } from '../../../lib/ui';
import BackupManager from '../../../components/backups/BackupManager.svelte';
// [/SECTION]
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="container mx-auto p-4 max-w-6xl">
<PageHeader title={$t.nav.tools_backups} />
<div class="mt-6">
<BackupManager />
</div>
</div>
<!-- [/SECTION] -->
<!-- [/DEF:BackupPage:Component] -->

View File

@@ -1,25 +0,0 @@
<!-- [DEF:SearchPage:Component] -->
<!--
@SEMANTICS: search, page, tool
@PURPOSE: Page for the dataset search tool.
@LAYER: UI
-->
<script>
import SearchTool from '../../../components/tools/SearchTool.svelte';
import TaskRunner from '../../../components/TaskRunner.svelte';
import { PageHeader } from '$lib/ui';
</script>
<div class="max-w-7xl mx-auto p-6">
<PageHeader title="Dataset Search" />
<div class="grid grid-cols-1 lg:grid-cols-3 gap-8">
<div class="lg:col-span-2">
<SearchTool />
</div>
<div class="lg:col-span-1">
<TaskRunner />
</div>
</div>
</div>
<!-- [/DEF:SearchPage:Component] -->

View File

@@ -0,0 +1,212 @@
<!-- [DEF:StoragePage:Component] -->
<!--
@SEMANTICS: storage, files, management
@PURPOSE: Main page for file storage management.
@LAYER: Feature
@RELATION: DEPENDS_ON -> storageService
@RELATION: CONTAINS -> FileList
@RELATION: CONTAINS -> FileUpload
@INVARIANT: Always displays tabs for Backups and Repositories.
-->
<script lang="ts">
// [SECTION: IMPORTS]
import { onMount } from 'svelte';
import { page } from '$app/stores';
import { listFiles, deleteFile } from '../../../services/storageService';
import { addToast } from '../../../lib/toasts';
import { t } from '../../../lib/i18n';
import FileList from '../../../components/storage/FileList.svelte';
import FileUpload from '../../../components/storage/FileUpload.svelte';
// [/SECTION: IMPORTS]
// [DEF:loadFiles:Function]
/**
* @purpose Fetches the list of files from the server.
* @post Updates the `files` array with the latest data.
*/
let files = [];
let isLoading = false;
let activeTab = 'backups';
let currentPath = 'backups'; // Relative to storage root
async function loadFiles() {
isLoading = true;
try {
const category = activeTab;
// If we have a currentPath, we use it.
// But if user switched tabs, we should reset currentPath to category root
let effectivePath = currentPath;
if (category && !currentPath.startsWith(category)) {
effectivePath = category;
currentPath = category;
}
// API expects path relative to category root if category is provided
const subpath = (category && effectivePath.startsWith(category))
? effectivePath.substring(category.length).replace(/^\/+/, '')
: effectivePath;
files = await listFiles(category, subpath);
} catch (error) {
addToast($t.storage.messages.load_failed.replace('{error}', error.message), 'error');
} finally {
isLoading = false;
}
}
// [/DEF:loadFiles:Function]
// [DEF:handleDelete:Function]
/**
* @purpose Handles the file deletion process.
* @param {CustomEvent} event - The delete event containing category and path.
*/
async function handleDelete(event) {
const { category, path, name } = event.detail;
if (!confirm($t.storage.messages.delete_confirm.replace('{name}', name))) return;
try {
await deleteFile(category, path);
addToast($t.storage.messages.delete_success.replace('{name}', name), 'success');
await loadFiles();
} catch (error) {
addToast($t.storage.messages.delete_failed.replace('{error}', error.message), 'error');
}
}
// [/DEF:handleDelete:Function]
// [DEF:handleNavigate:Function]
/**
* @purpose Updates the current path and reloads files when navigating into a directory.
* @param {CustomEvent} event - The navigation event containing the new path.
*/
function handleNavigate(event) {
currentPath = event.detail;
loadFiles();
}
// [/DEF:handleNavigate:Function]
// [DEF:navigateUp:Function]
/**
* @purpose Navigates one level up in the directory structure.
* @pre currentPath is set and deeper than activeTab root.
* @post currentPath is moved up one directory level.
*/
function navigateUp() {
if (!currentPath || currentPath === activeTab) return;
const parts = currentPath.split('/');
parts.pop();
currentPath = parts.join('/') || '';
loadFiles();
}
// [/DEF:navigateUp:Function]
onMount(() => {
const pathParam = $page.url.searchParams.get('path');
if (pathParam) {
currentPath = pathParam;
if (pathParam.startsWith('repositorys')) {
activeTab = 'repositorys';
} else {
activeTab = 'backups';
}
}
loadFiles();
});
$: if (activeTab) {
// Reset path when switching tabs
if (!currentPath.startsWith(activeTab)) {
currentPath = activeTab;
}
loadFiles();
}
</script>
<!-- [SECTION: TEMPLATE] -->
<div class="container mx-auto p-4 max-w-6xl">
<div class="mb-6">
<h1 class="text-2xl font-bold text-gray-900">{$t.storage.management}</h1>
{#if currentPath}
<div class="flex items-center mt-2 text-sm text-gray-500">
<button on:click={() => { currentPath = activeTab; loadFiles(); }} class="hover:text-indigo-600">{$t.storage.root}</button>
{#each currentPath.split('/').slice(1) as part, i}
<span class="mx-2">/</span>
<button
on:click={() => { currentPath = currentPath.split('/').slice(0, i + 1).join('/'); loadFiles(); }}
class="hover:text-indigo-600 capitalize"
>
{part}
</button>
{/each}
</div>
{/if}
</div>
<div class="flex justify-end mb-4">
<button
on:click={loadFiles}
class="inline-flex items-center px-4 py-2 border border-gray-300 rounded-md shadow-sm text-sm font-medium text-gray-700 bg-white hover:bg-gray-50"
disabled={isLoading}
>
{isLoading ? $t.storage.refreshing : $t.storage.refresh}
</button>
</div>
<div class="grid grid-cols-1 lg:grid-cols-3 gap-6">
<!-- Main Content: File List -->
<div class="lg:col-span-2 space-y-4">
<!-- Tabs -->
<div class="border-b border-gray-200">
<nav class="-mb-px flex space-x-8">
<button
on:click={() => activeTab = 'backups'}
class="py-4 px-1 border-b-2 font-medium text-sm {activeTab === 'backups' ? 'border-indigo-500 text-indigo-600' : 'border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300'}"
>
{$t.storage.backups}
</button>
<button
on:click={() => activeTab = 'repositorys'}
class="py-4 px-1 border-b-2 font-medium text-sm {activeTab === 'repositorys' ? 'border-indigo-500 text-indigo-600' : 'border-transparent text-gray-500 hover:text-gray-700 hover:border-gray-300'}"
>
{$t.storage.repositories}
</button>
</nav>
</div>
<div class="flex items-center mb-2">
{#if currentPath && currentPath !== activeTab}
<button
on:click={navigateUp}
class="mr-4 inline-flex items-center px-3 py-1 border border-gray-300 shadow-sm text-xs font-medium rounded text-gray-700 bg-white hover:bg-gray-50"
>
<svg class="h-4 w-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 19l-7-7m0 0l7-7m-7 7h18" />
</svg>
Back
</button>
{/if}
</div>
<FileList {files} on:delete={handleDelete} on:navigate={handleNavigate} />
</div>
<!-- Sidebar: Upload -->
<div class="lg:col-span-1">
<FileUpload
category={activeTab}
path={currentPath}
on:uploaded={loadFiles}
/>
</div>
</div>
</div>
<!-- [/SECTION: TEMPLATE] -->
<style>
/* ... */
</style>
<!-- [/DEF:StoragePage:Component] -->

View File

@@ -0,0 +1,109 @@
// [DEF:storageService:Module]
/**
* @purpose Frontend API client for file storage management.
* @layer Service
* @relation DEPENDS_ON -> backend.api.storage
* @SEMANTICS: storage, api, client
*/
const API_BASE = '/api/storage';
// [DEF:listFiles:Function]
/**
* @purpose Fetches the list of files for a given category and subpath.
* @param {string} [category] - Optional category filter.
* @param {string} [path] - Optional subpath filter.
* @returns {Promise<Array>}
* @PRE category and path should be valid strings if provided.
* @POST Returns a promise resolving to an array of StoredFile objects.
*/
export async function listFiles(category, path) {
const params = new URLSearchParams();
if (category) {
params.append('category', category);
}
if (path) {
params.append('path', path);
}
const response = await fetch(`${API_BASE}/files?${params.toString()}`);
if (!response.ok) {
throw new Error(`Failed to fetch files: ${response.statusText}`);
}
return await response.json();
}
// [/DEF:listFiles:Function]
// [DEF:uploadFile:Function]
/**
* @purpose Uploads a file to the storage system.
* @param {File} file - The file to upload.
* @param {string} category - Target category.
* @param {string} [path] - Target subpath.
* @returns {Promise<Object>}
* @PRE file must be a valid File object; category must be specified.
* @POST Returns a promise resolving to the metadata of the uploaded file.
*/
export async function uploadFile(file, category, path) {
const formData = new FormData();
formData.append('file', file);
formData.append('category', category);
if (path) {
formData.append('path', path);
}
const response = await fetch(`${API_BASE}/upload`, {
method: 'POST',
body: formData
});
if (!response.ok) {
const errorData = await response.json().catch(() => ({}));
throw new Error(errorData.detail || `Failed to upload file: ${response.statusText}`);
}
return await response.json();
}
// [/DEF:uploadFile:Function]
// [DEF:deleteFile:Function]
/**
* @purpose Deletes a file or directory from storage.
* @param {string} category - File category.
* @param {string} path - Relative path of the item.
* @returns {Promise<void>}
* @PRE category and path must identify an existing file or directory.
* @POST The specified file or directory is removed from storage.
*/
export async function deleteFile(category, path) {
const response = await fetch(`${API_BASE}/files/${category}/${path}`, {
method: 'DELETE'
});
if (!response.ok) {
const errorData = await response.json().catch(() => ({}));
throw new Error(errorData.detail || `Failed to delete: ${response.statusText}`);
}
}
// [/DEF:deleteFile:Function]
// [DEF:downloadFileUrl:Function]
/**
* @purpose Returns the URL for downloading a file.
* @param {string} category - File category.
* @param {string} path - Relative path of the file.
* @returns {string}
* @PRE category and path must identify an existing file.
* @POST Returns a valid API URL for file download.
*/
export function downloadFileUrl(category, path) {
return `${API_BASE}/download/${category}/${path}`;
}
// [/DEF:downloadFileUrl:Function]
export default {
listFiles,
uploadFile,
deleteFile,
downloadFileUrl
};
// [/DEF:storageService:Module]

View File

@@ -0,0 +1,22 @@
/**
* [DEF:BackupTypes:Module]
* @SEMANTICS: types, backup, interface
* @PURPOSE: Defines types and interfaces for the Backup Management UI.
*/
export interface Backup {
id: string;
name: string;
environment: string;
created_at: string;
size_bytes?: number;
status: 'success' | 'failed' | 'in_progress';
}
export interface BackupCreateRequest {
environment_id: string;
}
/**
* [/DEF:BackupTypes:Module]
*/

View File

@@ -130,7 +130,8 @@ class SemanticEntity:
self.compliance_issues.append(f"Missing Mandatory Tag: @{req_tag}")
# 3. Check for Belief State Logging (Python only)
if self.type == "Function" and self.file_path.endswith(".py"):
# Skip check for logger.py to avoid circular dependencies
if self.type == "Function" and self.file_path.endswith(".py") and "backend/src/core/logger.py" not in self.file_path:
if not getattr(self, 'has_belief_scope', False):
self.compliance_issues.append("Missing Belief State Logging: Function should use belief_scope context manager.")

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,117 @@
# Semantic Compliance Report
**Generated At:** 2026-01-26T11:41:28.355350
**Global Compliance Score:** 99.2%
**Scanned Files:** 108
## File Compliance Status
| File | Score | Issues |
|------|-------|--------|
| frontend/src/components/storage/FileList.svelte | 🟡 75% | [isDirectory] Missing Mandatory Tag: @PRE<br>[isDirectory] Missing Mandatory Tag: @POST<br>[isDirectory] Missing Mandatory Tag: @PRE<br>[isDirectory] Missing Mandatory Tag: @POST<br>[formatSize] Missing Mandatory Tag: @PRE<br>[formatSize] Missing Mandatory Tag: @POST<br>[formatSize] Missing Mandatory Tag: @PRE<br>[formatSize] Missing Mandatory Tag: @POST<br>[formatDate] Missing Mandatory Tag: @PRE<br>[formatDate] Missing Mandatory Tag: @POST<br>[formatDate] Missing Mandatory Tag: @PRE<br>[formatDate] Missing Mandatory Tag: @POST |
| frontend/src/routes/tools/storage/+page.svelte | 🟡 77% | [loadFiles] Missing Mandatory Tag: @PRE<br>[loadFiles] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @POST<br>[handleDelete] Missing Mandatory Tag: @PRE<br>[handleDelete] Missing Mandatory Tag: @POST<br>[handleNavigate] Missing Mandatory Tag: @PRE<br>[handleNavigate] Missing Mandatory Tag: @POST<br>[handleNavigate] Missing Mandatory Tag: @PRE<br>[handleNavigate] Missing Mandatory Tag: @POST<br>[navigateUp] Missing Mandatory Tag: @PRE<br>[navigateUp] Missing Mandatory Tag: @POST<br>[navigateUp] Missing Mandatory Tag: @PRE<br>[navigateUp] Missing Mandatory Tag: @POST |
| frontend/src/components/storage/FileUpload.svelte | 🟡 89% | [handleDrop] Missing Mandatory Tag: @PRE<br>[handleDrop] Missing Mandatory Tag: @POST<br>[handleDrop] Missing Mandatory Tag: @PRE<br>[handleDrop] Missing Mandatory Tag: @POST |
| frontend/src/components/git/CommitModal.svelte | 🟡 94% | [loadStatus] Missing Mandatory Tag: @POST<br>[loadStatus] Missing Mandatory Tag: @POST |
| frontend/src/components/DashboardGrid.svelte | 🟡 94% | [openGit] Missing Mandatory Tag: @PRE<br>[openGit] Missing Mandatory Tag: @POST<br>[openGit] Missing Mandatory Tag: @PRE<br>[openGit] Missing Mandatory Tag: @POST |
| backend/src/api/routes/settings.py | 🟡 95% | [get_storage_settings] Missing Mandatory Tag: @PRE<br>[get_storage_settings] Missing Mandatory Tag: @POST<br>[get_storage_settings] Missing Mandatory Tag: @PRE<br>[get_storage_settings] Missing Mandatory Tag: @POST<br>[update_storage_settings] Missing Mandatory Tag: @PRE<br>[update_storage_settings] Missing Mandatory Tag: @PRE |
| frontend/src/components/git/DeploymentModal.svelte | 🟡 96% | [loadEnvironments] Missing Mandatory Tag: @PRE<br>[loadEnvironments] Missing Mandatory Tag: @PRE |
| frontend/src/components/git/BranchSelector.svelte | 🟡 97% | [handleCheckout] Missing Mandatory Tag: @PRE<br>[handleCheckout] Missing Mandatory Tag: @PRE |
| backend/src/core/utils/dataset_mapper.py | 🟡 97% | [__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Mandatory Tag: @PRE<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
| generate_semantic_map.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__enter__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__enter__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__exit__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__exit__] Missing Belief State Logging: Function should use belief_scope context manager. |
| frontend/src/lib/stores.js | 🟢 100% | OK |
| frontend/src/lib/toasts.js | 🟢 100% | OK |
| frontend/src/lib/api.js | 🟢 100% | OK |
| frontend/src/lib/ui/Select.svelte | 🟢 100% | OK |
| frontend/src/lib/ui/index.ts | 🟢 100% | OK |
| frontend/src/lib/ui/PageHeader.svelte | 🟢 100% | OK |
| frontend/src/lib/ui/Card.svelte | 🟢 100% | OK |
| frontend/src/lib/ui/Button.svelte | 🟢 100% | OK |
| frontend/src/lib/ui/Input.svelte | 🟢 100% | OK |
| frontend/src/lib/ui/LanguageSwitcher.svelte | 🟢 100% | OK |
| frontend/src/lib/i18n/index.ts | 🟢 100% | OK |
| frontend/src/routes/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/+page.ts | 🟢 100% | OK |
| frontend/src/routes/tasks/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/migration/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/migration/mappings/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/tools/search/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/tools/mapper/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/tools/debug/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/settings/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/settings/+page.ts | 🟢 100% | OK |
| frontend/src/routes/settings/connections/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/settings/git/+page.svelte | 🟢 100% | OK |
| frontend/src/routes/git/+page.svelte | 🟢 100% | OK |
| frontend/src/pages/Dashboard.svelte | 🟢 100% | OK |
| frontend/src/pages/Settings.svelte | 🟢 100% | OK |
| frontend/src/services/connectionService.js | 🟢 100% | OK |
| frontend/src/services/gitService.js | 🟢 100% | OK |
| frontend/src/services/toolsService.js | 🟢 100% | OK |
| frontend/src/services/taskService.js | 🟢 100% | OK |
| frontend/src/services/storageService.js | 🟢 100% | OK |
| frontend/src/components/PasswordPrompt.svelte | 🟢 100% | OK |
| frontend/src/components/MappingTable.svelte | 🟢 100% | OK |
| frontend/src/components/TaskLogViewer.svelte | 🟢 100% | OK |
| frontend/src/components/Footer.svelte | 🟢 100% | OK |
| frontend/src/components/MissingMappingModal.svelte | 🟢 100% | OK |
| frontend/src/components/Navbar.svelte | 🟢 100% | OK |
| frontend/src/components/TaskHistory.svelte | 🟢 100% | OK |
| frontend/src/components/Toast.svelte | 🟢 100% | OK |
| frontend/src/components/TaskRunner.svelte | 🟢 100% | OK |
| frontend/src/components/TaskList.svelte | 🟢 100% | OK |
| frontend/src/components/DynamicForm.svelte | 🟢 100% | OK |
| frontend/src/components/EnvSelector.svelte | 🟢 100% | OK |
| frontend/src/components/tools/ConnectionForm.svelte | 🟢 100% | OK |
| frontend/src/components/tools/ConnectionList.svelte | 🟢 100% | OK |
| frontend/src/components/tools/MapperTool.svelte | 🟢 100% | OK |
| frontend/src/components/tools/DebugTool.svelte | 🟢 100% | OK |
| frontend/src/components/tools/SearchTool.svelte | 🟢 100% | OK |
| frontend/src/components/git/CommitHistory.svelte | 🟢 100% | OK |
| frontend/src/components/git/ConflictResolver.svelte | 🟢 100% | OK |
| frontend/src/components/git/GitManager.svelte | 🟢 100% | OK |
| backend/delete_running_tasks.py | 🟢 100% | [delete_running_tasks] Missing Belief State Logging: Function should use belief_scope context manager.<br>[delete_running_tasks] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/app.py | 🟢 100% | OK |
| backend/src/dependencies.py | 🟢 100% | OK |
| backend/src/core/superset_client.py | 🟢 100% | OK |
| backend/src/core/config_manager.py | 🟢 100% | OK |
| backend/src/core/scheduler.py | 🟢 100% | OK |
| backend/src/core/config_models.py | 🟢 100% | OK |
| backend/src/core/database.py | 🟢 100% | OK |
| backend/src/core/logger.py | 🟢 100% | OK |
| backend/src/core/plugin_loader.py | 🟢 100% | OK |
| backend/src/core/migration_engine.py | 🟢 100% | [_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager.<br>[_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager.<br>[_transform_yaml] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/core/plugin_base.py | 🟢 100% | OK |
| backend/src/core/utils/fileio.py | 🟢 100% | [replacer] Missing Belief State Logging: Function should use belief_scope context manager.<br>[replacer] Missing Belief State Logging: Function should use belief_scope context manager.<br>[replacer] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/core/utils/network.py | 🟢 100% | OK |
| backend/src/core/utils/matching.py | 🟢 100% | [suggest_mappings] Missing Belief State Logging: Function should use belief_scope context manager.<br>[suggest_mappings] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/core/task_manager/persistence.py | 🟢 100% | OK |
| backend/src/core/task_manager/manager.py | 🟢 100% | OK |
| backend/src/core/task_manager/models.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/core/task_manager/cleanup.py | 🟢 100% | [__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager.<br>[__init__] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/core/task_manager/__init__.py | 🟢 100% | OK |
| backend/src/api/auth.py | 🟢 100% | [get_current_user] Missing Belief State Logging: Function should use belief_scope context manager.<br>[get_current_user] Missing Belief State Logging: Function should use belief_scope context manager. |
| backend/src/api/routes/git.py | 🟢 100% | OK |
| backend/src/api/routes/connections.py | 🟢 100% | OK |
| backend/src/api/routes/environments.py | 🟢 100% | OK |
| backend/src/api/routes/migration.py | 🟢 100% | OK |
| backend/src/api/routes/plugins.py | 🟢 100% | OK |
| backend/src/api/routes/mappings.py | 🟢 100% | OK |
| backend/src/api/routes/git_schemas.py | 🟢 100% | OK |
| backend/src/api/routes/storage.py | 🟢 100% | OK |
| backend/src/api/routes/tasks.py | 🟢 100% | OK |
| backend/src/models/git.py | 🟢 100% | OK |
| backend/src/models/task.py | 🟢 100% | OK |
| backend/src/models/connection.py | 🟢 100% | OK |
| backend/src/models/mapping.py | 🟢 100% | OK |
| backend/src/models/storage.py | 🟢 100% | OK |
| backend/src/models/dashboard.py | 🟢 100% | OK |
| backend/src/services/git_service.py | 🟢 100% | OK |
| backend/src/services/mapping_service.py | 🟢 100% | OK |
| backend/src/plugins/backup.py | 🟢 100% | OK |
| backend/src/plugins/debug.py | 🟢 100% | OK |
| backend/src/plugins/search.py | 🟢 100% | OK |
| backend/src/plugins/mapper.py | 🟢 100% | OK |
| backend/src/plugins/git_plugin.py | 🟢 100% | OK |
| backend/src/plugins/migration.py | 🟢 100% | OK |
| backend/src/plugins/storage/plugin.py | 🟢 100% | OK |
| backend/tests/test_models.py | 🟢 100% | OK |
| backend/tests/test_logger.py | 🟢 100% | OK |

File diff suppressed because it is too large Load Diff

View File

@@ -15,7 +15,7 @@
## Phase 3: [US1] Scheduled Backups
- [x] T009 [US1] Implement schedule loading and registration logic in `SchedulerService`
- [x] T010 [US1] Update `Environment` settings API to handle `backup_schedule` updates in `backend/src/api/routes/environments.py`
- [x] T011 [P] [US1] Add schedule configuration fields to Environment edit form in `frontend/src/components/EnvSelector.svelte` (or appropriate component)
- [ ] T011 [P] [US1] Add schedule configuration fields to Environment edit form in `frontend/src/components/EnvSelector.svelte` (or appropriate component)
- [x] T012 [US1] Implement validation for Cron expressions in backend and frontend
## Phase 4: [US2] Unified Task Management UI
@@ -34,8 +34,8 @@
- [x] T021 [US4] Integrate log viewer into TaskList or as a separate modal/page
## Final Phase: Polish & Cross-cutting concerns
- [x] T022 Implement task cleanup/retention policy (e.g., delete tasks older than 30 days)
- [ ] T023 Add real-time updates for task status using WebSockets (optional/refinement)
- [ ] T022 Implement task cleanup/retention policy (e.g., delete tasks older than 30 days)
- [x] T023 Add real-time updates for task status using WebSockets (optional/refinement)
- [x] T024 Ensure consistent error handling and logging across scheduler and task manager
## Dependencies

View File

@@ -0,0 +1,34 @@
# Specification Quality Checklist: File Storage Management & UI
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-01-24
**Feature**: [Link to spec.md](../spec.md)
## Content Quality
- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed
## Requirement Completeness
- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified
## Feature Readiness
- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification
## Notes
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`

View File

@@ -0,0 +1,35 @@
# Checklist: File Storage UX & Configuration
**Purpose**: Validate implementation of User Experience and Configuration Flexibility requirements.
**Created**: 2026-01-24
**Feature**: [File Storage Management & UI](../spec.md)
## User Experience (File Management)
- [x] CHK001 Are loading states displayed while fetching the file list? [Completeness]
- [x] CHK002 Is visual feedback provided immediately after file upload starts? [Clarity]
- [x] CHK003 Are error messages user-friendly when upload fails (e.g., "File too large" vs "Error 413")? [Clarity]
- [x] CHK004 Is a confirmation modal shown before permanently deleting a file? [Safety]
- [x] CHK005 Does the UI clearly distinguish between "Backups" and "Repositories" tabs? [Clarity]
- [x] CHK006 Is the file list sortable by Date and Name? [Usability]
- [x] CHK007 Are file sizes formatted in human-readable units (KB, MB, GB)? [Usability]
- [x] CHK008 Is the download action easily accessible for each file item? [Accessibility]
- [x] CHK009 Does the upload component support drag-and-drop interactions? [Usability]
- [x] CHK010 Is the "Upload" button disabled or hidden when no category is selected? [Consistency]
## Configuration Flexibility
- [x] CHK011 Can the storage root path be configured to any writable directory on the server? [Flexibility]
- [x] CHK012 Does the system support defining custom directory structures using variables like `{environment}`? [Flexibility]
- [x] CHK013 Does the system support defining custom filename patterns using variables like `{timestamp}`? [Flexibility]
- [ ] CHK014 Are the supported pattern variables (e.g., `{dashboard_name}`) clearly documented in the UI? [Clarity]
- [x] CHK015 Can the configuration be updated without restarting the application? [Usability]
- [ ] CHK016 Is the current resolved path shown as a preview when editing patterns? [Usability]
- [ ] CHK017 Does the system allow reverting configuration to default values? [Recovery]
## Edge Cases & Error Handling
- [x] CHK018 Is the UI behavior defined for an empty file list (zero state)? [Coverage]
- [ ] CHK019 Is the behavior defined when the configured storage path becomes inaccessible? [Resilience]
- [x] CHK020 Are long filenames handled gracefully in the UI (e.g., truncation with tooltip)? [Layout]
- [x] CHK021 Is the behavior defined for uploading a file with a duplicate name? [Conflict Resolution]

View File

@@ -0,0 +1,74 @@
# API Contracts: File Storage Management & UI
## Endpoints
### GET /api/storage/files
List all files in the storage system.
**Query Parameters:**
- `category` (optional): Filter by category (`backup` or `repository`).
**Response:**
- `200 OK`: List of `StoredFile` objects.
```json
[
{
"name": "dashboard_backup_20260124.zip",
"path": "backups/dashboard_backup_20260124.zip",
"size": 102400,
"created_at": "2026-01-24T12:00:00Z",
"category": "backup",
"mime_type": "application/zip"
}
]
```
### POST /api/storage/upload
Upload a file to the storage system.
**Form Data:**
- `file`: The file content.
- `category`: Target category (`backup` or `repository`).
**Response:**
- `201 Created`: The uploaded `StoredFile` object.
- `400 Bad Request`: Invalid category or file.
### DELETE /api/storage/files/{category}/{filename}
Delete a file from storage.
**Path Parameters:**
- `category`: `backup` or `repository`.
- `filename`: Name of the file to delete.
**Response:**
- `204 No Content`: File deleted successfully.
- `404 Not Found`: File does not exist.
### GET /api/storage/download/{category}/{filename}
Download a file.
**Path Parameters:**
- `category`: `backup` or `repository`.
- `filename`: Name of the file to download.
**Response:**
- `200 OK`: File stream.
- `404 Not Found`: File does not exist.
### GET /api/settings/storage
Get current storage configuration.
**Response:**
- `200 OK`: `StorageConfig` object.
### PUT /api/settings/storage
Update storage configuration.
**Body:**
- `StorageConfig` object.
**Response:**
- `200 OK`: Updated `StorageConfig`.
- `400 Bad Request`: Invalid path or not writable.

View File

@@ -0,0 +1,34 @@
# Data Model: File Storage Management & UI
## Entities
### StorageConfig
*Configuration for the storage system.*
| Field | Type | Description | Constraints |
|---|---|---|---|
| `root_path` | `string` | Absolute path to the storage root directory. | Must be a valid, writable path. Default: `../ss-tools-storage` |
| `backup_structure_pattern` | `string` | Pattern for backup directory structure. | Default: `{category}/` |
| `repo_structure_pattern` | `string` | Pattern for repository directory structure. | Default: `{category}/` |
| `filename_pattern` | `string` | Pattern for filenames. | Default: `{name}_{timestamp}` |
### StoredFile
*Representation of a file in the storage system.*
| Field | Type | Description | Constraints |
|---|---|---|---|
| `name` | `string` | Name of the file (including extension). | No path separators. |
| `path` | `string` | Relative path from storage root. | |
| `size` | `integer` | Size of the file in bytes. | >= 0 |
| `created_at` | `datetime` | Creation timestamp. | |
| `category` | `enum` | Category of the file. | `backup`, `repository` |
| `mime_type` | `string` | MIME type of the file. | Optional |
## Directory Structure
```text
{root_path}/
├── backups/
│ └── {filename}.zip
└── repositories/
└── {filename}.zip

View File

@@ -0,0 +1,106 @@
# Implementation Plan: File Storage Management & UI
**Branch**: `014-file-storage-ui` | **Date**: 2026-01-24 | **Spec**: [specs/014-file-storage-ui/spec.md](../014-file-storage-ui/spec.md)
**Input**: Feature specification from `specs/014-file-storage-ui/spec.md`
## Summary
This feature implements a managed file storage system for dashboard backups and exported repositories. It introduces a configurable storage root (defaulting to outside the workspace) and a Web UI to list, upload, download, and delete files. The system enforces a structured layout with `backups/` and `repositories/` subdirectories to keep artifacts organized. The UI supports hierarchical folder navigation (e.g., `backups/SS2/DashboardName`), allowing users to browse, download, and manage files within nested directories.
## Technical Context
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
**Primary Dependencies**: FastAPI (Backend), SvelteKit (Frontend)
**Storage**: Local Filesystem (for artifacts), Config (for storage path)
**Testing**: pytest (Backend), vitest/playwright (Frontend - implied)
**Target Platform**: Linux server
**Project Type**: Web application
**Performance Goals**: File list load < 1s for 100 files, supports 50MB+ uploads
**Constraints**: Must prevent path traversal, must not pollute git repo
**Scale/Scope**: ~2-3 backend endpoints, 1-2 frontend pages/components
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
- **I. Semantic Protocol Compliance**:
- **Status**: PASSED
- **Check**: Will use `[DEF]` anchors and `@RELATION` tags.
- **Check**: Will follow File Structure Standard.
- **II. Causal Validity (Contracts First)**:
- **Status**: PASSED
- **Check**: Contracts will be defined in `specs/014-file-storage-ui/contracts/` before implementation.
- **III. Immutability of Architecture**:
- **Status**: PASSED
- **Check**: No changes to immutable architectural constraints expected.
- **IV. Design by Contract (DbC)**:
- **Status**: PASSED
- **Check**: Functions will define `@PRE` and `@POST` conditions.
- **V. Belief State Logging**:
- **Status**: PASSED
- **Check**: Will use standard logging patterns.
- **VI. Fractal Complexity Limit**:
- **Status**: PASSED
- **Check**: Feature scope is small, unlikely to exceed complexity limits.
- **VII. Everything is a Plugin**:
- **Status**: PASSED
- **Check**: New functionality will be implemented as a `StoragePlugin` (or similar) inheriting from `PluginBase`.
## Project Structure
### Documentation (this feature)
```text
specs/014-file-storage-ui/
├── plan.md # This file
├── research.md # Phase 0 output
├── data-model.md # Phase 1 output
├── quickstart.md # Phase 1 output
├── contracts/ # Phase 1 output
└── tasks.md # Phase 2 output
```
### Source Code (repository root)
```text
backend/
├── src/
│ ├── api/
│ │ └── routes/
│ │ └── storage.py # New route handler
│ ├── plugins/
│ │ └── storage.py # New plugin implementation
│ └── models/
│ └── storage.py # Pydantic models (StoredFile, StorageConfig)
└── tests/
└── test_storage.py # Backend tests
frontend/
├── src/
│ ├── routes/
│ │ └── storage/
│ │ └── +page.svelte # Main storage UI
│ ├── components/
│ │ └── storage/
│ │ ├── FileList.svelte # Component for listing files and folders (explorer view)
│ │ ├── Breadcrumbs.svelte # Component for navigation
│ │ └── FileUpload.svelte # Component for uploading
│ └── services/
│ └── storageService.js # Frontend API client
```
**Structure Decision**: Standard Web Application structure (Backend + Frontend) with Plugin architecture for the backend logic.
## Complexity Tracking
> **Fill ONLY if Constitution Check has violations that must be justified**
| Violation | Why Needed | Simpler Alternative Rejected Because |
|-----------|------------|-------------------------------------|
| N/A | | |

View File

@@ -0,0 +1,31 @@
# Quickstart: File Storage Management & UI
## Usage Guide
1. **Access File Storage**: Navigate to the "Tools" > "File Storage" section in the main navigation.
2. **View Files**: You will see two tabs: "Backups" and "Repositories". Click on a tab to view files in that category.
3. **Upload File**:
- Click the "Upload" button.
- Select a file from your computer.
- Choose the target category (Backup or Repository).
- Click "Upload" to start the transfer.
4. **Download File**: Click the "Download" icon next to any file in the list.
5. **Delete File**: Click the "Trash" icon next to any file to delete it permanently.
6. **Configure Storage Path**:
- Go to "Settings".
- Locate the "File Storage" section.
- Enter a new absolute path for the storage root.
- Click "Save". The system will verify write access to the new path.
## Development
### Backend
- **Plugin**: `backend/src/plugins/storage.py`
- **API Routes**: `backend/src/api/routes/storage.py`
- **Models**: `backend/src/models/storage.py`
### Frontend
- **Page**: `frontend/src/routes/tools/storage/+page.svelte`
- **Components**: `frontend/src/components/storage/*`

View File

@@ -0,0 +1,17 @@
# Research: File Storage Management & UI
**Decision**: Use Python's built-in `pathlib` and `shutil` for filesystem operations.
**Rationale**: Standard library, robust, cross-platform (though target is Linux), and sufficient for local file management. No external dependencies needed.
**Alternatives considered**: `os` module (lower level, less ergonomic), `pyfilesystem2` (external dependency, unnecessary overhead for simple local storage).
**Decision**: Use `pydantic` for configuration and file metadata models.
**Rationale**: Already used in the project, provides validation and serialization.
**Decision**: Use `multipart/form-data` for file uploads.
**Rationale**: Standard web practice for file uploads. FastAPI supports `UploadFile` natively.
**Decision**: Default storage path strategy.
**Rationale**: The default path will be `../ss-tools-storage` (relative to workspace root) or similar to ensure it sits outside the git repository by default, but allows configuration override.
**Decision**: Path Traversal Prevention.
**Rationale**: Will use `os.path.commonpath` or `pathlib.Path.resolve()` to strictly validate that any accessed file path is a child of the configured storage root.

View File

@@ -0,0 +1,97 @@
# Feature Specification: File Storage Management & UI
**Feature Branch**: `014-file-storage-ui`
**Created**: 2026-01-24
**Status**: Draft
**Input**: User description: "Я хочу проработать механизм хранения файлов и доступа к ним - бекапов дашбордов и репозиториев дашбордов. Во первых, нужно иметь указывать место хранения, по умолчанию оно должно быть за файловой системой сервера (чтобы не влиять на git репозиторий. Во вторых, нужен web ui для базового доступа ко всем файлам - возможность скачивания/удаления, информация о датах создания, возможность загрузки"
## User Scenarios & Testing *(mandatory)*
### User Story 1 - File Management Dashboard (Priority: P1)
Users need a visual interface to manage the artifacts generated by the system (dashboard backups, exported repositories) without needing direct server access. Users must be able to navigate through the folder structure (e.g., `backups/SS2/Sales Dashboard`) to locate specific files.
**Why this priority**: Core functionality requested. Without the UI, the storage mechanism is opaque and hard to use.
**Independent Test**: Can be fully tested by opening the new "File Storage" page, navigating into a subdirectory, uploading a test file, verifying it appears in the list, downloading it, and then deleting it.
**Acceptance Scenarios**:
1. **Given** the File Storage page is open, **When** I view the list, **Then** I see the top-level folders (e.g., `backups`, `repositories`) or files.
2. **Given** I am viewing a folder, **When** I click a subfolder name, **Then** the view updates to show the contents of that subfolder.
3. **Given** I am in a subfolder, **When** I click "Download" on a file, **Then** the file is downloaded to my local machine.
4. **Given** a file exists in the list, **When** I click "Delete" and confirm, **Then** the file is removed from the list and the server filesystem.
5. **Given** I have a file locally, **When** I drag and drop it or use the "Upload" button, **Then** the file is uploaded to the current directory and appears in the list.
---
### User Story 2 - Storage Location Configuration (Priority: P2)
Administrators need to control where potentially large or sensitive files are stored. Crucially, these files must not accidentally pollute the source code repository or the application's working directory.
**Why this priority**: Essential for system stability and cleanliness (preventing git pollution), but the system could theoretically start with a hardcoded safe default.
**Independent Test**: Change the storage path in Settings, generate a file (or upload one), and verify it exists in the new location on the server disk.
**Acceptance Scenarios**:
1. **Given** I am in the Settings page, **When** I enter a new absolute path for "File Storage Path" and save, **Then** the system updates the configuration.
2. **Given** the default configuration, **When** the system starts, **Then** the storage path defaults to a location outside the project's git scope (or is properly ignored).
3. **Given** an invalid path (e.g., no write permissions), **When** I try to save, **Then** the system shows an error message.
---
### Edge Cases
- **File Name Conflicts**: What happens when uploading a file that already exists? (System should likely rename or ask to overwrite).
- **Storage Quota/Disk Space**: What happens if the disk is full during upload?
- **Path Traversal**: Ensure users cannot access files outside the configured storage directory via the API.
- **Large Files**: Handling uploads/downloads of large backup archives (e.g., > 100MB).
## Clarifications
### Session 2026-01-24
- Q: Should the system enforce a structure to keep backups and repositories separate? → A: **Structured**: System creates and enforces `backups/` and `repositories/` folders; UI separates them.
### Session 2026-01-24 (Update)
- Q: Should the system allow advanced configuration of file structure and naming conventions? → A: **Yes**: Users should be able to configure the directory structure (e.g., include dashboard/environment names) and file naming patterns (e.g., include timestamps).
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: System MUST allow configuring a local filesystem root path for storing artifacts.
- **FR-002**: The default storage path MUST be configured such that it does not interfere with the application's git repository (e.g., a directory outside the workspace or explicitly git-ignored).
- **FR-003**: System MUST enforce a directory structure within the storage root: `backups/` for dashboard backups and `repositories/` for exported repositories.
- **FR-004**: System MUST provide a Web UI to list files and folders, organized by their type (Backup vs Repository).
- **FR-005**: System MUST display file metadata in the UI: Filename, Size, Creation Date.
- **FR-006**: System MUST allow users to download files from the storage directory (including subdirectories) via the Web UI.
- **FR-007**: System MUST allow users to delete files from the storage directory via the Web UI.
- **FR-008**: System MUST allow users to upload files to the specific folder in the storage directory via the Web UI.
- **FR-013**: System MUST support navigating through the directory hierarchy within the allowed categories.
- **FR-014**: System MUST display breadcrumbs or similar navigation aid to show current path.
- **FR-009**: System MUST validate that the configured storage path is accessible and writable.
- **FR-010**: System MUST prevent access to files outside the configured storage directory (Path Traversal protection).
- **FR-011**: System MUST allow configuring the directory structure pattern for backups and repositories (e.g., `{environment}/{dashboard_name}/`).
- **FR-012**: System MUST allow configuring the filename pattern for generated files (e.g., `{dashboard_name}_{timestamp}.zip`).
### Key Entities
- **StorageConfig**: Settings defining the root directory path.
- **StoredFile**: Conceptual representation of a file (Name, Path, Size, CreatedAt, MimeType).
### Assumptions
- The application server has access to a writable local filesystem.
- Users utilizing this feature have appropriate permissions within the application to manage system-wide storage settings and files.
## Success Criteria *(mandatory)*
### Measurable Outcomes
- **SC-001**: Users can successfully upload and then download a file of at least 50MB size.
- **SC-002**: Files created or uploaded via the system do not appear in the application's `git status` output by default.
- **SC-003**: File list loads in under 1 second for a directory containing 100 files.
- **SC-004**: Users can delete a file via UI and confirm it is physically removed from the disk.

View File

@@ -0,0 +1,96 @@
# Tasks: File Storage Management & UI
**Branch**: `014-file-storage-ui` | **Spec**: [specs/014-file-storage-ui/spec.md](../014-file-storage-ui/spec.md)
## Phase 1: Setup
*Goal: Initialize backend plugin structure and frontend route scaffolding.*
- [x] T001 Create storage plugin directory and `__init__.py` in `backend/src/plugins/storage/`
- [x] T002 Create storage models file `backend/src/models/storage.py` with `StorageConfig` and `StoredFile` Pydantic models
- [x] T003 Create empty storage route handler `backend/src/api/routes/storage.py` and register in `backend/src/api/routes/__init__.py`
- [x] T004 Create frontend storage route directory `frontend/src/routes/tools/storage/` and empty `+page.svelte`
- [x] T005 Create frontend service `frontend/src/services/storageService.js` stub
## Phase 2: Foundational
*Goal: Implement core backend logic for storage management, configuration, and security.*
- [x] T006 Implement `StoragePlugin` class in `backend/src/plugins/storage/plugin.py` inheriting from `PluginBase`
- [x] T007 Implement `get_storage_root()` method in `StoragePlugin` with default path logic (`../ss-tools-storage`)
- [x] T008 Implement `ensure_directories()` method to create `backups/` and `repositories/` subfolders on init
- [x] T009 Implement path traversal protection helper `validate_path(path)` in `StoragePlugin`
- [x] T010 Implement `list_files(category)` method in `StoragePlugin` returning `StoredFile` objects
- [x] T011 Implement `save_file(file, category)` method in `StoragePlugin` handling uploads
- [x] T012 Implement `delete_file(category, filename)` method in `StoragePlugin`
- [x] T013 Implement `get_file_path(category, filename)` method in `StoragePlugin` for downloads
- [x] T014 Register `StoragePlugin` in `backend/src/core/plugin_loader.py` (if manual registration needed)
## Phase 3: User Story 1 - File Management Dashboard (Priority: P1)
*Goal: Enable users to list, upload, download, and delete files via Web UI.*
### Backend Endpoints
- [x] T015 [US1] Implement `GET /api/storage/files` endpoint in `backend/src/api/routes/storage.py` using `StoragePlugin.list_files`
- [x] T016 [US1] Implement `POST /api/storage/upload` endpoint in `backend/src/api/routes/storage.py` using `StoragePlugin.save_file`
- [x] T017 [US1] Implement `DELETE /api/storage/files/{category}/{filename}` endpoint in `backend/src/api/routes/storage.py`
- [x] T018 [US1] Implement `GET /api/storage/download/{category}/{filename}` endpoint in `backend/src/api/routes/storage.py`
### Frontend Implementation
- [x] T019 [US1] Implement `listFiles`, `uploadFile`, `deleteFile`, `downloadFileUrl` in `frontend/src/services/storageService.js`
- [x] T020 [US1] Create `frontend/src/components/storage/FileList.svelte` to display files in a table with metadata
- [x] T021 [US1] Create `frontend/src/components/storage/FileUpload.svelte` with category selection and drag-drop support
- [x] T022 [US1] Implement main logic in `frontend/src/routes/tools/storage/+page.svelte` to fetch files and handle tabs (Backups vs Repositories)
- [x] T023 [US1] Integrate `FileList` and `FileUpload` components into `+page.svelte`
## Phase 4: User Story 2 - Storage Location Configuration (Priority: P2)
*Goal: Allow administrators to configure the storage root path via Settings.*
### Backend
- [x] T024 [US2] Add `storage_path` field to main configuration model in `backend/src/core/config_models.py` (if not using separate storage config)
- [x] T025 [US2] Implement `GET /api/settings/storage` and `PUT /api/settings/storage` endpoints in `backend/src/api/routes/settings.py` (or `storage.py`)
- [x] T026 [US2] Update `StoragePlugin` to read root path from global configuration instead of hardcoded default
- [x] T027 [US2] Add validation logic to `PUT` endpoint to ensure new path is writable
### Frontend
- [x] T028 [US2] Add `getStorageConfig` and `updateStorageConfig` to `frontend/src/services/storageService.js`
- [x] T029 [US2] Create configuration section in `frontend/src/routes/settings/+page.svelte` (or dedicated Storage Settings component)
- [x] T030 [US2] Implement form to update storage path with validation feedback
- [x] T031 [US2] Add configuration fields for directory structure and filename patterns in `backend/src/models/storage.py` and `frontend/src/routes/settings/+page.svelte`
- [x] T032 [US2] Implement logic in `StoragePlugin` to resolve dynamic paths based on configured patterns
## Phase 5: Polish & Cross-Cutting
*Goal: Finalize UI/UX and ensure robustness.*
- [x] T033 Add link to "File Storage" in main navigation `frontend/src/components/Navbar.svelte`
- [x] T034 Add error handling toasts for failed uploads or file operations
- [x] T035 Verify large file upload support (50MB+) in Nginx/FastAPI config if applicable
- [x] T036 Add confirmation modal for file deletion
## Phase 6: Folder Structure Support (Refactor)
*Goal: Enable hierarchical navigation, nested file management, and downloading.*
- [x] T037 Refactor `StoragePlugin.list_files` in `backend/src/plugins/storage/plugin.py` to accept `subpath` and return directories/files
- [x] T038 Refactor `StoragePlugin` methods (`save_file`, `delete_file`, `get_file_path`) to support nested paths
- [x] T039 Update backend endpoints in `backend/src/api/routes/storage.py` (`GET /files`, `POST /upload`, `DELETE /files`, `GET /download`) to accept `path` parameter
- [x] T040 Update `frontend/src/services/storageService.js` to pass `path` argument in all API calls
- [x] T041 Update `frontend/src/components/storage/FileList.svelte` to display folder icons, handle navigation events, and show breadcrumbs
- [x] T042 Update `frontend/src/components/storage/FileUpload.svelte` to upload to the currently active directory
- [x] T043 Update `frontend/src/routes/tools/storage/+page.svelte` to manage current path state and handle navigation logic
## Dependencies
1. **Phase 1 (Setup)**: No dependencies.
2. **Phase 2 (Foundational)**: Depends on Phase 1.
3. **Phase 3 (US1)**: Depends on Phase 2.
4. **Phase 4 (US2)**: Depends on Phase 2. Can run parallel to Phase 3.
5. **Phase 5 (Polish)**: Depends on Phase 3 and 4.
6. **Phase 6 (Refactor)**: Depends on Phase 3.
## Parallel Execution Examples
- **Backend/Frontend Split**: T015-T018 (Backend Endpoints) can be developed in parallel with T020-T021 (Frontend Components) using mock data.
- **Story Split**: US1 (File Management) and US2 (Configuration) are largely independent after Phase 2 is complete.
## Implementation Strategy
1. **MVP**: Complete Phases 1, 2, and 3. This delivers a working file manager with a default storage location.
2. **Full Feature**: Complete Phase 4 to allow path configuration.
3. **Polish**: Complete Phase 5 for better UX.

View File

@@ -0,0 +1,34 @@
# Specification Quality Checklist: Frontend Navigation Redesign
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: 2026-01-26
**Feature**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
## Content Quality
- [x] No implementation details (languages, frameworks, APIs)
- [x] Focused on user value and business needs
- [x] Written for non-technical stakeholders
- [x] All mandatory sections completed
## Requirement Completeness
- [x] No [NEEDS CLARIFICATION] markers remain
- [x] Requirements are testable and unambiguous
- [x] Success criteria are measurable
- [x] Success criteria are technology-agnostic (no implementation details)
- [x] All acceptance scenarios are defined
- [x] Edge cases are identified
- [x] Scope is clearly bounded
- [x] Dependencies and assumptions identified
## Feature Readiness
- [x] All functional requirements have clear acceptance criteria
- [x] User scenarios cover primary flows
- [x] Feature meets measurable outcomes defined in Success Criteria
- [x] No implementation details leak into specification
## Notes
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`

View File

@@ -0,0 +1,18 @@
# Backup Contracts
## Component: BackupManager
### Props
None (Top-level page component)
### Events
- `on:backup-create`: Triggered when user requests a new backup.
- `on:backup-restore`: Triggered when user requests a restore.
### Data Dependencies
- `GET /api/environments`: Fetch list of available environments.
- `GET /api/storage/files?category=backups`: Fetch list of backup files.
- `POST /api/tasks`: Create new backup task.
- Body: `{ plugin_id: 'superset-backup', params: { environment_id: string } }`
- `PUT /api/environments/{id}/schedule`: Update backup schedule.
- Body: `{ enabled: boolean, cron_expression: string }`

View File

@@ -0,0 +1,47 @@
# Data Model: Frontend Navigation Redesign
## Plugin Configuration
The `PluginConfig` model is extended to support backend-driven navigation.
```python
class PluginConfig(BaseModel):
"""Pydantic model for plugin configuration."""
id: str = Field(..., description="Unique identifier for the plugin")
name: str = Field(..., description="Human-readable name for the plugin")
description: str = Field(..., description="Brief description of what the plugin does")
version: str = Field(..., description="Version of the plugin")
ui_route: Optional[str] = Field(None, description="Frontend route for the plugin UI")
input_schema: Dict[str, Any] = Field(..., description="JSON schema for input parameters", alias="schema")
```
### ui_route
- **Type**: `Optional[str]`
- **Description**: Specifies the client-side route (URL path) where the plugin's custom UI is hosted.
- **Behavior**:
- If `None` (default): The dashboard will open the plugin using the generic `DynamicForm` modal.
- If set (e.g., `"/tools/mapper"`): The dashboard will navigate (`goto`) to this route when the plugin card is clicked.
## Backup Management (New)
### Backup Types
```typescript
// frontend/src/types/backup.ts
export interface BackupFile {
name: string; // e.g., "prod-dashboard-export-2024.zip"
path: string; // Relative path in storage
size: number; // Bytes
created_at: string; // ISO Date
category: 'backups'; // Fixed category
mime_type?: string;
}
export interface BackupState {
isLoading: boolean;
files: BackupFile[];
error: string | null;
selectedBackup: BackupFile | null;
}

View File

@@ -0,0 +1,88 @@
# Implementation Plan: Frontend Navigation Redesign
**Branch**: `015-frontend-nav-redesign` | **Date**: 2026-01-26 | **Spec**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
**Input**: Feature specification from `specs/015-frontend-nav-redesign/spec.md`
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/commands/plan.md` for the execution workflow.
## Summary
This feature redesigns the frontend navigation to shift from a Navbar-heavy approach to a Dashboard-centric model. Key changes include moving tool access (Mapper, Storage, Backups) to the Dashboard, simplifying the Navbar to global contexts (Tasks, Settings), removing deprecated features (Dataset Search, Environments), and implementing a dedicated Backup Management UI based on backend capabilities from feature 009.
Additionally, the navigation architecture is refactored to be backend-driven. Plugins now expose a `ui_route` property, allowing the frontend to dynamically determine the correct navigation path without hardcoded mapping.
## Technical Context
**Language/Version**: Python 3.9+ (Backend), Node.js 18+ (Frontend)
**Primary Dependencies**: FastAPI (Backend), SvelteKit + Tailwind CSS (Frontend)
**Storage**: N/A (UI reorganization and API integration)
**Testing**: Playwright (E2E - if available), Vitest (Unit)
**Target Platform**: Web Browser
**Project Type**: Web Application (Frontend + Backend)
**Performance Goals**: Instant navigation (<100ms), fast dashboard load
**Constraints**: Must maintain responsive design; Backup UI must interface with existing backend endpoints
**Scale/Scope**: ~5-10 file modifications, 1 new major component (BackupManager)
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
- [x] **Semantic Protocol Compliance**: All new components will follow Svelte component header standards.
- [x] **Causal Validity**: Contracts (props/events) will be defined before implementation.
- [x] **Immutability of Architecture**: No core architectural changes; only UI reorganization.
- [x] **Design by Contract**: New Backup component will define clear interface contracts.
- [x] **Everything is a Plugin**: N/A (Frontend changes primarily, backend remains plugin-based).
- [x] **Unified Frontend Experience**: All new UI components will use standardized components and internationalization (i18n).
## Project Structure
### Documentation (this feature)
```text
specs/015-frontend-nav-redesign/
├── plan.md # This file
├── research.md # Phase 0 output
├── data-model.md # Phase 1 output
├── quickstart.md # Phase 1 output
├── contracts/ # Phase 1 output
└── tasks.md # Phase 2 output
```
### Source Code (repository root)
```text
backend/
├── src/
│ ├── core/
│ │ ├── plugin_base.py # (Modify: Add ui_route property)
│ │ └── plugin_loader.py # (Modify: Populate ui_route in PluginConfig)
│ ├── plugins/ # (Modify: Implement ui_route in all plugins)
│ └── api/routes/ # (Verify backup routes exist)
frontend/
├── src/
│ ├── components/
│ │ ├── Navbar.svelte # (Modify: Simplify items)
│ │ ├── DashboardGrid.svelte # (Modify: Add tool links)
│ │ └── backups/ # (New: Backup UI)
│ │ ├── BackupManager.svelte
│ │ └── BackupList.svelte
│ ├── pages/
│ │ └── Dashboard.svelte # (Modify: Layout updates)
│ └── routes/
│ ├── +layout.svelte # (Check global nav injection)
│ ├── +page.svelte # (Modify: Use plugin.ui_route for navigation)
│ └── tools/
│ └── backups/ # (New Route)
│ └── +page.svelte
```
**Structure Decision**: Standard SvelteKit structure. New `backups` component directory for the complex backup UI. Route added under `tools/` to match existing pattern (mapper, storage).
## Complexity Tracking
> **Fill ONLY if Constitution Check has violations that must be justified**
| Violation | Why Needed | Simpler Alternative Rejected Because |
|-----------|------------|-------------------------------------|
| N/A | | |

View File

@@ -0,0 +1,29 @@
# Quickstart: Frontend Navigation Redesign
## Overview
This feature reorganizes the application navigation to be dashboard-centric and introduces a dedicated UI for Backup Management.
## New Routes
- `/tools/backups`: The new Backup Management interface.
- `/`: Dashboard (updated with new tool links).
## Removed Routes
- `/tools/search`: Deprecated and removed.
- `/settings/environments`: Deprecated and removed.
## Development
### Running the Backup UI
1. Ensure the backend is running: `cd backend && uvicorn src.app:app --reload`
2. Start the frontend: `cd frontend && npm run dev`
3. Navigate to `http://localhost:5173/tools/backups`
### Key Components
- `frontend/src/components/backups/BackupManager.svelte`: Main container for backup operations.
- `frontend/src/components/DashboardGrid.svelte`: Updated grid with new tool cards.
- `frontend/src/components/Navbar.svelte`: Simplified navigation bar.
## Verification
1. Check Dashboard: Should see cards for Mapper, Storage, and Backups.
2. Check Navbar: Should ONLY show Tasks and Settings.
3. Check Backup Tool: Should load and display backup status/controls.

View File

@@ -0,0 +1,23 @@
# Research: Frontend Navigation Redesign
## Decisions
### 1. Backup Management UI Strategy
**Decision**: Create a dedicated `BackupManager` component in `frontend/src/components/backups/`.
**Rationale**: The requirement is to have a "full component" accessible from the dashboard. Separating it into its own directory ensures modularity and keeps the Dashboard component clean. It will consume the existing backup APIs (likely `/api/tasks` with specific backup types or a dedicated backup endpoint if one exists - *to be verified in Phase 1*).
**Alternatives considered**: Embedding backup controls directly in the Dashboard (rejected: clutters the main view), reusing the TaskRunner component (rejected: need specific backup context/history view).
### 2. Navigation State Management
**Decision**: Use SvelteKit's layout system (`+layout.svelte`) and simple component props for Navbar state.
**Rationale**: The Navbar changes are global. Removing items is a static change to the `Navbar.svelte` component. No complex state management (stores) is needed for this structural change.
**Alternatives considered**: Dynamic config-based menu (rejected: overkill for this specific redesign).
### 3. Deprecation Strategy
**Decision**: Hard removal of "Dataset Search" and "Deployment Environments" components and routes.
**Rationale**: The spec explicitly calls for removal. Keeping dead code increases maintenance burden.
**Alternatives considered**: Hiding behind a feature flag (rejected: requirement is explicit removal).
### 4. Dashboard Grid Layout
**Decision**: Update `DashboardGrid.svelte` to include new cards for Mapper, Storage, and Backups.
**Rationale**: Reusing the existing grid component maintains consistency.
**Alternatives considered**: Creating a separate "Tools" page (rejected: spec requires access from main Dashboard).

View File

@@ -0,0 +1,97 @@
# Feature Specification: Frontend Navigation Redesign
**Feature Branch**: `015-frontend-nav-redesign`
**Created**: 2026-01-26
**Status**: Draft
**Input**: User description: "Я хочу провести редизайн фронта в части навигации. 1. Удалить Dataset Search (из Navbar и дашборда), Deployment Environments 2. Вкладку Tasks оставить для просмотра всех задач - убрать оттуда кнопку Run backup 3. Должен быть полноценный компонент бэкапов, как мы разрабатывали в 009-backup-scheduler. Доступ - из дашборда главного 4. Перенести ссылку на Dataset mapper из Navbar на дашборд 5. Перенести ссылку на Storage manager на дашборд Общая логика - на дашборде должны быть ссылки на полноценные инструменты, навбар - для настроек и общей Tasks"
## Clarifications
### Session 2026-01-26
- Q: Do I need to build the Backup Management UI from scratch? → A: Yes, create the UI for backup using data from task 009-backup-scheduler.
## User Scenarios & Testing *(mandatory)*
### User Story 1 - Centralized Tool Access via Dashboard (Priority: P1)
As a user, I want to access all main tools (Backups, Mapper, Storage) from the main Dashboard so that I have a central hub for operations.
**Why this priority**: This is the core of the redesign, shifting the navigation paradigm to a dashboard-centric model for tools.
**Independent Test**: Can be tested by verifying the Dashboard contains links/cards for Backups, Mapper, and Storage, and that clicking them navigates to the correct full-page tools.
**Acceptance Scenarios**:
1. **Given** I am on the main Dashboard, **When** I look at the available tools, **Then** I see options for "Backup Manager", "Dataset Mapper", and "Storage Manager".
2. **Given** I am on the Dashboard, **When** I click "Backup Manager", **Then** I am taken to the full Backup management interface.
3. **Given** I am on the Dashboard, **When** I click "Dataset Mapper", **Then** I am taken to the Mapper tool.
4. **Given** I am on the Dashboard, **When** I click "Storage Manager", **Then** I am taken to the Storage tool.
---
### User Story 2 - Simplified Navigation Bar (Priority: P1)
As a user, I want a clean Navbar containing only global context items (Tasks, Settings) so that the interface is less cluttered and navigation is distinct from tool usage.
**Why this priority**: Enforces the separation of concerns between "Global Status/Settings" (Navbar) and "Operational Tools" (Dashboard).
**Independent Test**: Can be tested by inspecting the Navbar across the application to ensure removed items (Search, Mapper, Environments) are gone and only Tasks and Settings remain.
**Acceptance Scenarios**:
1. **Given** I am on any page, **When** I view the Navbar, **Then** I do NOT see links for "Dataset Search", "Dataset Mapper", or "Deployment Environments".
2. **Given** I am on any page, **When** I view the Navbar, **Then** I see links for "Tasks" and "Settings".
3. **Given** I am on the Tasks page, **When** I look for the "Run backup" button, **Then** it is NOT present (as it belongs in the Backup tool).
---
### User Story 3 - Deprecation of Unused Features (Priority: P2)
As a user, I want removed features (Dataset Search, Deployment Environments) to be inaccessible so that I don't use deprecated workflows.
**Why this priority**: Cleans up the UI and prevents confusion with features that are being removed or hidden.
**Independent Test**: Verify that UI elements for Dataset Search and Deployment Environments are removed from both Navbar and Dashboard.
**Acceptance Scenarios**:
1. **Given** I am on the Dashboard, **When** I look for "Dataset Search", **Then** it is not visible.
2. **Given** I am on the Dashboard or Navbar, **When** I look for "Deployment Environments", **Then** it is not visible.
### Edge Cases
- **Direct URL Access**: If a user attempts to access the URL of a removed page (e.g., `/search` or `/environments`) via bookmark or history, they should be redirected to the Dashboard or shown a 404 page (standard app behavior).
- **Mobile View**: The simplified Navbar must remain responsive; with fewer items, it should likely avoid collapsing into a hamburger menu unless necessary on very small screens.
### Assumptions
- The backend logic and UI components for the "Backup Scheduler" (Feature 009) are available and ready to be integrated into the main Dashboard view.
- Existing tools (Dataset Mapper, Storage Manager) function independently of the Navbar context and will work correctly when accessed via the Dashboard.
## Requirements *(mandatory)*
### Functional Requirements
- **FR-001**: System MUST display "Dataset Mapper" entry point on the main Dashboard.
- **FR-002**: System MUST display "Storage Manager" entry point on the main Dashboard.
- **FR-003**: System MUST display "Backup Scheduler" (or similar name) entry point on the main Dashboard.
- **FR-004**: System MUST provide access to the full Backup Management component (newly created based on feature 009 data) via the Dashboard link.
- **FR-005**: Navbar MUST NOT contain links to "Dataset Search", "Dataset Mapper", or "Deployment Environments".
- **FR-006**: Dashboard MUST NOT contain "Dataset Search" widget or link.
- **FR-007**: Tasks page MUST NOT show the "Run backup" button (backup initiation moves to Backup tool).
- **FR-008**: Navbar MUST retain "Tasks" and "Settings" links.
- **FR-009**: Backup Manager MUST support configuring automated backup schedules (enabled/disabled, cron expression) per environment.
- **FR-010**: Backup List MUST provide a "Go to Storage" action that navigates to the Storage Manager with the correct path selected.
### Key Entities
- **Dashboard**: The main landing page serving as the registry for tools.
- **Navbar**: The persistent top navigation for global application state/config.
## Success Criteria *(mandatory)*
### Measurable Outcomes
- **SC-001**: Users can navigate to Dataset Mapper, Storage Manager, and Backup Manager within 1 click from the Dashboard.
- **SC-002**: Navbar contains strictly 0 links to operational tools (Mapper, Search, Storage), containing only Tasks and Settings.
- **SC-003**: "Run backup" action is successfully performed via the new Dashboard -> Backup route.

View File

@@ -0,0 +1,116 @@
# Tasks: Frontend Navigation Redesign
**Feature Branch**: `015-frontend-nav-redesign`
**Spec**: [specs/015-frontend-nav-redesign/spec.md](../spec.md)
**Plan**: [specs/015-frontend-nav-redesign/plan.md](../plan.md)
## Phase 1: Setup
*Goal: Initialize project structure for the new feature.*
- [x] T001 Create backup component directory structure
- Path: `frontend/src/components/backups/`
- [x] T002 Create backup page route directory
- Path: `frontend/src/routes/tools/backups/`
## Phase 2: Foundational
*Goal: Prepare core components for integration and verify backend connectivity.*
- [x] T003 Verify backend API endpoints for backups (via `009-backup-scheduler`)
- Path: `backend/src/api/routes/tasks.py` (or relevant backup route)
- [x] T004 Define Backup types and interfaces in frontend
- Path: `frontend/src/types/backup.ts`
## Phase 3: User Story 1 - Centralized Tool Access via Dashboard
*Goal: Update the main dashboard to include all tool entry points.*
**User Story**: As a user, I want to access all main tools (Backups, Mapper, Storage) from the main Dashboard so that I have a central hub for operations. (P1)
- [x] T005 [US1] Update DashboardGrid to include "Backup Manager" card
- Path: `frontend/src/components/DashboardGrid.svelte`
- [x] T006 [US1] Update DashboardGrid to ensure "Dataset Mapper" and "Storage Manager" cards are present
- Path: `frontend/src/components/DashboardGrid.svelte`
- [x] T007 [US1] Remove "Dataset Search" card from DashboardGrid
- Path: `frontend/src/components/DashboardGrid.svelte`
## Phase 4: User Story 2 - Simplified Navigation Bar
*Goal: Clean up the Navbar to show only global context items.*
**User Story**: As a user, I want a clean Navbar containing only global context items (Tasks, Settings) so that the interface is less cluttered and navigation is distinct from tool usage. (P1)
- [x] T008 [US2] Remove "Dataset Search" link from Navbar
- Path: `frontend/src/components/Navbar.svelte`
- [x] T009 [US2] Remove "Dataset Mapper" link from Navbar
- Path: `frontend/src/components/Navbar.svelte`
- [x] T010 [US2] Remove "Deployment Environments" link from Navbar
- Path: `frontend/src/components/Navbar.svelte`
- [x] T011 [US2] Verify "Tasks" and "Settings" links remain in Navbar
- Path: `frontend/src/components/Navbar.svelte`
- [x] T012 [US2] Remove "Run backup" button from Tasks page
- Path: `frontend/src/routes/tasks/+page.svelte` (or relevant component)
## Phase 5: Backup Management UI
*Goal: Implement the dedicated Backup Management interface.*
**User Story**: (Implicit P1 from FR-004) System MUST provide access to the full Backup Management component via the Dashboard link.
- [x] T013 [US1] Create BackupList component to display existing backups (Must use `src/lib/ui` components and `src/lib/i18n`)
- Path: `frontend/src/components/backups/BackupList.svelte`
- [x] T014 [US1] Create BackupManager main component (container) (Must use `src/lib/ui` components and `src/lib/i18n`)
- Path: `frontend/src/components/backups/BackupManager.svelte`
- [x] T015 [US1] Implement "Create Backup" functionality in BackupManager (Must use `src/lib/ui` components and `src/lib/i18n`)
- Path: `frontend/src/components/backups/BackupManager.svelte`
- [x] T016 [US1] Implement "Restore Backup" functionality (if supported by backend) (Must use `src/lib/ui` components and `src/lib/i18n`)
- Path: `frontend/src/components/backups/BackupManager.svelte`
- [x] T017 [US1] Create Backup page to host the manager (Must use `src/lib/ui` components and `src/lib/i18n`)
- Path: `frontend/src/routes/tools/backups/+page.svelte`
- [x] T017b [US1] Implement Backup Schedule configuration in BackupManager
- Path: `frontend/src/components/backups/BackupManager.svelte`
- [x] T017c [US1] Implement "Go to Storage" navigation in BackupList
- Path: `frontend/src/components/backups/BackupList.svelte`
## Phase 6: User Story 3 - Deprecation
*Goal: Remove deprecated routes and code.*
**User Story**: As a user, I want removed features (Dataset Search, Deployment Environments) to be inaccessible so that I don't use deprecated workflows. (P2)
- [x] T018 [US3] Delete Dataset Search route
- Path: `frontend/src/routes/tools/search/` (delete directory)
- [x] T019 [US3] Delete Deployment Environments route
- Path: `frontend/src/routes/settings/environments/` (delete directory)
- [x] T020 [US3] Delete Dataset Search component (if not used elsewhere)
- Path: `frontend/src/components/tools/SearchTool.svelte`
- [x] T021 [US3] Delete EnvSelector component (if not used elsewhere)
- Path: `frontend/src/components/EnvSelector.svelte`
## Phase 7: Polish & Cross-Cutting
*Goal: Final verification and cleanup.*
- [x] T022 Verify all navigation links work correctly
- Path: `frontend/src/components/Navbar.svelte`
- [x] T023 Verify responsive layout of new Dashboard grid
- Path: `frontend/src/components/DashboardGrid.svelte`
- [x] T024 Ensure i18n strings are extracted for new Backup UI
- Path: `frontend/src/lib/i18n/` (or relevant locale files)
- [x] T025 Verify "Run backup" action successfully triggers backup job (Manual/E2E check)
- Path: `frontend/src/components/backups/BackupManager.svelte`
## Dependencies
1. **Phase 1 & 2** must be completed first.
2. **Phase 3 (Dashboard)** and **Phase 4 (Navbar)** can be done in parallel.
3. **Phase 5 (Backup UI)** depends on Phase 1 & 2.
4. **Phase 6 (Deprecation)** should be done last to ensure no regressions before removal.
## Implementation Strategy
1. **Setup**: Create the new directory structure.
2. **Dashboard & Navbar**: Quick wins to reshape the navigation.
3. **Backup UI**: The core development effort. Connect to existing backend.
4. **Cleanup**: Remove old code once the new flows are verified.

File diff suppressed because it is too large Load Diff