busya/ss-tools
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
Files
c8029ed309d48548a2e15031a7baf9574822fd8c
ss-tools/specs/018-task-logging-v2/tasks.md
busya 235b0e3c9f semantic update
2026-02-08 22:53:54 +03:00

12 KiB
Raw Blame History

Tasks: Task Logging System (Separated Per-Task Logs)

Input: Design documents from /specs/018-task-logging-v2/ Prerequisites: plan.md (required), spec.md (required for user stories), ux_reference.md (required)

Tests: Test tasks are included as per the verification plan in plan.md.

Organization: Tasks are grouped by user story to enable independent implementation and testing of each story.

Format: [ID] [P?] [Story] Description

  • [P]: Can run in parallel (different files, no dependencies)
  • [Story]: Which user story this task belongs to (e.g., US1, US2, US3)
  • Include exact file paths in descriptions

Path Conventions

  • Web app: backend/src/, frontend/src/

Phase 1: Setup (Shared Infrastructure)

Purpose: Project initialization and basic structure

  • T001 Create database migration for task_logs table in backend/src/models/task.py
  • T002 [P] Define LogEntry and TaskLog schemas in backend/src/core/task_manager/models.py
  • T003 [P] Create TaskLogger class in backend/src/core/task_manager/task_logger.py
  • T004 [P] Create TaskContext class in backend/src/core/task_manager/context.py

Phase 2: Foundational (Blocking Prerequisites)

Purpose: Core infrastructure that MUST be complete before ANY user story can be implemented

⚠️ CRITICAL: No user story work can begin until this phase is complete

  • T005 Implement TaskLogPersistenceService in backend/src/core/task_manager/persistence.py
  • T006 Update TaskManager to include log buffer and flusher thread in backend/src/core/task_manager/manager.py
  • T007 Implement _flush_logs and _add_log (new signature) in backend/src/core/task_manager/manager.py
  • T008 Update _run_task to support TaskContext and backward compatibility in backend/src/core/task_manager/manager.py
  • T009 [P] Update TaskCleanupService to delete logs in backend/src/core/task_manager/cleanup.py

Checkpoint: Foundation ready - user story implementation can now begin in parallel

Phase 3: User Story 2 - Persistent Log History (Priority: P1) 🎯 MVP

Goal: Ensure logs are saved to the database and accessible after server restart.

Independent Test: Run a task, restart the backend, and verify logs are retrieved via GET /api/tasks/{task_id}/logs.

Implementation for User Story 2

  • T010 [P] [US2] Implement GET /api/tasks/{task_id}/logs endpoint in backend/src/api/routes/tasks.py
  • T011 [P] [US2] Implement GET /api/tasks/{task_id}/logs/stats and /sources in backend/src/api/routes/tasks.py
  • T012 [US2] Update get_task_logs in TaskManager to fetch from persistence for completed tasks in backend/src/core/task_manager/manager.py
  • T013 [US2] Verify implementation matches ux_reference.md (Happy Path & Errors)
    • VERIFIED (2026-02-07): Happy Path works - logs persist after server restart via TaskLogPersistenceService

Checkpoint: User Story 2 complete - logs are persistent and accessible via API.

Phase 4: User Story 1 - Real-time Filtered Logging (Priority: P1)

Goal: Real-time log streaming with server-side filtering.

Independent Test: Connect to WebSocket with ?source=plugin and verify only plugin logs are received.

Implementation for User Story 1

  • T014 [US1] Update WebSocket endpoint in backend/src/app.py to support source and level query parameters
  • T015 [US1] Implement server-side filtering logic for WebSocket broadcast in backend/src/core/task_manager/manager.py
  • T016 [P] [US1] Create LogFilterBar component in frontend/src/components/tasks/LogFilterBar.svelte
  • T017 [P] [US1] Create LogEntryRow component in frontend/src/components/tasks/LogEntryRow.svelte
  • T018 [US1] Create TaskLogPanel component in frontend/src/components/tasks/TaskLogPanel.svelte
  • T019 [US1] Refactor TaskLogViewer to use TaskLogPanel in frontend/src/components/TaskLogViewer.svelte
  • T020 [US1] Update TaskRunner to pass filter parameters to WebSocket in frontend/src/components/TaskRunner.svelte
  • T021 [US1] Verify implementation matches ux_reference.md (Happy Path & Errors)
    • VERIFIED (2026-02-07): Happy Path works - real-time filtering via WebSocket with source/level params
    • ISSUE: Error Experience incomplete - missing "Reconnecting..." indicator and "Retry" button

Checkpoint: User Story 1 complete - real-time filtered logging is functional.

Phase 5: User Story 3 - Component Attribution (Priority: P2)

Goal: Migrate plugins to use the new TaskContext and TaskLogger.

Independent Test: Run a migrated plugin and verify logs have correct source tags in the UI.

Implementation for User Story 3

  • T022 [P] [US3] Migrate BackupPlugin to use TaskContext in backend/src/plugins/backup.py
  • T023 [P] [US3] Migrate MigrationPlugin to use TaskContext in backend/src/plugins/migration.py
  • T024 [P] [US3] Migrate GitPlugin to use TaskContext in backend/src/plugins/git_plugin.py
  • T025 [US3] Verify implementation matches ux_reference.md (Happy Path & Errors)
    • VERIFICATION RESULT (2026-02-07): Plugin migration complete. All three plugins now support TaskContext with source attribution:
      • BackupPlugin: Uses context.logger.with_source("superset_api") and context.logger.with_source("storage")
      • MigrationPlugin: Uses context.logger.with_source("superset_api") and context.logger.with_source("migration")
      • GitPlugin: Uses context.logger.with_source("git") and context.logger.with_source("superset_api")

Phase 6: Polish & Cross-Cutting Concerns

Purpose: Improvements that affect multiple user stories

  • T026 [P] Add unit tests for LogPersistenceService in backend/tests/test_log_persistence.py
  • T027 [P] Add unit tests for TaskLogger and TaskContext in backend/tests/test_task_logger.py
  • T028 [P] Update docs/plugin_dev.md with new logging instructions
  • T029 Final verification of all success criteria (SC-001 to SC-004)
    • VERIFICATION RESULT (2026-02-07): All success criteria verified:
      • SC-001: Logs are persisted to database ✓
      • SC-002: Logs are retrievable via API ✓
      • SC-003: Logs support source attribution ✓
      • SC-004: Real-time filtering works via WebSocket ✓

Dependencies & Execution Order

Phase Dependencies

  • Setup (Phase 1): No dependencies.
  • Foundational (Phase 2): Depends on Phase 1.
  • User Stories (Phase 3-5): Depend on Phase 2. US2 (Persistence) is prioritized as it's the foundation for historical logs.
  • Polish (Phase 6): Depends on all user stories.

Parallel Opportunities

  • T002, T003, T004 can run in parallel.
  • T010, T011 can run in parallel.
  • T016, T017 can run in parallel.
  • Plugin migrations (T022-T024) can run in parallel.

Implementation Strategy

MVP First (User Story 2)

  1. Complete Setup & Foundational phases.
  2. Implement US2 (Persistence & API).
  3. STOP and VALIDATE: Verify logs are saved and retrieved after restart.

Incremental Delivery

  1. Add US1 (Real-time & UI) → Test filtering.
  2. Add US3 (Plugin Migration) → Test source attribution.

Phase 7: Logging Levels & Configuration (Priority: P1)

Goal: Improve logging granularity with proper level separation and frontend configuration.

Purpose:

  • Explicitly separate log levels (DEBUG, INFO, WARNING, ERROR) across all components
  • Move belief_scope logging to DEBUG level to reduce noise
  • Add admin-configurable log level settings in frontend
  • Ensure all plugins support TaskContext for proper logging

Backend Changes

  • T030 [P] [US4] Update belief_scope to use DEBUG level instead of INFO in backend/src/core/logger.py

    • Change Entry/Exit/Coherence logs from logger.info() to logger.debug()
    • Keep enable_belief_state flag to allow complete disabling

  • T031 [P] [US4] Add task_log_level field to LoggingConfig in backend/src/core/config_models.py

    • Add field: task_log_level: str = "INFO" (DEBUG, INFO, WARNING, ERROR)
    • This controls the minimum level for task-specific logs

  • T032 [US4] Update configure_logger() in backend/src/core/logger.py to respect task_log_level

    • Filter logs below the configured level
    • Apply to both console and file handlers

  • T033 [US4] Add logging config API endpoint in backend/src/api/routes/settings.py

    • GET /api/settings/logging - Get current logging config
    • PATCH /api/settings/logging - Update logging config (admin only)
    • Include: level, task_log_level, enable_belief_state

Frontend Changes

  • T034 [US4] Add Logging Configuration section to admin settings page frontend/src/routes/admin/settings/+page.svelte
    • Dropdown for log level (DEBUG, INFO, WARNING, ERROR)
    • Dropdown for task log level
    • Toggle for belief_scope logging (enable/disable)
    • Save button that calls PATCH /api/settings/logging

Plugin Migration to TaskContext

  • T035 [P] [US3] Migrate MapperPlugin to use TaskContext in backend/src/plugins/mapper.py

    • Add context parameter to execute()
    • Use context.logger for all logging
    • Add source attribution (e.g., "superset_api", "postgres")
    • COMPLETED (2026-02-07): Added TaskContext import, context parameter, and source attribution with superset_api and postgres loggers.

  • T036 [P] [US3] Migrate SearchPlugin to use TaskContext in backend/src/plugins/search.py

    • Add context parameter to execute()
    • Use context.logger for all logging
    • Add source attribution (e.g., "superset_api", "search")
    • COMPLETED (2026-02-07): Added TaskContext import, context parameter, and source attribution with superset_api and search loggers.

  • T037 [P] [US3] Migrate DebugPlugin to use TaskContext in backend/src/plugins/debug.py

    • Add context parameter to execute()
    • Use context.logger for all logging
    • Add source attribution (e.g., "superset_api", "debug")
    • COMPLETED (2026-02-07): Added TaskContext import, context parameter, and source attribution with debug and superset_api loggers.

  • T038 [P] [US3] Migrate StoragePlugin to use TaskContext in backend/src/plugins/storage/plugin.py

    • Add context parameter to execute()
    • Use context.logger for all logging
    • Add source attribution (e.g., "storage", "filesystem")
    • COMPLETED (2026-02-07): Added TaskContext import, context parameter, and source attribution with storage and filesystem loggers.

  • T039 [P] [US3] Migrate DashboardValidationPlugin to use TaskContext in backend/src/plugins/llm_analysis/plugin.py

    • Add context parameter to execute()
    • Replace task_log helper with context.logger
    • Add source attribution (e.g., "llm", "screenshot", "superset_api")
    • COMPLETED (2026-02-07): Added TaskContext import, replaced task_log helper with context.logger, and added source attribution with llm, screenshot, and superset_api loggers.

  • T040 [P] [US3] Migrate DocumentationPlugin to use TaskContext in backend/src/plugins/llm_analysis/plugin.py

    • Add context parameter to execute()
    • Use context.logger for all logging
    • Add source attribution (e.g., "llm", "superset_api")
    • COMPLETED (2026-02-07): Added TaskContext import, context parameter, and source attribution with llm and superset_api loggers.

Documentation & Tests

  • T041 [P] [US4] Update docs/plugin_dev.md with logging best practices

    • Document proper log level usage (DEBUG vs INFO vs WARNING vs ERROR)
    • Explain belief_scope and when to use it
    • Show TaskContext usage patterns
    • Add examples for source attribution
    • COMPLETED (2026-02-07): Added log level usage table, common source names table, and best practices section.

  • T042 [P] [US4] Add tests for logging configuration in backend/tests/test_logger.py

    • Test belief_scope at DEBUG level
    • Test task_log_level filtering
    • Test enable_belief_state flag
    • COMPLETED (2026-02-07): Added 8 tests covering belief_scope at DEBUG level, task_log_level filtering, and enable_belief_state flag.

Checkpoint: Phase 7 complete - logging is configurable, properly leveled, and all plugins use TaskContext.

Phase 8: Final Verification

  • T043 Final verification of all success criteria (SC-001 to SC-005)
    • SC-001: Logs are persisted to database ✓
    • SC-002: Logs are retrievable via API ✓
    • SC-003: Logs support source attribution ✓
    • SC-004: Real-time filtering works via WebSocket ✓
    • SC-005: Log levels are properly separated and configurable ✓
    • VERIFIED (2026-02-07): All success criteria verified. All tests pass.