# Feature Specification: Task Logging System (Separated Per-Task Logs)

**Feature Branch**: `018-task-logging-v2`  
**Reference UX**: `specs/018-task-logging-v2/ux_reference.md`
**Created**: 2026-02-07  
**Status**: Draft  
**Input**: User description: "Implement a separated per-task logging system with source attribution, persistence, and frontend filtering."

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Real-time Filtered Logging (Priority: P1)

As a user, I want to see logs from a running task filtered by their source so that I can focus on specific component behavior without noise.

**Why this priority**: This is the core value proposition—isolating component logs during execution.

**Independent Test**: Start a task that emits logs from multiple sources (e.g., `plugin` and `system`), apply a source filter in the UI, and verify only matching logs are displayed in real-time.

**Acceptance Scenarios**:

1. **Given** a task is running and emitting logs from "plugin" and "superset_api", **When** I select "superset_api" in the Source filter, **Then** only logs with the "superset_api" tag are visible.
2. **Given** a filter is active, **When** a new log matching the filter arrives via WebSocket, **Then** it is immediately appended to the view.

---

### User Story 2 - Persistent Log History (Priority: P1)

As a user, I want my task logs to be saved permanently so that I can review them even after the server restarts.

**Why this priority**: Current logs are in-memory and lost on restart, which is a major pain point for debugging past failures.

**Independent Test**: Run a task, restart the backend server, and verify that the logs for that task are still accessible via the UI.

**Acceptance Scenarios**:

1. **Given** a task has completed, **When** I restart the server and navigate to the task details, **Then** all logs are loaded from the database.
2. **Given** a task was deleted via the cleanup service, **When** I check the database, **Then** all associated logs are also removed.

---

### User Story 3 - Component Attribution (Priority: P2)

As a developer, I want to use a dedicated logger that automatically tags my logs with the correct source.

**Why this priority**: Simplifies plugin development and ensures consistent data for filtering.

**Independent Test**: Update a plugin to use `context.logger.with_source("custom")` and verify that logs appear with the "custom" tag.

**Acceptance Scenarios**:

1. **Given** a plugin uses the new `TaskContext`, **When** it calls `logger.info()`, **Then** the log is recorded with `source="plugin"` by default.
2. **Given** a plugin creates a sub-logger with `with_source("api")`, **When** it logs a message, **Then** the log is recorded with `source="api"`.

---

### Edge Cases

- **High Volume Logs**: How does the system handle a task generating 10,000+ logs in a few seconds? (Requirement: Batching and virtual scrolling).
- **Database Connection Loss**: What happens if the log flusher cannot write to the DB? (Requirement: Logs should remain in-memory as a fallback until the next flush).
- **Concurrent Tasks**: Ensure logs from Task A never leak into the view or database records of Task B.

## Requirements *(mandatory)*

### Functional Requirements

- **FR-001**: System MUST persist task logs in a dedicated `task_logs` database table.
- **FR-002**: Each log entry MUST include: `task_id`, `timestamp`, `level`, `message`, `source`, and optional `metadata`.
- **FR-003**: System MUST provide a `TaskLogger` via a `TaskContext` to all plugins.
- **FR-004**: System MUST support batch-writing logs to the database (e.g., every 2 seconds) to maintain performance.
- **FR-005**: Backend MUST support server-side filtering of logs by `level`, `source`, and text `search` via REST and WebSocket.
- **FR-006**: Frontend MUST provide a UI for filtering and searching logs within the task details view.
- **FR-007**: System MUST maintain backward compatibility for plugins using the old `execute` signature.
- **FR-008**: System MUST automatically delete logs when their associated task is deleted.
- **FR-009**: Task logs MUST follow the same retention policy as task records (logs are kept as long as the task record exists).
- **FR-010**: The default log level filter in the UI MUST be set to INFO and above (hiding DEBUG logs by default).
- **FR-011**: System MUST support filtering logs by specific keys within the `metadata` JSON (e.g., `dashboard_id`, `database_uuid`).

## Clarifications

### Session 2026-02-07
- Q: Should task logs follow the same retention period as the task records themselves? → A: Same as Task Records.
- Q: What should be the default log level filter when a user opens the task log panel? → A: INFO and above.
- Q: Do we need to support searching or filtering inside the JSON metadata? → A: Searchable keys (e.g., `dashboard_id` should be filterable).

### Key Entities *(include if feature involves data)*

- **TaskLog**: Represents a single log message.
    - Attributes: `id` (PK), `task_id` (FK), `timestamp`, `level` (Enum/String), `source` (String), `message` (Text), `metadata` (JSON).
- **TaskContext**: Execution context provided to plugins.
    - Attributes: `task_id`, `logger`, `config`.

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001**: Log retrieval for a task with 1,000 entries takes less than 200ms.
- **SC-002**: Database write overhead for logging does not increase task execution time by more than 5%.
- **SC-003**: 100% of logs generated by a task are available after a server restart.
- **SC-004**: Users can filter logs by source and see results in under 100ms on the frontend.