ss-tools/specs/022-sync-id-cross-filters/spec.md

Feature Specification: ID Synchronization and Cross-Filter Recovery

Feature Branch: 022-sync-id-cross-filters
Reference UX: N/A Created: 2026-02-25
Status: Draft
Input: User description: "Техническое задание: Сервис синхронизации ID и восстановления кросс-фильтрации..."

User Scenarios & Testing (mandatory)

User Story 1 - Transparent Migration with Working Cross-Filters (Priority: P1)

As a user migrating a dashboard equipped with cross-filters, I want the system to automatically remap internal Chart and Dataset IDs to the target environment's IDs, so that cross-filtering works immediately after migration without manual reconstruction.

Why this priority: Core value proposition. Superset breaks cross-filter links during migration because integer IDs change; fixing this is the primary goal of this feature.

Independent Test: Can be fully tested by migrating a dashboard containing native cross-filters to a blank environment with "Fix Cross-Filters" checked, and verifying filters function natively on the target without opening them in edit mode.

Acceptance Scenarios:

1. Given a dashboard with cross-filters pointing to Source IDs, When migrated to a Target environment where charts are new (UUIDs missing), Then the system transparently does a dual-import, fetches new IDs, patches the metadata, and imports again so the resulting dashboard uses correct Target IDs.
2. Given a dashboard with cross-filters where charts already exist on the Target, When migrated, Then the system looks up IDs locally and patches the dashboard metadata directly without an intermediary import.

---

User Story 2 - UI Option for Cross-Filter Recovery (Priority: P2)

As a user initiating a migration, I want a clear option to toggle cross-filter repair ("Исправить связи кросс-фильтрации"), so I can bypass the double-import logic if I know cross-filters aren't used or if I require a strict, unmodified file import.

Why this priority: Gives users control over the migration pipeline and reduces target system load if patching isn't needed.

Independent Test: Can be tested by opening the migration modal/interface and verifying the checkbox state, label, and help description are correct.

Acceptance Scenarios:

1. Given the migration launch UI, When it opens, Then the "Исправить связи кросс-фильтрации" is visible, enabled by default, and has a descriptive tooltip.

---

User Story 3 - Settings UI and Mapping Table (Priority: P2)

As a system administrator, I want to configure the synchronization schedule via Cron and view the current mapping table in the Settings UI so that I can monitor the agent's status and debug missing IDs.

Why this priority: Required for operational visibility and configuration flexibility.

Independent Test: Open the settings page, adjust the Cron schedule, and view the mapping table. Verify the table displays environment names, UUIDs, integer IDs, and resource names.

Acceptance Scenarios:

1. Given the Settings UI, When navigated to the ID Mapping section, Then I see an input to configure the Cron string for the background job.
2. Given the ID Mapping section, When viewed, Then I see a table or list showing Environment, UUID, Integer ID, and Resource Name for all synchronized objects.

---

Edge Cases

• What happens when the background sync process fails to reach a specific Environment API (e.g., due to network issues or authentication)?
• How does the system handle migrating dashboards where some UUIDs map properly while others don't (partial existence footprint)?
• What happens if the intermediary 'Technical Import' step fails (e.g., due to duplicate names or target environment constraints)?

Requirements (mandatory)

Functional Requirements

• FR-001: System MUST maintain a scheduled background synchronization process to poll all active Environments, configurable via a Cron expression in the UI.
• FR-002: System MUST catalog Chart, Dataset, and Dashboard objects from each Environment, mapping their universal UUIDs to environment-specific IDs and capturing their human-readable names.
• FR-003: System MUST persist this identity catalog locally for rapid access.
• FR-004: System MUST provide a pre-migration verification mechanism to determine if required Target IDs are already known in the local catalog.
• FR-005: System MUST perform an intermediary "Technical Import" and immediate re-synchronization when migrating objects to a fresh environment, effectively forcing the Target system to generate and reveal new IDs.
• FR-006: System MUST patch the dashboard configuration metadata before the final import step, ensuring all cross-filter associations exclusively use the correct Target IDs.
• FR-007: System MUST provide a user interface option to explicitly toggle the Cross-Filter recovery feature during migration execution.
• FR-008: System MUST provide a Settings UI view displaying the universal UUID, the environment-specific integer ID, the environment name, and the resource name for all cataloged objects.

Key Entities

• Resource Mapping: Represents the unified reference between an Environment, a Resource Type (Chart/Dataset/Dashboard), its universal UUID, its environment-specific Integer ID, and its last sync timestamp.

Success Criteria (mandatory)

Measurable Outcomes

• SC-001: 100% of native cross-filters on migrated dashboards continue to function perfectly on the Target environment without manual user intervention.
• SC-002: Background sync completes across a minimum of 3 standard environments in under 5 minutes without causing API rate-limiting.
• SC-003: A single "Fix Cross-Filters" migration takes no more than 2x the execution time of a standard migration.

---

Test Data Fixtures (recommended for CRITICAL components)

Fixtures

resource_mapping_record:
  description: "Example state of the mapping database after a successful sync step"
  data:
    environment_id: "prod-env-1"
    resource_type: "chart"
    uuid: "123e4567-e89b-12d3-a456-426614174000"
    remote_integer_id: 42
    last_synced_at: "2026-02-25T10:00:00Z"
    # UI settings table and Cron schedule requirements
    # This section describes the data structure for resource mapping and sync process
    # It's not a fixture itself, but rather a description of the data collected.
    # For UI settings and Cron, separate fixtures or requirements would be needed.
    # The following is a description of the data points collected during sync:
    # 1. Извлечение (Extraction):
    #    *   **Charts:** `id`, `uuid`, `slice_name`
    #    *   **Datasets:** `id`, `uuid`, `table_name`
    #    *   **Dashboards:** `id`, `uuid`, `slug`
    # 2. Сохранение (Upsert): Сохраняет данные в локальную БД сервиса миграции в таблицу маппинга.
    #    *   *Ключ уникальности:* `environment_id` + `uuid`.
    #    *   *Обновляемое поле:* `remote_integer_id` (актуальный ID на конкретном стенде) и `resource_name` (для отображения в UI).
    #    *   *Метаданные:* `last_synced_at`.

missing_target_scenario:
  description: "Target check indicates partial or entirely missing IDs"
  data:
    all_uuids_present: false
    missing_uuids:
      - "123e4567-e89b-12d3-a456-426614174000"