## ADDED Requirements ### Requirement: Engine I/O is outside the database transaction Timetabler SHALL send the Scheduling Engine request and wait for/parse/validate its complete response before opening the database transaction that applies final state. It SHALL NOT hold Timetabler database locks while waiting on the engine or any Resource Booking/adapter network call. #### Scenario: Engine response is invalid - **WHEN** the engine response cannot be parsed or fails complete-result validation - **THEN** no final-application transaction opens, no Timetabler allocation changes, and no integration outbox row is created #### Scenario: Engine response is valid - **WHEN** a complete valid result is received - **THEN** Timetabler constructs the full change set before acquiring database locks ### Requirement: Final schedule application is one atomic database unit Timetabler SHALL apply a final schedule change set inside one outer `transaction.atomic()` block that locks affected rows in deterministic ID order, applies Staff/Location relation replacement, scheduled fields, variants/weeks/deletions, recalculates database resource maps, records idempotency, and inserts the outbound outbox records before commit. #### Scenario: Failure after each database stage - **WHEN** a test injects an exception after receipt insertion, relation deletion, relation insertion, scheduled-field update, variant/week processing, resource-map recalculation, or outbox insertion - **THEN** the entire final change rolls back, no outbox remains, and no Redis/WebSocket/Kafka/HTTP effect has run #### Scenario: Nested resource-map transaction runs - **WHEN** `api/utils.py::recalculate_resource_map()` is called inside the outer final-application transaction - **THEN** its nested atomic block behaves as a savepoint, sees the same connection's uncommitted relation changes, and its output rolls back with the outer transaction on failure #### Scenario: Concurrent change sets overlap - **WHEN** two final applications target overlapping activities or resources - **THEN** deterministic lock ordering avoids inconsistent partial results, and any lock timeout/deadlock causes one complete transaction to retry/fail without a partial commit ### Requirement: Processing exceptions propagate as failure The final-application service SHALL allow database/application exceptions to escape the atomic scope. Consumer/API exception handlers SHALL run outside that scope and SHALL NOT convert a rolled-back schedule into a successfully acknowledged response. #### Scenario: Schedule application raises - **WHEN** any database stage raises an unexpected exception - **THEN** the transaction rolls back, the consumer does not commit/store the Kafka offset, and the message remains eligible for controlled retry #### Scenario: Manual direct writer raises - **WHEN** manual constraint-break or another synchronous direct writer fails during atomic apply - **THEN** the endpoint reports failure and does not emit a success WebSocket/Kafka effect ### Requirement: Engine response application is idempotent Timetabler SHALL keep a durable applied-response receipt keyed by engine request ID and response hash/protocol version. A matching redelivery SHALL be a no-op; a reused request ID with conflicting response content SHALL be rejected and alerted. #### Scenario: Duplicate response is redelivered - **WHEN** the same request ID and response hash is consumed after a prior successful commit - **THEN** no database allocation or outbox event is duplicated and post-commit recovery may resume from durable records #### Scenario: Request ID is reused with another payload - **WHEN** an already-recorded request ID arrives with a different response hash - **THEN** Timetabler does not apply it, quarantines/logs the mismatch, and raises an operator alert ### Requirement: Kafka offsets are acknowledged after durable success The Scheduling Engine response consumer SHALL disable automatic offset commit and automatic offset store. It SHALL store/commit a consumed response offset only after the Timetabler database/outbox commit succeeds and required post-commit durable delivery records exist. #### Scenario: Worker crashes before database commit - **WHEN** the consumer crashes before the transaction commits - **THEN** no mutation/outbox/receipt is durable and the uncommitted Kafka response is redelivered for a full retry #### Scenario: Worker crashes after database commit before offset commit - **WHEN** the consumer crashes after the database/outbox/receipt commit but before the response offset commits - **THEN** redelivery resolves to the existing receipt, does not reapply the mutation, and recovers pending durable effects #### Scenario: Worker crashes after offset commit - **WHEN** the consumer crashes after both durable state and offset commit - **THEN** the publisher/retry records continue independently and no response replay is required for integration delivery ### Requirement: Valid rollback redeliveries are not age-discarded The consumer SHALL distinguish valid un-applied/redelivered responses from unknown or terminal late responses using durable request/receipt state rather than only comparing request age to the current 30-second timeout. #### Scenario: Old response is redelivered after rollback - **WHEN** a valid response is older than 30 seconds only because an earlier application rolled back and its offset was not committed - **THEN** the consumer attempts idempotent application instead of silently discarding it #### Scenario: Unknown late response arrives - **WHEN** a response has no valid request state or violates the agreed first-arrival expiry/protocol policy - **THEN** it is quarantined/recorded with an observable reason and is not silently treated as success ### Requirement: External effects occur after commit Redis refresh, WebSocket notification, `confirmed_schedule`, legacy/microservice Kafka, HTTP calls, and Resource Booking/adapter communication SHALL NOT run before the final Timetabler transaction commits. Required durable deliveries SHALL be backed by an outbox; `confirmed_schedule` SHALL remain feature-gated and SHALL NOT act as an RB acknowledgement. #### Scenario: Redis fails after commit - **WHEN** Redis refresh throws after the allocation/outbox commit - **THEN** canonical database state remains committed, retry/refresh occurs without reapplying it, and the Kafka response is acknowledged only according to the durable-effect policy #### Scenario: WebSocket fails after commit - **WHEN** a UI notification fails after commit - **THEN** the database/outbox state remains authoritative and the UI can recover by refetching without a duplicate schedule apply #### Scenario: confirmed_schedule is unavailable - **WHEN** the engine has no usable `confirmed_schedule` handler - **THEN** final Timetabler/RB integration correctness does not depend on that message ### Requirement: Timing and locking safeguards are measured The implementation SHALL configure lock/statement timeouts, deterministic locking, Kafka max-poll behavior, and measured single/bulk timing against the engine's approximately 30-second temporary reservation window. If the required percentile cannot meet the window, an engine reservation TTL extension/renewal SHALL be delivered before rollout. #### Scenario: Representative worst-case batch - **WHEN** the agreed maximum representative engine batch is applied under production-like load - **THEN** parse, lock wait, atomic apply, resource-map recalculation, commit, Redis refresh, and offset timings are recorded and meet the approved reservation/max-poll budget #### Scenario: Timing exceeds reservation TTL - **WHEN** measurements show the safe finalization path can exceed the engine reservation TTL - **THEN** rollout is blocked until an extension/renewal protocol and its tests are complete ### Requirement: Atomic application remains engine-protocol compatible only with safeguards The design SHALL treat one-transaction final application as compatible with the current complete-result engine protocol only when exception propagation, manual offsets, idempotency, post-commit effects, deterministic locks, and timing controls are implemented and verified. #### Scenario: Compatibility evidence is reviewed - **WHEN** Phase 1 seeks production approval - **THEN** the completion report contains evidence for each required safeguard and does not claim zero implementation risk