# Timetabler Backend Architecture

This document is a codeview of the current `timetabler-be` repository. It reflects the package layout, Django model graph, admin API flow, Kafka/Redis integration, and Socket.IO bridge present in the codebase.

## Package Map

| Package | Responsibility | Main files |
| --- | --- | --- |
| `backend` | Django project configuration, URL root, environment-backed settings, locale middleware, Redis client factory, Kafka producer wrapper. | `backend/settings.py`, `backend/urls.py`, `backend/kafka.py`, `backend/redis_client.py`, `backend/middleware/locale_middleware.py` |
| `api.urls` | Admin route registration under `/api/admin/`. Most route names double as permission keys. | `api/urls/admin.py` |
| `api.views.admin` | Admin REST controllers. Views inherit `AdminApiBase`, validate requests, read/write ORM models, sync Redis, send Kafka messages, write audit trails, and return normalized responses. | `api/views/admin/*.py` |
| `api.models` | Django ORM domain model for users, roles, permissions, timetabling entities, resource constraints, audit tables, Kafka logs, and join tables. | `api/models/*.py`, `api/models/relations.py` |
| `api.validator` | Laravel-style rule validator used by views for request validation. | `api/validator.py` |
| `api.utils` | Cross-cutting helpers for AES, logging, audit deltas, Redis JSON sync, slot math, resource maps, websocket HTTP posts, and code/name generation. | `api/utils.py` |
| `kafka_consumer` | Standalone Django-aware Kafka consumers that process scheduler responses, update SQL and Redis, push websocket notifications, and forward microservice updates. | `kafka_consumer/tt_response.py`, `kafka_consumer/schedule_response.py`, `kafka_consumer/preschedule_response.py` |
| `node` | Express and Socket.IO broadcaster. Receives HTTP callbacks from Django workers and emits events to the browser socket room. | `node/broadcast.js`, `node/conf.js` |

## Package Interaction

```mermaid
flowchart LR
    FE["Frontend client"] -->|HTTP POST /api/admin/*| URLs["backend.urls + api.urls.admin"]
    URLs --> Views["api.views.admin"]
    Views --> Base["AdminApiBase"]
    Base --> Auth["TokenAuthentication"]
    Auth --> Tokens["AccessToken"]
    Base --> Perms["CheckPermissionBackend"]
    Perms --> Roles["UserRole, RolePermission, UserPermission"]
    Base --> RequestLog["IncomingApiAdmin"]
    Views --> Validator["BaseValidator"]
    Views --> Models["api.models domain graph"]
    Views --> Audit["AuditTrail + AuditTrailDetails"]
    Views --> Redis["Redis JSON cache"]
    Views --> KafkaProducer["backend.kafka.send_request"]
    KafkaProducer --> KafkaLog["KafkaLog"]
    KafkaProducer --> Kafka["Kafka request topic"]
    Kafka --> Scheduler["Scheduler / timetabling microservice"]
    Scheduler --> ResponseTopic["Kafka response topic"]
    ResponseTopic --> Consumers["kafka_consumer workers"]
    Consumers --> Models
    Consumers --> Redis
    Consumers --> KafkaLog
    Consumers --> NodeBridge["node/broadcast.js"]
    NodeBridge -->|Socket.IO event| FE
```

The synchronous path is the admin HTTP request. The asynchronous path starts when a view publishes a Kafka request and ends when a consumer updates persistence/cache state and calls the Node broadcaster for the relevant `socket_id`.

## Request Lifecycle

1. `backend.urls` routes `/api/admin/` to `api.urls.admin`.
2. `api.urls.admin` selects the view class and route name, for example `schedule;request`.
3. `AdminApiBase.initial()` authenticates with `TokenAuthentication`, verifies route permission unless the route is in `NO_PERMISSION`, records `IncomingApiAdmin`, and requires `timestamp` and `signature`.
4. The concrete view calls `BaseValidator` for endpoint rules and then performs ORM work.
5. Mutating views usually update Redis through `bulk_sync_to_redis()`, publish Kafka messages through `send_request()`, and write `AuditTrailDetails`.
6. Responses return through `AdminApiBase.api_response()`, which also updates the API log.

```mermaid
classDiagram
direction LR
class APIView
class BaseAuthentication
class AdminApiBase {
  +initial(request)
  +basic_validation(data)
  +api_response(data)
  +sign(data, secret)
  +update_m2m_field(model, field_name, old_ids, new_ids, old_new_data)
  +update_m2m_field_bulk(models, field_name, new_ids, old_new_data)
}
class TokenAuthentication {
  +authenticate(request)
}
class BaseValidator {
  +validate()
  +rule_required(field, value, param)
  +rule_exists(field, value, param)
  +rule_array(field, value, param)
}
class CheckPermissionBackend {
  +get_all_permissions(user_obj)
  +has_perm(user_obj, perm)
}
class IncomingApiAdmin {
  +insert_log(user_id, request_data, url, ip)
  +update_log(id, outgoing_data, scode)
}
class AuditTrail
class AuditTrailDetails {
  +custom_insert(audit_trail, action, remark_param)
}

APIView <|-- AdminApiBase
BaseAuthentication <|-- TokenAuthentication
AdminApiBase --> TokenAuthentication : authenticates
AdminApiBase --> CheckPermissionBackend : calls user.has_perm()
AdminApiBase --> IncomingApiAdmin : logs request and response
AdminApiBase --> BaseValidator : endpoint validation
AdminApiBase --> AuditTrail : writes admin audit type
AuditTrail "1" --> "*" AuditTrailDetails : has details
```

## Domain Model Packages

The model package is best read in functional clusters. Full class diagrams live under [`docs/uml`](./uml/).

| Model cluster | Classes |
| --- | --- |
| Identity and access | `User`, `AccessToken`, `Role`, `RolePermission`, `UserPermission`, `UserRole` |
| Observability and audit | `IncomingApiAdmin`, `AuditTrail`, `AuditTrailDetails`, `ErrorLog`, `KafkaLog` |
| Organization and geography | `TtDepartment`, `TtZone`, `TtLocation`, `TtJourney`, `TtTravelTable` |
| Calendar and academic structure | `TtAcademicTerm`, `TtWeek`, `TtWeekPattern`, `TtModule`, `TtModuleGroup`, `TtPos`, `TtPathway`, `TtStudentSet` |
| Preferences and constraints | `TtAvailability`, `TtStartPreference`, `TtUsagePreference`, `TtSuitability`, `TtConstraintProfile`, `TtTimeBlock`, `TtFreeBlock`, `TtResourceBreak`, `TtMaximumHour`, `TtMaximumWorkspan` |
| Activity scheduling | `TtActivityType`, `TtActivityTemplate`, `TtActivity`, plus join/resource-map models in `api.models.relations` |
| Configuration and tagging | `TtSetting`, `TtTag`, `TtTagRelation` |

```mermaid
classDiagram
direction TB
class TtDepartment
class TtZone
class TtAcademicTerm
class TtWeek
class TtWeekPattern
class TtModuleGroup
class TtModule
class TtPos
class TtPosModuleGroup
class TtPosModuleGroupModule
class TtPathway
class TtStudentSet
class TtActivityType
class TtActivityTemplate
class TtActivity

TtDepartment --> TtDepartment : parent department
TtZone --> TtZone : parent zone
TtDepartment --> TtZone : default zone
TtZone --> TtDepartment : owner department
TtAcademicTerm "*" -- "*" TtWeek : TtAcademicTermWeek
TtWeekPattern "*" -- "*" TtWeek : TtWeekPatternWeek
TtModule "*" --> "1" TtAcademicTerm
TtModule "*" -- "*" TtWeek : TtModuleWeek
TtPos "*" --> "1" TtAcademicTerm
TtPos "*" -- "*" TtModuleGroup : TtPosModuleGroup
TtPosModuleGroup --> TtPos
TtPosModuleGroup --> TtModuleGroup
TtPosModuleGroup "*" -- "*" TtModule : TtPosModuleGroupModule
TtPathway --> TtPos
TtPathway --> TtAcademicTerm
TtPathway "*" -- "*" TtPosModuleGroupModule : selected modules
TtStudentSet --> TtPos
TtStudentSet --> TtAcademicTerm
TtStudentSet "*" -- "*" TtModule : TtStudentSetModule
TtActivityTemplate --> TtActivityType
TtActivityTemplate --> TtModule
TtActivity --> TtActivityTemplate
TtActivity --> TtActivityType
TtActivity --> TtModule
TtActivity "*" -- "*" TtStudentSet : TtStudentSetActivity
TtActivity "*" -- "*" TtWeek : TtActivityWeek
```

```mermaid
classDiagram
direction LR
class TtStaff
class TtLocation
class TtSuitability
class TtAvailability
class TtStartPreference
class TtUsagePreference
class TtConstraintProfile
class TtTimeBlock
class TtResourceBreak
class TtFreeBlock
class TtMaximumHour
class TtMaximumWorkspan
class TtTravelTable
class TtJourney
class TtActivity

TtStaff --> TtAvailability
TtStaff --> TtStartPreference
TtStaff --> TtUsagePreference
TtLocation --> TtAvailability
TtLocation --> TtStartPreference
TtLocation --> TtUsagePreference
TtStaff "*" -- "*" TtSuitability : TtStaffSuitability
TtLocation "*" -- "*" TtSuitability : TtLocationSuitability
TtStaff "*" -- "*" TtConstraintProfile : TtStaffConstraintProfile
TtLocation "*" -- "*" TtConstraintProfile : TtLocationConstraintProfile
TtConstraintProfile "*" -- "*" TtResourceBreak : resource breaks
TtConstraintProfile "*" -- "*" TtFreeBlock : free blocks
TtConstraintProfile "*" -- "*" TtMaximumHour : maximum hours
TtConstraintProfile "*" -- "*" TtMaximumWorkspan : maximum workspans
TtConstraintProfile "*" -- "*" TtTravelTable : travel rules
TtResourceBreak "*" -- "*" TtTimeBlock
TtFreeBlock "*" -- "*" TtTimeBlock
TtMaximumHour "*" -- "*" TtTimeBlock
TtMaximumWorkspan "*" -- "*" TtTimeBlock
TtTravelTable "1" --> "*" TtJourney
TtJourney --> TtLocation : from/to location
TtActivity "*" -- "*" TtStaff : assigned staff
TtActivity "*" -- "*" TtLocation : assigned location
```

## Async Scheduling Flow

Scheduling, prescheduling, and swap requests all use the same integration pattern: persist/log the request, publish to Kafka with a `method` header, consume a response, update SQL/Redis, then push a Socket.IO notification.

```mermaid
sequenceDiagram
autonumber
participant FE as Frontend
participant API as ScheduleRequest or Swap view
participant KafkaP as backend.kafka.send_request
participant Log as KafkaLog
participant ReqTopic as Kafka request topic
participant Engine as Scheduler service
participant RespTopic as Kafka response topic
participant Worker as kafka_consumer.tt_response
participant DB as Django models
participant Redis as Redis JSON cache
participant Node as node/broadcast.js

FE->>API: POST with Token, timestamp, signature, socket_id
API->>API: validate activity ids and optional slot/resources
API->>KafkaP: send_request(topic, payload, socket_id, method)
KafkaP->>ReqTopic: produce JSON with request_id and method header
KafkaP->>Log: insert request row
API-->>FE: 200 accepted style response
ReqTopic->>Engine: consume scheduling request
Engine->>RespTopic: publish result with request_id
RespTopic->>Worker: poll response
Worker->>Log: update matching response row
Worker->>DB: update activities, assignments, variants, resource maps
Worker->>Redis: bulk_sync_to_redis()
Worker->>Node: POST websocket endpoint with socket_id and data
Node-->>FE: emit schedule_response, preschedule_response, swap, or related event
```

See the focused sequence files in [`docs/uml`](./uml/) for authentication, generic admin mutations, imports, allocation, and scheduling.

## GitLab Mermaid Files

The following files contain standalone diagrams and their responsibilities/flow descriptions:

- [`uml/class-admin-api.md`](./uml/class-admin-api.md)
- [`uml/class-domain-curriculum.md`](./uml/class-domain-curriculum.md)
- [`uml/class-resource-constraints.md`](./uml/class-resource-constraints.md)
- [`uml/class-integration-observability.md`](./uml/class-integration-observability.md)
- [`uml/sequence-authentication.md`](./uml/sequence-authentication.md)
- [`uml/sequence-admin-mutation.md`](./uml/sequence-admin-mutation.md)
- [`uml/sequence-scheduling.md`](./uml/sequence-scheduling.md)
- [`uml/sequence-import-allocation.md`](./uml/sequence-import-allocation.md)