feat(sprint10): Phase 1 — Data model + bank statement parsers (MT940, CAMT.053, CSV)

Implements the Sprint 10 Phase 1 foundation for the Smart Payment Import feature: Domain layer: - 3 new enums: BankFormat (MT940, CAMT053, CSV), ImportSessionStatus, MatchStatus - StaffPermission.FINANCE_IMPORT - AuditEventType: BANK_IMPORT_STARTED/COMPLETED/FAILED + BANK_PAYMENT_CONFIRMED - NotificationType.BANK_IMPORT_COMPLETED - ConsentType.BANK_DATA (DSGVO consent for IBAN storage) - 3 new entities: BankImportSession, BankTransaction, CsvColumnMapping - Member: + iban (VARCHAR 34) + ibanConsentDate - MemberStatus.LEFT (semantic alias for RESIGNED, referenced by Sprint 9 RetentionService) Persistence: - V30__bank_import_sessions.sql - V31__bank_transactions.sql - V32__csv_column_mappings.sql (also adds iban + iban_consent_date to members) - 3 Spring Data repositories Parser infrastructure (cannamanage-service/src/main/java/de/cannamanage/service/bankimport): - BankStatementParser interface (Strategy pattern, Spring-injected list) - ParsedTransaction + ParseResult records - BankStatementParseException (parse errors) - Mt940Parser: custom state machine, CENTURY_BOUNDARY=70 for YY→YYYY, proprietary header tolerance (skips lines before first :20: for StarMoney/WISO/Hibiscus wrappers) - Camt053Parser: StAX streaming with XXE hardening (IS_SUPPORTING_EXTERNAL_ENTITIES, SUPPORT_DTD, IS_REPLACING_ENTITY_REFERENCES all false); supports camt.053.001.02 and camt.053.001.08 namespaces - CsvBankParser: Apache Commons CSV with configurable columns per club; German number format ("1.234,56"); ISO-8859-1 default encoding - BankStatementParserService: filename-extension hint + content probe; throws UnrecognizedFormatException when no parser claims the file Build verified via Docker (cannamanage-api:sprint10-phase1). Sprint 9 fix (incidental, required to compile): - Added MemberStatus.LEFT (Sprint 9 RetentionService referenced it but the enum value was missing) - MemberListRegistryGenerator: added LEFT to formatStatus() switch (mapped to "Ausgetreten", same as RESIGNED) Sprint 10 docs: analysis, plan, plan-review, testplan. Co-Authored-By: Lumen <lumen@cannamanage.de>
2026-06-15 17:21:55 +02:00
parent 57f418f7c9
commit 55110c95af
31 changed files with 4456 additions and 6 deletions
@@ -0,0 +1,25 @@
+-- Sprint 10: Bank statement import sessions
+-- Each upload of a bank statement creates one session, which is then matched + reviewed by an admin.
+-- Status flow: PENDING → IN_REVIEW → COMPLETED (or FAILED at any point).
+-- Once COMPLETED, the session is immutable per GoBD requirements (§147 AO).
+CREATE TABLE bank_import_sessions (
+    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    tenant_id UUID NOT NULL,
+    club_id UUID NOT NULL REFERENCES clubs(id) ON DELETE CASCADE,
+    filename VARCHAR(255) NOT NULL,
+    format VARCHAR(20) NOT NULL,                 -- MT940, CAMT053, CSV
+    total_transactions INTEGER NOT NULL DEFAULT 0,
+    matched_count INTEGER NOT NULL DEFAULT 0,
+    confirmed_count INTEGER NOT NULL DEFAULT 0,
+    skipped_count INTEGER NOT NULL DEFAULT 0,
+    status VARCHAR(20) NOT NULL DEFAULT 'PENDING', -- PENDING, IN_REVIEW, COMPLETED, FAILED
+    uploaded_by UUID NOT NULL REFERENCES users(id),
+    error_message TEXT,
+    created_at TIMESTAMP NOT NULL DEFAULT NOW(),
+    completed_at TIMESTAMP
+);
+
+CREATE INDEX idx_bank_import_sessions_tenant ON bank_import_sessions(tenant_id);
+CREATE INDEX idx_bank_import_sessions_club ON bank_import_sessions(club_id);
+CREATE INDEX idx_bank_import_sessions_status ON bank_import_sessions(club_id, status);
+CREATE INDEX idx_bank_import_sessions_created ON bank_import_sessions(club_id, created_at DESC);
@@ -0,0 +1,32 @@
+-- Sprint 10: Parsed bank transactions
+-- One row per transaction in an uploaded bank statement.
+-- amount_cents: positive = incoming (potential member payment), negative = outgoing (expense).
+-- match_status drives the review UI: UNMATCHED/SUGGESTED/MATCHED/CONFIRMED/SKIPPED.
+-- CASCADE on session delete: discarding a draft session also deletes its parsed rows.
+-- SET NULL on member/payment delete: history is preserved even if the matched entity is removed.
+CREATE TABLE bank_transactions (
+    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    tenant_id UUID NOT NULL,
+    session_id UUID NOT NULL REFERENCES bank_import_sessions(id) ON DELETE CASCADE,
+    club_id UUID NOT NULL REFERENCES clubs(id) ON DELETE CASCADE,
+    booking_date DATE NOT NULL,
+    value_date DATE,
+    amount_cents INTEGER NOT NULL,                        -- positive = incoming, negative = outgoing
+    currency VARCHAR(3) NOT NULL DEFAULT 'EUR',
+    reference_text TEXT,                                  -- Verwendungszweck
+    counterparty_name VARCHAR(300),
+    counterparty_iban VARCHAR(34),
+    bank_reference VARCHAR(100),
+    match_status VARCHAR(20) NOT NULL DEFAULT 'UNMATCHED',-- UNMATCHED, SUGGESTED, MATCHED, CONFIRMED, SKIPPED
+    match_confidence INTEGER,                             -- 0-100, only populated when match_status != UNMATCHED
+    matched_member_id UUID REFERENCES members(id) ON DELETE SET NULL,
+    matched_payment_id UUID REFERENCES payments(id) ON DELETE SET NULL,
+    skip_reason VARCHAR(100),
+    created_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+CREATE INDEX idx_bank_transactions_tenant ON bank_transactions(tenant_id);
+CREATE INDEX idx_bank_transactions_session ON bank_transactions(session_id);
+CREATE INDEX idx_bank_transactions_club_status ON bank_transactions(club_id, match_status);
+CREATE INDEX idx_bank_transactions_member ON bank_transactions(matched_member_id);
+CREATE INDEX idx_bank_transactions_payment ON bank_transactions(matched_payment_id);
@@ -0,0 +1,31 @@
+-- Sprint 10: CSV column mapping templates + member IBAN fields
+-- CSV files have no standard layout — each bank uses different columns/encodings.
+-- Admins create a named mapping per bank (e.g. "Sparkasse Export") that the parser reuses.
+CREATE TABLE csv_column_mappings (
+    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    tenant_id UUID NOT NULL,
+    club_id UUID NOT NULL REFERENCES clubs(id) ON DELETE CASCADE,
+    name VARCHAR(100) NOT NULL,                              -- e.g. "Sparkasse Export"
+    date_column INTEGER NOT NULL,
+    amount_column INTEGER NOT NULL,
+    reference_column INTEGER,
+    counterparty_column INTEGER,
+    iban_column INTEGER,
+    delimiter VARCHAR(5) NOT NULL DEFAULT ';',
+    date_format VARCHAR(20) NOT NULL DEFAULT 'dd.MM.yyyy',
+    decimal_separator VARCHAR(1) NOT NULL DEFAULT ',',
+    skip_header_rows INTEGER NOT NULL DEFAULT 1,
+    encoding VARCHAR(20) NOT NULL DEFAULT 'ISO-8859-1',
+    is_default BOOLEAN NOT NULL DEFAULT FALSE,
+    created_at TIMESTAMP NOT NULL DEFAULT NOW()
+);
+
+CREATE INDEX idx_csv_column_mappings_tenant ON csv_column_mappings(tenant_id);
+CREATE INDEX idx_csv_column_mappings_club ON csv_column_mappings(club_id);
+
+-- Add optional IBAN fields to members.
+-- Both columns are intentionally NULLABLE — IBAN is only populated after explicit
+-- BANK_DATA consent (DSGVO Art. 6(1)(a)). ibanConsentDate records when consent was given.
+-- PostgreSQL adds nullable columns instantly (no table rewrite), safe for production.
+ALTER TABLE members ADD COLUMN IF NOT EXISTS iban VARCHAR(34);
+ALTER TABLE members ADD COLUMN IF NOT EXISTS iban_consent_date TIMESTAMP;