Skip to content

BrownRook Ledger solution architecture

BrownRook Ledger turns receipt files and electronic purchase records into canonical expenses, categorized line items, review data, and budget reports. PostgreSQL is the system of record. Receipt files and OCR cache artifacts stay on shared storage; RabbitMQ carries work references only.

Infrastructure context

The BrownRook infrastructure rendered diagram and editable Draw.io source show the wider network, edge, DNS, identity, PKI, and K3s platform around Ledger. This page focuses on components and data flows owned by the Ledger application. Keeping those views separate lets infrastructure changes remain authoritative in brownrook-infra while application behavior remains authoritative here.

Runtime components and data flow

Authenticated web access

flowchart TB
    publicUser[Public Ledger user]
    privateUser[Corporate LAN user]
    entra[Microsoft Entra ID]
    publicEdge[Public NGINX to Traefik]
    privateEdge[Private DNS to Traefik]

    subgraph cluster[K3s home-budget namespace]
        publicOauth[Public OAuth2 Proxy]
        privateOauth[Private OAuth2 Proxy]
        web[Ledger web application]
        db[(PostgreSQL)]
        files[(Shared receipt files and OCR cache)]
    end

    publicUser -->|idc.brownrook.com/ledger| publicEdge --> publicOauth
    privateUser -->|ledger.brownrook.net/ledger| privateEdge --> privateOauth
    publicOauth <-->|sign in| entra
    privateOauth <-->|sign in| entra
    publicOauth -->|authenticated identity| web
    privateOauth -->|authenticated identity| web
    web -->|read reports and audited corrections| db
    web -->|receipt preview| files

Public and private browser routes use separate OAuth2 Proxy deployments because their callback hosts and secure cookies differ. Both enforce the same Microsoft Entra tenant and access policy.

Receipt processing

flowchart TB
    sources[Scans, PDFs, and electronic receipts]
    files[(Shared receipt files and OCR cache)]
    mode{Deployment mode}
    cron[Receipt processor CronJob]
    publisher[Receipt publisher]
    rabbit[(RabbitMQ work, retry, and dead queues)]
    worker[Receipt worker]
    results[(OCR result, retry, and dead queues)]
    collector[OCR Results Collector]
    db[(PostgreSQL system of record)]
    enrich[Product enrichment job]
    providers[Brave Search, local Qwen and optional paid models]
    review[Ledger recommendation review]
    graph[(Neo4j evidence projection)]

    sources --> files --> mode
    mode -->|base| cron
    cron -->|discover, OCR, normalize, persist| db
    mode -->|queued| publisher
    publisher -->|confirmed versioned message| rabbit
    rabbit -->|manual-ACK delivery| worker
    worker -->|read and verify source| files
    worker -->|OCR, normalize, reconcile| results
    results -->|manual-ACK delivery| collector
    collector -->|validate and persist| db
    worker -->|transient retry or terminal failure| rabbit
    db -->|pending items| enrich
    enrich -->|product evidence and saved collaboration| db
    enrich -->|search and optional query expansion| providers
    db -->|saved recommendation| review
    review -->|explicit acceptance and audit event| db
    db -->|rebuildable evidence projection| graph

The base Kubernetes deployment runs receipt discovery and processing in one CronJob every 15 minutes. The optional deploy/rabbitmq overlay changes that CronJob into a publisher and adds RabbitMQ, a KEDA-scaled worker, and an OCR Results Collector. Workers have no PostgreSQL credentials; collectors own the durable result transaction. The private-LAN overlays add a second authenticated Ledger route; the RabbitMQ private variant also exposes only the broker management UI through private HTTPS. PostgreSQL and AMQP application traffic remain internal. See the network access guide for these boundaries.

The main processing path is:

  1. Discover a source file and calculate its SHA-256 hash.
  2. Check PostgreSQL processing and evidence state for completion or exhaustion.
  3. Run OCR, parse and normalize the receipt, and reconcile totals.
  4. Store OCR evidence, the canonical expense, line items, and final processing status in PostgreSQL.
  5. Apply deterministic and approved learned category rules, with optional AI classification only after rule-based matching fails.
  6. Present receipts, expenses, review queues, reconciliation, duplicates, and category reports in Ledger.

Post-import product enrichment has separate modes. The original enrichment path can persist sufficiently strong matches automatically. Collaborative jobs pool retained OCR/image evidence, Qwen proposals, Brave results and model verification, and save a recommendation for human review. Production collaboration tries Qwen first and enables paid fallback only by explicit configuration. Comparison mode calls OpenAI and Qwen to record paired results without accepting products.

Ledger acceptance updates the selected item's product description and URL in one PostgreSQL transaction, preserving imported item text and amounts and recording the actor and lineage event. Supported accepted interpretations lead the UI; imported text remains in collapsed history. Related OCR lines retain their source citations but do not establish exact item-row identity. Neo4j projects retained evidence and acceptance events; it does not choose or overwrite canonical products. See the user guide, collaboration manual and graph reference.

In queued mode, delivery is at least once. Shared-volume source locks and cache request markers prevent concurrent or repeated OCR for one refresh, while collector-side PostgreSQL keys make result replay idempotent. A worker acknowledges only after confirmed result publication or retry/dead-letter routing; a collector acknowledges only after database commit. See the RabbitMQ design for those boundaries in detail.

Observability boundary

Production can collect the same home-budget stdout/stderr through two independent paths: an in-cluster Fluent Bit DaemonSet reading node CRI files and an external Grafana Alloy service reading the Kubernetes pods/log API. Both write to Loki using separate private-CA client identities and distinguish their copies with collection="fluent-bit" or collection="kubernetes-api".

The Kubernetes log collection design documents the topology, vendor boundary, mTLS identities, failure semantics, label contract, GitOps ownership, and comparison method. The log forwarding runbook contains deployment and recovery commands, and the comparison results keep measured behavior separate from intended behavior.

Persistence boundaries

Component Stores Role
Shared storage Original receipt files and OCR cache JSON Durable source evidence and reusable extraction artifacts
PostgreSQL Processing state, OCR runs, evidence, canonical expenses, line items, categories, audit history, and analytics views Authoritative application state
RabbitMQ Versioned source hash and relative-path work messages Optional transient work distribution, retry delay, and dead-letter inspection

Ledger read queries use read-only database transactions. Category overrides, category rules, and item description/product-link corrections use explicit transactions and record the authenticated user in audit history.

Build and deployment flow

flowchart LR
    change[Pull request] --> checks[GitHub Actions application tests]
    change --> doccheck[Strict MkDocs documentation build]
    checks --> merge[Merge to main]
    doccheck --> merge
    merge --> image[Build amd64 application image]
    image --> registry[(GitHub Container Registry)]
    image --> update[Commit immutable image tag to k8s/kustomization.yaml]
    update --> argo[Argo CD automated sync]
    argo --> migrate[PreSync database setup job]
    migrate --> workloads[PostgreSQL, Ledger web, receipt jobs, and optional queue worker]
    merge --> docs[Build versioned Markdown with MkDocs Material]
    docs --> pages[GitHub Pages online manual]

The PostgreSQL init container uses the stable schema-bootstrap image for a new volume. Existing databases are upgraded by the versioned PreSync job before application workloads roll forward. Kubernetes secrets and storage credentials are created outside Git.

For operator commands and recovery procedures, see the operations runbook. For end-user review workflows, see the Ledger user guide.