03 Tech stack¶
Source page: 03-stack.html
3. Technology stack (v1)¶
v1 architecture decisions (this revision): single PostgreSQL with the TimescaleDB extension as system of record, event log, analytics store, and vector store · transactional Postgres event log instead of Kafka (RabbitMQ evaluated and rejected; Kafka/Redpanda deferred with triggers) · RudderStack for tracking ingestion, landing in Postgres (ClickHouse deferred with triggers) · GraphQL-primary public API with a thin REST facade · Cassandra and SurrealDB evaluated and rejected as system of record.
3.1 Backend — Phoenix / Elixir / Erlang
| Concern | Choice | Rationale |
|---|---|---|
| Language / runtime | Elixir on the BEAM (Erlang VM) | Fault isolation, lightweight processes, massive concurrency; one tenant's failure cannot cascade |
| Web framework | Phoenix | Mature, productive, first-class real-time support |
| Internal web UI | Phoenix LiveView | Server-rendered, real-time UI for boards, dashboards, job folders, and settings with minimal JavaScript |
| Interactive canvases | React islands (React Flow) inside LiveView pages | The automation canvas and quote designer are too interaction-heavy for LiveView alone |
| Public API | GraphQL-primary (Absinthe) + thin REST facade | One Absinthe schema is the source of truth and serves mobile, web islands, and public partners. A thin REST facade (Phoenix controllers delegating to the same contexts) covers the top 10–15 partner operations — create lead, fetch job, attach document, webhook management — because the long tail of integrators (suppliers, Zapier-class tools, small agencies) speaks REST + webhooks. Public GraphQL ships with query-cost analysis, depth limits, and persisted queries; webhooks are REST-shaped regardless. |
| Background jobs & workflows | Oban Pro (Postgres-backed; commercial license — see §6) | Durable, multi-day workflow execution with retries, scheduling, and workflow/batch primitives — replaces Temporal, BullMQ, and Redis queues |
| Real-time | Phoenix Channels + PubSub | Live board updates, presence, texting UI — built into the framework |
| Domain eventing | Transactional Postgres event log (append-only, outbox-style) | Every state change inserts an immutable event row in the same transaction as the change — stronger consistency than a broker can give without outbox machinery. Consumers (automation engine via Oban, audit trail, projections) track cursors; PubSub broadcasts after commit; replay is a WHERE id > cursor query |
| Caching | ETS / Cachex | In-BEAM caching removes a Redis dependency |
Why the BEAM fits this product: the automation engine is the core of the product, and OTP supervision trees give per-tenant isolation with clean restarts; real-time collaboration is pervasive and Phoenix makes it a language feature; the event surface (Twilio, suppliers, measurements, payments, web tracking) is large and BEAM concurrency absorbs bursts gracefully; and PubSub, queues, caching, scheduling, and now eventing all live inside the runtime or the primary database, collapsing infrastructure to a minimum.
3.2 Data layer (v1)
| Concern | Choice | Rationale |
|---|---|---|
| System of record | PostgreSQL + TimescaleDB extension | JSONB for per-tenant custom fields; row-level security for multi-tenancy; transactions across state + events; backs Oban and pgvector. TimescaleDB is a Postgres extension, so the entire Ecto/Oban/pgvector toolchain is unchanged |
| Event log | Append-only events hypertable (monthly chunks) | Automatic time partitioning from day one; native compression on old chunks keeps growth manageable |
| Analytics & tracking data | RudderStack landing tables in Postgres/Timescale + continuous aggregates | Incrementally-maintained rollups power the reporting dashboards and per-lead website-activity panels without a separate OLAP system. Run on a hypertable-enabled instance that can start shared and later split from OLTP |
| Vector store | pgvector | Embeds contact communication history to ground AI selling playbooks |
| Geospatial | PostGIS | Parcel geometries, storm-swath polygons, radius/polygon segment queries, and territory routing for the prospect data engine — same database, same toolchain |
| Prospect warehouse | Separate schema/database on the same Postgres+Timescale+PostGIS cluster | Physically colocated for shared tooling, logically isolated from CRM OLTP; parcels and storm events are naturally geospatial/time-series |
| Object storage | S3 + CloudFront | Photos/documents dominate storage; lifecycle to Glacier for closed jobs |
| Search | OpenSearch (Apache 2.0; preferred over Typesense's GPL server — see §6) | Typo-tolerant global search |
Deferred infrastructure — reintroduction triggers
| Component | Status | Trigger to introduce |
|---|---|---|
| ClickHouse | Deferred | Tracking-event volume or dashboard query latency exceeds what Timescale continuous aggregates handle economically; columnar scans become decisively cheaper. Reporting queries live behind a repository interface so the swap is a data migration, not a rewrite |
| Kafka / Redpanda | Deferred | Event-log write volume contends with OLTP; retention growth outpaces partition+compression; or consumer fan-out (dozens of cursors polling) gets clumsy where consumer groups are effortless. Consumers already think "read events after cursor," so migration is mechanical. Note Redpanda is BSL (see §6); Apache Kafka is Apache 2.0 |
Rejected alternatives
- RabbitMQ — queue semantics, not log semantics: no replay/backfill for late or buggy consumers, so the "one stream, consumers never disagree" guarantee breaks. Its strengths (work queues, routing) are already covered by Oban.
- Cassandra — no multi-row transactions (breaks the transactional event log and money-adjacent flows), no joins and query-first modeling (hostile to a CRM's flexible per-tenant filtering), and it forfeits the entire Postgres ecosystem (Oban, pgvector, JSONB, RLS, Timescale). Earns its cost only at scales far beyond this market. If a single table ever outgrows Postgres, it gets a dedicated store; the system of record does not migrate.
- SurrealDB — young database (1.0 in late 2023) with a thin production track record and near-zero ecosystem fit for this stack (no mature Ecto adapter, no Oban/pgvector/Timescale, no RudderStack destination, immature ops tooling). Its built-in live queries duplicate what LiveView + PubSub already do. Asymmetric risk for a payments-adjacent, multi-tenant SaaS.
3.3 AI / personality layer — locally hosted, purpose-tuned LLM Changed in v4
- Local model instead of a hosted GenAI API. Personality profiles, selling playbooks, and matched-draft generation run on a self-hosted model fine-tuned for these tasks. Motivations: the unknown future cost and stake of hosted LLM/GenAI APIs (pricing volatility, usage-based cost exposure at scale, vendor dependence and terms drift), plus data control — personality and behavioral data never leave company infrastructure.
- Candidate base models (Hugging Face), to be evaluated on task data: Qwen2.5-14B/7B-Instruct (Apache 2.0), Mistral-7B-Instruct / Mistral-Small (Apache 2.0), Phi-4 (MIT), Llama-3.1-8B-Instruct (Llama Community License — permitted for this use but carries use restrictions and attribution terms), Gemma-2-9B-It (Gemma Terms of Use). Apache-2.0/MIT weights preferred (see §6).
- Fine-tuning: LoRA/QLoRA adapters trained on labeled playbook and profile examples; periodic re-tuning as outcome data accumulates. Evaluation harness with held-out rubric scoring is a first-class deliverable, not an afterthought.
- Serving: vLLM (Apache 2.0) on a small GPU pool (e.g., L4/A10-class instances) behind the AI OTP application; batch generation for profile refreshes, low-QPS interactive generation for drafts.
- Embeddings: open embedding model (e.g., nomic-embed-text or bge — Apache 2.0/MIT) into pgvector; retrieval grounds playbooks in what the prospect actually said and browsed.
- Escape hatch retained: the AI layer sits behind an internal adapter interface, so a hosted API (any vendor) can be swapped in per-task if local quality lags or economics flip; the adapter also enables side-by-side evals.
- Honest trade-offs: new fixed GPU hosting cost replaces variable API spend; tuning/eval engineering burden is real; small local models trail frontier hosted models on open-ended reasoning — acceptable here because the tasks are narrow, templated, and tunable.
- Implemented as a discrete OTP application with its own data-access policy, consent checks, and retention rules
3.4 Website tracking
- Self-hosted RudderStack (Elastic License 2.0 — free for this use; see §6) replaces the custom snippet and ingestion endpoint: JS SDK, server SDKs, identity stitching, consent hooks, destination fan-out
- Event flow: snippet → RudderStack → Postgres/Timescale landing tables → continuous aggregates → dashboards and lead-activity panels
- The tracking firehose stays on this path and does not flow through the domain event log, keeping OLTP write volume clean
- Consent management via OneTrust/Osano integration or a built-in CMP; consent state gates collection in the snippet itself
- Heatmaps & session capture (new in v4): self-hosted PostHog (MIT-licensed core; some premium features proprietary — see §6) provides autocapture, click/move/scroll heatmaps, and session summaries; identity-stitched with RudderStack profiles so per-visitor heatmaps render on the Contact 360 view; capture is consent-gated by the same CMP signal, with aggregate heatmaps available even where individual identification is not consented
3.5 Mobile
- React Native (Expo) for iOS and Android; WatermelonDB or SQLite + sync layer for offline-first operation
- Native modules: camera with annotation overlay, offline photo queue, GPS, push notifications
- Mobile clients consume the same Absinthe GraphQL schema as the public API
- Trade-off accepted: no code sharing with the Elixir backend; validation and pricing logic maintained in both languages with contract tests guarding the boundary
3.6 Third-party services
| Function | Provider |
|---|---|
| Payments | Stripe (cards, ACH, text-to-pay) — or embedded payments via Payrix/Finix for a revenue share |
| Consumer financing | Wisetack API; Sunlight Financial secondary |
| E-signing | Dropbox Sign or Documenso embedded APIs |
| Telephony / SMS | Twilio (two-way texting, caller ID, A2P 10DLC) |
| Postmark (transactional) + Nylas (two-way inbox sync) | |
| PDF generation | Playwright/Chromium render farm |
| Accounting | Intuit API for QBO; lightweight web connector for QuickBooks Desktop |
| Measurements | Direct REST integrations behind an internal adapter interface |
3.7 Infrastructure & DevOps
- AWS (us-east + us-west); ECS Fargate initially, EKS + Karpenter if scale demands
- Postgres hosting constraint: TimescaleDB's licensed features (continuous aggregates, compression) are unavailable on AWS RDS — self-host on EC2 or use Timescale Cloud (see §6)
- Elixir releases in Docker; BEAM clustering via libcluster for distributed PubSub
- OpenTofu for IaC (drop-in Terraform replacement; Terraform moved to BUSL — see §6); GitHub Actions for CI/CD
- Grafana stack + Telemetry/PromEx (BEAM-native metrics) or Datadog; Sentry for errors; OpenFeature (+ flagd) or LaunchDarkly for flags
3.8 Security & compliance
- Auth via WorkOS or Auth0 — SSO/SCIM for enterprise, email/password for small shops; RBAC mapped to Admin/Sales/Field pricing roles
- SOC 2 Type II program (Vanta/Drata) from early stage; GDPR/CCPA data-subject-request workflows; personality profiles carry explicit consent and retention policies
- Third-party license review at upgrade time (see §6 — license-drift risk)
3.9 Data acquisition & activation services (prospect data engine) New in v3
| Function | Source / provider | Cost model |
|---|---|---|
| Parcel & ownership data | County assessor/recorder feeds directly, or an aggregator (Regrid, ATTOM, CoreLogic) for normalized multi-county coverage | Free-to-modest per county (labor-intensive) or aggregator subscription |
| Building permits | Municipal portals or permit aggregators (e.g., Shovels) | Free per portal or API subscription |
| Storm swaths | NOAA/NWS public hail and wind data | Free |
| Contact enrichment | Broker append APIs (Melissa, BatchData, Endato, PropertyRadar) behind an internal adapter interface — not realistically self-sourceable | Pay-per-record, on demand, segment-gated |
| Address hygiene | USPS CASS/NCOA processing | Per-batch service fee |
| DNC / suppression | National + state DNC scrub services | Subscription |
| Direct mail | Mail-house API (Lob, PostGrid) | Per-piece |
| Ad audiences | Meta / Google customer-match APIs (hashed uploads) | Ad spend |
| Offline attribution | Per-campaign tracking numbers (Twilio), UTM/QR codes | Per-number/usage |
Ongoing external costs are unavoidable for contact enrichment, address hygiene, DNC scrubbing, and channels — the economics shift from renting bulk lists annually to owning the public-records base layer and paying retail only for enrichment on demand. Realistic accuracy expectations: county data authoritative but weeks-to-months lagged; appended emails 60–80% deliverable and phones worse, industry-wide. Enrichment contracts should be negotiated for perpetual rights to appended records before signing (see §5).