Reliable payment systems start with strong TechOps.

Overview

DAE is an operational analytics workspace for turning transaction exports, device logs and controller evidence into validated findings and decision-ready reports. A sidebar-navigated workspace (Home, Data, Insights, Logs, Config) replaces the old single-file dashboard, with a dedicated Joins → Analysis → Reports flow and a Datasets library for everything already imported.

Stack

Data sources

Reconciliation engine

DAE's analysis engine runs transaction analytics and batching checks, response-code analytics, device-log investigation, connectivity analytics, decline-reason deep dives, and device-event analysis — each dataset is profiled, joined and validated before a finding is allowed into a report.

Built for defensible operational analysis

• Deterministic analysis — numbers and findings are computed from source rows; the analytical path doesn't depend on an LLM.
• Governed data — S3-compatible object storage and a MySQL backend track ownership, checksums, retention and lineage.
• Durable execution — lease-based background jobs support retries, idempotency and work that survives process boundaries.

What it replaced

• Manual cross-checks against the bank's settlement export → automatic transaction and amount-difference flags on every batch.
• Eyeballing timestamps to guess which terminals were settling late → automatic Daily / Irregular / Force-Batch classification.
• Decline-code knowledge living in one or two people's heads → five named, numbered response playbooks with clear trigger conditions.
• A single-file HTML dashboard → a sidebar-navigated, multi-page workspace (Home / Data / Insights / Logs / Config) with a dedicated Datasets library.

Companion layer

Six Grafana dashboards — overview, drill-down, processing overview, error investigation, session trace, and application support — read the same MySQL database directly, with no separate export/ETL step.

Infrastructure

Migrated off Fly.io onto a free-tier-optimized AWS stack: a single EC2 instance runs the app and worker containers, RDS (MySQL) is the production database, S3 holds application artifacts and database backups, and SSM Parameter Store manages secrets and config — all provisioned through Terraform, with remote state locked in S3 and DynamoDB.

Stage 1: Durable Jobs & Governed Storage

Four epics plus the closing gate shipped through seven merged pull requests over a total span of 10 days and 6 hours. The gate closed on a real cross-machine production upload and profile — one machine wrote an object, another profiled it successfully in production.

What's next

Forecasting and predictive insights are intentionally paused — Stage 1 focused on durable ingestion and governed storage; predictive capability activates once that foundation is proven.

Overview

A mobile-first PWA that analyses South Africa's six national lottery games — scoring balls, generating ensemble predictions, and surfacing heatmaps and draw history. It ships alongside BEP, a private, embedded engineering platform used to run the product's own development lifecycle, in a single Next.js app to share components and avoid a second build pipeline.

Stack

Analytics engine

Each ball is scored across 9 dimensions — frequency, gap, recency, pair, pattern, momentum, overdue, ensemble and composite — rolled into a 1–5 star verdict, alongside HOT / OVERDUE / COLD / NORMAL status tags and 4 AI pick strategies (Ensemble Primary, Frequency Bias, Gap Balanced, Hot Momentum). An auto-fetcher scrapes the official lottery site twice weekly and 8 analytics engines process the results into the API the frontend consumes.

BEP — the self-built engineering platform

A lightweight GitHub + Linear + Azure DevOps hybrid, covering planning, build & test, deployment (an 8-stage pipeline with approval gates), release and ops — including live DORA metrics computed client-side. Runs entirely on localStorage with no backend, using a custom DOM event for cross-tab reactivity, and keeps an append-only audit trail that survives deletion of the underlying record.

Release snapshot (v1.0.0)

Grafana Monitoring

Four dashboard folders cover analytics accuracy, backend operations, Fly.io infrastructure and a support runbook — all reading from the same API and Prometheus endpoint.

Dashboard overview — Analytics, Backend Ops, Fly Infrastructure & Support Runbook

System health tiles and prediction-strategy accuracy per game

Overdue-ball hit-rate gauges and what the model is watching right now

Full ball rankings by model confidence, verified against recent draws

Fetcher status, data version and per-game draw coverage

Instance uptime, memory timeline and CPU tracked in real time

Known issues

Links

Overview

A real-time market data platform ingesting live crypto, stock and financial-news feeds around the clock, publishing everything through a shared mapping/validation pipeline into one MySQL-backed store — with a per-feed Grafana dashboard and a plain-language Insights panel sitting on top of the same data.

Stack

Three feeds, one pipeline

Crypto trades stream continuously from Binance's WebSocket API, stock trades pull from Finnhub during market hours, and Alpha Vantage news is polled hourly and scored for sentiment — all three land through the same ingest → map → validate → publish pipeline and the same admin tooling, rather than three one-off scripts.

Insights panel

Four standing questions — what to watch in crypto, which stock is trending, the market mood from recent news, whether markets are moving together — answered live from whatever's actually been ingested, and explicitly labelled informational rather than investment advice.

Field note — routing around a 451

The crypto worker got a straight HTTP 451 from Binance when running out of Fly's Johannesburg region — confirmed by probing from other regions — so it runs from Amsterdam while the rest of the app, including the other two feeds, stays in Johannesburg.

Admin control, with a hard guardrail

An admin panel can start, stop and restart the streaming workers directly through the Fly Machines API — with a hardcoded denylist that refuses to touch the web process group no matter what calls it, so a bug or a crafted request further up the stack can't take the app down through that path.

Grafana Monitoring

Four dashboards — one per feed plus a pipeline overview — all reading from the same MySQL store for continuous refresh.

Crypto Stream — three independently-scaled price panels, volume, momentum and volatility

News Stream — sentiment breakdown and top mentioned tickers

Pipeline overview — files ingested, rows published, exceptions

Stocks Stream — Finnhub trade volume and recent trades

Known issues

What's next

Porting the crypto dashboard's per-symbol axis fix to the stocks feed, and closing out the DB memory verification window.

Links

Overview

AutoSiteLoader automates provisioning payment terminals for a merchant site on an admin portal. An operator drops a site's terminal loadsheet in, and the tool drives the portal through Playwright to provision each terminal, issue it to the site, export the resulting terminal list, and add a standard product catalog — with a dry-run mode that exercises the whole flow without saving, and a downloadable exception report for anything that couldn't be processed.

Stack

Four-stage pipeline

• Provision — create each terminal asset.
• Issue — assign it to the merchant site.
• Export — save the resulting portal records.
• Add products — apply the standard catalog to every provisioned terminal.

Dry run before live

Every loadsheet can be run in Dry run mode — the full Playwright flow executes without clicking Save — before a Live submit commits anything on the real portal.

Known issues

Overview

P24Dashboard (P24 Operations Dashboard) was a day-to-day investigation and operations tool covering terminal health, bank settlement, transaction performance and support tickets across a payment terminal fleet, paired with a companion Grafana layer for real-time alerting. A twice-daily scheduled pipeline logged into the acquiring bank's portal, scraped and merged settlement exports, ran reconciliation, and wrote the result into a shared database so the dashboard and Grafana always agreed.

Stack

Twice-daily recon pipeline

A scheduled pipeline ran at 07:00 and 12:00 SAST: Playwright scraped the acquirer's settlement export, imported it, and ran a 14-strategy waterfall matching engine against bank markoff files — resolving every transaction to a Strong Match, Partial Match, or Unmatched using identifiers like transaction ID, RRN, amount, TID, MID and card BIN/last-4. Amount and date alone was never treated as sufficient; at least one hard identifier had to line up before a match was accepted.

Panels covered

Companion layer

Five Grafana dashboards (fleet overview, per-terminal drill-down, settlement, alerts & thresholds, and pipeline health) sat alongside the custom dashboard, backed by 4 alert rules — including one watching for the pipeline itself going quiet for over 13 hours, catching a broken pipeline before anyone noticed stale data. A separate real-time ingestion path also streamed live Bitcoin blockchain events over WebSocket into the same platform.

What it became

Its analytics-engine module was later extracted into its own standalone platform — see the DAE project card above — which now carries that governed-ingestion work forward independently.