Production systems

Metered API billing platform

Billing is the one subsystem where a rounding error is a legal problem and a duplicated event is a refund. This is a full usage-based billing stack, covering event ingestion through invoicing, payment, and two dashboards, built around the assumption that every input arrives twice, out of order, and occasionally hostile.

Type
Personal project, open source
Backend
Django 5 · DRF · PostgreSQL
Frontend
Two React + TypeScript SPAs (Vite)
Tests
125+ backend tests against live PostgreSQL

View on GitHub

The problem

Usage-based billing looks like a counting problem and is actually a correctness problem. The same usage event will be delivered more than once. Events arrive late, after the period they belong to has already been invoiced. Two background jobs will try to close the same month at the same time. And every one of those failures moves money.

I built this to work through those boundaries deliberately rather than discover them in production. The repository ships with a design document covering the data model, the idempotency strategy, and a threat model.

How it works

Events are ingested idempotently and rolled up hourly. Invoices are generated from the rollups with tiered pricing, and every state change writes to an audit log that the database itself refuses to let you rewrite.

USAGE events INGEST dedupe · late-safe ROLLUP hourly INVOICE tiered · half-up signed webhook dashboards APPEND-ONLY AUDIT LOG · ENFORCED BY DB TRIGGER

Two front ends sit on the same API: a customer dashboard for usage and invoices, and an operations console for credits, overrides, and anomaly review.

Decisions that mattered

The audit log is append-only at the database level, not in application code.

An audit log a service can quietly update is not an audit log. Enforcing it with a trigger means the guarantee survives a bug, a migration, or someone with a database console.

Ingestion is idempotent by design, with explicit late-event handling.

At-least-once delivery is the norm for usage pipelines. Deduplication at the boundary is the only place you can do it once instead of in every downstream consumer.

Money uses exact arithmetic with half-up rounding, never floats.

Binary floating point cannot represent most decimal amounts. In billing, the accumulated error is not a rounding curiosity; it is a discrepancy someone has to reconcile.

Webhooks are HMAC-signed with a replay window.

A signature proves origin; the window is what stops a captured-and-replayed payload from being accepted forever. Both are needed, and the second is the one people skip.

Concurrent background jobs take a lock before touching a period.

Two schedulers closing the same month simultaneously is a race that produces double invoices. The lock makes the failure mode impossible rather than unlikely.

Tested against a real database

The suite runs against live PostgreSQL rather than mocks. That is deliberate: the trigger, the unique constraints, and the locking behaviour are the parts most worth testing, and every one of them disappears if you mock the database away. Anomaly detection, a usage spike measured against a thirty-day baseline, is covered the same way.

Other work