Distributed Multi-Tenant Email Job Scheduler

Introduction

Distributed Multi-Tenant Email Job Scheduler is an enterprise-grade job scheduling architecture designed to handle bulk, recurring, and transactional email campaigns with strict isolation across tenants, rate limits, and failure recovery.

Why I Built It

Email delivery at scale introduces significant architectural challenges:

• High volume campaigns can easily overwhelm SMTP servers or trigger ISP rate limits.
• In multi-tenant environments, one tenant's spike in volume should never degrade performance for other tenants.
• System crashes or transient network failures must not cause duplicate sends or lost jobs.

This project was built to address these challenges through distributed queues, idempotency keys, and token-bucket throttling.

Core Features

• Multi-Tenant Architecture: Organization-scoped campaigns, JWT and OAuth authentication, and strict tenant-level database row authorization.
• Distributed Job Pipeline: Powered by BullMQ and Redis with configurable worker concurrency, delayed job execution, and stalled-job recovery.
• Resilience & Idempotency: Exponential backoff retries with cryptographic idempotency safeguards preventing duplicate message deliveries.
• Token-Bucket Rate Limiting: Redis-backed rate limiting with per-tenant and per-sender SMTP quotas, absorbing burst traffic without blocking unrelated tenants.
• Analytics & Observability: Real-time log indexing in Elasticsearch, execution tracking, and automated Slack webhook alerts for system incidents.
• Containerized Infrastructure: Production Docker and Docker Compose setup for straightforward local testing and deployment across Vercel and Render.

Tech Stack & Architecture

• Backend: Node.js, Express.js, TypeScript, BullMQ, Redis
• Frontend & Admin: Next.js, Tailwind CSS
• Database & ORM: PostgreSQL with Prisma ORM
• Search & Monitoring: Elasticsearch, Slack Webhook API
• DevOps: Docker Compose, Render, Vercel

Explore the source code on GitHub