Distributed Task Queue System

Fault-tolerant distributed system for scalable, asynchronous background processing

FastAPI Redis PostgreSQL Docker Kubernetes
Get Started System Architecture View Demo

See TaskFlow in Action

Watch how TaskFlow handles distributed task processing at scale

Live Demo

TaskFlow Demo

Key Features

🚀

Distributed Processing

Horizontally scalable worker nodes process tasks in parallel across multiple instances

⚡

High Performance

Built on FastAPI for lightning-fast task submission and monitoring with async capabilities

Reliable Scheduling

Redis-based locking prevents race conditions and ensures reliable task distribution

📊

Observable

Complete audit trail of task states via REST endpoints for full visibility

Production Ready

Docker Compose and Kubernetes deployment configurations included

Persistent Storage

PostgreSQL ensures durable persistence of task history and results

Quick Start

Docker Deployment

curl -sSL https://raw.githubusercontent.com/dhruvkshah75/TaskFlow/main/install.sh | bash

One-line installation with Docker Compose

Kubernetes Deployment

kubectl create namespace taskflow
kubectl apply -f k8s/

Deploy to production Kubernetes cluster

Built With

FastAPI
Python 3.11+
Redis
PostgreSQL
Docker
Kubernetes

Ready to Scale Your Task Processing?

Get started with TaskFlow today and handle millions of background jobs with ease

View on GitHub