NextFlow

Distributed Visual AI Workflow Orchestrator

---

What It Does

NextFlow is a production-grade, highly-scalable visual node-based AI workflow orchestrator.

It empowers you to build complex, multi-step AI and compute pipelines (Directed Acyclic Graphs or DAGs) using an intuitive, beautifully designed drag-and-drop canvas. NextFlow abstracts away the complexity of connecting LLMs, vector databases, document parsers, and media processing (FFmpeg), giving you a visual approach to logic design.

Instead of writing brittle scripts that time out or fail unpredictably, NextFlow leverages a distributed event-driven backend. Once your pipeline is built, NextFlow compiles your DAG and orchestrates the execution entirely in the background via dedicated cloud workers.

NextFlow visual node editor in action.

---

Features & Architecture

Building a distributed node-editor required tackling incredibly complex state management and infrastructure challenges:

• True Distributed Execution: Bypasses serverless timeouts by fully offloading heavy compute (LLM processing, FFmpeg video frame extraction, PDF chunking, Embedding generation) to dedicated Node 22 background workers via Trigger.dev.
• Topological DAG Sorting: Calculates proper execution order, automatically waits for upstream node resolutions, dynamically maps output payloads to downstream inputs, and halts execution loops with cycle detection.
• Advanced State Management: Employs a multi-slice Zustand architecture handling real-time graph synchronization, complex UI layers (sidebars, toolbars), and live execution tracking.
• Visual Execution Inspector: Features a timeline-based run inspector. Hovering over past execution logs instantly highlights the active nodes and reveals input/output payloads in real-time, making debugging AI hallucinations trivial.
• Integrated RAG Pipeline: Native support for Chunking, Embedding (Gemini), Storing (PostgreSQL pgvector via Supabase), and semantic Retrieval & Reranking.

---

The Tech Stack

Frontend

• Framework: Next.js 15 (App Router)
• Visual Canvas: React Flow (@xyflow/react)
• State Management: Zustand (Multi-slice stores)
• Styling: TailwindCSS, Lucide Icons, highly customized glassmorphic modern UI
• Components: Radix UI primitives (where applicable)

Backend & Cloud Services

• API Runtime: Next.js Serverless Route Handlers
• Authentication (Clerk): Handles secure user onboarding, session management, and protects backend API routes from unauthorized execution requests.
• Database (Supabase PostgreSQL): Provides a highly scalable, hosted Postgres database. NextFlow utilizes Supabase's Transaction Pooler (PgBouncer) to prevent connection exhaustion from Next.js serverless functions, and leverages the native pgvector extension to store and query high-dimensional embeddings for the RAG pipeline.
• ORM: Prisma + Prisma Postgres Adapter
• Background Orchestration (Trigger.dev v3): The backbone of the distributed execution engine. Trigger.dev offloads heavy computational tasks (like FFmpeg video processing and long-running LLM prompts) from Vercel's strict serverless timeout windows to dedicated, persistent background cloud workers.
• AI / LLM (Google Gemini SDK): Powers the intelligent nodes within the workflow. Utilizes gemini-2.5-flash for fast, cost-effective multimodal reasoning and gemini-embedding-2 for generating dense vector representations of text.
• Media Processing: FFmpeg (fluent-ffmpeg) for video frame extraction and media manipulation.
• Deployment (Vercel): Hosts the Next.js web application and API layer, providing global edge caching, environment variable management, and seamless CI/CD integration.

---

Project Structure

nextflow-app/
├── prisma/                  # Prisma schema and migrations (incl. pgvector)
├── public/                  # Static assets and placeholders
├── src/
│   ├── app/                 # Next.js App Router (Pages, API Routes)
│   │   ├── api/             # REST Endpoints (DAG trigger, graph saving, webhooks)
│   │   ├── workflow/        # Dynamic workflow canvas routes
│   │   └── page.tsx         # User workspace / Dashboard
│   ├── components/          # Reusable UI Architecture
│   │   ├── canvas/          # React Flow custom nodes, Canvas UI, Execution Inspector
│   │   ├── sidebar/         # Left (Node Palette) & Right (Run History) Sidebars
│   │   └── toolbar/         # Workspace controls and triggers
│   ├── lib/                 # Singletons (Prisma client, Clerk auth helpers)
│   ├── stores/              # Zustand State Management (workflow-store, run-store)
│   ├── trigger/             # Trigger.dev v3 Cloud Workers
│   │   ├── orchestrator.ts  # Master Task: Traverses DAG and routes execution
│   │   └── tasks.ts         # Worker definitions (Gemini, FFmpeg, PDF-Parse, pgvector)
│   ├── types/               # TypeScript interfaces and robust payload typing
│   └── utils/               # DAG utilities (topological sort, cycle detection)
├── trigger.config.ts        # Trigger.dev deployment configuration
└── next.config.ts           # Next.js bundler optimizations (ServerExternalPackages)

---

Live Demo

Check it out live: https://nextflow-gamma-sable.vercel.app

(Note: Ensure you are logged in to save your workflows and run background tasks.)

---

How to Run Locally

If you'd like to spin this up on your local machine:

1. Environment Setup

Populate your .env.local with the following keys:

• NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY & CLERK_SECRET_KEY
• DATABASE_URL (Supabase Transaction Pooler) & DIRECT_URL
• TRIGGER_SECRET_KEY (Trigger.dev v3 Dev Key)
• GEMINI_API_KEY (Google Gemini)

2. Install & Start Frontend

npm install
npm run dev

(Runs the Next.js app on localhost:3000)

3. Start the Background Task Worker

In a separate terminal tab, boot up the Trigger.dev dev worker:

npx trigger.dev@latest dev

(Listens for the tasks triggered by your localhost app)

---