Madlab

---

What is this?

Madlab is a self-hosted tool for fine-tuning language models on your own hardware. It handles the annoying parts: dataset formatting, training config, GGUF conversion, and evaluation. Works with any HuggingFace model.

Stack: React frontend + Node.js backend + Python training scripts (PyTorch/Transformers)

---

Features

Training

• LoRA/QLoRA fine-tuning with configurable rank, alpha, dropout
• NEFTune - Noisy embeddings for +3-5% quality boost
• DoRA - Weight-decomposed LoRA for better quality at same rank
• Config presets - Quick Test, Small Model, Large Model, Quality Fine-tune
• Job queue - Queue multiple training runs, auto-start next
• Checkpoint resume - Save progress, resume on crash
• Graceful cancellation - SIGINT allows checkpoint save before stop
• Real-time loss chart - Visual training progress in Monitoring tab
• Progress timeline - ETA based on epoch duration

Dataset Management

• Multi-format import - CSV, Parquet, JSON, JSONL
• Dataset profiling - Row count, duplicates, length stats
• Dataset versioning - Track changes, rollback to previous versions
• Dataset preview - Inspect samples + validation before training
• Validation rules - Custom quality checks (min/max length, regex, etc.)
• LLM-powered augmentation - Expand datasets via paraphrasing

Evaluation

• BLEU/ROUGE metrics - Fuzzy matching alongside exact match
• LLM Judge - AI-powered output quality scoring
• Auto test case generator - Generate adversarial/edge case inputs
• Failure analysis - LLM explains why model failed specific samples

LLM-Powered Features (via LM Studio)

• Magic Import - Auto-format any HuggingFace dataset
• Dataset quality analyzer - LLM rates training sample quality
• Hyperparameter advisor - LLM suggests optimal training config
• Synthetic data generation - Generate training data from examples

Infrastructure

• Health dashboard - CPU, memory, disk, GPU monitoring
• Audit logging - Track all sensitive operations
• Structured logging - JSON logs with request timing metrics
• Project backup/restore - Export/import full project state
• Model lineage tracking - Full provenance for reproducibility
• Response caching - LRU cache for LLM proxy requests
• LM Studio health probe - Circuit breaker for upstream failures

UI/UX

• Command palette - Ctrl/Cmd+K for quick actions
• Keyboard shortcuts - Ctrl+1/2/3/4 for tab navigation
• Dark/Light theme - Toggle in header
• Toast notifications - Consistent feedback across panels
• WebSocket status - Connection indicator in header
• Export bundle - One-click ZIP of model + config + dataset

Desktop App (Experimental)

• Electron packaging - Run as standalone desktop app

---

Quick Start

# Clone
git clone https://github.com/yourusername/madlab.git
cd madlab

# Backend
cd madlab-backend
npm install
cd trainer && python -m venv venv && venv\Scripts\activate && python setup.py && cd ..
npm run build && npm start

# Frontend (new terminal)
cd madlab-frontend
npm install && npm run dev

Open http://localhost:5173

---

Requirements

Dependency	Version	Notes
Node.js	18+	Backend server
Python	3.10+	Training scripts
CUDA	11.8+	Optional, for GPU training
LM Studio	Any	Optional, for Magic Import/Judge features

Hardware:

• CPU training: Works, but slow. Fine for small models (<1B params)
• GPU training: NVIDIA with 8GB+ VRAM recommended

---

Setup

Backend

cd madlab-backend
npm install

Python environment (inside trainer/ folder):

cd trainer
python -m venv venv

# Windows
venv\Scripts\activate

# Linux/Mac
source venv/bin/activate

# For GPU training - first check your CUDA version: nvidia-smi
# Then install PyTorch with matching CUDA in setup.py:

python setup.py

cd ..

Build and run:

npm run build
npm start

Backend runs on http://localhost:8080

Frontend

cd madlab-frontend
npm install
npm run dev

Frontend runs on http://localhost:5173

---

Configuration

Backend (.env)

Create madlab-backend/.env:

PORT=8080
LM_STUDIO_URL=http://localhost:1234
ALLOWED_ORIGINS=http://localhost:5173,http://localhost:3000

# Optional: Increase timeouts for slower systems (in milliseconds)
# FETCH_TIMEOUT=60000    # Default: 60s - for HuggingFace API calls
# LLM_TIMEOUT=300000     # Default: 5min - for LLM inference calls

Frontend (.env)

Create madlab-frontend/.env:

VITE_API_URL=http://localhost:8080
VITE_WS_URL=ws://localhost:8080/events

---

Usage

Basic Workflow

1. Pick a model - Enter a HuggingFace model ID (e.g., TinyLlama/TinyLlama-1.1B-Chat-v1.0) or browse
2. Get data - Three options:
   • Import from HuggingFace datasets
   • Upload your own .jsonl files
   • Generate synthetic data from examples
3. Configure - Set epochs, batch size, learning rate, device (CPU/CUDA)
4. Train - Hit start, watch logs in Monitoring tab
5. Convert - Export to GGUF (f16 or q8_0 quantized)
6. Evaluate - Run against validation set

Dataset Format

Your .jsonl files need input and target fields:

{"input": "What is 2+2?", "target": "4"}
{"input": "Capital of France?", "target": "Paris"}

Magic Import tries to auto-convert other formats using an LLM.

LM Studio Features

If you have LM Studio running locally:

• Magic Import - Auto-formats any HuggingFace dataset
• Magic Judge - LLM-based evaluation of model outputs
• Synthetic Data - Generate training data from examples

Point LM_STUDIO_URL to your instance (default: http://localhost:1234)

---

Project Structure

madlab/
├── madlab-backend/
│   ├── src/
│   │   ├── routes/        # API endpoints
│   │   ├── services/      # Training, conversion logic
│   │   ├── utils/         # Security, fetch utilities
│   │   ├── types/         # TypeScript interfaces
│   │   ├── config.ts      # Centralized configuration
│   │   └── server.ts      # Express server
│   ├── trainer/           # Python scripts
│   │   ├── train.py       # Fine-tuning script
│   │   ├── data_tools.py  # Dataset utilities
│   │   └── evaluate_gguf.py
│   └── data/              # Datasets stored here
│
├── madlab-frontend/
│   └── src/
│       ├── components/    # React components
│       ├── types.ts       # TypeScript interfaces
│       └── App.tsx

---

Troubleshooting

llama-cpp-python installation fails

Replace the entire content of your madlab-backend/trainer/requirements.txt with this block:

transformers
accelerate
peft
bitsandbytes
scipy
huggingface_hub[hf_xet]
protobuf
sentencepiece
datasets
pandas
mistral-common[image,audio]
matplotlib

# Use custom llama-cpp-python wheel
https://github.com/rookiemann/llama-cpp-python-py314-cuda131-wheel-or-python314-llama-cpp-gpu-wheel/releases/download/v0.3.16-cuda13.1-py3.14/llama_cpp_python-0.3.16-cp314-cp314-win_amd64.whl

# Install gguf directly from llama.cpp repo
https://github.com/ggml-org/llama.cpp/archive/refs/heads/master.zip#subdirectory=gguf-py

"CUDA not available"

• Check nvidia-smi works in terminal - note the CUDA version shown
• Make sure you installed PyTorch with matching CUDA support (see Setup section)
• Reinstall PyTorch with your CUDA version:
  • First, uninstall any existing installation:
    • pip uninstall torch torchvision torchaudio -y
  • CUDA 11.8 (older GPUs, GTX 10xx/16xx, RTX 20xx):
    • pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
  • CUDA 12.1 (RTX 30xx, RTX 40xx):
    • pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
  • CUDA 12.4 (RTX 40xx, newer drivers):
    • pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
  • CUDA 12.6 (latest stable, RTX 40xx/50xx):
    • pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
  • CUDA 13.0 (nightly, experimental support for RTX PRO 6000 Blackwell):
    • pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130
• Or just use CPU: set device to "CPU" in the UI (slower but works)

"Model not found"

• Check the HuggingFace model ID is correct
• Some models require authentication: huggingface-cli login

Training is slow

• Use GPU if possible
• Reduce max_seq_len (default 512)
• Reduce batch size if running out of memory

Timeout errors / Operations timing out

• Increase timeouts in .env file (see Configuration section)
• Set FETCH_TIMEOUT=120000 for slow HuggingFace API calls
• Set LLM_TIMEOUT=600000 for very slow LLM inference

"Failed to connect to LM Studio"

• Start LM Studio and load a model
• Check the URL in .env matches your LM Studio server
• Magic features work without it, just no auto-formatting

WebSocket disconnects

• Backend might have crashed, check terminal
• Refresh the page

Port already in use

• Change PORT in backend .env
• Update VITE_API_URL and VITE_WS_URL in frontend .env to match

---

API Endpoints

Endpoint	Method	Description
/health	GET	Server health check
/datasets	GET	List all datasets
/datasets/import	POST	Import from HuggingFace
/datasets/upload	POST	Upload .jsonl file
/train/start	POST	Start training
/train/stop	POST	Stop training
/train/status	GET	Training status
/train/config	GET/POST	Get/update training config
/train/artifacts	GET	List model artifacts
/instillations	GET/POST/PUT/DELETE	Manage instillations

---

License

AGPLv3 - See LICENSE file.

Copyright (C) 2025 David Bentler