EchoNotes

EchoNotes is a Python-based application that monitors a folder for new files, extracts the content (text, audio, video), summarizes it using a local instance of an LLM model, and saves the summarized output back to disk. It supports offline operation and can handle multiple file formats, including PDFs, Word documents, text files, video/audio files.

Features

• Monitors a directory for new files (PDF, DOCX, TXT, common audio formats, and video formats).
• Text Extraction:
  • PDF files (via PyPDF2 and Tesseract for OCR)
  • Word documents (via python-docx)
  • Plain text files
  • Audio files such as MP3, WAV, M4A, AAC, FLAC, OGG, OPUS, WMA, AIFF, MP2, AMR, and AC3
  • Video files such as MP4, AVI, MOV, MKV, WEBM, and M4V
  • Non-MP3 audio is normalized to MP3 via FFmpeg before transcription
• Summarization:
  • Supports explicit LLM providers instead of assuming Ollama-style APIs.
  • Supported providers: Open WebUI, Ollama, OpenAI, Claude/Anthropic, OpenRouter, and a legacy generic generate endpoint.
  • Supports customizable markdown prompts.
• Offline Operation:
  • All processing (text extraction, transcription, summarization) can be done offline.
  • Pre-downloads WhisperX ASR models and handles everything locally.
• Background Worker Pool:
  • Files are queued immediately by the watcher and processed by persistent workers.
  • Each worker keeps its model loaded to avoid per-file startup costs.
  • The container is intended to run continuously while monitoring the mounted incoming/ folder.
• Automatic Transcript Formatting:
  • Audio and video transcripts can be reformatted into readable Markdown before saving.
  • Formatting falls back to the raw transcript if the formatter fails.
• Speaker Labels:
  • WhisperX diarization can label timestamped transcript lines as Speaker 1, Speaker 2, and so on.
  • Speaker diarization requires a Hugging Face token for the pyannote diarization model.
• Automatic Chunking:
  • Large transcripts are chunked and reduced automatically so long meetings do not overflow model context windows.
• Obsidian Export:
  • Audio/video jobs can copy the final MP3, transcript, summary, and an Obsidian note into vault/.
  • The Obsidian note now includes structured front matter, linked entities, and deterministic sections rendered by Python.
  • The Obsidian note template and extraction prompt are loaded from the config area, with image-shipped fallbacks.
• Logging: Extensive logging to help track operations and errors.

Requirements

Quick Start via Docker

Create a config directory and place your runtime files there:

mkdir -p config incoming vault model-cache
cp config.sample.yml config/config.yml
cp summarize-notes.md config/summarize-notes.md

Edit config/config.yml for your LLM endpoint, model, tokens, and any diarization settings.

Run the persistent worker container:

docker run -d --name echonotes \
  -v /path/to/incoming:/app/incoming \
  -v /path/to/vault:/app/vault \
  -v /path/to/config:/app/config \
  -v /path/to/model-cache:/app/model-cache \
  echonotes:latest

Example from the repo root:

docker run -d --name echonotes \
  -v "$(pwd)/incoming:/app/incoming" \
  -v "$(pwd)/vault:/app/vault" \
  -v "$(pwd)/config:/app/config" \
  -v "$(pwd)/model-cache:/app/model-cache" \
  echonotes:latest

The model cache mount is optional but recommended for local/dev use. If /app/model-cache is empty, EchoNotes downloads the configured WhisperX model at startup. If whisper_model is not configured, it defaults to base on CPU and small on GPU.

Published image tags follow this pattern:

• echonotes:latest: CPU image
• echonotes:latest-cuda12.8: default GPU image
• echonotes:gpu: alias for the default GPU image

Versioned releases follow the same split:

• echonotes:1.4.1
• echonotes:1.4.1-cuda12.8

Installation from source, via docker.

Docker Setup

1. Build the Docker Images:

   EchoNotes currently supports two local build variants:

   • CPU: echonotes:latest
   • GPU: echonotes:latest-cuda12.8

   Build the CPU image:

   docker build -t echonotes:latest .
   

   The default Docker build is the CPU variant. It uses the official PyTorch CPU wheel path so the image does not pull the CUDA package set.

   Build the GPU image:

   docker build \
     --build-arg IMAGE_VARIANT=gpu \
     --build-arg GPU_BASE_IMAGE=nvidia/cuda:12.8.1-cudnn-runtime-ubuntu22.04 \
     --build-arg TORCH_INDEX_URL=https://download.pytorch.org/whl/cu128 \
     --build-arg TORCH_EXTRA_INDEX_URL= \
     --build-arg TORCH_PACKAGE_SPEC=torch==2.8.0 \
     --build-arg TORCHAUDIO_PACKAGE_SPEC=torchaudio==2.8.0 \
     --build-arg TORCH_INSTALL_NO_DEPS=1 \
     --build-arg TORCH_PYTHON_DEPS=filelock,fsspec,jinja2,markupsafe,mpmath,networkx,sympy,typing-extensions \
     -t echonotes:latest-cuda12.8 .
   

   Optional local alias for the GPU image:

   docker tag echonotes:latest-cuda12.8 echonotes:gpu
   

   If you want both local variants available in one pass, run both commands above.

   The GPU build uses the NVIDIA CUDA runtime image directly and keeps only the extra CUDA libraries that PyTorch still needs beyond that base, which avoids duplicating the full nvidia-* pip wheel bundle.

2. Run the Docker Container:

   Run the long-lived worker container with the three runtime mounts:

   docker run -d --name echonotes \
     -v /path/to/incoming:/app/incoming \
     -v /path/to/vault:/app/vault \
     -v /path/to/config:/app/config \
     -v /path/to/model-cache:/app/model-cache \
     echonotes:latest
   

3. Pre-download WhisperX Models:

   The recommended way to pre-download WhisperX models is to mount a host cache directory to /app/model-cache and warm it before your first long run.

   Build the CPU image and warm an empty local cache:

   ./build.sh --model-cache-dir ./model-cache
   

   Build the GPU image and warm an empty local cache through the NVIDIA runtime:

   ./build.sh --gpu --model-cache-dir ./model-cache
   

   Those commands build the image first, then warm the cache. build.sh only performs the automatic warmup step when the target cache directory is empty, unless you explicitly pass --model.

   To warm the cache without building:

Docker Compose Example

You can use Docker Compose to manage the container:

version: '3.8'
services:
  echonotes:
    image: echonotes:latest
    volumes:
      - ./incoming:/app/incoming
      - ./vault:/app/vault
      - ./config:/app/config
      - ./model-cache:/app/model-cache
    restart: unless-stopped

Run the service with:

docker-compose up -d

For a GPU host, switch the image tag to echonotes:gpu or echonotes:latest-cuda12.8 and add the appropriate GPU runtime settings for your Docker installation.

Usage

EchoNotes monitors the /app/incoming directory continuously. When it detects a new file, it processes it according to the file type:

• PDF: Extracts text using PyPDF2 or OCR via Tesseract if needed.
• Word Documents (DOCX): Extracts text using python-docx.
• Text Files (TXT): Reads the plain text.
• Audio Files: Common audio inputs are converted to MP3 with FFmpeg when needed, then transcribed with WhisperX.
• Video Files: Extracts audio using FFmpeg, then transcribes it with WhisperX.

Once the text is extracted, it can be summarized by sending the text and a customizable markdown prompt to a configured LLM provider. If no LLM provider is configured, EchoNotes will still extract and transcribe files, but it will skip LLM-based formatting and summarization.

Files placed in hidden folders or dot-paths such as .obsidian, .stfolder, or .syncthing* are ignored. Temporary partial-download files such as .part, .tmp, and .crdownload are also ignored.

Pipeline Overview

Audio

• Detect supported audio file in incoming folder
• Wait until file size becomes stable
• Move file into working folder
• Convert to MP3 if needed
• Transcribe with WhisperX
• Align timestamps and diarize speakers
• Format transcript with LLM if configured
• Generate summary with LLM if configured
• Create Obsidian note and copy vault artifacts
• Move originals and outputs to completed

Video

• Detect supported video file in incoming folder
• Wait until file size becomes stable
• Move file into working folder
• Extract audio to MP3 with FFmpeg
• Transcribe with WhisperX
• Align timestamps and diarize speakers
• Format transcript with LLM if configured
• Generate summary with LLM if configured
• Create Obsidian note and copy vault artifacts
• Move originals and outputs to completed

Documents

• Detect supported document file in incoming folder
• Wait until file size becomes stable
• Move file into working folder
• Extract text from PDF, DOCX, or TXT
• Use OCR fallback for image PDFs
• Generate summary with LLM if configured
• Move originals and outputs to completed

Configuration

The application is configured via /app/config/config.yml. The image also includes baked defaults in /app/config-defaults, so if a prompt file is missing from the mounted config directory EchoNotes falls back to the image default where available. An example configuration file is shown below:

path_to_watch: "/app/incoming"
incoming_rescan_interval_seconds: 5 # Periodic fallback scan for missed filesystem events; set 0 to disable

llm:
  provider: "openwebui"
  model: "gpt-4o-mini"
  base_url: "http://openwebui:3000/api"
  api_key: "your_api_token_here"
  timeout_seconds: null
  max_tokens: 2048

whisper_model: "base" # Optional; defaults to 'base' on CPU and 'small' on GPU when omitted
whisper_batch_size: null # Optional; defaults to 16 on GPU and 4 on CPU
whisper_min_batch_size: 1 # On GPU, OOM retries back off down to this batch size
gpu_oom_fallback: "cpu" # One of: cpu, fail
worker_count: 2 # Number of background workers to run concurrently; on GPU start with 1
diarization_enabled: true # Enable Whisp