SpiderCamera 🕷️📷

High-performance RAW camera capture library for Raspberry Pi

SpiderCamera is a C++ wrapper around libcamera designed for high-speed RAW frame capture with Python bindings via pybind11. It targets Raspberry Pi boards and provides in-memory burst capture with planned support for hot parameter updates and GPIO triggers.

Features

• 🎥 High-speed YUV420 streaming (for real-time analysis, bypassing slow RAW decompression).
• 🐍 Python interface via pybind11.
• 🚀 ~14 fps burst capture performance at 3840x2400.
• 💾 In-memory "burst" frame buffering via get_burst_frames().
• ✅ Hot parameter changes (ISO, exposure, resolution, focus) via Python setters.
• ✅ Per-frame GPIO hardware trigger support (for flash/strobe sync).
• ✅ Hardware-level FPS Calculation via get_last_series_fps() (eliminates Python timing jitter).

Requirements

• Raspberry Pi with libcamera support
• Python 3.11+
• libcamera-dev
• libgpiod-dev (for v0.4+)
• pybind11
• opencv-python-headless (for processing/saving frames in Python)

Quick Start

SpiderCamera/ ├── include/ │ └── spider_camera.hpp # C++ API interface ├── src/ │ ├── spider_camera.cpp # Core implementation │ ├── pisp_decompress.cpp # (DEPRECATED: Old v0.2 RAW logic) │ └── frame_buffer.cpp # (DEPRECATED: Old v0.2 RAW logic) ├── bindings/ │ └── pybind_spider.cpp # Python bindings (pybind11) ├── examples/ │ ├── demo_python.py # Test script for v0.3 (YUV Burst) │ └── demo_config.json # v0.3 config file (ISO, Exp, Res, Focus) ├── build.sh # Build script └── Specification.md # Technical specification

# Clone repository
git clone https://github.com/YOUR_USERNAME/SpiderCamera.git
cd SpiderCamera

# Setup virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install pybind11 numpy opencv-python-headless

# Build
chmod +x build.sh
./build.sh

# Test
python3 examples/demo_python.py

⚠️ Data Processing & Stride Handling (v0.6+)

Starting from v0.6, the library performs a raw memory copy to prevent segmentation faults on Raspberry Pi 5 (due to ISP alignment/padding). The byte arrays returned by get_burst_frames() contain padding bytes at the end of every row.

Correct way to process frames:

# 1. Get properties including Stride
width, height, fmt, stride = cam.get_frame_properties()

# 2. Calculate dimensions
# For YUV420/NV12, physical height in memory is 1.5x the image height
yuv_height = int(height * 1.5)

for flat_frame in frames:
    # A. Determine how many stride-aligned rows are in the buffer
    rows = flat_frame.size // stride
    
    # B. Reshape using Stride (NOT width)
    # This aligns the data correctly in 2D space
    view_2d = flat_frame[:rows*stride].reshape((rows, stride))
    
    # C. Crop the valid data
    # Remove padding on the right (:width) and extra rows at bottom (:yuv_height)
    image_data = view_2d[:yuv_height, :width]
    
    # D. Ensure memory is contiguous for OpenCV
    import numpy as np
    image_contiguous = np.ascontiguousarray(image_data)
    
    # E. Convert
    bgr = cv2.cvtColor(image_contiguous, cv2.COLOR_YUV2BGR_I420)

Application Pipeline: Super-Resolution & Photometry

The primary goal of this library is to enable advanced surface analysis by combining high-speed burst capture with synchronized lighting.

The intended pipeline consists of two stages:

1. Capture Stage (Real-time, ~1 sec):

   • The camera captures a high-speed burst (~14 fps) of YUV 3840x2400 frames while in motion.
   • A per-frame GPIO trigger fires exactly in sync with each frame's exposure, activating external photometric lighting.

2. Processing Stage (Offline, ~5-10 sec):

   • The captured burst of ~14 frames is passed to Python for analysis.
   • Task A (Reference Frame): A subset of frames (e.g., 4-6) are fed into a Multi-Frame Super-Resolution (MFSR) algorithm. The slight (sub-pixel) motion between frames provides the necessary data to reconstruct a single, high-detail reference image (e.g., ~7680x4800).
   • Task B (Photometry): The full set of synchronized frames is fed into a neural network, which analyzes the precise light reflections (photometry) to detect surface inequalities and defects.

API Reference

Core Methods (v0.1)

• set_cam(id) — select camera by index.

• get_cam() — get current camera ID.

• be_ready() — prepare camera (state transition: 0 → 1).

• stop() — stop and release camera (state transition: * → 0).

• get_state() — get current state:

  • 0 — off
  • 1 — ready
  • 2 — streaming
  • 4 — error

Streaming Methods (v0.2+)

• go() — start YUV frame capture (state: 1 → 2).
• pause() — pause streaming, keep camera configured (state: 2 → 1).
• get_burst_frames() -> list[np.ndarray] — return all buffered frames as a list of flat NumPy arrays (includes hardware padding/stride).
• get_frame_properties() -> tuple — (UPDATED in v0.6) returns metadata required to reconstruction the image:
  • Returns: (width, height, format, stride)
  • stride (int): The actual number of bytes per row in memory (including hardware padding). Critical for reshaping.
• get_last_series_fps() -> float — (NEW in v0.7) Returns the precise average FPS of the last capture burst, calculated using C++ hardware timestamps (excludes warmup frames).

Hot Parameters (v0.3)

Implementation: These setters store values which are applied during be_ready() and go().

• set_iso(iso: int) — Sets target ISO (e.g., 4000). C++ calculates AnalogueGain.
• set_exposure(us: int) — Sets shutter time in microseconds (e.g., 100).
• set_resolution(w: int, h: int) — Sets target resolution (e.g., 3840, 2400).
• set_focus(value: float) — Sets manual focus value (e.g., 0.0 for infinity).

GPIO Trigger (v0.4)

Status: ✅ Complete and tested

Features:

• set_frame_trigger_pin(pin: int): Configures the BCM GPIO pin for output (e.g., pin 21 on gpiochip4 for RPi 5).
• enable_frame_trigger(enabled: bool): Enables/disables the trigger logic.
• Trigger Logic: Generates a precise HIGH/LOW pulse for each frame.
  • GPIO LOW: Set at the start of handle_request_complete (end of frame N exposure).
  • GPIO HIGH: Set at the end of handle_request_complete (immediately before queuing frame N+1).
• Implementation: Uses the stable C-API (gpiod.h) for libgpiod to ensure compatibility and avoid "undefined symbol" linking errors.

SpiderCamera/
├── include/
│   └── spider_camera.hpp        # C++ API interface
├── src/
│   ├── spider_camera.cpp        # Core implementation
│   ├── pisp_decompress.cpp      # (DEPRECATED: Old v0.2 RAW logic)
│   └── frame_buffer.cpp         # (DEPRECATED: Old v0.2 RAW logic)
├── bindings/
│   └── pybind_spider.cpp        # Python bindings (pybind11)
├── examples/
│   ├── demo_python.py           # CLI Burst Capture (Headless)
│   ├── demo_python_preview.py   # Headless Preview (saves to temp/spider_preview.jpg)
│   ├── spider_preview_gui.py    # GUI Control (PyQt5) with real-time preview
│   └── demo_config.json         # Shared configuration file
├── build.sh                     # Build script
└── Specification.md             # Technical specification

Demo & Testing Tools

The library includes three Python scripts to verify functionality and performance:

1. demo_python.py (Burst Benchmark)

   • Use case: Performance testing and headless capture.
   • Action: Loads config, captures a high-speed burst (e.g., 1.5s), calculates actual FPS, and saves frames to temp/.
   • Run: python3 examples/demo_python.py

2. demo_python_preview.py (Headless Preview)

   • Use case: Monitoring the feed without a display attached.
   • Action: Continuously fetches frames and updates a single JPEG file (temp/spider_preview.jpg). Useful for checking focus/framing via a file browser or web server.
   • Run: python3 examples/demo_python_preview.py

3. spider_preview_gui.py (GUI Control Panel)

   • Use case: Interactive configuration and real-time feedback.
   • Features:
     • Live viewfinder (~2 FPS visualization).
     • Hot-swappable parameters: Sliders for ISO, Exposure, and Focus.
     • Resolution switcher (triggers soft-restart).
     • Saves current settings to demo_config.json.
   • Run: python3 examples/spider_preview_gui.py
   • Requires: python3-pyqt5

Git History

v0.1 — Basic Initialization (2025-10-31)

Status: ✅ Complete and tested

Features:

• Camera manager initialization.
• Camera selection via set_cam().
• Basic RAW configuration and state management.

v0.2 — Burst Frame Streaming (2025-11-10)

Status: ✅ Complete and tested

Features:

• go() / pause() methods for streaming control.
• New architecture: burst capture mode.
• get_burst_frames() — C++-side decompression and delivery of all buffered frames to Python as a list of NumPy arrays.
• PISP_COMP1 8-bit compressed RAW decompression via LUT.
• Increased buffer count (-bufferCount 8) to prevent pipeline stalls.
• Performance: stable ~13–14 fps at 4608×2592 (12 MP) resolution.
• Hard-coded capture settings in C++ loop (14 fps, ISO 4000, 100 µs exposure).

v0.3 — Hot Parameters (2025-11-11)

Status: ✅ Complete and tested

Features:

• Implemented Python setters: set_iso, set_exposure, set_focus, set_resolution.
• Parameters are now passed from Python (e.g., read from a JSON config) to C++.
• Refactored go() to use these variables instead of hard-coded values.
• Corrected gain logic: now uses AeEnable=false with full AnalogueGain (e.g., 40.0) and DigitalGain=1.0, successfully "exorcising" the auto-exposure "poltergeist".
• Confirmed stable manual exposure at 100us (verified with/without external light).

v0.4 — GPIO Trigger (2025-11-15)

Status: ✅ Complete and tested

Features:

• set_frame_trigger_pin() and enable_frame_trigger() implementation.
• Hardware sync: GPIO HIGH on frame start, LOW on frame end.
• Switched to C-API (libgpiod) for stability on RPi 5.

v0.5 — GUI & Stability (2025-11-20)

Status: ✅ Complete and tested

Features:

• PyQt5 GUI: Created spider_preview_gui.py for real-time parameter tuning.
• Stability Fix: Fixed race conditions in stop() method to prevent library crashes during rapid parameter changes.
• Headless Preview: Added demo_python_preview.py for continuous monitoring.
• YUV Pipeline: Fully transitioned to YUV420 for high-speed burst capture (replacing raw decompression).

v0.6 — Stride & Stability Fix (2025-11-21)

Status: ✅ Complete and tested

Features:

• Critical Fix for RPi 5: Solved "green screen" artifacts, diagonal shearing, and Segmentation Faults caused by hardware ISP memory padding.
• Raw Memory Copy: C++ core now performs a fast memcpy of the entire buffer (including padding) instead of attempting to strip it row-by-row. This significantly improves stability.
• API Update: get_frame_properties() now returns a 4-tuple: (width, height, format, stride).
• Python Handling: All example scripts (demo_python.py, demo_python_preview.py, spider_preview_gui.py) updated to implement Stride-aware reshaping and cropping.
• Cleaned Codebase: Removed deprecated RAW10 unpacking logic and unused legacy code.

v0.7 — Hardware FPS & Precision (2025-12-11)

Status: ✅ Complete and tested

Features:

• Physical FPS Calculation: Implemented logic in C++ to measure frame rates using hardware monotonic clocks (std::chrono::steady_clock).
• Precision: Calculates FPS based on the exact interval between the first valid frame (after warmup) and the last frame, eliminating Python GIL latency and context-switching jitter.
• **New API