AI Winter School for Ag — Image Analysis Workshops

This repository contains the hands-on coding workshops for the AI-in-Ag Winter School, focused on practical computer vision techniques for agricultural plot imagery — from exploratory analysis through to deep learning-based vegetation segmentation.

The workshops form a single narrative arc: each one builds on the output of the last.

#	Notebook	Topic	Output used by next workshop
1	01_image_diversity_analysis	Image auditing, PCA & UMAP, diverse image sampling	A small, visually diverse image subset
2	02_colour_spaces_and_thresholding	Colour spaces (RGB/HSV/ExG) & threshold-based segmentation	Tuned intuition for what "vegetation" looks like in pixel space
3	03_VegAnn	Deep learning segmentation (VegAnn U-Net + SegVeg XGBoost)	Full vegetation health pipeline

Each notebook comes in two versions:

• _PARTICIPANT.ipynb — what you work through during the session. Core logic is left as # YOUR CODE HERE exercises; plotting and evaluation code is pre-written so you can focus on the analysis.
• _SOLUTION.ipynb — completed reference implementation. Use it to check your work or get unstuck, not as a starting point.

---

Workshop 1 — Image Inspection, PCA & UMAP for Plot Image Variability

File: 01_image_diversity_analysis_PARTICIPANT.ipynb

You'll audit a heterogeneous set of plot images, build an efficient batch loading pipeline, and use PCA to understand visual variability across the dataset. You'll then implement Farthest Point Sampling (FPS) on the PCA embedding to select a small, maximally diverse subset of images — far better than naively picking the "most extreme" images, which tend to be near-duplicates. Finally, you'll compare PCA (linear) against UMAP (non-linear) as alternative ways to view the same image space.

You will learn to:

• Audit image resolution/aspect ratio at scale using PIL's lazy loading
• Build batch image-loading pipelines with pathlib and pre-allocated numpy arrays
• Apply PCA to flattened pixel data and interpret a scree plot
• Implement Farthest Point Sampling for diverse image selection
• Compare PCA and UMAP projections of the same data

Dataset expected: a folder named plot_images/ in the same directory as the notebook.

Output: the diverse image subset you sample here should be saved to segmentation_sample_set/ — this feeds directly into Workshop 2.

---

Workshop 2 — Colour Spaces & Threshold-Based Vegetation Segmentation

File: 02_colour_spaces_and_thresholding_PARTICIPANT.ipynb

Using the diverse subset from Workshop 1, you'll explore how the same image looks in RGB, HSV, and Excess Green (ExG) colour spaces, and implement three classic threshold-based segmentation methods. You'll tune manual RGB/HSV bounds by hand, compare them against automatic Otsu thresholding on ExG, and quantify where the methods agree and disagree across the dataset. An optional section revisits PCA from Workshop 1 in HSV space, and explains why Hue's circular nature breaks the linear assumptions PCA relies on.

You will learn to:

• Convert between RGB, HSV, and ExG and reason about why each helps (or hurts) vegetation segmentation
• Implement Otsu auto-thresholding and manual inRange thresholding
• Visualise and diagnose segmentation quality (false positives/negatives, boundary disagreement)
• Compute per-pixel multi-method agreement maps across a dataset
• Understand a key limitation of Euclidean PCA on circular features (Hue)

Dataset expected: a folder named segmentation_sample_set/ (the output of Workshop 1) in the same directory as the notebook.

---

Workshop 3 — Deep Learning Segmentation: VegAnn & SegVeg

File: 03_VegAnn_PARTICIPANT.ipynb

Threshold rules break down under lighting changes, shadows, senescence, and mixed-pixel canopy edges — this workshop replaces them with pre-trained deep learning models. You'll run a cascaded three-stage pipeline: a U-Net (VegAnn) for binary vegetation/background segmentation, followed by two XGBoost pixel classifiers (SegVeg) that further split vegetation into health categories. Because plot images are larger than the models' training tile size, you'll implement mirror-padded, overlapping patch inference with probability averaging to avoid boundary artefacts.

Pipeline:

Image → Stage 1: VegAnn (U-Net)      → vegetation vs background
      → Stage 2: SegVeg-XGBoost      → Soil | Healthy | Chlorotic
      → Stage 3: SegVeg-Necrosis     → Soil | Healthy | Necrosis | Chlorosis

You will learn to:

• Load pre-trained PyTorch and XGBoost checkpoints for inference (model.eval(), torch.no_grad())
• Implement mirror padding and overlapping-patch inference with probability averaging
• Apply ImageNet preprocessing required by a pre-trained encoder
• Build a mask-gated cascaded pipeline (only classify pixels already flagged as vegetation)
• Compute and visualise per-class coverage statistics across an image set

Dataset expected: a folder named SegVeg_samples_plot_clips/, plus model checkpoints under SegVeg_data/ (VegAnn.ckpt, XGBoost, Necrosis.pkl). Reference: Serouart et al. (2022), Data in Brief / Plant Phenomics — see in-notebook citations.

This notebook also depends on the local module VegAnn.py (in this repository) — keep it in the same directory as the notebook.

---

Setup

Install dependencies from requirements.txt:

pip install -r requirements.txt

Workshop 1 additionally needs scikit-learn and umap-learn; Workshop 2 needs opencv-python (already listed). Each notebook's first cell also lists its specific pip install line — run that first if you hit an import error.

How to work through each notebook

1. Read the markdown context above each exercise — it explains why, not just what.
2. Fill in every # YOUR CODE HERE block. Don't change function signatures or return structures — later cells and plotting code depend on them.
3. Test on a single image/small example first (each notebook gives you one), then run on the full dataset.
4. Use the ✅ Checklist at the end of each notebook to confirm you've understood the key takeaways before moving to the next workshop.
5. If you get stuck, check the corresponding _SOLUTION.ipynb — but try the exercise yourself first.

Suggested order

Work through the notebooks in numeric order (1 → 2 → 3); each one's output dataset is the next one's input.

References

The code and model checkpoints are based on the SegVeg repo: https://github.com/mserouar/SegVeg

Madec, S., Irfan, K., Velumani, K. et al. VegAnn, Vegetation Annotation of multi-crop RGB images acquired under diverse conditions for segmentation. Sci Data 10, 302 (2023). https://doi.org/10.1038/s41597-023-02098-y

Serouart Mario Madec Simon David Etienne Velumani Kaaviya Lopez Lozano Raul Weiss Marie Baret Frédéric. SegVeg: Segmenting RGB Images into Green and Senescent Vegetation by Combining Deep and Shallow Methods. Plant Phenomics (2022). https://doi.org/10.34133/2022/9803570

Jonas Anderegg, Radek Zenkl, Achim Walter, Andreas Hund, Bruce A. McDonald. Combining High-Resolution Imaging, Deep Learning, and Dynamic Modeling to Separate Disease and Senescence in Wheat Canopies. Plant Phenomics (2023). https://doi.org/10.34133/plantphenomics.0053