🤖 For AI Agents

IMPORTANT: Before implementing any changes, consult Status.md

• What features are already implemented
• Current implementation status of each component
• Known issues and limitations

Instructions for AI assistants:

1. Before starting work: Check Status.md to verify feature status
2. During implementation: Update status from ❌ to ⏳ (in progress)
3. After completion: Update status to ✅ and document any new features or changes
4. Always maintain: Keep Status.md synchronized with actual implementation state

📋 Overview

A low-cost hobby project that enables semantic Q&A over PDF documents. Upload PDFs, generate embeddings locally, and ask questions in natural language.

Key Features:

• Semantic search using vector embeddings (RAG over PDF documents)
• Function calling over structured fund data (performance, ownership, categories)
• Hybrid answers — LLM autonomously combines both approaches
• Natural language Q&A with source citations
• Local embedding generation (LM Studio/Ollama)
• Switchable vector storage: in-memory (default) or Azure Cosmos DB

🏗️ Architecture

flowchart LR
    subgraph PdfTextExtractor ["PdfTextExtractor (extraction)"]
        PDF[PDF Files] --> PTE[PdfTextExtractor]
        PTE -->|native| PDFPIG[PdfPig]
        PTE -.->|OCR| LMS[LM Studio / OpenAI]
        PTE --> TXT[Text Files]
    end

    subgraph Preprocessor ["Preprocessor (embedding)"]
        TXT --> PRE[Preprocessor]
        PRE --> EMB[embeddings.json]
        PRE -.->|local| LOCAL[Ollama / LM Studio]
        PRE -.->|cloud| OAI_EMB[OpenAI Embeddings]
    end

    subgraph YieldRaccoon ["YieldRaccoon (data collection)"]
        YR[YieldRaccoon] -->|WebView2| FP[Fund Provider]
        YR --> DB[(SQLite\nFundProfile, FundHistoryRecord,\nCountry, Sector,\nFundCountryAllocation, FundSectorAllocation)]
        YR -.->|DualWrite| API
    end

    subgraph Runtime ["Backend + Frontend (retrieval & generation)"]
        FE[Frontend] --> API[Backend API]
        EMB -->|default| API
        COSMOS[(Cosmos DB)] -.->|optional| API
        AZSQL[(Azure SQL)] -.->|optional| API
        API -->|RAG + Function Calling| OAI[OpenAI API]
        API -.->|LLM alt| GROQ[Groq API]
    end

Components

🚀 Quick Start

Prerequisites

Configuration: See Configuration & Secrets Guide for complete setup instructions.

1. Extract Text from PDFs

cd PdfTextExtractor
dotnet run --project PdfTextExtractor.Wpf
# Use the WPF GUI to extract text files from your PDFs

See PdfTextExtractor README for CLI and API usage.

2. Generate Embeddings

cd Preprocessor/Preprocessor
# Ensure text files from step 1 are in ./pdfs folder alongside the PDFs
dotnet run

3. Start Backend

cd backend/Backend.API
dotnet user-secrets set "BackendOptions:OpenAIApiKey" "your-key"
dotnet user-secrets set "BackendOptions:LlmProvider" "OpenAI"
cp ../../Preprocessor/Preprocessor/bin/Debug/net9.0/output.json Data/embeddings.json
dotnet run

Optional: Add Azure SQL for structured fund queries (AzureSqlConnectionString), or use Groq as a free LLM alternative. See Backend README.

4. Start Frontend

cd frontend
npm install && cp .env.example .env.local
npm run dev

Open http://localhost:3000

📁 Project Structure

.
├── Preprocessor/           # Embedding generation from pre-extracted text
├── backend/                # ASP.NET Core API
│   ├── Backend.API/        # Main API project
│   └── Backend.Tests/      # Unit tests
├── frontend/               # Next.js web app
├── PdfTextExtractor/       # PDF extraction library (native + OCR)
│   ├── PdfTextExtractor.Core/       # Core library (DDD, extractors, events)
│   ├── PdfTextExtractor.Wpf/        # WPF desktop application
│   └── PdfTextExtractor.Core.Tests/ # NUnit tests
├── YieldRaccoon/           # Fund price crawler (WPF desktop)
│   ├── YieldRaccoon.Domain/         # Business logic, entities, value objects
│   ├── YieldRaccoon.Application/    # Use-case orchestration, interfaces
│   ├── YieldRaccoon.Infrastructure/ # EF Core, event stores, services
│   └── YieldRaccoon.Wpf/           # WPF UI with WebView2
├── docs/                   # Guides
│   ├── AZURE-DEPLOYMENT.md
│   └── SECRETS-MANAGEMENT.md
└── azure-setup.sh          # Azure deployment script

☁️ Deployment

Deploy to Azure with near-zero cost (~$0.03/month):

Quick Deploy

# 1. Create Azure resources (backend + frontend)
./azure-setup.sh

# 2. Add secrets to GitHub (from script output)
#    - AZURE_WEBAPP_PUBLISH_PROFILE
#    - AZURE_STATIC_WEB_APPS_API_TOKEN

# 3. Add variable to GitHub
#    - NEXT_PUBLIC_API_URL

# 4. Push to main - triggers automatic deployment
git push origin main

GitHub Workflows

See Azure Deployment Guide for complete documentation.

📚 Documentation

🛠️ Tech Stack

📄 License

This is a hobby project for learning Semantic Kernel and AI integration.