🧾 LayoutLMv3 Fine-Tuning for Invoice Extraction

Welcome to the LayoutLMv3 Fine-Tuning project! 🚀 This project focuses on extracting structured data from invoices and PDFs using LayoutLMv3, PaddleOCR, and Label Studio. The system extracts key fields like invoice number, date, vendor GSTIN, PAN, product description, rate, quantity, and amount.

📄 Overview

This repository contains code to fine-tune LayoutLMv3 for extracting fields from invoice PDFs. It supports various formats, including scanned documents, by leveraging PaddleOCR for text extraction and Label Studio for annotation.

🔧 Key Features

• Fine-Tunes LayoutLMv3 to extract structured data from PDFs.
• OCR Integration using PaddleOCR for text extraction from scanned PDFs.
• Annotation with Label Studio to create labeled datasets.
• Handles Multiple Formats of invoices with different label placements.
• Extracts Multiple Fields:
  • 📅 Invoice Number and Date
  • 🏢 Vendor GSTIN and PAN
  • 📦 Product Description
  • 💲 Rate, Quantity, and Amount
  • 📜 HSN Codes
  • 📬 Client Address

🛠️ Installation

1. Clone the repository:

 git clone https://github.com/YashSonar/LayoutLMv3-Invoice-Extraction.git

2. Install dependencies:

 pip install -r requirements.txt

3. Download pre-trained LayoutLMv3 weights:

 mkdir input && cd input
 wget <layoutlmv3-pretrained-model-url>

4. Install PaddleOCR for text extraction:

 pip install paddleocr

5. Install Label Studio for annotation:

 pip install label-studio

🚀 Usage

1. Annotate PDFs with Label Studio to generate JSON datasets.
2. Fine-tune LayoutLMv3 using the provided main.py.
3. Extract text from scanned PDFs using PaddleOCR.
4. Run the model to extract fields from invoices:

 python main.py

📈 Model Training

• The model is fine-tuned on annotated data created using Label Studio.