Document Verification System

Introduction

Goals

• Automated Document Processing: Utilize computer vision and OCR to automatically classify and extract data from uploaded government documents.
• Fraud Detection: Enable companies to verify the authenticity of documents by comparing them with records in a government database.
• Ease of Use: Provide a user-friendly interface for document submission and verification.
• Accuracy: Enhance the system's accuracy in detecting fraud through improved OCR and document classification.
• Integration: Seamlessly integrate with existing systems and databases used by organizations for document verification.

Contributors

• Rahul Dev Banjara

Project Architecture

1. Frontend: User interface for document upload and verification results.
2. Backend: Server-side processing, including OCR, document classification, and interaction with the government database.
3. Document Processing Engine: Storage for processed documents and verification records.

Status

Known Issues

• OCR Accuracy on Low-Quality Scans: The OCR models may have difficulty with documents that are poorly scanned or distorted.
• Document Classification Confusion: There may be issues with misclassifying documents that have similar formats but different types.
• Scalability Limitations: The system's performance may degrade with high volumes of simultaneous uploads or very high-resolution images.

High-Level Next Steps

1. Multi-Language Support: Implement support for multiple languages in the OCR and text extraction processes to accommodate diverse user needs and enhance usability across different regions.
2. Custom OCR Fine-Tuning: Fine-tune the OCR model with custom datasets for improved accuracy on specific document types.
3. Improve Data Quality: Increase the quality and quantity of training data by incorporating a larger set of original and high-quality documents. This will enhance the accuracy and robustness of OCR and classification models.
4. Develop Production-Grade Software: Refactor the system to meet production-grade standards, including improving code quality, documentation, testing, and reliability. Implement best practices for software development to ensure robustness and maintainability.
5. Integrate Relational Databases: Replace the current dummy JSON database with a relational database system (e.g., PostgreSQL, MySQL) to better manage and query large volumes of data. This will improve data integrity, scalability, and performance.

Usage

Installation

Prerequisites

• Python 3.10 is required for this project. We recommend using a virtual environment to ensure compatibility.

Virtual Environment Setup

1. Create a virtual environment:

   python3 -m venv venv
   

2. Activate the virtual environment:

   • On Linux/macOS:

     source venv/bin/activate
     

   • On Windows:

     venv\Scripts\activate
     

Download Models

Before proceeding with installation, download the required models from the provided link. You will need the following models:

• Classification Model
• License Model
• Passport Model
• Citizenship Model

Setting Up

1. Clone the repository:

   git clone https://github.com/devrahulbanjara/Document-and-Record-Management
   

2. Create a directory for the models:

   mkdir "trained models"
   

3. Place the downloaded model files into the trained_models directory.

4. Navigate to the project directory:

   cd Document-and-Record-Management
   

5. Install dependencies:

   pip install -r requirements.txt
   

6. Run the application:

   streamlit run src/app.py
   

Data Source

• Synthetic Datasets: Text manually placed on template images to simulate real documents.
• Lost and Found Groups: Documents sourced from Facebook, Instagram, X, and similar platforms.
• Government Samples: Document samples provided by government sources for validation and comparison.

Code Structure

Frontend

• Path: src/app.py
• Description: Provides the user interface for document upload, classification, and verification using Streamlit. Users can upload documents via webcam or file, which are then classified and verified.

Backend

Classification

• Path: src/classification.py
• Description: Classifies documents into categories such as citizenship, license, passport, and others.

Detection

• Path: src/detection.py
• Description: Detects regions of interest (e.g., name, date of birth) in classified documents using YOLOv8 models.

OCR

• Path: src/ocrtext.py
• Description: Extracts text from detected regions using Optical Character Recognition (OCR).

Filtering

• Paths:
  • src/filter_citizenship.py
  • src/filter_license.py
  • src/filter_passport.py
• Description: Filters the text extracted via OCR to retrieve relevant details like name, citizenship number, etc., for different document types (citizenship, license, passport).

Image Preprocessing

• Path: src/process_image.py
• Description: Performs image preprocessing tasks, including resizing, orientation correction, and binarization, before classification and detection.

Validation

• Paths:
  • src/validate_citizenship.py
  • src/validate_document.py
• Description: Validates the extracted data against records stored in the database to ensure document authenticity.

Plotting

• Path: src/plot_details.py
• Description: Generates visual plots and results for detected regions and classified data.

Database Models

• Path: src/database.py
• Description: Defines operations to interact with the database.json file, which acts as a dummy database for storing and verifying document details.

Notebooks

• Path: notebooks/
• Description: Contains Jupyter notebooks for model experimentation, including:
  • Classification model training and inference (notebooks/Classification/)
  • YOLOv8 model training and inference (notebooks/YOLOv8/)
  • OCR experiments using EasyOCR (notebooks/OCR/)
  • Text filtering experiments after OCR output (notebooks/Text Filtering/)

Tests

• Path: tests/test_template.py
• Description: Unit tests to ensure the correctness of various components in the document verification system.

Results

Classification Model Evaluation

• Model Accuracy: 0.98

YOLOv8 Models Evaluation

Citizenship Model

License Model

Passport Model

Overall System Evaluation

• Overall Accuracy: 82.61%