MMRAG — Multimodal Retrieval-Augmented Generator Operations Console

LU // Architectural Engine Scope Qdrant Store Engine

Why MMRAG Outperforms Standard RAG: Standard RAG handles single-modality textual processing exclusively, dropping layout architecture metadata completely. MMRAG matches visual page geometry alongside text content targets under a single Qdrant Collection identity via separate named vector tracks. This ensures contextual extraction does not fracture across multi-column tables or embedded engineering manual diagrams.

📝 TEXT COGNITION LAYER

• Model Designation: BAAI/bge-base-en-v1.5

• Dimensional Axis: 768 Vectors | Metric: Cosine Similarity

🖼️ VISUAL GEOMETRY CANVAS TRACK

• Model Designation: CLIP ViT-B/32

• Dimensional Axis: 512 Vectors | Metric: Cosine Similarity

Dual-Vector Configuration Interface: Co-indexed parameters are mapped using named embeddings configurations, bypassing collection duplication:

vectors_config={
    "text": VectorParams(size=768, distance=Distance.COSINE),
    "image": VectorParams(size=512, distance=Distance.COSINE)
}

RU // Continuous Delta Ingestion Track auto_ingest.py

🛡️ MD5 Incremental Protection Layer (registry.json)
Every source document gets an isolated MD5 file hash computed before text layout extraction. If the target file matches an identical hash signature inside the tracking ledger, the processing thread triggers an instant [SKIP] command. Existing records in Qdrant stay completely untouched—avoiding vector point duplication and unnecessary embedding recomputes.

Concurrency Design Pattern: Because Qdrant manages data transactions point-by-point via unique UUID fields generated out of payload markers, **Ingestion and Query routines can run simultaneously hand-in-hand without locking issues**:

Isolated Points: Point IDs utilize tracking hashes: hashlib.md5((text + str(page_no) + source_file).encode()).hexdigest().
Lock-Free Store: Query and upsert actions execute concurrently across separate system workers.

# Core Delta Filtering Blueprint (auto_ingest.py)
current_hash = file_hash(pdf_path)
if pdf_name in registry and registry[pdf_name] == current_hash:
    print(f"[SKIP] {pdf_name} - Vector Index Fresh")
    continue

LB // Dynamic Processing Flow & State Machine Concurrent Runtime Matrix

RB // Multi-Modal Client Search Desk app.py Interface

💡 Semantic

🔤 Keyword

📖 Metadata

🖼️ Reverse Image

🎙️ Voice/Audio Track

Cosine Threshold: 0.70

Query Concept String

Filters results dynamically based on threshold.

Streamlit Render Outputs

MMRAG Workstation Console