Multimodal RAG Platform  ·  View 05 of 16

Multimodal Ingestion Pipeline

One lane per modality across five stages. The lanes run in parallel and converge on a single chunk record.

Editable source SVG draw.io All views
Intake
Intake
Extract
Extract
Interpret
Interpret
Chunk & enrich
Chunk & enrich
Index
Index
Text & Markdown
Text & Markdown
Upload / crawl
Upload / crawl
Normalise encoding
Normalise encoding
Heading structure
Heading structure
Semantic chunks
512 tok, 15% lap
Semantic chunks...
Dense + sparse
Dense + sparse
PDF & Office
PDF & Office
Upload / connector
Upload / connector
Layout parse
reading order
Layout parse...
OCR fallback
scanned pages
OCR fallback...
Page-anchored chunks
Page-anchored chunks
Dense + sparse
Dense + sparse
Images & charts
Images & charts
Standalone or embedded
Standalone or embedded
Region crop
bbox retained
Region crop...
VLM description
Claude vision
VLM description...
Caption + OCR text
Caption + OCR text
Text + image vector
Text + image vector
Tables & sheets
Tables & sheets
Upload / extracted
Upload / extracted
Table detect
cell structure
Table detect...
Header + type infer
Header + type infer
Markdown linearise
Markdown linearise
Vector + typed rows
Vector + typed rows
Audio & video
Audio & video
Upload / archive pull
Upload / archive pull
Demux + keyframes
Demux + keyframes
ASR + diarisation
word timings
ASR + diarisation...
Timecoded windows
Timecoded windows
Vector + frame refs
Vector + frame refs
Multimodal Ingestion Pipeline — One Lane per Modality
Multimodal Ingestion Pipeline — One Lane per Modality
Every lane emits the same chunk record: text projection, provenance, modality tag.
Every lane emits the same chunk record: text projection, provenance, modality tag.
v 1.0 · owner Solution Architecture
v 1.0 · owner Solution ArchitectureText is not SVG - cannot display

Per-modality treatment

  • PDF and Office — layout parse with OCR fallback for scans
  • Images and charts — VLM description plus OCR, bounding box kept
  • Tables — cell-structure detection, then both prose and typed rows
  • Audio and video — transcription with word-level timings

The convergence

  • Every lane emits the same record shape
  • Text projection, provenance pointer, modality tag
  • That uniformity is what makes cross-modal retrieval possible

Provenance

  • Page and character offsets for documents
  • Bounding box for image regions
  • Timecode for audio and video
  • Provenance is what turns a retrieval into a citation
Solution Architecture v1.0 · Enterprise Architecture · 2026-0805-multimodal-ingestion-pipeline