RAG – Dealing with Real world Data
- When we build RAG’s for enterprise, We might endup with different forms of data
- Pure Text Data
- Multi Modal Knowledge
- Structure Data (Databases, Tables, CSV)
- Semi Structure Data
- Source Code
- Images & Visual only data
- Audio and Video
- Web & Dynamic Content
- Multi-source Enterprises
Multi Modal Knowledge
-
We have a text where the knowledge is in two places
- Text Knowledge:
- Paragraphs, definitions, proofs, steps
- Visual Knowledge
- diagrams, charts, figures, flowgraphs
- Text Knowledge:
-
Lets try building a RAG for a ncert class X science book
NCERT – RAG
- Download the text book from Refer Here
- When we loaded text from ncert not all sections are getting captured
-
So we need to find a way to
- parse the document correctly
- Deal with images
- Stores images as metadata to document to return images
- Capture the image caption and convert that into embedding to store in vector along with image metadata (OCR)
- Caption images using Vision model and embedd the meaning into vector with image metadata
-
Lets try parsing PDF Files:
- Find out nuances of parsing a PDF File
- Parse the pdf by extracting images as well
- Try multiple libraries
-
Refer Here for the notebook and we will be using this to build a RAG based on text book.
