RAG – Productionization
- To productionize the RAG we need to dive deeper on
- chunking
- vector stores
- RAG scoring (measure to ensure the results are good enough)
- Gaurdrails
- Variety of data
- text
- images
Idea – CBSE Teacher
- Look at the mathematics book Refer Here
- We need to extract text and images from pdf and ensure they are organized by
- chapters
- sections
Extraction of Text and images – Filtering out noise
Proof – of – Concept (POC) – Extraction
- The current code which we have is extracting text and images
- We have some issues
- blank images
- There are some images which are flowcharts or tables which are not captured correctly
-
Exercise: Dig into pdf and find out the type of the flow charts
-
Refer Here for the code
