Tag: artificial-intelligence

Why not use ChatGPT or Gemini Directly All the LLM (Large Language Models) are pre-trained on publicly available information. Then why don’t i use for my needs: LLMS donot have access to private data: My organization has technical manuals, I want to create a chatgpt/gemini kind of solution around user manuals. LLMs can hallucinate LLMs… Continue reading Gen-AI Developer Classroom notes 06/May/2025

Langchain: Memory We have following classes ConversationBufferMemory ConversationBufferWindowMemory ConversationTokenBufferMemory ConversationSummaryMemory Refer Here for the notebook Langchain Primitives The most basic langchain abstract interface is langchain_core.runnables.base.Runnable‘ Refer Here for notebook

Chains in Langchain Refer Here for chains api reference Lets explore chains First chain: Lets write a prompt pass the prompt to llm Second chain: Lets write a prompt pass the prompt to llm convert the output to some format Third chain Use Vector database as retriever write a prompt with context Pass the prompt… Continue reading Gen-AI Developer Classroom notes 26/Apr/2025

Link to the notebook Refer Here Refer Here for retriever Refer Here for the implementation of text book rag with text Exercise Try building a retriever for a Directai.blog with atleast 10 to 15 pages as source

Document Loaders Document Loaders in langchain Document loaders can read documents and give Document Object Every Document loader is derived from BaseLoader Look into jupyter notebook for text markdown word Many PDF loaders are supported All loaders Chunking Documents using Text Splitters Refer Here for Text Splitters All splitters can split documents based on chunk… Continue reading Gen-AI Developer Classroom notes 22/Apr/2025

Google AI Studio Lets use google AI Studio to get api keys and use it with langchain GCP Account Creation Lets create a free trail gcp account Set up your account and enabling Compute Engine API Vertex API Install gcloud sdk cli

LangChain Chat Models Refer Here for official docs Runnable: This is kind of what almost everything in langchain is derived from Chat Models offer similar interface for different models from differnt providers Providers and Model Names Refer Here for samples

References Text Models: Restrictive functionality and not all the model are part of text generation models and Refer Here for all possible Text Completion models Prompt Template Refer Here for official docs So lets use a string prompt template Refer Here Refer Here fo the sample do

Langchain components LangChain is a powerful framework for building applications that use Large Language Models (LLMs). It modularizes LLM workflows into reusable and extensible components, allowing developers to create complex applications like chatbots, agents, and RAG pipelines. 1. Models (LLMs, ChatModels) Purpose: To interact with language models like OpenAI’s GPT, Anthropic, HuggingFace, etc. Types: LLM:… Continue reading Gen-AI Developer Classroom notes 15/Apr/2025

Version Control Systems A version Control System enables us to store code (Repository) Which enables multi user Which enables multiple versions Git is most popular version control system create by Linus Torvalds Git is a distributed version control system Initial configuration of git hint: Use any git cheatsheet Git commit requires username and email address… Continue reading Gen-AI Developer Classroom notes 10/Apr/2025

Models in Hugging face Explore SpaceHub Models: Refer Here Embeddings from hugging face models Try writing a simple code using hugging face transformers for question answering from transformers import pipeline model_id = ” pipeline(‘<nlp-question>’, model=model_id, …) lets try question answering and image classification Refer Here for the Colab Notebook

NLP (Natural Language Processing) Basic Tasks: Sentiment Analysis Summarizations Question Answering NER Hugging Face Account Creation Hugging face transformers are based on deep learning and to run deep learning models, we need to install torch Tensorflow Running Python code Locally Google Colab Basic coding approach Create a pipeline proceed with some nlp task Simple Setup… Continue reading Gen-AI Developer Classroom notes 08/Apr/2025

Problem We have a text book, primary with text and also some images. We need to parse the text from pdf file and also images Relation between chunk and token A document is broken down into chunks. Chunk will have multiple tokens Note: If using a llm model ensure chunk does not have more tokens… Continue reading Gen-AI Developer Classroom notes 07/Apr/2025

Lets Build an Agent for Schools (For the future sessions) Basic Idea: Make AI agent prepare question papers Also make it analyze the results and guide student with improvement areas and references to work on Lets Build a RAG for Students Basic Idea: RAG which is based on standard text books of boards A student… Continue reading Gen-AI Developer Classroom notes 04/Apr/2025

Vector Databases Vector Databases store vectors in higher dimensions The text/images are converted into chunks and then into vectors by embeddings. Vectors are stored in the vector databases which can now help in similarity search. Usecases of Vector Databases Product search by uploading image Recommendation engines Facial recognitions How do Vector Databases search for similar… Continue reading Gen-AI Developer Classroom notes 03/Apr/2025

Demonstration of a Very simple RAG Watch classroom recording for sbi home insurance policy alibaba 40 theives harry potter Installing ollama We will be using ollama to run smaller models on our laptops Refer Here for downloading ollama Now lets run a simple model on ollama

RAG (Retrieval Augemented Generation) This concept was termed by facebook. Basic LLM (Prompt/Chat interface) Most of the LLMs are trained on publicly available information (Website, Books, articles …..) I want LLM to generate responses on my organization data Building a LLM: Costly RAG’s kind of solve this problem RAG Conceptually has two layer Retriever Generator:… Continue reading Gen-AI Developer Classroom notes 29/Mar/2025

Prompting Techniques In-context learning Most LLMs maintain context that is information about the generated content we can refine the subsequent prompts with enhancements In the classroom, we generated python code to insert products into databases and then asked it to implement softdelete We have done similar stuff for reworking on French learning plan. Anthropic Anthropic… Continue reading Gen-AI Developer Classroom notes 27/Mar/2025

AI Artificial General Intellingence is field where we have systems which behave like humans AI (Narrow AI) is field where we have system which thinks like human in a specific domain Narrow AI became popular with a subfield of AI called as Machine Learning. ML can predict or classify or clustering all based of mathematical/statistical… Continue reading Gen-AI Developer Classroom notes 25/Mar/2025