Gen-AI Developer Classroom notes 14/Oct/2025

AI Engineering

Statistical language model: They use tokens and they have access to vocabulary. This is a language model which can do majorly two types of things
- Fill in the blanks: Masked language model
  Kids were ___ on the ground
- predict next word: Autoregressive language model
  kids were screaming loud in the classroom and then arrived the _____
These language models were trained on very large volumes of data which require a very large infra.
This helps the models to generate text

Hello Good morning in french

This is what we call as Large Language Model (LLM)
Creating a LLM is not possible for every org, Few Organizations started the concept of model as service.
LLMs started supporting multi modality (Text, Images, Audio, Video) and also good at general purpose activities (Foundational model)
For enterprises barrier for developing AI enabled applications is very less. This led to AI Engineering.
Problems:
- Fine Tuning: Brand Themes, Solving specific problems
- AI Agents: Perform tasks/actions
- RAG (chatbots, knowledge base AI)

I am trying to understand Artificial Intelligence and came across word called as entropy, Can you explain in simple terms