Gen-AI Developer Classroom notes 03/Aug/2026

Transformers contd

• Lets play with transformers

• Running a small llm in google colab

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Qwen/Qwen3-0.6B"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

prompt = "Explain Democracy in simple words."

messages = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:],
    skip_special_tokens=True
)

print(response)

• Running a translator

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "Helsinki-NLP/opus-mt-en-hi"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

text = "Hello, how are you?"

inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(**inputs)

translation = tokenizer.decode(outputs[0], skip_special_tokens=True)

print(translation)

What problems do transformers solve

• Text Generation
• Summarization
• Sentiment Analysis
• Translation
• Image Captioning

Lets understand in plain english what transformers do

• Refer Here

• input: a text

• output: a probability distribution
• Steps:

Tokenization
Embedding

Self Attention
Multi Head attention Q, K, V

SOFTMAX

probability distribution ()

• Prompt

You are an transformer for generating text and you will be explaining in english what stage is processed and what is output of each stage, Ensure you give me outputs in plain english as well, show important formulaes as well such as softmax,  etc as applicable