Introduction to Generative AI — Generative Models, GPT & Transformers, Language Modeling, Ethics, and a Real Text-Generation App (with Code)
A developer's intro to Generative AI: model families, GPT & transformers, language modeling, AI ethics, and a real text-generation app with guardrails.
- Author
- Randhir Jassal
- Published
- Reading time
- 34 min read
- Views
- 7 views
Introduction to Generative AI — Generative Models, GPT & Transformers, Language Modeling, Ethics, and a Real Text-Generation App (with Code)
"Generative AI" is the phrase on every roadmap, but most introductions either drown you in math or hand-wave with "it predicts the next word." Neither helps you actually build something or reason about when it''s the wrong tool.
This guide is a developer''s on-ramp. We''ll cover what makes a model "generative" (vs the classifiers you already know), the model families (autoregressive, diffusion, GANs, VAEs) and when each is used, how GPT and transformers actually work at an intro level, how text generation and language modeling fit together, the ethical considerations you must handle before shipping, and finally a complete, runnable text-generation application — backend, streaming, guardrails, and a UI. Beginner-friendly framing, real code throughout.
TL;DR
- Generative models learn to produce data (text, images, audio), where discriminative models only label it. The difference is "draw me a cat" vs "is this a cat?"
- Four families matter: autoregressive (LLMs/GPT — text), diffusion (Stable Diffusion — images), GANs (older image gen), VAEs (compression/latent spaces). For text, autoregressive transformers dominate.
- GPT = a decoder-only transformer trained to predict the next token. Generation is a loop: predict → sample → append → repeat.
- Language modeling is the objective; text generation is the application. Decoding strategy (temperature, top-p) shapes the output as much as the model.
- Ethics is not optional. Hallucination, bias, copyright, privacy, misuse, and environmental cost are real, shippable risks — handle them at the boundary.
- The project: a FastAPI text-generation service with streaming, PII redaction, a content guardrail, and a Gradio UI — works with either a hosted API or a local open model.
1. What "generative" actually means
You already know discriminative models — they draw boundaries between classes. "Spam or not spam." "Cat or dog." They learn P(label | input).
Generative models learn the distribution of the data itself — P(data) — so they can produce new samples that look like the training data. Instead of "is this a cat?", a generative model can "draw me a cat that has never existed."
Discriminative: input → label (classify, score, detect)
Generative: prompt → new data (write, draw, compose, synthesize)
The practical consequence: generative models don''t have a single "right answer." Given "Once upon a time," there are billions of valid continuations. That''s both their power (creativity) and their danger (they can confidently produce plausible nonsense).
2. Understanding generative models — the four families
Not all generative AI is an LLM. Four families cover the landscape; knowing them helps you pick the right tool.
2.1 Autoregressive models (the LLM family)
Generate data one piece at a time, each conditioned on everything before it.
P(x1, x2, ..., xn) = P(x1) * P(x2|x1) * P(x3|x1,x2) * ... * P(xn|x1..xn-1)
For text, "one piece" = one token. This is how GPT, Llama, Claude, and every modern text model work. Best for: text, code, any sequence.
2.2 Diffusion models
Start with pure noise and iteratively denoise it into a sample. Trained by adding noise to real data and learning to reverse the process.
noise → denoise step 1 → denoise step 2 → ... → a coherent image
This is Stable Diffusion, DALL·E 3, Midjourney, and most 2026 image/video/audio generation. Best for: images, video, audio.
2.3 GANs (Generative Adversarial Networks)
Two networks compete: a generator makes fakes, a discriminator tries to spot them. They improve together until the fakes are convincing.
Generator → fake image → Discriminator → "real or fake?" → both learn
GANs dominated image generation 2015–2021; mostly superseded by diffusion now, but still used for fast, specialized generation (faces, super-resolution). Best for: fast, narrow image tasks.
2.4 VAEs (Variational Autoencoders)
Compress data into a smooth latent space, then decode points in that space back into data. The latent space is continuous, so you can interpolate ("morph one face into another").
data → encoder → latent vector → decoder → reconstructed data
Rarely the final product alone, but VAEs power the latent space inside diffusion models (Stable Diffusion is a "latent diffusion model"). Best for: representation learning, the compression layer of bigger systems.
2.5 The picking guide
| You want to generate… | Use |
|---|---|
| Text, code, structured output | Autoregressive transformer (LLM) |
| Images, video, audio | Diffusion |
| Fast narrow images (faces, upscaling) | GAN |
| A smooth latent space / compression | VAE |
The rest of this guide focuses on text — the autoregressive transformer family — because it''s where most developers start and where you can ship the fastest.
3. Introduction to GPT and Transformer Architectures
3.1 What GPT stands for
Generative Pre-trained Transformer:
- Generative — it produces text, doesn''t just classify.
- Pre-trained — it learned general language from a huge corpus before you ever touched it.
- Transformer — the neural network architecture underneath.
3.2 The transformer in one picture
A transformer processes a sequence of tokens through stacked blocks. Each block has two sub-layers:
tokens → [embed + position] →
┌─────────────────────────────┐
│ Self-Attention │ ← each token looks at every other token
│ + residual + LayerNorm │
├─────────────────────────────┤
│ Feed-Forward Network │ ← per-token transformation
│ + residual + LayerNorm │
└─────────────────────────────┘ × N blocks (GPT-3 = 96, Llama-8B = 32)
→ predict next token
3.3 Self-attention — the key idea, intuitively
For each token, the model computes how much to "pay attention to" every other token, then builds a new representation weighted by that attention. This is how it figures out that in "the trophy didn''t fit in the suitcase because it was too big," it refers to the trophy.
The mechanism, simplified: each token produces a Query ("what am I looking for?"), a Key ("what do I offer?"), and a Value ("what I contribute"). Attention weight = how well a query matches each key; output = weighted sum of values.
import torch
import torch.nn.functional as F
import math
def attention(Q, K, V):
# Q, K, V: (seq_len, d)
scores = Q @ K.T / math.sqrt(Q.size(-1)) # how relevant each token is to each other
weights = F.softmax(scores, dim=-1) # turn scores into a distribution
return weights @ V # weighted blend of values
That single operation — match, weight, blend — repeated across many "heads" and many layers, is the entire reason transformers work.
3.4 Why "decoder-only" for GPT
GPT uses only the decoder half of the original transformer, with a causal mask that prevents each token from seeing the future. This is what makes it a generator: at every position it can only use the past to predict the next token — exactly what you need to write text left-to-right.
4. Text Generation and Language Modeling
4.1 Language modeling is the training objective
A language model is trained on one task: given the previous tokens, predict the next one. Cross-entropy loss measures how surprised the model is by the true next token.
import torch.nn.functional as F
def lm_loss(logits, input_ids):
# logits: (batch, seq, vocab) — model''s prediction at each position
# Shift by one: predict token t+1 from tokens [0..t]
return F.cross_entropy(
logits[:, :-1, :].reshape(-1, logits.size(-1)),
input_ids[:, 1:].reshape(-1),
)
Astonishingly, this single objective — applied to trillions of tokens — is enough to produce models that write code, summarize documents, and answer questions. The capability emerges from scale.
4.2 Perplexity — measuring a language model
Perplexity is the standard metric: roughly, "on average, how many tokens is the model choosing between?" Lower is better.
import torch
def perplexity(loss_value: float) -> float:
return float(torch.exp(torch.tensor(loss_value)))
# A well-trained model on in-domain text: perplexity 5–20.
# Random model: perplexity ≈ vocab_size (tens of thousands).
4.3 Generation is a sampling loop
def generate(model, tokenizer, prompt, max_new_tokens=100):
ids = tokenizer.encode(prompt)
for _ in range(max_new_tokens):
logits = model(ids) # predict next-token distribution
next_id = sample(logits[-1]) # pick one token
ids.append(next_id) # append and repeat
if next_id == tokenizer.eos_token_id:
break
return tokenizer.decode(ids)
4.4 Decoding strategies
The same model produces wildly different text depending on how you sample:
import torch
import torch.nn.functional as F
def sample(logits, temperature=0.8, top_p=0.9, top_k=None):
logits = logits / max(temperature, 1e-5) # temperature: <1 sharper, >1 wilder
if top_k is not None: # top-k: keep only k most likely
v, _ = torch.topk(logits, top_k)
logits[logits < v[-1]] = float("-inf")
probs = F.softmax(logits, dim=-1)
if top_p is not None: # top-p (nucleus): smallest set summing to p
sorted_probs, sorted_idx = torch.sort(probs, descending=True)
cumulative = sorted_probs.cumsum(dim=-1)
mask = cumulative - sorted_probs > top_p
sorted_probs[mask] = 0.0
sorted_probs /= sorted_probs.sum()
choice = torch.multinomial(sorted_probs, 1)
return sorted_idx[choice].item()
return torch.multinomial(probs, 1).item()
Rules of thumb:
- Factual / code:
temperature=0.0–0.3— deterministic, fewer mistakes. - Chat / general:
temperature=0.7, top_p=0.9— coherent but not robotic. - Creative writing:
temperature=0.9–1.1— more surprising, more risk of going off the rails.
This is why "the model gave a weird answer" is often a decoding problem, not a model problem.
5. Ethical Considerations in AI
This section is not a footnote — it''s the part that decides whether your generative app is safe to ship. Generative models introduce risks that classifiers never had, because they produce content rather than just labeling it.
5.1 Hallucination — confident falsehood
LLMs predict plausible text, not true text. They will state wrong facts, invent citations, and fabricate APIs with total confidence.
Why it happens: the model optimizes for "what''s a likely continuation," not "what''s true." Truth isn''t in the loss function.
Mitigations:
- Ground with retrieval (RAG) — give the model real source documents and instruct it to answer only from them.
- Cite sources and show them to the user.
- Constrain the task — "answer only from the context below; if it''s not there, say you don''t know."
- Never use raw LLM output for high-stakes facts (medical, legal, financial) without human review.
GROUNDED_PROMPT = """Answer the question using ONLY the context below.
If the answer is not in the context, reply exactly: "I don''t have that information."
Context:
{context}
Question: {question}
Answer:"""
5.2 Bias and fairness
Models learn the biases of their training data — gender, race, geography, language. A model trained mostly on English internet text will be worse at, and biased about, other languages and cultures.
Mitigations:
- Test outputs across demographic groups before shipping.
- Don''t use generative models for consequential decisions about people (hiring, lending) without rigorous fairness audits.
- Be explicit in prompts about neutrality where it matters.
5.3 Copyright and intellectual property
Models are trained on copyrighted text and code. Output can reproduce training data verbatim, and the legal status of AI-generated content derived from copyrighted training data is still unsettled in 2026.
Mitigations:
- Don''t present generated content as wholly original where provenance matters.
- For code, check generated snippets against licenses (some models can emit GPL-licensed code).
- Keep a human in the loop for anything you''ll publish or sell.
5.4 Privacy and PII
Two risks: (1) users paste sensitive data into your app, which then goes to a third-party API; (2) the model may have memorized PII from training data.
Mitigations:
- Redact PII before sending to any external model (the boundary is your responsibility).
- Use enterprise endpoints (Azure OpenAI, AWS Bedrock) that contractually don''t train on your inputs.
- Don''t log raw prompts containing user PII.
import re
PII_PATTERNS = {
"email": re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"),
"phone": re.compile(r"\b(?:\+?\d{1,3}[-.\s]?)?\(?\d{3,5}\)?[-.\s]?\d{3,4}[-.\s]?\d{3,4}\b"),
"pan": re.compile(r"\b[A-Z]{5}\d{4}[A-Z]\b"), # Indian PAN
"ssn": re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
}
def redact_pii(text: str) -> str:
for label, pattern in PII_PATTERNS.items():
text = pattern.sub(f"[REDACTED_{label.upper()}]", text)
return text
5.5 Misuse — deepfakes, misinformation, spam
The same model that writes a helpful email writes convincing phishing, fake reviews, and propaganda at scale.
Mitigations:
- Rate-limit and authenticate your generation endpoints.
- Add usage policies and monitor for abuse patterns.
- Consider watermarking / disclosure that content is AI-generated.
5.6 Environmental and economic cost
Training a frontier model emits significant carbon and costs tens of millions of dollars; inference at scale adds up too.
Mitigations:
- Use the smallest model that does the job — don''t call GPT-4-class models for sentiment analysis a logistic regression could do.
- Cache responses for repeated queries.
- Batch requests; use efficient serving (vLLM).
5.7 The shipping checklist for ethics
Before a generative feature goes live:
- PII is redacted before leaving your network.
- High-stakes outputs are grounded (RAG) and/or human-reviewed.
- The model is the smallest one that meets quality needs.
- Generation endpoints are authenticated + rate-limited.
- Users know they''re talking to AI (disclosure).
- You don''t log raw PII-containing prompts.
- You''ve tested for obvious bias on representative inputs.
6. Project: a real Text-Generation Application
Now we build it. A FastAPI service that generates text with streaming, PII redaction, a simple content guardrail, and a Gradio UI. It works with either a hosted API (OpenAI/Azure) or a local open-weight model — so you can run it free on your laptop or scale it in production.
6.1 Architecture
Browser (Gradio UI)
│ prompt
▼
FastAPI /generate (SSE streaming)
│ 1. redact PII 2. content guardrail 3. rate limit
▼
LLM backend ── option A: OpenAI/Azure API
└─ option B: local Hugging Face model
│ streamed tokens
▼
Browser renders tokens as they arrive
6.2 Project structure
textgen-app/
├── app/
│ ├── main.py # FastAPI service
│ ├── llm.py # pluggable LLM backend (API or local)
│ ├── safety.py # PII redaction + content guardrail
│ └── config.py # settings
├── ui.py # Gradio frontend
├── requirements.txt
└── .env
6.3 app/safety.py — guardrails
import re
PII_PATTERNS = {
"email": re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"),
"phone": re.compile(r"\b(?:\+?\d{1,3}[-.\s]?)?\(?\d{3,5}\)?[-.\s]?\d{3,4}[-.\s]?\d{3,4}\b"),
"pan": re.compile(r"\b[A-Z]{5}\d{4}[A-Z]\b"),
}
BLOCKED_PATTERNS = [
re.compile(r"\b(make|build|synthesize)\s+(a\s+)?(bomb|explosive|bioweapon)\b", re.I),
# extend with your policy; for real apps use a moderation model (OpenAI moderation, Llama Guard)
]
def redact_pii(text: str) -> str:
for label, pattern in PII_PATTERNS.items():
text = pattern.sub(f"[REDACTED_{label.upper()}]", text)
return text
def is_blocked(text: str) -> bool:
return any(p.search(text) for p in BLOCKED_PATTERNS)
6.4 app/llm.py — pluggable backend
import os
from collections.abc import AsyncIterator
BACKEND = os.getenv("LLM_BACKEND", "openai") # "openai" or "local"
# ---------- Option A: hosted API (OpenAI / Azure) ----------
if BACKEND == "openai":
from openai import AsyncOpenAI
client = AsyncOpenAI() # reads OPENAI_API_KEY
async def stream_generate(prompt: str, temperature: float, max_tokens: int) -> AsyncIterator[str]:
stream = await client.chat.completions.create(
model=os.getenv("OPENAI_MODEL", "gpt-4o-mini"),
messages=[{"role": "user", "content": prompt}],
temperature=temperature,
max_tokens=max_tokens,
stream=True,
)
async for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
yield delta
# ---------- Option B: local open-weight model ----------
else:
import torch
from threading import Thread
from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer
model_id = os.getenv("LOCAL_MODEL", "Qwen/Qwen2.5-1.5B-Instruct")
_tok = AutoTokenizer.from_pretrained(model_id)
_model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
async def stream_generate(prompt: str, temperature: float, max_tokens: int) -> AsyncIterator[str]:
messages = [{"role": "user", "content": prompt}]
inputs = _tok.apply_chat_template(
messages, return_tensors="pt", add_generation_prompt=True
).to(_model.device)
streamer = TextIteratorStreamer(_tok, skip_prompt=True, skip_special_tokens=True)
Thread(target=_model.generate, kwargs=dict(
input_ids=inputs, streamer=streamer,
max_new_tokens=max_tokens, do_sample=temperature > 0,
temperature=max(temperature, 1e-5), top_p=0.9,
)).start()
for token in streamer:
yield token
6.5 app/main.py — the FastAPI service
import json
import time
from collections import defaultdict
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from app.llm import stream_generate
from app.safety import redact_pii, is_blocked
app = FastAPI(title="Text Generation API")
# --- Naive in-memory rate limit (use Redis in production) ---
_hits: dict[str, list[float]] = defaultdict(list)
RATE_LIMIT = 20 # requests
WINDOW = 60 # seconds
def check_rate_limit(ip: str) -> None:
now = time.time()
_hits[ip] = [t for t in _hits[ip] if now - t < WINDOW]
if len(_hits[ip]) >= RATE_LIMIT:
raise HTTPException(status_code=429, detail="Rate limit exceeded")
_hits[ip].append(now)
class GenRequest(BaseModel):
prompt: str = Field(min_length=1, max_length=4000)
temperature: float = Field(default=0.7, ge=0.0, le=2.0)
max_tokens: int = Field(default=256, ge=1, le=1024)
@app.post("/generate")
async def generate(req: GenRequest, request: Request):
ip = request.client.host if request.client else "unknown"
check_rate_limit(ip)
# 1. Safety guardrail
if is_blocked(req.prompt):
raise HTTPException(status_code=400, detail="Prompt violates usage policy")
# 2. Redact PII before the prompt leaves our process
safe_prompt = redact_pii(req.prompt)
# 3. Stream tokens back as Server-Sent Events
async def event_stream():
try:
async for token in stream_generate(safe_prompt, req.temperature, req.max_tokens):
yield f"data: {json.dumps({''token'': token})}\n\n"
yield f"data: {json.dumps({''done'': True})}\n\n"
except Exception as e: # never leak internals to the client
yield f"data: {json.dumps({''error'': ''generation failed''})}\n\n"
print(f"[generate] error: {e}")
return StreamingResponse(event_stream(), media_type="text/event-stream")
@app.get("/health")
def health():
return {"ok": True}
6.6 ui.py — Gradio frontend with streaming
import gradio as gr
import httpx
import json
API_URL = "http://localhost:8000/generate"
def generate_stream(prompt, temperature, max_tokens):
out = ""
with httpx.stream("POST", API_URL, json={
"prompt": prompt, "temperature": temperature, "max_tokens": int(max_tokens),
}, timeout=120) as r:
if r.status_code != 200:
yield f"Error: {r.read().decode()}"
return
for line in r.iter_lines():
if not line or not line.startswith("data: "):
continue
data = json.loads(line[len("data: "):])
if data.get("token"):
out += data["token"]
yield out
elif data.get("error"):
yield out + f"\n\n[error: {data[''error'']}]"
with gr.Blocks(title="Text Generation Demo") as demo:
gr.Markdown("# Text Generation\nStreaming generation with PII redaction + guardrails.")
with gr.Row():
with gr.Column():
prompt = gr.Textbox(label="Prompt", lines=4, placeholder="Write a short story about...")
temp = gr.Slider(0.0, 1.5, value=0.7, step=0.1, label="Temperature")
mt = gr.Slider(32, 1024, value=256, step=32, label="Max tokens")
go = gr.Button("Generate", variant="primary")
with gr.Column():
output = gr.Textbox(label="Output", lines=14)
go.click(generate_stream, inputs=[prompt, temp, mt], outputs=output)
if __name__ == "__main__":
demo.launch()
6.7 Run it
# requirements.txt
# fastapi uvicorn pydantic httpx gradio
# openai # if LLM_BACKEND=openai
# transformers torch # if LLM_BACKEND=local
pip install -r requirements.txt
# Choose a backend
export LLM_BACKEND=local # free, runs on your laptop
# export LLM_BACKEND=openai && export OPENAI_API_KEY=sk-... # hosted
# Terminal 1 — API
uvicorn app.main:app --reload --port 8000
# Terminal 2 — UI
python ui.py # opens http://localhost:7860
Type a prompt, watch tokens stream in. You now have a real generative-AI application — with the three things most demos skip: streaming, PII redaction, and a safety guardrail.
6.8 What you''d add for production
- Real moderation: swap the regex
is_blockedfor OpenAI''s moderation endpoint or Llama Guard. - Redis rate limiting instead of the in-memory dict.
- Auth on
/generate(API keys or JWT). - Observability: log prompt/response metadata (not PII), token counts, latency, cost — Langfuse for LLM-specific tracing.
- Caching identical prompts.
- vLLM for the local backend if you serve real traffic (continuous batching, paged attention).
- A golden eval set to catch quality regressions when you change the prompt or model.
7. Where to go next
- RAG (Retrieval-Augmented Generation) — connect the model to your own documents so it answers from real sources (and hallucinates less).
- Agents and tool use — let the model call functions, query databases, browse.
- Fine-tuning (LoRA/QLoRA) — adapt an open model to your domain or style cheaply.
- Multimodal — image (diffusion), audio (TTS/ASR), and combined text+image models.
- Evaluation — build eval harnesses so you can measure quality, not just vibe-check it.
Each of these builds on the foundation here: a generative model, wrapped in a sampling loop, behind a safe boundary.
8. Closing — the right mental model
Generative AI feels magical, but the mental model is simple: a model that learned the shape of data well enough to produce new samples of it, run inside a loop, shaped by your decoding choices, and made safe by the guardrails you put around it.
Three habits that make you good at generative AI fast:
- Pick the smallest model that works. Capability beyond your need is wasted cost and carbon.
- Treat the boundary as your responsibility. Redact PII, ground facts, rate-limit, disclose. The model won''t do this for you.
- Measure, don''t vibe. Build an eval set before you tune prompts or swap models — otherwise you can''t tell improvement from noise.
Internalize those, ship the project, and generative AI stops being a buzzword — it becomes a tool you can reason about, build with, and deploy responsibly.
Further reading
- Attention Is All You Need — the transformer paper that started it all.
- The Illustrated Transformer — Jay Alammar. The best visual explainer.
- Andrej Karpathy — "Let''s build GPT" — build a generative text model from scratch.
- Hugging Face course — free, hands-on, end-to-end.
- Stable Diffusion / latent diffusion paper — if you want to go beyond text into image generation.
Building a generative feature and not sure how to make it safe or which model to pick? Email randhir.jassal@gmail.com with what you''re building and I''ll point you in the right direction.
Get the next issue
A short, curated email with the newest posts and questions.