← All categories

LLMs & Generative AI

Large language models from the ground up: transformer architecture, attention, tokenization, prompting, fine-tuning, embeddings, safety, and production cost optimization — covering the 2026 landscape of Claude 4, GPT-4o, Gemini 2.x, Llama 4, and mainstream reasoning models.

LLMs & Generative AI

What is a large language model?

Transformer architecture overview

Attention mechanism — self-attention and multi-head attention

Tokenization — BPE, WordPiece, SentencePiece

Context window and why it matters

Temperature, top-p, and top-k sampling

Hallucination — causes and mitigation

Prompt engineering — zero-shot, few-shot, chain-of-thought, system prompts

RLHF — Reinforcement Learning from Human Feedback

Instruction tuning vs base models

Fine-tuning approaches — full fine-tune, LoRA, QLoRA, PEFT

Embeddings — what they are, use cases, cosine similarity

Model quantization — INT8, INT4, GGUF

Evaluation metrics — BLEU, ROUGE, perplexity, human eval

Safety & alignment — red-teaming, constitutional AI, guardrails

LLM pricing and cost optimisation

Multimodal models

Context length extension techniques

Reasoning models — chain-of-thought, o3-style, and extended thinking

Model distillation

RAG architecture and production considerations

Mixture-of-Experts (MoE) architecture

Structured output and function calling