Free CSR Workshop · Part 1 · Phase 1: Foundations

Building an LLM from Scratch — Part 1

Every large language model starts with the same building blocks: text becomes tokens, tokens become vectors, and attention lets those vectors talk to each other. In this hands-on 3-hour session you'll write each of those pieces yourself in Python and PyTorch — no black-box libraries, no hand-waving.

DateSaturday, 7 November 2026
Time10:00 AM – 1:00 PM IST
Duration3 hours, hands-on
FeeFree · CSR initiative
Register Free See the Agenda
What Phase 1 Covers

The Input Side of a GPT, Built by Hand

Part 1 takes you from raw text to a working multi-head attention module — the foundation every later phase builds on.

🧭

How an LLM Really Works

Next-token prediction, the decoder-only GPT architecture, and the three stages of building a model: data & architecture, pretraining, fine-tuning. A map you'll use for the whole series.

✂️

Tokenization from Scratch

Why models don't read words. Build a simple tokenizer, then implement Byte-Pair Encoding (BPE) step by step — vocabulary, merges, special tokens and handling unknown text.

🧱

Training Data Pipeline

Turn a text corpus into input–target pairs with a sliding window, then wrap it in a PyTorch Dataset and DataLoader with context length, stride and batching.

📐

Token & Positional Embeddings

Map token IDs to learnable vectors, add positional information so order matters, and see what an embedding layer actually is under the hood.

🎯

Self-Attention, Step by Step

Start with a simple attention score, then add trainable query, key and value weights and scaled dot-product attention — computed by hand, then in code.

🧠

Causal & Multi-Head Attention

Mask the future so the model can't cheat, add dropout, and stack heads into a reusable multi-head attention module — the heart of the transformer.

3-Hour Agenda

Saturday, 7 November · Phase 1

Each block mixes a short concept walkthrough with live coding. You leave with a notebook that runs end to end.

  1. 10:00 – 10:20
    Kick-off: the LLM big pictureWhat a language model is, how GPT-style models generate text one token at a time, and the roadmap for the series. Environment check (local Python or Google Colab).
  2. 10:20 – 10:55
    Working with text: tokenizationWord vs character vs sub-word tokens. Build a basic tokenizer, then implement Byte-Pair Encoding from scratch and compare it with a production tokenizer.
  3. 10:55 – 11:25
    Building the data pipelineSliding-window sampling, input–target pairs for next-token prediction, and a PyTorch DataLoader ready for training.
  4. 11:25 – 11:35
    Break
  5. 11:35 – 12:00
    Embeddings: from IDs to vectorsToken embedding layers, absolute positional embeddings, and the final input tensor the transformer receives.
  6. 12:00 – 12:45
    Coding attention from scratchSimple self-attention → scaled dot-product attention with Q, K, V → causal masking and dropout → a multi-head attention class you can reuse in later phases.
  7. 12:45 – 1:00
    Wrap-up & Q&ARecap of Phase 1, common pitfalls, take-home exercises, and what comes next in Part 2.
Who It's For

Engineers and Students Who Want to Look Inside

This workshop is for software engineers, data scientists, ML practitioners and college students who use LLMs but want to understand how they are actually built.

Prerequisites: comfortable with basic Python (functions, classes, lists). Some familiarity with NumPy or PyTorch helps but isn't required — we explain every tensor operation we use.

Bring: a laptop with Python 3.10+ and PyTorch installed, or simply a Google account to run everything in Google Colab.

  • A working BPE tokenizer you wrote and understand line by line.
  • A PyTorch data pipeline that turns raw text into training batches.
  • Embedding layers for tokens and positions, wired together.
  • A multi-head causal attention module ready to plug into the transformer in Part 2.
  • Session notebook & exercises to practise before the next phase.
The Series

Part 1 Is Where It Starts

Building an LLM from Scratch runs in phases. Each part builds directly on the code from the one before.

This workshop · 7 Nov

Part 1 — Phase 1: Foundations

Tokenization, data pipeline, embeddings and attention: everything the model needs before the transformer block.

Coming soon

Part 2 and beyond

Later phases continue from the attention module you build here. Dates will be announced after Part 1 — registered participants hear first.

Register

Reserve Your Free Seat

This workshop is part of the OPLEXUS CSR initiative and is free to attend. Fill in your details and we'll email your confirmation along with joining details and setup instructions before the session.

📅

When

Saturday, 7 November 2026
10:00 AM – 1:00 PM IST (3 hours)

✉️
Are you working or in college?

We use these details only to manage your registration and send workshop updates. See our Privacy Policy.