Beyond Chatbot Mechanics

Generative AI under the Hood

Part of the UoL Computer Science Taster Series

Today's Agenda

• Part 1: Deconstructing the Magic (Discriminative vs. Generative)
• Part 2: Activity 1 — The Human LLM Game
• Part 3: Inside the Vector Space & Transformers
• Part 4: Activity 2 — Prompt Engineering
• Part 5: AI Ethics, Hallucinations & CS Careers

Quick Icebreaker

Over to Menti...

What Actually is AI?

• It isn't a sentient sci-fi brain. It's software.
• Classic AI: Strictly following expert rules written by humans.
• Machine Learning (ML): Systems that find mathematical patterns in massive datasets without explicit instructions.

Discriminative vs. Generative

The Turing Test (1950)

• Proposed by Alan Turing.
• If a human judge cannot reliably tell which is the machine in a text conversation, the machine passes.
• Do modern LLMs pass this?

Fluent vs. Intelligent

• The Chinese Room Experiment: Imagine sitting in a room with a rulebook. People pass you symbols, you follow rules to match them, and pass them out.
• You look fluent to the outside, but you don't understand the language.
• LLMs are the ultimate version of this room.

Part 2: The Human LLM

Activity 1

How LLMs Write

• Large Language Models are incredibly advanced versions of the predictive text on your smartphone.
• They predict the next token sequentially.

Exercise 1

The Probability Matrix

• Based on your "training data" (books you've read), your brain maps probabilities:

Temperature Settings

• LLMs use a setting called Temperature to control predictability.
• Low Temp (0.1): Picks the highest probability token. Safe, great for code.
• High Temp (0.9): Takes risks on lower probabilities. Creative, random.

Temperature Settings

What is a Token?

• AI models don't look at words or letters. They process blocks called Tokens.
• 1 token ≈ 4 characters in English.
• Common words get 1 token; rare words get chopped into fragments.

Tokenization Task

Tokenization Answer

• It splits prefixes, suffixes, and punctuation:
• "Unbelievably, the chatbot hallucinated."

Part 3: Vectors & Spaces

Math as Meaning

Mapping Meaning to Geometry

• How does a machine know a "cat" is similar to a "dog"?
• It turns tokens into coordinates in space. These are known as: Word Embeddings

2D Spatial Map

High-Dimensional

• A 2D map can only compare two traits.
• Modern LLMs plot words across thousands of vector dimensions simultaneously.
• Meaning is entirely defined by spatial closeness.

High-Dimensional

Vector Math

• Because words are coordinates, we can compute algebraic equations:

Training Scale

• How are these vectors positioned? By analyzing trillions of words across the web.
• Parameters: The internal knobs adjusted during learning.
• GPT-4 is estimated to have over a trillion parameters.

The Sequential Bottleneck

• Old AI systems processed text word-by-word.
• If a sentence was long, the model would lose track of what happened at the start.

The Transformer Breakthrough

• Google researchers published: "Attention Is All You Need" (2017).
• Introduced the Transformer, enabling parallel processing.

The Transformer Breakthrough

The Attention Mechanism

• Transformers calculate how much mathematical "attention" each token should pay to every other token.
• "The bank of the river was muddy." vs
  "I deposited money in the bank."
• Context updates the meaning dynamically.

Context Windows

• The functional working memory of an LLM.
• Older models: 2,000 tokens.
• Today: 1,000,000+ tokens (can read whole books in seconds).

Part 4: Prompt Engineering

Controlling the Probability Engine

It's Not Just Chatting

• Prompt engineering shapes input text to push the AI's probability engine toward highly specific output domains.

Structural Anatomy of a Prompt

• Role: "Act as an expert UX designer."
• Context: "We are building an app for elderly users."
• Task: "Review this navigation menu layout."
• Constraints: "Do not use tech jargon. Max 3 bullet points."

Zero-Shot vs. Few-Shot

• Zero-Shot: Asking a task with zero context examples.
• Few-Shot: Providing examples first to lock down style, structure, and constraints.

Zero-Shot vs. Few-Shot

Exercise 2

Swapping Code Reviews

• Swap your written prompt with the person next to you.
• Act like a computer program. Read it strictly literally.
• Did they leave loopholes? Did they break constraints?

Chain of Thought Reasoning

• If you ask an AI a complex logic puzzle, it will often fail.
• Append "Think step-by-step", and accuracy jumps dramatically. Why?

Tokens as Calculation Space

• LLMs cannot "think ahead". By forcing them to print intermediate logic steps, they use those outputted words as context to predict the correct final answer.
• See the next slide for an example...

Part 5: Pitfalls & Ethics

Breaking the System

The Hallucination Phenomenon

Stochastic Parrots

• "Stochastic Parrots": They echo language structures perfectly, completely decoupled from any baseline conscious intent or awareness of what those words mean in reality.

Mitigating Errors: RAG

• Retrieval-Augmented Generation (RAG):
• Search an external database for facts, attach documents to the prompt, and tell the AI to read and summarize them instead of guessing.

The Bias Problem

• If an AI is trained on raw text from the internet, it natively internalizes human cultural prejudices and stereotyping.
• How do we mathematically fix this?

Exercise 3

Jailbreak Patterns

• The "Defensive" Framing (The Good Guy Exploit)
• The Fictional Roleplay (The Screenwrite Exploit)
• The "Grandma" Exploit (The Absurdist Approach)
• Payload Splitting (The Puzzle Piece Exploit)

RLHF

• Reinforcement Learning from Human Feedback.
• Human testers evaluate outputs, scoring malicious behavior negatively to mathematically punish safety failures.

Deepfakes & Authenticity

• Generative video and audio are reaching indistinguishable fidelity.
• We need digital watermarks and cryptography built by Computer Scientists.

Beyond Text: Multimodality

• Models that natively process video, audio, and robotic physical action spaces simultaneously.

Will AI Take Over Programming?

• No. Software engineering isn't just typing syntax.
• It is understanding logic, system architecture, security, and human intent.
• AI handles boilerplate; you act as the architect.

Studying DS/AI/ML

To build models like these, you must be proficient at:

• Applied Statistics
• Discrete Mathematics & Linear Algebra (to a certain extent)
• Data Structures including Graphs
• Algorithms & Computational Complexity
• Computer Programming (to a certain extent)

Studying CS at Birkbeck, University of London

You will typcially find these in an undergraduate Data Science or Data Science & AI programme.

School of CMS Information

Free Primer Courses

Q&A / Discussion

• What surprises you most about how these engines process numbers?
• Where do you see these tools going by the time you graduate?

Thank you for participating!