Code in Motion:

Engineering the Autonomous Future

University of London
Computer Science Taster Session
School of Computing and Mathematical Sciences
Birkbeck, University of London

Agenda for Today

  • 1: The Physical vs. Digital Divide
  • 2: Computer Vision (How machines see)
  • 3: Unplugged Activity: Edge Detection
  • 4: Control Theory (How machines move)
  • 5: Unplugged Activity: Human Feedback Loops
  • 6: Immersive Tech & Campus Tour

The Big Questions

  • How does a self-driving car recognise a pedestrian in a rainstorm?
  • What allows a surgical robot to stitch a blood vessel with astonishing precision?
Autonomous Vehicle Sensor Suite

The Mindset Shift

  • Traditional Programming: Code runs in a perfect, predictable digital box. (If X, then Y).
  • Robotics: Code runs in the messy, unpredictable physical world.
  • Gravity, friction, wind, and bad lighting ruin perfect code.

The Sense-Compute-Act Loop

👀 SENSE
(Cameras/Sensors) ➔ 🧠 COMPUTE
(Algorithms) ➔ 🦾 ACT
(Motors/Brakes)

If this loop is too slow, the car crashes.

Part 1: SENSE

Computer Vision

How do we "See"?

  • Humans see a pedestrian.
  • Computers do not see people, cars, or trees.
  • They only see grids of numbers.

The Pixel Matrix

  • An image is a 2D matrix (a grid).
  • Each square is a pixel, represented by a number indicating brightness (0 = Black, 255 = White).
Pixel grid zooming in on an image

Adding Colour (RGB)

  • Colour images are just three matrices stacked on top of each other: Red, Green, and Blue.
  • A self-driving car processing a 4K camera feed at 60 frames per second is analysing roughly 1.5 billion numbers every second.

The Pedestrian Problem

  • How do we find a person in that sea of 1.5 billion numbers?
  • Step 1:
    • We have to find the boundaries between objects.
    • We need to find the edges.

Finding Edges Mathematically

  • An "edge" is simply a place in the image where the numbers suddenly change from very high (bright) to very low (dark).
  • We find this using a mathematical operation called Convolution.

The Kernel Filter

  • We slide a tiny 3x3 grid of numbers (a kernel) across the entire image.
  • It multiplies the pixels and highlights areas with sharp contrast.
[ -1, -1, -1 ]
[ -1, 8, -1 ]
[ -1, -1, -1 ]

Convolution in Action

Input Matrix (RGB)

âž”
Apply
Kernel

Output: Edge Map

Unplugged Activity 1

Edge Detection

The Setup

You have a 1D strip of pixels (numbers 0 to 9).
0 is black, 9 is white.

[1, 1, 2, 8, 9, 8, 2, 1]

Where is the "edge" of the object?

The Algorithm

  • To find the edge, we will calculate the derivative (the rate of change).
  • Task: Subtract the left pixel from the right pixel. Write the difference below.
1 - 1 = 0
2 - 1 = 1
8 - 2 = 6
(Massive change!)

The Debrief

  • You just acted as a Convolutional Neural Network (CNN) layer.
  • By scanning for massive numerical differences, the car's computer identifies the outline of a pedestrian against the pavement.

Beyond Edges: Bounding Boxes

  • Once edges are found, Machine Learning models (like YOLO) draw bounding boxes around recognized shapes.
  • The car calculates if the box's trajectory intersects with the car's trajectory.
[Pedestrian: 98.7%]

YOLO (You Only Look Once)

A popular, state-of-the-art machine learning algorithm used for real-time object detection in computer vision.

Key characteristics:

  • Speed
  • Global Context
  • Continuous Evolution

What if the camera is blinded?

  • Cameras fail in fog, heavy rain, or blinding sunlight.
  • Autonomous vehicles use Sensor Fusion: Combining Camera pixels with LiDAR (lasers) and Radar (radio waves).

LiDAR (Light Detection and Ranging)

LiDAR ACTIVE
3D Point Cloud
Pts/sec: 1.5M

LiDAR creates a perfect 3D point cloud, allowing the car to measure exact distances to the millimeter.

LiDAR (Light Detection and Ranging)

Part 2: ACT

Control Theory & Robotics

The Precision Problem

  • We've seen the pedestrian. Now we must brake.
  • Or, in the case of a surgical robot: The surgeon moves a joystick 25 millimeters; the robotic scalpel moves exactly 1 millimeter inside a patient.
  • How do we ensure the motor stops exactly where we want it to?

Precision in Action

  • The da Vinci Surgical System translates human hand movements into micro-movements inside the patient.
  • Without closed-loop feedback, this level of stability is impossible.

Precision in Action

Open-Loop vs. Closed-Loop

  • Open-Loop: You tell the system what to do, and hope it works. (e.g., A toaster. It heats for 2 minutes, regardless of if the bread is frozen or already burnt).
  • Closed-Loop: The system constantly checks its own progress and adjusts. (e.g., A central heating thermostat).

The Feedback Loop

Goal (Stop at Line) âž” Error (10m away) âž” Action (Apply Brakes) âž” Measure (Now 8m away) âž” (Repeat)

The Brain of the Robot:
PID Controller

  • Nearly every robot, drone, and autonomous car uses a PID Controller to move smoothly.
  • Proportional
  • Integral
  • Derivative

Proportional (P)
- The Present

  • The motor output is proportional to the error.
  • Analogy: If you are driving 100 miles to a destination, you drive fast. When you are 1 mile away, you slow down.
  • The closer you get to the target, the less power you apply.

The Overshoot Problem

  • If you only use "P", the robot moves so fast to correct the error that momentum carries it past the target.
  • It ends up oscillating—wobbling back and forth forever trying to find the exact center.

Derivative (D)
- The Future

  • Calculates the rate of change of the error.
  • If the robot is approaching the target very fast, "D" acts as a shock absorber. It applies a negative force to slow it down before it overshoots.

Integral (I)
- The Past

  • What if a drone is hovering, but a gentle breeze pushes it 10 cm down?
  • "P" might not be strong enough to overcome gravity.
  • "I" adds up all past errors over time.
  • If the drone is slightly off target for too long, "I" slowly builds up enough power to nudge it back.

The Mathematical Elegance

Computer Scientists write this behavior into software using calculus. The formula for the control signal u(t) is:

u(t) = Kpe(t) + Ki t ∫ 0 e(τ) dτ + Kd de(t) dt

(Don't worry, you don't need to memorize this today! But this is the equation that saves lives in surgical robotics.)

Visualising PID

Image of PID controller block diagram showing error processing

Unplugged Activity 2

The Human PID Controller

The Setup

The Blindfolded Robot

I need two volunteers.

  • Volunteer 1 (The Robot): Will close their eyes. Their goal is to stand exactly on a specific tile.
  • Volunteer 2 (The Sensor/Algorithm): Can only say two words: "Forward" or "Backward", followed by a number of steps.

The Execution

  • If the sensor says "Forward 5" (High Proportional error), the robot takes large steps.
  • As the robot gets closer, the sensor must say "Forward 1" or "Forward 0.5" to prevent overshoot (Derivative damping).

The Debrief

  • You just acted out a Closed-Loop Feedback system.
  • Latency (the delay in Volunteer 2 speaking) causes crashes. If the sensor data is delayed, the robot acts on old information.

Part 3: The Real World

When Systems Fail

Latency & Edge Computing

  • A self-driving car cannot send camera data to a cloud server in London, wait for the AI to process it, and wait for the "Brake" command to return.
  • The latency (delay) would be fatal.
  • Edge Computing: The processing must happen locally, physically on the car's motherboard, in milliseconds.

Latency & Edge Computing

The Ethics of Autonomy

  • When writing software that interacts with human lives, Computer Scientists must make ethical choices.
  • If an autonomous vehicle loses its brakes, how does the software decide what to steer into?

The Digital Trolley Problem

  • Does the car prioritise the safety of the driver at all costs?
  • Or does it minimise overall harm, even if it means injuring the occupants?
  • These aren't just philosophy questions anymore—they are lines of code.
[PATH_A]
COST: HIGH
[PATH_B]
COST: LOW

Spatial Computing

  • Notice how the VR headsets use SLAM (Simultaneous Localization and Mapping).
  • They are constantly running edge-detection and feature-matching kernels in real-time to prevent you from walking into physical walls.

The "Chicken & Egg" Problem

  • How do you build a map without knowing your location?
  • How do you know your location without a map?

SLAM

Programming the Physical World

  • Because robotics requires extreme safety and memory management, we often move away from Python.
  • We use highly rigorous, memory-safe languages like Rust, or robust object-oriented systems in Java or C++.
  • A single memory leak in a surgical robot is catastrophic.

Next Steps

Computer Science @ Birkbeck

... and some freebies!

Studying at Birkbeck

  • We don't just teach you how to write code; we teach you how to engineer solutions.
  • Modules cover algorithms, data structures, software engineering, and AI.
  • Flexibility: We offer both daytime and evening teaching, allowing you to build your career or work in London's tech sector while you study.

A message from George Birkbeck...

The Immersive Learning Lab

  • Next, you will head to the Immersive Learning Lab.
  • You will experience the latest VR (Virtual Reality) and AR (Augmented Reality) hardware.
  • These tools use the exact same Computer Vision algorithms we just discussed to track your hand movements and map the room.

Studying CS at Birkbeck, University of London

CMS QR Code

School of CMS Information

Feedback QR Code

Free Primer Courses

Thank You!
Any Questions?

  • Computer vision, robotics, or the math behind the machine?
  • Studying Computer Science at Uni?
  • Doesn't have to be here.
  • ...
  • Who's going to win the FIFA World Cup?