Level 3 · Real Programs

Regular Expressions 🔍

A tiny language for describing shapes of text. It looks like someone sat on a keyboard, and it will one day save you four hours in four minutes.

Why bother

You can find a phone number in a document with string methods. It takes forty lines and misses cases. With a pattern it is one line. The trade is that the line is unreadable until you learn the notation, so this lesson teaches the 20% that does 80% of the work.

import re

text = "Call Stan on 555-0199 or Elaine on 555-0100 before Friday."

print(re.findall(r"\d{3}-\d{4}", text))

['555-0199', '555-0100']

\d{3}-\d{4} means "three digits, a hyphen, four digits". That is the whole idea: you describe a shape, and Python finds every piece of text with that shape.

🅁 Always use raw strings

Write patterns as r"...". Regex uses backslashes constantly, and so do Python strings, so without the r you end up writing "\\\\d" to mean one \d. The r prefix turns that off. Every regex you ever see in real code has it.

The pieces worth memorising

Pattern	Matches	Example
`.`	any single character except newline	`a.c` matches abc, a7c
`\d`	a digit	`\d\d` matches 42
`\w`	a letter, digit or underscore	`\w+` matches a word
`\s`	any whitespace	space, tab, newline
`[abc]`	any one of these	`[aeiou]` is a vowel
`[^abc]`	anything except these
`[a-z]`	a range	`[A-Za-z]` is any letter
`*`	zero or more of the thing before	`ab*` matches a, ab, abbb
`+`	one or more	`\d+` is a number
`?`	zero or one, so optional	`colou?r` matches both spellings
`{{3}}`	exactly three	`\d{{3}}`
`{{2,4}}`	between two and four
`^` / `$`	start / end of the text	`^Dear`
`\|`	either	`cat\|dog`
`( )`	a group you want to capture

The five functions

import re

text = "Guybrush scored 95, Elaine scored 88, Otis scored 42."

print(re.findall(r"\d+", text))
print(re.search(r"\d+", text).group())
print(re.match(r"Guybrush", text) is not None)
print(re.sub(r"\d+", "??", text))
print(re.split(r",\s*", text))

['95', '88', '42']
95
True
Guybrush scored ??, Elaine scored ??, Otis scored ??.
['Guybrush scored 95', 'Elaine scored 88', 'Otis scored 42.']

Function	Does	Returns
`re.findall`	every match	a list of strings
`re.search`	the first match anywhere	a match object, or None
`re.match`	a match at the very start only	a match object, or None
`re.sub`	find and replace	a new string
`re.split`	split on a pattern	a list
`re.finditer`	every match, lazily, with positions	match objects

search and match return None when there is no match, and None.group() raises AttributeError. Always test before using the result.

Groups: pulling pieces out

import re

log = "2026-08-17 09:30:00 ERROR disk full"

pattern = r"(\d{4})-(\d{2})-(\d{2}) (\d{2}:\d{2}:\d{2}) (\w+) (.+)"
found = re.search(pattern, log)

if found:
    print(found.group(0))
    print(found.group(1), found.group(2), found.group(3))
    print(found.group(5), "->", found.group(6))
    print(found.groups())

2026-08-17 09:30:00 ERROR disk full
2026 08 17
ERROR -> disk full
('2026', '08', '17', '09:30:00', 'ERROR', 'disk full')

Numbered groups get unreadable fast. Name them:

import re

log = "2026-08-17 09:30:00 ERROR disk full"

pattern = (r"(?P<date>\d{4}-\d{2}-\d{2}) "
           r"(?P<time>\d{2}:\d{2}:\d{2}) "
           r"(?P<level>\w+) "
           r"(?P<message>.+)")

found = re.search(pattern, log)
if found:
    parts = found.groupdict()
    print(parts["level"], "on", parts["date"])
    print(parts)

ERROR on 2026-08-17
{'date': '2026-08-17', 'time': '09:30:00', 'level': 'ERROR', 'message': 'disk full'}

Greedy versus lazy: the classic surprise

import re

html = "<b>bold</b> and <i>italic</i>"

print(re.findall(r"<.+>", html))
print(re.findall(r"<.+?>", html))

['<b>bold</b> and <i>italic</i>']
['<b>', '</b>', '<i>', '</i>']

LOGIC[Formidable: Success]

+ and * are greedy: they take as much as they possibly can while still allowing a match. The first pattern matched from the very first < to the very last >, which is technically correct and completely useless.

Adding ? after them makes them lazy: take as little as possible. When a pattern matches far more than you expected, this is nearly always why.

Substitution with groups

import re

dates = "Due 15/10/1990, meeting 20/12/1991."

iso = re.sub(r"(\d{2})/(\d{2})/(\d{4})", r"\3-\2-\1", dates)
print(iso)

def shout(match):
    return match.group(0).upper()

print(re.sub(r"\b\w{4}\b", shout, "this is a test of four char words"))

Due 1990-10-15, meeting 1991-12-20.
THIS is a TEST of FOUR CHAR words

\3-\2-\1 in the replacement means "group 3, then group 2, then group 1". And when the replacement needs logic, pass a function: it receives each match and returns the replacement text.

Useful patterns to steal

import re

text = """Contact elaine@melee.gov or stan@usedships.example.
Visit https://rustyschool.com/python for the course.
Ring 555-0199. Order #A-1042 shipped 2026-08-17."""

print(re.findall(r"[\w.+-]+@[\w-]+\.[\w.]+", text))
print(re.findall(r"https?://[^\s]+", text))
print(re.findall(r"\d{4}-\d{2}-\d{2}", text))
print(re.findall(r"#[A-Z]-\d+", text))

['elaine@melee.gov', 'stan@usedships.example.']
['https://rustyschool.com/python']
['2026-08-17']
['#A-1042']

Spot the trailing full stop on that second address. The pattern's final [\w.]+ includes dots, so it kept going past the domain and took the end of the sentence with it. Patterns match text, not meaning, and this is exactly the kind of near-miss that survives a quick eyeball and breaks later.

📧 On validating email addresses

That email pattern is fine for finding addresses in text. It is not a validator. The real specification for a valid email address is monstrous, and the standard regex that implements it is thousands of characters long. The industry answer is: check there is an @ with something either side, then send a confirmation message. That is the only real test anyway.

Compile patterns you reuse

import re

pattern = re.compile(r"\berror\b", re.IGNORECASE)

lines = ["Error: disk full", "all fine", "ERROR again", "terrorist"]
for line in lines:
    if pattern.search(line):
        print(f"match: {line}")

match: Error: disk full
match: ERROR again

re.compile parses the pattern once instead of on every call, and gives you somewhere to hang a name and a comment. \b is a word boundary, which is why "terrorist" does not match. Flags like re.IGNORECASE and re.MULTILINE go here.

When not to use regex

Parsing HTML or XML. Use a parser. HTML is not a regular language and cannot be correctly matched by a regular expression. This is a mathematical fact, not an opinion.
Parsing CSV. Use the csv module. Lesson 23 showed you why.
When a string method does it. "x" in text beats re.search for a literal substring, and is faster and clearer.
When the pattern needs a comment to be readable and you only use it once. A short loop can be kinder to the next reader.

Exercise 1

Extract and total

Pull every price out of a receipt and total them.

Reveal solution

import re

receipt = """Grog        £4.50
Rubber chicken   £12.00
Map        £3.25
Sword      £8.75"""

prices = re.findall(r"£(\d+\.\d{2})", receipt)
print(prices)
print(f"Total: £{sum(float(p) for p in prices):.2f}")

['4.50', '12.00', '3.25', '8.75']
Total: £28.50

The group around the number means findall returns just the digits, not the currency symbol. Groups control what you get back, which is half the reason to use them.

Exercise 2

Redact sensitive data

Replace every email address and phone number in a message with [redacted], keeping everything else intact.

Reveal solution

import re

message = """From: elaine@melee.gov
Call me on 555-0199 or reach stan@usedships.example.
The meeting is Tuesday."""

redacted = re.sub(r"[\w.+-]+@[\w-]+\.[\w.]+", "[redacted email]", message)
redacted = re.sub(r"\d{3}-\d{4}", "[redacted phone]", redacted)

print(redacted)

From: [redacted email]
Call me on [redacted phone] or reach [redacted email]
The meeting is Tuesday.

Look closely at the second line: the full stop after usedships.example has vanished. The pattern ends with [\w.]+, which happily includes dots, so it swallowed the sentence's punctuation along with the domain.

That is greedy matching biting you in a way that is easy to miss, because the output still looks plausible. Ending the pattern with [\w-]+\.[A-Za-z]{2,} would stop at the top-level domain instead. Always read what a regex actually matched, not what you meant.

Exercise 3

Parse a log file

Turn each line of a log into a dictionary, and count how many of each level there were. Skip any line that does not match.

Reveal solution

import re
from collections import Counter

log = """2026-08-17 09:30:00 INFO server started
2026-08-17 09:31:12 ERROR disk full
this line is garbage
2026-08-17 09:31:45 WARN retrying
2026-08-17 09:32:00 ERROR still full"""

pattern = re.compile(
    r"(?P<date>\d{4}-\d{2}-\d{2}) (?P<time>[\d:]+) (?P<level>\w+) (?P<message>.+)"
)

entries = []
for line in log.splitlines():
    found = pattern.match(line)
    if found:
        entries.append(found.groupdict())

print(f"{len(entries)} parsed, 1 skipped")
for level, count in Counter(e["level"] for e in entries).most_common():
    print(f"  {level:6} {count}")

for entry in entries:
    if entry["level"] == "ERROR":
        print(f"{entry['time']} {entry['message']}")

4 parsed, 1 skipped
  ERROR  2
  INFO   1
  WARN   1
09:31:12 disk full
09:32:00 still full

Silently skipping unparseable lines is a decision, not a default. In a real tool you would count them and report the number, because a log format that quietly changed is exactly the sort of thing you want to hear about.

+100 XP

← PreviousLesson 24: Dates and Times Next →Lesson 26: Virtual Environments and pip