Modules and the Standard Library 📚
Everything so far has lived in one file. Real programs are many files, and Python ships with hundreds of them already written for you.
import
import random
import math
print(math.sqrt(16))
print(math.pi)
random.seed(42) # makes the "random" repeatable
print(random.randint(1, 100))
print(random.choice(["grog", "map", "sword"]))
4.0
3.141592653589793
82
grog
import math makes the module available, and you reach into it with a dot.
random.seed(42) fixes the sequence so the same "random" numbers come out
every time, which is invaluable for testing and for lessons like this one that promise
you an exact output.
The four import styles
import math
from math import sqrt
from math import sqrt as square_root
import statistics as stats
print(math.sqrt(9))
print(sqrt(9))
print(square_root(9))
print(stats.mean([1, 2, 3, 4]))
3.0
3.0
3.0
2.5
| Style | When |
|---|---|
import x | The default. Always obvious where a name came from |
from x import y | When you use y constantly and the origin is obvious |
import x as y | For long names, or by convention (import pandas as pd) |
from x import * | Never. It dumps unknown names into your file and hides collisions |
Writing your own module
A module is just a .py file. Make pirate_tools.py:
"""Helpers for pirate arithmetic."""
INSULT_LIMIT = 8
def format_name(first, last):
"""Return a properly capitalised full name."""
return f"{first.title()} {last.title()}"
def can_duel(insults):
"""True if this pirate knows enough insults to duel."""
return insults >= INSULT_LIMIT
Then, in another file in the same folder:
import pirate_tools
print(pirate_tools.format_name("guybrush", "threepwood"))
print(pirate_tools.can_duel(9))
print(pirate_tools.INSULT_LIMIT)
That is the whole mechanism. When your file passes about two hundred lines, or when a group of functions clearly belong together, split them out. Future you will be grateful.
The __main__ guard
def add(a, b):
return a + b
if __name__ == "__main__":
print("Running directly, so here is a demo:")
print(add(2, 3))
Running directly, so here is a demo:
5
Python sets __name__ to "__main__" when a file is run directly,
and to the module's name when it is imported. So that block runs when you type
python3 thing.py and stays quiet when another file imports it.
Without the guard, importing a module runs everything in it, including your test prints and, memorably for someone, an entire database migration. Put definitions at the top level and actions inside the guard. It is one of the strongest conventions in Python.
A tour of the batteries
This is the part people mean by "batteries included". Every one of these ships with Python and needs no installation.
Numbers, chance and time
import random, statistics, datetime
random.seed(7)
rolls = [random.randint(1, 6) for _ in range(10)]
print(rolls)
print(f"mean {statistics.mean(rolls)}, median {statistics.median(rolls)}")
born = datetime.date(1990, 10, 15)
print(f"Monkey Island was released on a {born.strftime('%A')}")
[3, 2, 4, 6, 1, 1, 5, 1, 3, 5]
mean 3.1, median 3.0
Monkey Island was released on a Monday
Text and data formats
import json, textwrap
data = {"ship": "Sea Monkey", "crew": ["Otis", "Meathook"]}
encoded = json.dumps(data)
print(encoded)
print(json.loads(encoded)["crew"][0])
long = "The rubber chicken with a pulley in the middle is arguably the finest item in adventure gaming."
print(textwrap.fill(long, width=45))
{"ship": "Sea Monkey", "crew": ["Otis", "Meathook"]}
Otis
The rubber chicken with a pulley in the
middle is arguably the finest item in
adventure gaming.
Collections and iteration tools
from collections import Counter, deque
from itertools import combinations
print(Counter("mississippi").most_common(2))
queue = deque(["a", "b"])
queue.appendleft("start")
print(list(queue))
print(list(combinations(["grog", "map", "sword"], 2)))
[('i', 4), ('s', 4)]
['start', 'a', 'b']
[('grog', 'map'), ('grog', 'sword'), ('map', 'sword')]
Files, paths and the system
import sys, pathlib, os
print(sys.version_info.major, sys.version_info.minor)
p = pathlib.Path("data") / "crew.json"
print(p)
print(p.suffix, p.stem, p.parent)
3 13
data/crew.json
.json crew data
The genuinely useful oddities
| Module | For |
|---|---|
hashlib | Hashing: checksums, and password storage done properly |
secrets | Cryptographically safe random values for tokens and passwords |
uuid | Unique identifiers |
argparse | Command-line arguments (Lesson 27) |
logging | Grown-up printing (Lesson 28) |
unittest | Testing without installing anything (Lesson 29) |
sqlite3 | A real SQL database, in one file, built in (Lesson 45) |
csv | Spreadsheet data, with the quoting rules handled (Lesson 23) |
re | Regular expressions (Lesson 25) |
timeit | Measuring how slow something really is (Lesson 51) |
import hashlib, secrets, uuid
print(hashlib.sha256(b"swordfish").hexdigest()[:16])
print(len(secrets.token_hex(16)))
print(len(str(uuid.uuid4())))
b9f195c5cc7ef6af
32
36
There are over two hundred modules in there. Nobody knows them all, and nobody needs to. The valuable habit is not memorisation, it is suspicion: before you write forty lines to parse a date or shuffle a deck, spend thirty seconds searching the standard library index. The answer is there surprisingly often, and it has been tested by millions of people.
Where do modules come from?
import sys
# Python looks in these places, in order, for anything you import
for entry in sys.path[:3]:
print(repr(entry))
sys.path is the search list: your script's folder first, then the standard
library, then installed packages. This is why a file of your own called
random.py breaks everything: yours is found first, and the real one becomes
unreachable. Do not name your files after standard modules. Everyone does it once.
Dice statistics
Roll two dice a thousand times and report how often each total appeared, as a percentage, using the standard library. Seed with 1 so your answer matches.
Reveal solution
import random
from collections import Counter
random.seed(1)
totals = Counter(random.randint(1, 6) + random.randint(1, 6) for _ in range(1000))
for total in sorted(totals):
pct = totals[total] / 10
print(f"{total:2} {'#' * int(pct):18} {pct:.1f}%")
2 ## 2.6%
3 ##### 5.2%
4 ####### 7.0%
5 ########### 11.0%
6 ############## 14.7%
7 ################ 16.8%
8 ############# 13.6%
9 ########### 11.4%
10 ####### 7.7%
11 ####### 7.1%
12 ## 2.9%The shape is the famous bell curve: seven is the most common total because there are six ways to roll it and only one way to roll two.
Build a module
Write textstats.py with three functions (word count, average word length, most common word) and a __main__ block that demonstrates them. Then describe how another file would use it.
Reveal solution
"""Simple statistics about a piece of text."""
from collections import Counter
def word_count(text):
"""How many words in text."""
return len(text.split())
def average_word_length(text):
"""Mean length of the words in text, to one decimal place."""
words = text.split()
if not words:
return 0.0
return round(sum(len(w) for w in words) / len(words), 1)
def most_common_word(text):
"""The word that appears most often, lowercased."""
words = [w.strip(".,!?").lower() for w in text.split()]
return Counter(words).most_common(1)[0][0]
if __name__ == "__main__":
sample = "The rubber chicken. The pulley. The middle."
print(word_count(sample))
print(average_word_length(sample))
print(most_common_word(sample))
7
5.3
theAnother file would write import textstats then textstats.word_count(essay), and the demo block would stay silent.
Find the module
For each task, name the standard library module that already does it. No code required, just the search skill.
- Work out how many days until Christmas.
- Generate a secure password reset token.
- Read a spreadsheet exported as CSV where some fields contain commas.
- Zip up a folder for backup.
- Pretty-print a deeply nested dictionary.
- Find every file ending in .jpg in a folder tree.
Reveal solution
datetime, subtracting two dates gives atimedelta.secrets. Notrandom, which is predictable and explicitly documented as unsuitable for security.csv. Splitting on commas by hand breaks on the first quoted field, guaranteed.shutil.make_archive, orzipfilefor more control.pprint, orjson.dumps(x, indent=2).pathlib:Path('.').rglob('*.jpg').
The skill being trained here is the reflex to look before you build. It is worth more than any individual module.
Lists, tuples, dictionaries, sets, nested data, comprehensions, functions, arguments, scope and modules. You now have the toolbox that the rest of Python is built out of. Take the Level 2 quiz, and build something from the workshop before Level 3 turns your scripts into software.