Level 2 · The Toolbox

Modules and the Standard Library 📚

Everything so far has lived in one file. Real programs are many files, and Python ships with hundreds of them already written for you.

import

import random
import math

print(math.sqrt(16))
print(math.pi)

random.seed(42)                      # makes the "random" repeatable
print(random.randint(1, 100))
print(random.choice(["grog", "map", "sword"]))
4.0
3.141592653589793
82
grog

import math makes the module available, and you reach into it with a dot. random.seed(42) fixes the sequence so the same "random" numbers come out every time, which is invaluable for testing and for lessons like this one that promise you an exact output.

The four import styles

import math
from math import sqrt
from math import sqrt as square_root
import statistics as stats

print(math.sqrt(9))
print(sqrt(9))
print(square_root(9))
print(stats.mean([1, 2, 3, 4]))
3.0
3.0
3.0
2.5
StyleWhen
import xThe default. Always obvious where a name came from
from x import yWhen you use y constantly and the origin is obvious
import x as yFor long names, or by convention (import pandas as pd)
from x import *Never. It dumps unknown names into your file and hides collisions

Writing your own module

A module is just a .py file. Make pirate_tools.py:

"""Helpers for pirate arithmetic."""

INSULT_LIMIT = 8


def format_name(first, last):
    """Return a properly capitalised full name."""
    return f"{first.title()} {last.title()}"


def can_duel(insults):
    """True if this pirate knows enough insults to duel."""
    return insults >= INSULT_LIMIT

Then, in another file in the same folder:

import pirate_tools

print(pirate_tools.format_name("guybrush", "threepwood"))
print(pirate_tools.can_duel(9))
print(pirate_tools.INSULT_LIMIT)

That is the whole mechanism. When your file passes about two hundred lines, or when a group of functions clearly belong together, split them out. Future you will be grateful.

The __main__ guard

def add(a, b):
    return a + b


if __name__ == "__main__":
    print("Running directly, so here is a demo:")
    print(add(2, 3))
Running directly, so here is a demo:
5

Python sets __name__ to "__main__" when a file is run directly, and to the module's name when it is imported. So that block runs when you type python3 thing.py and stays quiet when another file imports it.

🎯 Why this matters

Without the guard, importing a module runs everything in it, including your test prints and, memorably for someone, an entire database migration. Put definitions at the top level and actions inside the guard. It is one of the strongest conventions in Python.

A tour of the batteries

This is the part people mean by "batteries included". Every one of these ships with Python and needs no installation.

Numbers, chance and time

import random, statistics, datetime

random.seed(7)
rolls = [random.randint(1, 6) for _ in range(10)]
print(rolls)
print(f"mean {statistics.mean(rolls)}, median {statistics.median(rolls)}")

born = datetime.date(1990, 10, 15)
print(f"Monkey Island was released on a {born.strftime('%A')}")
[3, 2, 4, 6, 1, 1, 5, 1, 3, 5]
mean 3.1, median 3.0
Monkey Island was released on a Monday

Text and data formats

import json, textwrap

data = {"ship": "Sea Monkey", "crew": ["Otis", "Meathook"]}
encoded = json.dumps(data)
print(encoded)
print(json.loads(encoded)["crew"][0])

long = "The rubber chicken with a pulley in the middle is arguably the finest item in adventure gaming."
print(textwrap.fill(long, width=45))
{"ship": "Sea Monkey", "crew": ["Otis", "Meathook"]}
Otis
The rubber chicken with a pulley in the
middle is arguably the finest item in
adventure gaming.

Collections and iteration tools

from collections import Counter, deque
from itertools import combinations

print(Counter("mississippi").most_common(2))

queue = deque(["a", "b"])
queue.appendleft("start")
print(list(queue))

print(list(combinations(["grog", "map", "sword"], 2)))
[('i', 4), ('s', 4)]
['start', 'a', 'b']
[('grog', 'map'), ('grog', 'sword'), ('map', 'sword')]

Files, paths and the system

import sys, pathlib, os

print(sys.version_info.major, sys.version_info.minor)
p = pathlib.Path("data") / "crew.json"
print(p)
print(p.suffix, p.stem, p.parent)
3 13
data/crew.json
.json crew data

The genuinely useful oddities

ModuleFor
hashlibHashing: checksums, and password storage done properly
secretsCryptographically safe random values for tokens and passwords
uuidUnique identifiers
argparseCommand-line arguments (Lesson 27)
loggingGrown-up printing (Lesson 28)
unittestTesting without installing anything (Lesson 29)
sqlite3A real SQL database, in one file, built in (Lesson 45)
csvSpreadsheet data, with the quoting rules handled (Lesson 23)
reRegular expressions (Lesson 25)
timeitMeasuring how slow something really is (Lesson 51)
import hashlib, secrets, uuid

print(hashlib.sha256(b"swordfish").hexdigest()[:16])
print(len(secrets.token_hex(16)))
print(len(str(uuid.uuid4())))
b9f195c5cc7ef6af
32
36
ENCYCLOPEDIA[Medium: Success]

There are over two hundred modules in there. Nobody knows them all, and nobody needs to. The valuable habit is not memorisation, it is suspicion: before you write forty lines to parse a date or shuffle a deck, spend thirty seconds searching the standard library index. The answer is there surprisingly often, and it has been tested by millions of people.

Where do modules come from?

import sys

# Python looks in these places, in order, for anything you import
for entry in sys.path[:3]:
    print(repr(entry))

sys.path is the search list: your script's folder first, then the standard library, then installed packages. This is why a file of your own called random.py breaks everything: yours is found first, and the real one becomes unreachable. Do not name your files after standard modules. Everyone does it once.

Exercise 1

Dice statistics

Roll two dice a thousand times and report how often each total appeared, as a percentage, using the standard library. Seed with 1 so your answer matches.

Reveal solution
import random
from collections import Counter

random.seed(1)

totals = Counter(random.randint(1, 6) + random.randint(1, 6) for _ in range(1000))

for total in sorted(totals):
    pct = totals[total] / 10
    print(f"{total:2}  {'#' * int(pct):18} {pct:.1f}%")
 2  ##                 2.6%
 3  #####              5.2%
 4  #######            7.0%
 5  ###########        11.0%
 6  ##############     14.7%
 7  ################   16.8%
 8  #############      13.6%
 9  ###########        11.4%
10  #######            7.7%
11  #######            7.1%
12  ##                 2.9%

The shape is the famous bell curve: seven is the most common total because there are six ways to roll it and only one way to roll two.

Exercise 2

Build a module

Write textstats.py with three functions (word count, average word length, most common word) and a __main__ block that demonstrates them. Then describe how another file would use it.

Reveal solution
"""Simple statistics about a piece of text."""

from collections import Counter


def word_count(text):
    """How many words in text."""
    return len(text.split())


def average_word_length(text):
    """Mean length of the words in text, to one decimal place."""
    words = text.split()
    if not words:
        return 0.0
    return round(sum(len(w) for w in words) / len(words), 1)


def most_common_word(text):
    """The word that appears most often, lowercased."""
    words = [w.strip(".,!?").lower() for w in text.split()]
    return Counter(words).most_common(1)[0][0]


if __name__ == "__main__":
    sample = "The rubber chicken. The pulley. The middle."
    print(word_count(sample))
    print(average_word_length(sample))
    print(most_common_word(sample))
7
5.3
the

Another file would write import textstats then textstats.word_count(essay), and the demo block would stay silent.

Exercise 3

Find the module

For each task, name the standard library module that already does it. No code required, just the search skill.

  1. Work out how many days until Christmas.
  2. Generate a secure password reset token.
  3. Read a spreadsheet exported as CSV where some fields contain commas.
  4. Zip up a folder for backup.
  5. Pretty-print a deeply nested dictionary.
  6. Find every file ending in .jpg in a folder tree.
Reveal solution
  1. datetime, subtracting two dates gives a timedelta.
  2. secrets. Not random, which is predictable and explicitly documented as unsuitable for security.
  3. csv. Splitting on commas by hand breaks on the first quoted field, guaranteed.
  4. shutil.make_archive, or zipfile for more control.
  5. pprint, or json.dumps(x, indent=2).
  6. pathlib: Path('.').rglob('*.jpg').

The skill being trained here is the reflex to look before you build. It is worth more than any individual module.

🎉 That is Level 2

Lists, tuples, dictionaries, sets, nested data, comprehensions, functions, arguments, scope and modules. You now have the toolbox that the rest of Python is built out of. Take the Level 2 quiz, and build something from the workshop before Level 3 turns your scripts into software.

+100 XP