Nested Data 🪆

The shape you will meet a thousand times

crew = [
    {"name": "Guybrush", "role": "captain", "skills": ["insults", "sailing"]},
    {"name": "Elaine", "role": "governor", "skills": ["politics", "swordplay", "rescue"]},
    {"name": "Otis", "role": "lookout", "skills": []},
]

print(len(crew))
print(crew[0]["name"])
print(crew[1]["skills"][0])
print(len(crew[2]["skills"]))

3
Guybrush
politics
0

A list of dictionaries. Every API response, every CSV file, every database query result and every JSON file you ever open will look roughly like this. Read the access left to right: crew[1] is a dictionary, ["skills"] is its list, [0] is the first of those.

Looping over it

crew = [
    {"name": "Guybrush", "role": "captain", "skills": ["insults", "sailing"]},
    {"name": "Elaine", "role": "governor", "skills": ["politics", "swordplay", "rescue"]},
    {"name": "Otis", "role": "lookout", "skills": []},
]

for member in crew:
    skills = ", ".join(member["skills"]) or "none listed"
    print(f"{member['name']:10} ({member['role']:9}) {skills}")

Guybrush   (captain  ) insults, sailing
Elaine     (governor ) politics, swordplay, rescue
Otis       (lookout  ) none listed

x or y gives you x unless it is falsy, in which case y. An empty string is falsy, so an empty skill list produces "none listed". That is the truthiness rule from Lesson 6 doing real work.

Filtering and summarising

crew = [
    {"name": "Guybrush", "role": "captain", "pay": 100},
    {"name": "Elaine", "role": "governor", "pay": 250},
    {"name": "Otis", "role": "lookout", "pay": 40},
    {"name": "Meathook", "role": "lookout", "pay": 45},
]

lookouts = [m for m in crew if m["role"] == "lookout"]
print([m["name"] for m in lookouts])

total = sum(m["pay"] for m in crew)
print(f"Total wages: {total}")

richest = max(crew, key=lambda m: m["pay"])
print(f"Best paid: {richest['name']}")

by_pay = sorted(crew, key=lambda m: m["pay"], reverse=True)
for m in by_pay:
    print(f"  {m['pay']:4}  {m['name']}")

['Otis', 'Meathook']
Total wages: 435
Best paid: Elaine
   250  Elaine
   100  Guybrush
    45  Meathook
    40  Otis

Those square-bracket lines are list comprehensions, which is the next lesson. Even before you can write them, you can read them: "the name of m, for every m in crew".

Grouping

crew = [
    {"name": "Guybrush", "role": "captain"},
    {"name": "Otis", "role": "lookout"},
    {"name": "Meathook", "role": "lookout"},
]

by_role = {}
for member in crew:
    role = member["role"]
    by_role.setdefault(role, []).append(member["name"])

for role, names in by_role.items():
    print(f"{role}: {', '.join(names)}")

captain: Guybrush
lookout: Otis, Meathook

setdefault(key, []) means "give me the list at this key, creating an empty one first if needed". It is the standard grouping idiom. The tidier alternative:

from collections import defaultdict

pairs = [("captain", "Guybrush"), ("lookout", "Otis"), ("lookout", "Meathook")]

by_role = defaultdict(list)
for role, name in pairs:
    by_role[role].append(name)

print(dict(by_role))

{'captain': ['Guybrush'], 'lookout': ['Otis', 'Meathook']}

Navigating safely when data is missing

response = {"user": {"profile": {"name": "Elaine"}}}
broken = {"user": {}}

print(response["user"]["profile"]["name"])

# this would raise KeyError on `broken`
name = broken.get("user", {}).get("profile", {}).get("name", "unknown")
print(name)

Elaine
unknown

Chained .get() calls with {} as the default let you walk a deep structure without exploding when a level is missing. Real API responses are missing fields constantly, and this pattern is the difference between a script that survives contact with reality and one that does not.

Shallow copies bite here

import copy

original = {"name": "ship", "crew": ["Otis", "Meathook"]}

shallow = original.copy()
deep = copy.deepcopy(original)

shallow["crew"].append("Stowaway")

print("original:", original["crew"])
print("deep:    ", deep["crew"])

original: ['Otis', 'Meathook', 'Stowaway']
deep:     ['Otis', 'Meathook']

Printing nested data readably

import json

game = {
    "title": "Monkey Island",
    "crew": [{"name": "Guybrush", "insults": 8}],
}

print(game)
print()
print(json.dumps(game, indent=2))

{'title': 'Monkey Island', 'crew': [{'name': 'Guybrush', 'insults': 8}]}

{
  "title": "Monkey Island",
  "crew": [
    {
      "name": "Guybrush",
      "insults": 8
    }
  ]
}

json.dumps(x, indent=2) is the fastest way to see the shape of confusing data. Keep it in your fingers. There is also pprint in the standard library, which does the same job while keeping Python's own notation.

films = [
    {"title": "Monkey Island", "ratings": [9, 8, 10]},
    {"title": "Disco Elysium", "ratings": [10, 10, 9]},
    {"title": "Grim Fandango", "ratings": [9, 9, 9]},
]

films = [
    {"title": "Monkey Island", "ratings": [9, 8, 10]},
    {"title": "Disco Elysium", "ratings": [10, 10, 9]},
    {"title": "Grim Fandango", "ratings": [9, 9, 9]},
]

def average(film):
    return sum(film["ratings"]) / len(film["ratings"])


for film in sorted(films, key=average, reverse=True):
    print(f"{film['title']:15} {average(film):.1f}")

Disco Elysium   9.7
Monkey Island   9.0
Grim Fandango   9.0

people = {
    "Guybrush": ["insults", "sailing"],
    "Elaine": ["politics", "sailing"],
    "Otis": ["complaining"],
}

people = {
    "Guybrush": ["insults", "sailing"],
    "Elaine": ["politics", "sailing"],
    "Otis": ["complaining"],
}

by_skill = {}
for person, skills in people.items():
    for skill in skills:
        by_skill.setdefault(skill, []).append(person)

for skill in sorted(by_skill):
    print(f"{skill:12} {', '.join(by_skill[skill])}")

complaining  Otis
insults      Guybrush
politics     Elaine
sailing      Guybrush, Elaine

users = [
    {"name": "Elaine", "address": {"city": "Melee"}},
    {"name": "Otis", "address": {}},
    {"name": "Guybrush"},
]

users = [
    {"name": "Elaine", "address": {"city": "Melee"}},
    {"name": "Otis", "address": {}},
    {"name": "Guybrush"},
]

for user in users:
    city = user.get("address", {}).get("city", "unknown")
    print(f"{user['name']:10} {city}")

Elaine     Melee
Otis       unknown
Guybrush   unknown