The Secret Life of Python: The Lazy Baker

Source: Dev.to

Why Your Python Code Runs Out of Memory, and How yield Fixes It

The library smelled faintly of ozone and burning plastic. Timothy was sitting in front of his laptop, but he wasn’t typing. He was staring blankly at a spinning cursor. The fan was screaming.

“It crashed again,” Timothy groaned.

Margaret marked her place in her book and looked over.

“What are you trying to build?”

“I need to simulate a dataset,” Timothy explained. “I need one hundred million random sensor readings to test my new analysis script. I wrote a function to generate them and put them in a list.”

He pointed to the frozen code on his screen.

Timothy’s “Eager” Baker

def produce_readings(n):
    results = []
    print("Starting production...")
    for i in range(n):
        # Imagine complex calculation here
        results.append(i * 1.5)
    print("Production complete!")
    return results

# Asking for 100 million items
data = produce_readings(100_000_000)

for reading in data:
    process(reading)

“I hit Run,” Timothy said, “and then the computer freezes. It never even gets to the process loop. It just runs out of memory.”

Margaret nodded knowingly.

“You have hired the Eager Baker.”

The Bakery Analogy

“Imagine a bakery. You walk in and ask for ten thousand loaves of bread.
The Eager Baker says, ‘Yes, sir!’ He immediately mixes ten thousand dough balls, bakes ten thousand loaves, piles them up, fills the counter, the floor, the back room… The bakery is bursting. He cannot give you a single loaf until all ten thousand are finished.”

“That is what your list is doing,” Margaret said. “You asked for one hundred million numbers. Python is trying to build them all, store them all in RAM, and then let you use the first one. Your computer does not have a warehouse big enough.”

“So I can’t process big data?” Timothy asked.

“You can,” Margaret smiled. “But you must fire the Eager Baker. You need the Lazy Baker.”

Margaret’s “Lazy” Baker (Generator)

def produce_readings_lazy(n):
    print("Starting production...")
    for i in range(n):
        yield i * 1.5
    print("Production complete!")

“Where is the list?” Timothy asked. “Where is results.append?”

“There is no list,” Margaret said. “There is only yield. This keyword turns the function into a generator.”

Using the Generator

# Create the generator object
baker = produce_readings_lazy(100_000_000)

print(f"Baker object: {baker}")

# Ask for the first loaf
print(next(baker))

# Ask for the second loaf
print(next(baker))

Output

Baker object: <generator object produce_readings_lazy at 0x...>
Starting production...
0.0
1.5

“Do you see?” Margaret whispered. “When we created baker, nothing happened. No code ran. No memory was used. The baker was waiting.”

“When we called next(baker), he woke up. He ran until he hit yield. He handed you one value, then paused, remembering exactly where he left off.”

“So he bakes them one at a time?” Timothy asked.

“On demand,” Margaret confirmed. “He bakes one, hands it to you, and waits for you to come back. He never stores more than one item at a time.”

Memory Comparison

Margaret imported sys to weigh the objects.

import sys

# The Eager List (small scale to avoid crashing)
eager_list = [i * 1.5 for i in range(1_000_000)]

# The Lazy Generator (same scale)
lazy_gen = (i * 1.5 for i in range(1_000_000))

print(f"List Size in Bytes:      {sys.getsizeof(eager_list)}")
print(f"Generator Size in Bytes: {sys.getsizeof(lazy_gen)}")

Result

List Size in Bytes:      8448728
Generator Size in Bytes: 104

“The list is ~8 MB. The generator is… 104 bytes? That’s smaller than a tweet.”

“Because the list holds a million numbers. The generator only holds the recipe for how to get the next number. It doesn’t matter if you ask for a million items or a trillion.”

“He can even bake forever,” Margaret added. “If you wrote while True: yield n, he would never run out of dough. He would bake infinitely, as long as you only ask for one at a time.”

The Catch

“One catch,” Margaret warned. “The Lazy Baker has no shelf space. Once he hands you a loaf, he forgets it. You cannot ask for ‘loaf number 5’ again unless you restart the whole process. Generators are a one‑way street.”

“But,” she noted, typing quickly, “you can chain them together into pipelines.”

# The Pipeline: Data flows through, one item at a time
raw_data   = (read_sensor() for _ in range(1_000_000))
clean_data = (x for x in raw_data if x > 0)
final_data = (process(x) for x in clean_data)

“No lists,” Timothy marveled. “Just a stream.”

Quick Reference (Margaret’s Notebook)

• The Eager Approach (Lists) – Computes everything upfront. Fast random access (my_list[5]), but eats memory.
• The Lazy Approach (Generators) – Computes one item at a time.
• yield Keyword – Pauses a function, returns a value, and saves its state.
• Generator Expressions – (x for x in data). Like list comprehensions, but with ().
• Trade‑off – Generators save massive memory, but they are single‑use only. You cannot index them (gen[0]) or iterate over them twice.
• Use Cases – Massive datasets, infinite streams, data pipelines.

Timothy replaced his brackets with parentheses and hit Run. The fan on his laptop instantly quieted down. The data began to flow.

“I never thought I’d say this,” Timothy grinned, “but I love laziness.”

“Efficiency, Timothy,” Margaret corrected, sipping her tea. “We call it efficiency.”

In the next episode, Margaret and Timothy will face “The Magic Bookshelf”—where they discover a data structure that d…

Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.