Python and Memory Management, Part 2: Heap Memory

Source: Dev.to

Python’s Memory Sections: Where Objects Live

When you run a Python program the operating system creates several memory regions for the process.
On Linux you can inspect those regions with the script below (save it to a file and execute it):

#!/usr/bin/env python3
import os, sys

def main():
    pid = os.getpid()
    with open(f"/proc/{pid}/maps") as f:
        for line in f:
            line = line.strip()
            parts = line.split()
            if len(parts) >>>> Found: {name} ({id(obj):#x}) in this block")

# Objects to track
vars = {
    "True": True,
    "small number": 4,
    "large number": 2**123,
    "small string": "a",
    "large string": "a" * 2000,
    "int class": int,
    "sys module": sys,
    "main function": main,
}

if __name__ == "__main__":
    main()

The script reads /proc/<pid>/maps, prints each memory block, and reports whether any of the objects in vars reside in that block.

Below is a brief overview of the most common memory regions you will see.

1. Executable‑Code Region (Read‑Only)

These blocks contain the Python interpreter binary and any shared libraries it loads.
Typical permissions are r—p (read‑only) or r‑xp (read‑execute).

00400000-0041f000 r--p 00000000 08:11 545993 /usr/bin/python3.14
0041f000-006d2000 r-xp 0001f000 08:11 545993 /usr/bin/python3.14
006d2000-00945000 r--p 002d2000 08:11 545993 /usr/bin/python3.14
...
7f3d6c6ab000-7f3d6c6d1000 r--p 00000000 08:11 541316 /usr/lib/x86_64-linux-gnu/libc.so.6
7f3d6c6d1000-7f3d6c827000 r-xp 00026000 08:11 541316 /usr/lib/x86_64-linux-gnu/libc.so.6

These sections are part of the Python binary or the shared objects it depends on.

2. Data Segment (Read‑Write)

This region holds read‑write data that is part of the Python executable itself, such as the singleton objects True, small integers, and built‑in types that are pre‑allocated when the interpreter starts.

00946000-00a85000 rw-p 00545000 08:11 545993 /usr/bin/python3.14
  >>>>> Found: True (0x95a0a0) in this block
  >>>>> Found: small number (0xa5bb08) in this block
  >>>>> Found: int class (0x958cc0) in this block

3. Heap Region (Dynamic Allocation)

The OS‑managed heap (brk/sbrk) is used for objects that require more than 512 bytes.
These allocations are performed with the C library’s malloc.

1039c000-10449000 rw-p 00000000 00:00 0  [heap]
  >>>>> Found: large string (0x10406840) in this block

4. Anonymous Memory Mappings (Python’s pymalloc)

Python’s private allocator (pymalloc) obtains large anonymous memory blocks called arenas.
Each arena is subdivided into pools that serve small objects.

>>>> Found: small string (0x7f75e62ec730) in this block
>>>> Found: main function (0x7f75e62d85e0) in this block
>>>> Found: large number (0x7f75e67e3f30) in this block
>>>> Found: sys module (0x7f75e67a2ca0) in this block

For a deeper dive into arenas, pools, and the overall strategy, see:

• Python memory management – Rushter
• Memory allocation strategies – GingerBill

5. Stack Region (Function Calls)

The stack holds C‑level call frames and local variables. Python objects themselves never live on the stack.

7fff20372000-7fff20394000 rw-p 00000000 00:00 0  [stack]

6. Kernel‑Interface Regions

These mappings ([vvar], [vdso]) are used by the kernel to provide fast system‑call interfaces.
They are not directly relevant to Python’s object allocation.

7fff203b8000-7fff203bc000 r--p 00000000 00:00 0  [vvar]
7fff203bc000-7fff203be000 r-xp 00000000 00:00 0  [vdso]

How Python Manages Garbage Collection with Reference Counting

Every CPython object contains a reference count (ob_refcnt). When the count drops to zero, the object’s memory is reclaimed.

/* CPython’s PyObject structure */
typedef struct _object {
    Py_ssize_t ob_refcnt;    /* reference count */
    PyTypeObject *ob_type;  /* pointer to type object */
    /* ... type‑specific fields follow ... */
} PyObject;

A Small Demo

import sys
import weakref

class Node:
    def __init__(self, name):
        self.name = name
        self.child = None

    def __repr__(self):
        return f"Node({self.name!r})"

# Create two nodes
a = Node('a')          # refcnt = 1
b = Node('b')          # refcnt = 1

# A weak reference does **not** increase the refcount
a_weak = weakref.ref(a)

print('a refcnt:', sys.getrefcount(a))   # usually 2 (one from getrefcount's argument)
print('a weakref alive?', a_weak() is not None)

# Create a reference cycle
a.child = b
b.child = a

print('a refcnt after cycle:', sys.getrefcount(a))
print('b refcnt after cycle:', sys.getrefcount(b))

# Break the strong references
del a
del b

# The objects are still alive because the cycle keeps their refcnt > 0,
# but the garbage collector will eventually collect them.
import gc
gc.collect()

• sys.getrefcount() shows the current reference count (the call itself adds a temporary reference).
• A weak reference (weakref.ref) lets you refer to an object without affecting its count.
• When a reference cycle is created (a.child = b; b.child = a), the reference counts never reach zero, so CPython’s cyclic‑garbage collector steps in to break the cycle and free the memory.

In the next section we’ll explore the details of that cyclic‑garbage collector.

Reference Counting and Weak References

import sys, weakref, gc

class Node:
    child = None

a = Node()
b = Node()
b_ref = weakref.ref(b)

# Create a cycle
a.child = b
b.child = a          # reference count = 2

print("All refs a, b, and a_ref assigned", f"{sys.getrefcount(a_ref())=}")  
# 2 (the two strong refs) + 1 for the argument to getrefcount = 3

del a
print("After variable a deleted", f"{sys.getrefcount(a_ref())=}")   # reference count = 1

del b
print("After both variables deleted", f"{sys.getrefcount(a_ref())=}")  
# reference count = 0, object freed – but `sys.getrefcount` still shows 1!
print("After both variables deleted", f"{sys.getrefcount(b_ref())=}")  
# same result

What’s happening?

Even after deleting a and b, a weak reference can still retrieve the object.
Add the following lines to see what is still referring to a:

import gc   # Garbage Collector Interface Module
print(f"Referrers to a: {gc.get_referrers(a_ref())}")

The output shows two references:

• a.child → b
• b.child → a

Although we deleted the variable a, the attribute a.child still exists, keeping b alive, and vice‑versa.

How does Python handle such cycles?

Python runs a mark‑and‑sweep garbage collector that:

1. Starts from a set of root objects (globals, locals, etc.).
2. Recursively marks every object reachable from those roots.
3. Sweeps away objects that were not marked (i.e., unreachable).

Running this algorithm after every assignment would be far too expensive, so CPython runs it periodically, based on generation thresholds.

import gc
print(f"GC generations thresholds: {gc.get_threshold()}")   # Default: (700, 10, 10)

Generations

This follows the generational hypothesis: most objects die young, while long‑lived objects tend to stay alive.

Because the cycle a ↔ b is still in generation 0 when we delete the variables, the collector hasn’t run yet, so the weak reference can still access the objects.

Forcing a collection

import sys, weakref, gc

class Node:
    child = None

a = Node()
b = Node()
b_ref = weakref.ref(b)

a.child = b
b.child = a

print("All refs a, b, and a_ref assigned", f"{sys.getrefcount(a_ref())=}")

del a
print("After variable a deleted", f"{sys.getrefcount(a_ref())=}")

del b
print("After both variables deleted", f"{sys.getrefcount(b_ref())=}")

# Force collection of generations 0, 1, and 2
n_collected = gc.collect(2)
print(f"Collected {n_collected} objects")   # → 2 objects

The GIL’s role in garbage collection

The Global Interpreter Lock (GIL) makes reference‑count updates atomic, ensuring thread‑safe garbage collection. Without the GIL, concurrent reference‑count modifications could cause race conditions, memory corruption, and break the collector.

---

I hope you enjoyed this deep dive into CPython’s memory management. Understanding reference counting, weak references, and the generational garbage collector helps explain many subtle behaviours you may encounter while writing Python code.