The Secret Life of Python: The Import System

Source: Dev.to

How Python finds your code (and why it sometimes gets lost)

Timothy slumped in his chair, staring at his terminal with the exhausted frustration of someone who’d been debugging the same problem for three hours.

“I don’t understand,” he muttered, running the script for what felt like the hundredth time. “I deleted the file. I physically removed old_utils.py from my project directory. I even restarted my terminal. But when I run my code…”

import utils

print(utils.get_version())
# Output: "Version 1.0 - DEPRECATED"

“It’s still loading the old version! The file doesn’t even exist anymore!”

Margaret walked over, a knowing smile on her face. “Your Python isn’t haunted. It just has a very good memory. You’re fighting the import system—and losing because you don’t understand the rules.”

The Cache: sys.modules

When you type import utils, what does Python do first?

“It looks for utils.py in my directory?”

Python is lazy—it avoids work whenever possible. Before searching the file system, it checks a dictionary called sys.modules.

import sys

print(type(sys.modules))   # <class 'dict'>
print(len(sys.modules))     # e.g., 347 (varies)

# Show a few entries
for name in list(sys.modules.keys())[:10]:
    print(name)
# sys, builtins, _frozen_importlib, _imp, _thread, ...

sys.modules maps module names to module objects. If 'utils' is already a key, Python returns that cached object immediately—without touching the file system.

import sys
import json

print('json' in sys.modules)          # True
print(sys.modules['json'])            # <module 'json' from '.../json/__init__.py'>
print(json.dumps({'test': 'data'}))   # {"test": "data"}
print(dir(sys.modules['json'])[:5])   # ['JSONDecodeError', 'JSONDecoder', ...]

Because the cache persists for the lifetime of the process, changes to a module’s source file are invisible until the interpreter restarts or the cache is refreshed.

Demonstration of the problem

# Create a simple module file
with open('example.py', 'w') as f:
    f.write('''
def greet():
    return "Hello, version 1"
''')

import example
print(example.greet())   # Hello, version 1

# Modify the file
with open('example.py', 'w') as f:
    f.write('''
def greet():
    return "Hello, version 2"
''')

# Re‑import (still cached)
import example
print(example.greet())   # Hello, version 1
print('example' in sys.modules)  # True

Reloading Modules

The Dangerous Way: Delete from sys.modules ❌

import sys

del sys.modules['example']   # Remove cached entry

import example               # Reloads from disk
print(example.greet())       # Hello, version 2

Why it’s dangerous: Other parts of the program may still hold references to the old module object, leading to inconsistent behavior.

import sys
if 'example' in sys.modules:
    del sys.modules['example']

# Write version 1
with open('example.py', 'w') as f:
    f.write('def greet(): return "Version 1"')

import example as ex1
from example import greet as greet1

# Replace file with version 2 and delete cache
del sys.modules['example']
with open('example.py', 'w') as f:
    f.write('def greet(): return "Version 2"')

import example as ex2
from example import greet as greet2

print(greet1())   # Version 1 (old reference)
print(greet2())   # Version 2
print(ex1 is ex2) # False

The Proper Way: importlib.reload() ✅

import importlib
import example

# Edit the file to version 2
with open('example.py', 'w') as f:
    f.write('def greet(): return "Version 2"')

importlib.reload(example)   # Updates the existing module object
print(example.greet())      # Version 2
print(id(example))          # Same memory address as before

reload() updates the module in place, so existing references see the new code. However, any from example import greet statements made before the reload will still point to the old function object.

The Map: sys.path

If a module isn’t already cached, Python searches for it on disk using sys.path, a list of directories that define the import search order.

import sys
print(sys.path)

Typical entries include:

1. The directory containing the script that was used to invoke the Python interpreter.
2. PYTHONPATH environment variable entries (if set).
3. Standard library directories.
4. Site‑packages (third‑party libraries).

Python stops at the first match it finds. Manipulating sys.path (e.g., inserting a project’s src folder at the front) can control which version of a module is imported.

import sys
sys.path.insert(0, '/path/to/my/project')
import mymodule   # Now Python looks in '/path/to/my/project' first

Takeaway:
sys.modules is Python’s in‑memory cache of imported modules; it makes imports fast but can cause stale code to persist. Use importlib.reload() for a safe reload, and be aware of sys.path when Python searches for modules on disk. When debugging import issues, always check whether the module is already cached.