Python by Structure: Context Managers and the 'with' Statement

Source: Dev.to

The Problem with Manual File Handling

Timothy was debugging a file‑processing script when Margaret noticed a “too many open files” error in the logs.

def process_config():
    f = open('config.txt', 'r', encoding='utf-8')
    data = f.read()
    # Process data...
    if 'error' in data:
        return None  # Oops – file never closed!
    f.close()
    return data

If the early return is taken, the file stays open. Relying on garbage collection is unreliable and can exhaust system resources.

Traditional Solution: try/finally

def process_config():
    f = open('config.txt', 'r', encoding='utf-8')
    try:
        data = f.read()
        if 'error' in data:
            return None
        return data
    finally:
        f.close()

The finally block guarantees that f.close() runs regardless of how the function exits, but the extra boilerplate is cumbersome.

The with Statement

Python’s with statement automates the cleanup:

def process_config():
    with open('config.txt', 'r', encoding='utf-8') as f:
        data = f.read()
        if 'error' in data:
            return None
        return data
# File is automatically closed here

The with block has a clear entry point (opening the file) and an exit point (the end of the indented block). Python guarantees that the file is closed when the block is left, whether by normal execution, an early return, or an exception.

How with Works: Context Managers

Objects used in a with statement must implement two special methods:

• __enter__(self): Called at the start of the block; its return value is bound to the variable after as.
• __exit__(self, exc_type, exc_val, exc_tb): Called when the block ends; receives exception information if one was raised.

Simple Context Manager Example

class FileLogger:
    def __init__(self, filename):
        self.filename = filename
        self.file = None

    def __enter__(self):
        print(f"Opening {self.filename}")
        self.file = open(self.filename, 'w', encoding='utf-8')
        return self.file

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Closing {self.filename}")
        if self.file:
            self.file.close()
        return False  # Do not suppress exceptions

# Usage
with FileLogger('output.log') as log:
    log.write('Starting process\n')
    log.write('Processing data\n')
# File automatically closed here

Output

Opening output.log
Closing output.log

Exception Handling in __exit__

If an exception occurs inside the with block, Python passes the exception details to __exit__. The method can decide whether to suppress the exception by returning True.

class ErrorLogger:
    def __enter__(self):
        print("Entering context")
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type is not None:
            print(f"Exception occurred: {exc_type.__name__}: {exc_val}")
            return True   # Suppress the exception
        print("Exiting normally")
        return False

with ErrorLogger():
    print("Working...")
    raise ValueError("Something went wrong!")
print("Continuing after exception")

Output

Entering context
Working...
Exception occurred: ValueError: Something went wrong!
Continuing after exception

Using Multiple with Statements

You can nest with statements or combine them in a single line (Python 3.1+).

# Nested
with open('input.txt', 'r', encoding='utf-8') as infile:
    with open('output.txt', 'w', encoding='utf-8') as outfile:
        outfile.write(infile.read())

# Combined
with open('input.txt', 'r', encoding='utf-8') as infile, \
     open('output.txt', 'w', encoding='utf-8') as outfile:
    outfile.write(infile.read())

Both files are guaranteed to close, in reverse order of acquisition (the last opened closes first).

Common Use Cases for Context Managers

• File operations – guaranteed close
• Database connections – guaranteed commit/rollback
• Locks and semaphores – guaranteed release
• Network connections – guaranteed disconnect
• Temporary state changes – guaranteed restore
• Any resource that needs cleanup

Example: Threading Lock

import threading

lock = threading.Lock()

# Without context manager – risky
lock.acquire()
try:
    # Critical section
    pass
finally:
    lock.release()

# With context manager – safe
with lock:
    # Critical section
    pass

The lock is automatically released even if an exception occurs inside the critical section.