Python File Handling Mastery: Ditch Common Pitfalls with Pathlib & Context Managers

Source: Dev.to

Introduction

File handling bugs can be elusive—often they don’t crash your program but silently corrupt data or leak resources. These issues become especially painful when a script works on your local machine but fails on a server due to encoding mismatches or exhausted file descriptors. Mastering Python’s file I/O, especially with modern tools like pathlib and context managers, leads to more reliable and cross‑platform code.

Using pathlib and Context Managers

The classic approach:

f = open('file.txt')
content = f.read()
f.close()

fails to guarantee that the file is closed if an exception occurs. The idiomatic solution is to use a context manager (with) together with pathlib for OS‑agnostic paths:

from pathlib import Path

file_path = Path("example.txt")
with file_path.open(encoding="utf-8") as f:
    content = f.read()
    print(content)

Specifying encoding="utf-8" ensures consistent behavior across Windows, macOS, and Linux.

Reading Files Efficiently

For large files, avoid loading the entire content into memory:

with file_path.open(encoding="utf-8") as f:
    for line in f:
        process(line.strip())

Iterating line‑by‑line streams the data, allowing you to handle files larger than available RAM.

Writing Files Safely

Always open files within a with block to guarantee flushing and closing:

from pathlib import Path

with Path("output.txt").open("w", encoding="utf-8") as f:
    f.write("Hello, Python world!\n")

If you forget the mode, Python defaults to read‑only, raising io.UnsupportedOperation. For appending logs:

with Path("log.txt").open("a", encoding="utf-8") as f:
    f.write(f"Error occurred at {timestamp}\n")

Working with Structured Formats

CSV

Use the csv module instead of manual string splitting:

import csv
from pathlib import Path

with Path("data.csv").open(encoding="utf-8", newline="") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row["column_name"])  # Works even if column order changes

The newline="" argument prevents extra blank lines on Windows.

JSON

Serialize and deserialize JSON safely:

import json
from pathlib import Path

data = {"key": "value"}
with Path("config.json").open("w", encoding="utf-8") as f:
    json.dump(data, f, indent=4)  # Human‑readable output

Common Pitfalls and Fixes

Further Reading

For an in‑depth guide covering production‑ready techniques, error handling, and advanced tools, see Mastering Python File I/O: How to Read and Write Files Easily.