The Async Core: Understanding Eventlet and Gevent in Flask-SocketIO

Source: Dev.to

The Blocking Problem

To understand the necessity of Eventlet and Gevent, one must first analyze why standard threading fails at scale. In a traditional WSGI deployment (e.g., Gunicorn with the sync or gthread worker), concurrency is mapped 1:1 with OS‑level threads or processes.

If a Flask‑SocketIO server were to use standard OS threads to manage WebSocket connections, it would encounter two primary bottlenecks:

• Memory Overhead: A standard Linux thread typically reserves a stack size of 8 MB. While virtual memory management mitigates the immediate physical cost, the commit charge and kernel structures (Thread Control Blocks) still impose a heavy footprint. Spawning 10,000 threads for 10,000 idle WebSocket clients would theoretically require ~80 GB of addressable memory space, leading to resource exhaustion long before CPU limits are reached.
• Context Switching Latency: The OS kernel scheduler manages thread execution using preemptive multitasking. As the number of threads rises, the scheduler spends an increasing percentage of CPU cycles simply deciding which thread to run next (context switching). This “thrashing” degrades throughput significantly.

Furthermore, Python’s Global Interpreter Lock (GIL) ensures that only one thread executes Python bytecode at a time. While I/O operations (like waiting for a socket message) release the GIL, the overhead of managing thousands of threads remains prohibitive.

Greenlets Explained

Eventlet and Gevent solve the blocking problem by implementing coroutines (cooperative user‑space threads) via the greenlet C‑extension library. Unlike OS threads, greenlets are managed entirely in user space without kernel intervention.

The Mechanism: Stack Slicing

The technical brilliance of greenlets lies in how they manage the call stack. The CPython interpreter uses the standard C stack for function calls. To pause a function in the middle of execution (which is necessary when a function blocks on I/O), the state of the stack must be preserved.

When a greenlet switches context (yields):

1. Stack Slicing: The library copies the current greenlet’s portion of the C stack from the CPU’s stack pointer into a buffer on the heap.
2. Stack Restoration: It copies the target greenlet’s saved stack from the heap back onto the C stack.
3. Instruction Pointer Update: It updates the instruction pointer to resume execution where the target greenlet left off.

This “trampoline” mechanism allows Python to pause execution deep inside nested function calls—even across C‑extension boundaries—without the C stack growing indefinitely.

Efficiency

Because greenlets share the same OS thread and process memory, a context switch involves only a memcpy operation rather than a system call. This reduces the context‑switch time from microseconds (OS threads) to nanoseconds. Additionally, a greenlet’s initial stack size is minuscule (often just a few kilobytes), allowing a single process to host tens of thousands of concurrent greenlets.

Monkey Patching: The “Magic” Integration

Standard Python libraries (like socket and time) are blocking. If you call time.sleep(10) or socket.recv() in a standard Flask route, the entire OS thread freezes. Since Eventlet/Gevent run on a single OS thread, one blocking call would halt the entire server, freezing all connected clients.

How It Works

Monkey patching dynamically modifies the standard library at runtime. When you execute eventlet.monkey_patch() or gevent.monkey.patch_all(), the library modifies sys.modules:

• Replaces the standard socket class with a “green” socket class.
• Replaces threading.Thread with a greenlet‑based equivalent.

Execution Flow of a “Green” Socket

1. Intercept: User code calls socket.recv(). Because of monkey patching, this invokes the Gevent/Eventlet version, not the OS version.
2. Register: The green socket registers a callback with the Hub (the central event loop). This watcher tells the Hub: “Wake me up when file descriptor X has data to read.”
3. Yield: The green socket calls greenlet.switch(), pausing the current request’s execution and yielding control to the Hub.
4. Wait: The Hub uses a high‑performance, non‑blocking polling mechanism (typically epoll on Linux or kqueue on macOS) to check for I/O events across all file descriptors.
5. Resume: When data arrives on the socket, the Hub sees the event, triggers the callback, and switches execution back to the original greenlet.

To the Flask developer, the code looks synchronous (data = sock.recv(1024)). Under the hood, the execution is asynchronous and non‑blocking.

The Risks of Monkey Patching

While powerful, monkey patching introduces significant engineering risks:

• C‑Extension Incompatibility: Libraries written in C that bypass the Python socket API (e.g., certain database drivers or old gRPC versions) perform direct OS system calls. These cannot be patched. If such a library blocks, it blocks the entire loop.
• Order of Operations: Patching must occur before any other modules import socket or threading. Late patching can result in a “split brain” scenario where some parts of the app use green sockets and others use blocking OS sockets, leading to deadlocks.

Choosing Your Fighter: Eventlet vs. Gevent vs. Threading

When configuring Flask‑SocketIO, you must choose an async_mode.

Threading

• Concurrency Model: Standard OS threads.
• Pros: Maximum compatibility. No monkey patching required. Works with all third‑party libraries.
• Cons: Poor scalability. Capable of handling hundreds of clients, but fails at thousands due to memory and context‑switching overhead.
• Use Case: Development, debugging, or low‑traffic internal tools.

Eventlet

• Concurrency Model: Greenlets.
• Architecture: Historically the default for Flask‑SocketIO. Uses a pure‑Python hub (mostly) wrapping epoll.
• Status (2024/2025): Deprecated. The Eventlet project is currently in maintenance mode (“life support”). New feature development has stalled, and compatibility with newer Python versions (3.10+) has historically lagged.
• Performance: High, but generally slightly slower than Gevent in raw throughput benchmarks.
• Use Case: Legacy applications. New projects should avoid Eventlet.

Gevent

• Concurrency Model: Greenlets.
• Architecture: Built on top of libev (a highly optimized C library) and Cython.
• Status: Active. Gevent remains well‑maintained and robust.
• Performance: Very high. The C‑based hub and loop provide superior performance and lower latency compared to Eventlet.
• Use Case: The recommended choice for production Flask‑SocketIO deployments requiring high concurrency.

Conceptual Benchmark Comparison

Under a workload of 5,000 concurrent WebSocket connections sending “heartbeat” messages… (benchmark details omitted).