MongoDB Replica Set에서 Secondary Nodes를 읽어 읽기 용량을 늘리는 함정

Source: Dev.to

Imagine we are responsible for managing the MongoDB cluster that supports our country’s national financial payment system, similar to Pix in Brazil. Our application was designed to be read‑heavy, with one write operation for every 20 read operations.

With Black Friday approaching, a critical period for our national financial payment system, we have been entrusted with creating a scaling plan for our cluster to handle the increased demand. Given that our system is read‑heavy, we are exploring ways to enhance the read performance and capacity of our cluster.

We are responsible for a system that powers roughly 60 % of all national transactions, so ensuring the highest availability of this MongoDB cluster is absolutely critical—it is the backbone of our economy.

AI‑generated recommendations

When we asked several AI models (GPT‑5, Grok Code Fast 1, Claude Sonnet 4, Gemini 2.5 Pro) “How can I increase read performance and capacity in a MongoDB replica set cluster?”, a common suggestion was to distribute reads to secondary nodes using the readPreference setting and to add more secondary nodes.

All models correctly warned that reads from secondaries can be stale, because replication is not instantaneous.

Example scenario

Assume a replica set with three nodes: one primary and two secondaries. Each node can handle up to 100 reads / s.

If a network outage removes one node, the remaining two nodes must each handle 120 reads / s, exceeding their capacity and risking further failures that could bring down the entire cluster.

Read capacity vs. read performance

• Read capacity – the number of read operations the cluster can sustain without overloading hardware or significantly increasing latency.
• Read performance – how quickly an individual read operation is fulfilled.

Using secondary nodes to boost read capacity can reduce availability: a single node failure may overload the remaining nodes.

Strategies to improve read performance without sacrificing availability

Proximity

Place secondary nodes closer to the application to reduce network latency.

Caching

Execute the same queries on the same node consistently, allowing its cache to retain the needed data and speed up subsequent reads.

Scaling the cluster safely

The most reliable way to increase read capacity while preserving high availability is to scale the cluster, either vertically (scale‑up) or horizontally (scale‑out).

Vertical scaling (scale‑up)

Increase resources of existing nodes (CPU, RAM, storage, IOPS).

Advantages

• Operational simplicity – no changes to data distribution or query routing.
• Minimal application changes – connection strings and query patterns usually stay the same.
• Immediate performance improvement, especially for CPU‑or memory‑bound workloads.

Disadvantages

• Upper limits – eventually you hit the maximum instance size.
• Non‑linear performance growth – doubling resources may not double throughput.
• Single‑node bottlenecks – hot documents, collections, or heavy aggregations can still contend for primary resources.
• MongoDB EA only: Obtaining additional resources may be easier on Atlas than on‑premises.

Horizontal scaling (scale‑out)

Add shards and partition data across them.

Advantages

• Near‑linear throughput growth – adding shards increases read/write capacity and total storage.
• Hotspot mitigation – a good shard key spreads load evenly.
• Geographic flexibility – zone sharding keeps data close to users and satisfies data‑residency requirements.

Disadvantages

• Design complexity – choosing an appropriate shard key is critical; a poor choice leads to imbalances or inefficient scatter‑gather queries.
• Operational overhead – chunk balancing, resharding, and managing cross‑shard queries/transactions add complexity.
• Query pattern considerations – to avoid fan‑out, applications often need to include the shard key in queries.
• MongoDB EA only: Obtaining additional resources may be easier on Atlas than on‑premises.

For more information on scaling in MongoDB, see the articles “A Guide to Horizontal vs Vertical Scaling” and “Database Scaling”, or consult the official MongoDB documentation on scaling strategies.

Read‑only and analytics nodes (MongoDB Atlas)

Some readers might suggest using read‑only or analytics nodes in Atlas to increase read capacity.

Key points from the Atlas documentation:

• Read‑only nodes “optimize local reads in the nodes’ respective service areas.”
• Read‑only nodes “don’t provide high availability because they don’t participate in elections.”

The first point indicates lower latency for local reads, but it does not increase overall read capacity. The second point confirms that read‑only nodes do not contribute to high availability, which is a critical requirement for our system. Consequently, these node types do not solve our capacity or availability challenges.