High memory usage in Postgres is good, actually

Source: PlanetScale Blog

Houseplants and Over‑watering

Houseplants often die from over‑watering, not neglect. It’s easy to project human needs onto them:

“If I am thirsty, they must be thirsty too.”

But many indoor plants actually benefit from drying out between waterings.

Database Empathy

Similarly, your empathy can lead to misinterpreting signals from your database. You don’t like feeling overwhelmed, so you don’t want your database overwhelmed either.

Note: Not all usage is created equal, and memory in computers can be uniquely complex to understand.

Interpreting PlanetScale Dashboard Metrics

• Memory usage may sit at 80 % on the PlanetScale dashboard.

  • This looks bad, but it can be representative of a healthy system.

• CPU usage is a different story.

  • Consistently high CPU is a problem: queries wait longer, the slowest queries get slower, and you have less headroom for spikes.

Why Postgres Wants Your Memory

Reading from disk is slower than reading from RAM, even with PlanetScale Metal’s locally attached NVMe drives. Postgres is designed to take advantage of that gap by caching as much data in memory as it can.

There are two layers of caching at work, and both consume RAM:

1. shared_buffers – Postgres’ own buffer pool.

   • When a query needs data, Postgres first checks this pool (fixed‑size 8 KB chunks by default) before reading from disk.
   • The more of your working data that fits here, the fewer disk reads Postgres needs to perform.
   • This parameter can be configured on the Cluster Configuration page of the PlanetScale dashboard. The default value should be sufficient for most workloads; it should not be your first step in troubleshooting memory usage.

2. OS page cache – The operating system’s second caching layer.

   • Even when Postgres goes to disk, the OS keeps a copy of the data it reads in RAM so the next access is faster.
   • This is standard Linux behavior, not a Postgres feature.
   • Postgres was designed with this in mind; its documentation notes that the OS cache handles data beyond what fits in shared_buffers.

Between these two layers, a healthy Postgres server will use most of the available RAM—that’s the goal, not a side effect.

• Reading a page from RAM is roughly 1,000 × faster than reading it from even a fast NVMe drive.
• When caching works well, the vast majority of page reads are served from memory without touching disk.
• If that ratio drops (e.g., the working dataset outgrows available memory), queries slow down as Postgres waits on disk more often.

See our documentation on “Normal operating ranges” to sense‑check the values you should see in Cluster Metrics for CPU, memory, and more.

Memory Usage vs. CPU Usage

Two Kinds of Memory Usage

The single “memory usage” percentage combines two different behaviors. In the Cluster Metrics page, memory is shown as a stacked chart over time with four categories:

1. Active cache – Recently accessed pages the OS wants to keep.
2. Inactive cache – Pages not accessed lately but still cached.
3. Memory‑mapped – Cached pages backed by real files on disk.
4. RSS (Resident Set Size) – Private memory used by Postgres processes.

These four categories can be grouped into two use‑cases:

1. Cache (active, inactive, memory‑mapped)

• Much of the “used” memory on a healthy host is cache (also called page cache).
• All three cache types are reclaimable by the OS and can be dropped when something else needs RAM.
• High cache = good – Frequently accessed data stays near the CPU for faster access.

2. Process Memory (RSS)

• RSS is the memory actually used by Postgres processes.
• This memory is not reclaimable by the OS and contributes to out‑of‑memory (OOM) risk.
• High RSS = memory pressure → restarts, degraded behavior.

What Is Resident Set Size?

• RSS ≈ amount of private memory allocated to a process (stack, heap, catalog/relcache caches, query execution memory such as sorts and hash tables).
• Because Postgres uses a process‑per‑connection model, each connection consumes a baseline amount of memory, but not every process will use the same amount.
• Some memory use is shared across processes (e.g., shared buffers), while RSS reflects the non‑shared portion.

Takeaways

• Don’t panic when you see high memory percentages on PlanetScale; it often means the system is efficiently caching data.
• Monitor CPU closely—sustained high CPU is a clear performance problem.
• Use the Cluster Metrics breakdown to differentiate between healthy cache usage and concerning RSS growth.
• Adjust shared_buffers only after confirming that cache sizing is the real bottleneck.

By understanding the distinction between cache and process memory, you can better interpret your dashboard and keep your database running smoothly.

RSS Increases

Calculating RSS usage is not as simple as adding up the memory usage of every process. RSS can increase for a number of reasons:

• Multiple work_mem allocations – Postgres may grant several work_mem blocks within a single query (see details below).
• Catalog bloat – Common in multi‑tenant schemas that use a table‑per‑tenant pattern.
• Memory allocator inefficiencies – The operating system’s allocator may not return memory to the process efficiently.
• Misbehaving extensions – Badly configured or buggy extensions can inflate RSS.
• Cached plans & prepared statements – These accumulate per‑session memory that isn’t released until the session ends or the statement is explicitly deallocated.

work_mem

The work_mem parameter’s default value is set relative to the amount of memory in your database cluster. It can be modified on the Cluster Configuration page of the PlanetScale dashboard.

Tuning work_mem might seem like an obvious lever—decrease it to reduce RSS, or increase it to prevent operations from spilling to disk. However, the allocation is per‑sort/hash node, per‑query, per‑backend:

• A single complex query can allocate work_mem multiple times.
• The allocation multiplies across every active connection.

Setting work_mem too low forces more disk I/O; setting it too high globally can cause total memory usage to spike unpredictably under load. Neither direction is a safe default change without first understanding your workload’s concurrency and query complexity.

Efficient Connection Pooling

Fewer active connections mean fewer copies of all per‑process overhead, which is often the most effective way to reduce RSS.

• PgBouncer on PlanetScale runs in transaction mode, returning connections to the pool after each transaction completes.
• See our blog post [Scaling Postgres connections with PgBouncer] for more details.

Investigating Memory Usage While Debugging Performance

If you’re experiencing degraded performance, the challenge is figuring out what drove the RSS growth.

• Query Insights helps you investigate query performance through CPU time, I/O, and latency, but it does not show per‑query memory. You may see OOM markers and slow‑query signals, but not query‑specific RSS usage.

• RSS is a per‑process metric, not a per‑query metric. You cannot read “RSS per query” directly from EXPLAIN or Query Insights. Instead, gather multiple signals and triangulate:

  1. Cluster Metrics – Identify when RSS rises.
  2. Query Insights (same time window) – Look for expensive patterns (high runtime, CPU, I/O, rows/blocks read) and OOM‑adjacent activity.
  3. Re‑run suspect queries with EXPLAIN (ANALYZE, BUFFERS, MEMORY) to inspect operator‑level memory usage.
  4. Connection counts – Many concurrent connection processes can increase RSS even when a single query is moderate.

• The [Out‑of‑Memory documentation] has more details on likely causes of, and how to prevent, OOM events.

In Summary

• Cached data is good. Ideally, your “hot dataset” fits in the page cache of your database cluster, maintaining fast performance. Too little cached data can lead to increased CPU usage and degraded performance.
• High memory usage isn’t automatically bad. If it’s due to cache, you typically have a healthy, performant database.
• Memory pressure is bad. Rising RSS toward limits, OOM kills, unexplained restarts, and tail‑latency spikes together with heavy disk I/O when the working set is tight on RAM are signals to act on.
• Sustained high CPU is a problem. It means you are out of headroom. Tune the workload (see Query Insights) or upgrade.
• Dashboard shows a high “% memory used.” Do not panic. Investigate the types of memory being used and check for OOM events before taking action.