Buffer cache, I/O and memory
Two caches between you and the disk
The idea in 30 seconds
PostgreSQL reads pages through its own cache, shared buffers, which sits on top of the operating system's file cache. In EXPLAIN, "hit" means the page was in shared buffers and "read" means it was fetched from the operating system, which may still be memory, not disk. Large sequential scans are deliberately kept from flooding shared buffers, so "read" there is expected, not a problem.
Helps to know first: How a table is stored on disk
On this page
Two layers of cache
When PostgreSQL needs a page, it looks in two places before the disk:
| Layer | Owned by | Size |
|---|---|---|
| Shared buffers | PostgreSQL, shared by all its processes | shared_buffers, 128 MB by default |
| Page cache | The operating system, for every file on the machine | Whatever RAM is free |
A page fetched from the operating system usually stays in both. That double caching is deliberate: the documentation suggests starting shared_buffers at about 25% of RAM on a dedicated server, and says more than 40% is unlikely to help, because PostgreSQL relies on the operating system’s cache too.
EXPLAIN (ANALYZE, BUFFERS) reports each page as a hit (found in shared buffers) or a read (requested from the operating system). A read is not necessarily a disk read: if the page was in the page cache, it came from memory.
Watch the cache work
We built a 52 MB table, small_t, restarted PostgreSQL to empty shared buffers, and counted its rows three times. (A restart does not empty the operating system’s cache.) PostgreSQL 18.3, shared_buffers at the default 128 MB, on a laptop:
| Run | Time | Hit | Read |
|---|---|---|---|
| 1, just after restart | 39 ms | 0 | 6,667 |
| 2 | 21 ms | 282 | 6,385 |
| 3 | 22 ms | 564 | 6,103 |
PredictA table that never warms up
The table is 52 MB and shared buffers are 128 MB, with nothing else running. Why did each repeated scan add only 282 hits, instead of the whole table becoming cached?
Large sequential scans use a small ring of buffers, by design. A scan that touches a large number of pages reuses a limited set of buffers in a circle, so it cannot push everything else out of shared buffers. A one-off report scanning a whole table would otherwise evict the pages your busy queries depend on.
The 359 MB table behaved the same way, and afterwards shared buffers held only 846 pages of each table. Runs 2 and 3 still counted almost everything as reads, yet took about 21 ms: those pages came from the operating system’s page cache, which is memory. Run 1 was the slowest; we did not clear the operating system’s cache, so this test cannot say how much of that was the disk.
When a table really should stay in shared buffers, you can load it explicitly with the pg_prewarm extension:
CREATE EXTENSION pg_prewarm;
SELECT pg_prewarm('small_t'); -- 6667 pagesAfter that, every page of the same count was a hit (6,667 hits, 0 reads), at about 26 ms. On this laptop that was no faster than reading through the warm operating system cache: both are memory. The difference matters when the operating system cache cannot hold everything, and pages would otherwise come from disk.
What effective_cache_size does
effective_cache_size sounds like a cache size and is not one. It allocates nothing. It tells the planner how much data it can assume is cached between shared buffers and the operating system, which makes index scans look cheaper or more expensive in its estimates. Set it to roughly the memory you expect to be available for caching; it never changes how much memory PostgreSQL uses.
Check yourself
EXPLAIN (ANALYZE, BUFFERS)showsread=6385. Did PostgreSQL go to disk 6,385 times?Show answer
Not necessarily. “Read” means the page was not in shared buffers and was requested from the operating system, which may have served it from its own page cache in memory.
Why do repeated scans of a large table not fill shared buffers with it?
Show answer
Sequential scans of large tables use a small ring of buffers that they reuse, so a big scan does not evict everything else from shared buffers.
What does raising
effective_cache_sizechange?Show answer
Only the planner’s cost estimates: it assumes more data is cached, so index scans look cheaper. It allocates no memory.
Finished reading?
Saved in this browser only.
Sources
Where each claim on this page comes from.
- 1.PostgreSQL documentation: 19.4 Resource ConsumptionPrimary· §19.4.1 Memory: shared_buffers
shared_buffers defaults to 128 MB; on a dedicated server, 25% of RAM is a reasonable start; because PostgreSQL also relies on the operating system cache, more than 40% is unlikely to help.
- 2.PostgreSQL documentation: Appendix M GlossaryPrimary· Buffer Access Strategy
Operations that touch many pages, including sequential scans of large tables, reuse a small ring of buffers so they do not evict too many pages from shared buffers.
- 3.PostgreSQL documentation: 19.7 Query Planning (effective_cache_size)Primary· §19.7.2 Planner Cost Constants: effective_cache_size
effective_cache_size is the planner's assumption about cache available to a query; it affects cost estimates, not memory allocation.
- 4.PostgreSQL documentation: F.30 pg_prewarmPrimary· F.30
pg_prewarm loads a relation's pages into the buffer cache.
Go further
What to read next, and why it is worth the time.
- 1.PostgreSQL documentation: 19.4 Resource ConsumptionPrimary· §19.4.1 Memory
The authoritative advice on shared_buffers, work_mem and their friends. Short, and better than most blog posts on the subject.
Applies to PostgreSQL 18. Draft: not yet fact-checked end to end.