This insightful interview with Tom Lane, a core developer celebrating twenty five years as a committer on the PostgreSQL project, looks at foundational architectural choices that have sustained the open source database over its thirty-year history
Postgres...A lot can be said about this wonder tool and here at IProgrammer know all about it. From those dozens of IProgrammer articles about Postgres, the most relevant to this interview has to be "The Enduring Influence Of Postgres"; a historical recollection of the Postgres project and its impact on the DBMS industry provides an insight into the key features of the object relational database as conceived by Mike Stonebraker. Those were:
- Supporting ADTs in a Database System
- Commercial Adaptations
- Open Source
- Support for a Variety of Language Models
- Support for Multiprocessors: XPRS
- Log centric Storage and Recovery
- Active Databases and Rule Systems
- Extensible access methods for new data types
This interview done by Elizabeth Garrett Christensen on behalf of Snowflake, is about the infrastructure that makes all that happen; the foundational architectural choices that have sustained the open source database over its thirty year history, including its unique process per connection model, write ahead logging (WAL) for crash recovery and multi version concurrency control (MVCC).
Along with that, Tom Lane also discusses the project's informal governance structure, the extensive reliance on C memory contexts for resource management and the ongoing balance between maintaining a stable core and supporting diverse extensions. Let's go through them.
1. Process per connection model
Unlike databases such as MySQL or SQL Server that use threads, Postgres spawns an isolated operating system process for each client connection. The benefits of this are code simplicity and crash isolation. Developers write straight line code without worrying about concurrent threads corrupting private memory structures. If a session crashes, the postmaster process kills all children and reinitializes shared memory safely. In the 1990s, avoiding unstandardized thread implementations gave Postgres simplicity, allowing new contributors to hack on the codebase easily.
But there's trade offs as well. Each process carries heavy catalog caches and private memory maps. High connection counts increase CPU context switching overhead, which led the ecosystem to rely on connection poolers like PgBouncer.
2. Write Ahead Logging (WAL) and recovery
When Postgres was released by UC Berkeley, it lacked a write ahead log entirely; a crash often meant restoring from backups. Introducing WAL (culminating in Postgres 8.0) transformed Postgres from an academic project into a production grade database.
Under it, changes are written sequentially to a single WAL stream before updating shared buffer pages. Transactions commit as soon as WAL hits disk, deferring random page syncs to background checkpoints. To guarantee reliability, crash recovery replay is intentionally kept as a simple, single process operation.
3. Multi Version Concurrency Control (MVCC)
Rather than overwriting rows or relying on undo logs to handle concurrent reads and writes, Postgres appends a new copy of the row with transaction metadata in the header. The
architectural advantage is that readers never block writers and writers never block readers. Maintenance work cleaning dead tuple versions via VACUUM and autovacuum is pushed into background processes rather than blocking transaction aborts or commits.
4. Memory context hierarchies
Instead of standard C malloc/free routines or language enforced ownership models, Postgres uses a tree of named memory contexts tied to specific lifespans (session, query, or row level). The impact is that when a query or transaction ends, its entire memory context tree is freed at once. This eliminates retail free() calls and prevents memory leaks, making C memory management robust across 1.5 million lines of code.
5. Extensibility as a core constraint
Extensibility was baked into Postgres from its Berkeley roots (custom types and index access methods). If a proposed core feature cannot be extended to work with custom extension types, the community rejects it until generalized. This philosophy enabled extensions without bloating the core engine or splitting the community. The core parser, built with Flex and Bison, remains intentionally static to preserve grammar correctness guarantees.
In "ElasticSearch Search Capabilities Baked Into PostgreSQL" we documented that capability:
pg_ivm - Materialised Views On Steroids
an extension module for PostgreSQL 14 that provides an Incremental View Maintenance (IVM) feature. That means that materialized views are updated immediately after a base table is modified.
pgsqlite - Load SQLite databases into PostgresSQL
a pure python module and command line tool that makes it simple to import a SQLite database into Postgres, saving a ton of time and hassle in the process.
pg_later - Native Asynchronous Queries Within Postgres
an interesting project and extension built by Tembo which enables Postgres to execute queries asynchronously. Fire your query - but don't forget to check later for the result.
Hydra Turns PostgreSQL Into A Column Store
Hydra is an open-source extension that adds columnar tables to Postgres for efficient analytical reporting.
Turn PostgreSQL Into A Vector Store
pg_vector is an extension for PostgreSQL that renders it a viable alternative to specialized vector stores used in LLMs.
6. Permissive licensing and informal governance
The liberal BSD/MIT style license allowed commercial vendors and cloud providers to build proprietary forks while maintaining touch with the community codebase.
As far as governance goes, the project operates without a formal legal entity or corporate steering committee. All changes occur via email discussions and consensus, producing deliberate, highly stable releases where database reliability comes first.
But that's all done; written history. The question is what will the future hold? where is the project heading?
In the interview, Tom emphasizes again the point that Postgres does not have a formal project roadmap or steering committee. Because development is decentralized, the project moves deliberately, prioritizing stability and data reliability over quick releases or trend chasing. However, several major architectural shifts, upcoming features and community policies are actively being worked on or discussed for the future:
Architectural and performance shifts
Transitioning to a Threaded Model: Developers are actively researching converting Postgres from its process per connection model to a threaded model to reduce CPU context switching overhead on high core hardware. To achieve this, engineers must solve the problem of sharing catalog caches across threads/sessions without risking data corruption.
Column organized tables: There is strong ongoing interest in supporting columnar storage alongside traditional row based tables. Doing so requires extending the existing Table Access Method (TAM) API so extensions or core features can support column oriented table formats.
Parallel crash recovery: While write ahead log (WAL) replay during crash recovery is intentionally kept to a single process for maximum simplicity and safety, it may eventually be parallelized in the future.
Extensible SQL parser: Because the parser relies on static Flex and Bison tables, extensions currently cannot add new SQL grammar. Tom notes that if a competitive tool can match Bison's correctness and ambiguity guarantees, the project would be open to making the parser extensible.
Vacuum improvements: Ongoing engineering continues on autovacuum heuristics and parallelizing maintenance (such as parallel vacuum in Postgres 19).
Core vs. extensions philosophy
Tom favors keeping core Postgres lightweight and lean. If a new feature or capability can work well as an extension, his stance is to keep it outside the core engine so core developer resources aren't stretched too thin.
Community policies on ai
The community is formulating official policies regarding AI assisted code. The current consensus is that a human committer must stand behind every patch and be capable of explaining every design decision.
The project is also experiencing a sharp influx of AI generated bug and security reports. While some are legitimate, many lack architectural understanding, so the community is working on better filtering mechanisms.
Finally, as far as Tom's future goes, he plans to continue working on Postgres as long as he remains engaged and relevant, noting he is content to keep working on core development for the foreseeable future.
Wrapping it up, I must say that the interview is pretty insightful and a must watch for anyone working with Postgres.
More Information
Tom Lane on the Architectural Decisions That Shaped 30 Years of Postgres
Celebrating 30 Years of Postgres: A Conversation with Tom Lane
Related Articles
The Enduring Influence Of Postgres
ElasticSearch Search Capabilities Baked Into PostgreSQL
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Facebook or Linkedin.
AG-UI Protocol Gets .NET SDK
01/10/2026
The .NET team has contributed a .NET SDK for AG-UI in collaboration with CopilotKit. The AG-UI .NET SDK is now kept in the AG-UI repository alongside the TypeScript and Python SDKs, and means any .NET [ ... ]
MariaDB Community Server 13 Now Generally Available
24/09/2026
MariaDB has announced the general availability of MariaDB Community Server 13.0. The company says this version pushes further into Oracle compatibility, gives developers a new way to retrieve data fro [ ... ]
- The CyberAgents Exchange
- MIT's Missing Semester of Your CS Education, 2026 Edition
- XAML.io Adds Full Coding Agent
- NetBeans 31 Released
- AI-901: Introduction to AI in Azure
- Meta VR Start Developer Competition Opens
- Robot Rabbits Used To Catch Florida Pythons
- Google Spanner Omni Released
- Julia 1.13 Improves Latency
- Robot Meltdown
- NVidia Introduces CUDA GPU Programming In Rust
- VS Code Expands Codex Harness
Comments
or email your comment to: comments@i-programmer.info