This is an engineering conversation around pg_lake – a new OSS Postgres extension that lets you query and manage Iceberg tables directly form Postgres.

Marco Slot, who has EXTENSIVE experience, shares with us various engineering internals, like:• how pg_lake makes analytics (literally) 100x faster• why Postgres is architecturally terrible at analytical queries (and how vectorized execution fixes this)• how (and why) pg_lake intercepts query plans and delegates parts of the query tree to DuckDB• Marco's hard-won experience through a decade+ career in Postgres• versatility as the real moat of Postgres• the practical differences in engineering b/w OLTP and OLAP• and a lot more——————————————————————–*TIMELINE*0:02 What is pg_lake?2:23 Postgres' 100x slower problem and columnar storage experiments they had to make Postgres fast for analytics6:00 practical examples and internals16:20 perf internals – vectorized execution & CPU Optimization23:00 pg_lake architecture (why DuckDB isn't embedded) and the connection-per-process issue29:16 how pg_lake intercepts the query plan tree and delegates parts to DuckDB41:09 Iceberg catalogs48:24 postgres to iceberg ingestion patterns (and pg_incremental)53:40 Marco's (long) career: early AWS, Citus, Microsoft, Crunchy Data & Snowflake1:04:20 Marco's observations around the merging between OLTP and OLAP (and the subtle dev differences there)1:15:30 reverse ETL1:33:08 Iceberg as the TCP/IP for tables1:35:00 Marco's thoughts on the ”Just Use Postgres” fever—————————————–*MARCO*You can find Marco on:- LinkedIn: https://www.linkedin.com/in/marcoslot/- X: https://x.com/marcoslot- GitHub: https://github.com/marcoslot

—————————————–

*pg_lake*

You can find the project on GitHub:- https://github.com/snowflake-labs/pg_lake

—————————————–*TRANSCRIPT*Feed this into your favorite AI for summarization, or to prompt it specific questions:https://gist.githubusercontent.com/stanislavkozlovski/65c037a8963e49d8121b25003ec94715/raw/4f51f5dcd562b42e8d511b8bc58f0fff6ad5302e/foo.md(or just send Gemini this video link and ask it)——————————————*OTHER PLATFORMS*Watch on YouTube here:https://youtu.be/Jd0DcX2fO_kApple Podcasts:https://podcasts.apple.com/us/podcast/topicpartition/id1814926834General RSS:https://anchor.fm/s/104fd76e0/podcast/rss—————————————–If you found anything useful from this episode, please consider supporting our growth (so we can continue delivering valuable content). You can do this by simply sending it to a friend. It takes 2 seconds to do, and recording/producing this takes us 8hrs+