You can query the entire internet, 100+ billion (!) rows, with @duckdb in under a minute. Yes, it's crazy, but you can basically select * the internet. You might be familiar with the Wayback Machine from @internetarchive. The @CommonCrawl is a similar project that also has all its data available in a S3 bucket. @dumkydewilde was interested to see how the 'vibe-coded' web has grown over the last few years. While @Lovable , @Vercel and @Cloudflare Pages have taken off tremendously, they still pale compared to the traditional Wordpress-Blogspot hegemony. See for yourself in the interactive Dive visualization, or read the full blog to do it yourself 👇. - Dive: - Blog:

On October 1, we’re bringing together a small group of NYC data leaders to compare notes on what’s actually working in agentic analytics, what isn’t, and what teams are still figuring out. Joining the conversation: → Ian Macomber, VP Data @ Ramp → Josh Hanson, Head of Data @ Clay → Ali Salihoglu, Head of Digital & Data @ Blank → Duncan Fraser, Data Lead Some topics we’ll get into include: 🦆 Semantic layer vs. context layer 🦆 Getting teams comfortable with AI tooling 🦆 Keeping answers consistent across Slack, BI, and wherever else data shows up 🦆 What changes when agents become consumers of company data No polished case studies. Just practitioners comparing notes with a room full of people living the same problems. Space is limited, register to join us here-

Text classification in MotherDuck just got ~50x faster at ~1% of the cost. prompt_jev() is a SQL function powered by Jev, TypeSafe's new system one model. 100k rows: 40s, $0.50, frontier-LLM accuracy. The LLM took 32 min and $37. Read on:

If you didn't have a chance to join us in person in the last few weeks, we'll be quacking in London, Seattle (x2), and NYC in the coming weeks (including for a panel with Hex and Clay on the future of agentic analytics. Oh, and we're hosting a webinar on how to build a data agent in 30 minutes. Check out our upcoming events here:

Change a few DuckLake defaults and you get a 30% speed boost. Partition and cluster well and the gains can be bigger. Matt Martin and I just finished the performance optimization chapter of the @OReillyMedia Definitive Guide, and we're giving it away!

The Data Outpost talk agenda is live. Nov. 4 is hands-on-keyboard: a full day of workshops for people building with data & AI. Nov. 5 is talks and panels with founders, engineers, product leaders, researchers, operators, and data architects making machine intelligence useful inside actual companies. We’re talking agents, interfaces and workflows, memory, trust, and the data infrastructure underneath all of it. 25+ speakers. 2 days. $499. 📍 San Francisco 🗓️ Nov 4–5 🔗

DuckDB Monthly #45# is out. Spotlight: Vladimir Gribanov, creator of the DuckDB MSSQL extension (native TDS, no ODBC). His bulk-load tuning took 38M rows into SQL Server from 933s to 96s. Plus DuckLabs joins AWS, MotherDuck acquires Tower, and v2.0-alpha lands.

DuckDB 2.0 rewrote recursive CTEs. Walking a 20,000-commit git history: 1.8 to 16 s on 1.5.5, 0.10 s on 2.0, every run. 1.5: re-read the whole table every round. 2.0: read once, build a lookup, touch only the frontier's rows. Deep parent/child chains are where it shows.

We're three weeks out from our next boat party with Modal and Braintrust for TECH WEEK by a16z. Spots are incredibly limited, come quack with fellow AI, data, if you'll be in SF!

Every database writes, stores, and reads data. So why do you need to pick between 50 different ones? The key is the layout and performance of that database. Do you want it to write fast or read fast? Do you want to know which user placed which order or do you want to sum and count all orders. @dumkydewilde tells you why you need another ducking database, and what kind of performance optimizations and tricks @duckdb uses to be both fast and cheap. In the end the useful question is not which database is best. It's which question you are going to ask a thousand times a day.

Last year at Big Data London. Next week, round two. Two booths this time (P60 and L104) and the party's back at Kindred on Sept 23 at 7 PM. Duck In, Duck Out: London Edition. Space is limited. Request your spot:

DuckDB 2.0 alpha reads Parquet and CSV from S3 2x to 3x faster, with zero query changes. 1.5: each worker downloads, waits, decodes, repeats. 2.0: a download pool fetches ahead, workers only decode. Same laptop, same query: 18.8 s to 7.7 s. On by default. Full numbers, plus recursive CTEs and VARIANT:

2022 plan: hack on DuckDB, learn Rust. What happened instead: investors funded the company before there was one. Our CEO Jordan Tigani on Beyond Coding with @PatrickAkil_ — Giving DuckDB Labs co-founder shares, two years to a product people would pay for, and whether dashboards survive the age of agents.

Vegas, here we come! The MotherDuck team is gearing up for dbt SUMMIT 26 next week at The Cosmopolitan. Stop by Booth #309# from September 15–18 to chat with our crew about all things data!

DATA OUTPOST: A 2-day conference for data people in a world being rebuilt by AI. If you are a founder, engineer, product leader, researcher, operator, or data architect making machine intelligence useful inside actual companies, consider this your forward station. Nov 4: Hands-on-keyboard workshops. Less talking about building, more building. Nov 5: Talks and panels including: Tristan Handy (Fivetran + dbt Labs) Stefanie Tignor, PhD (Clay) Sudeep Dasgupta (DoorDash) Tosh Rayadhurgam (Stripe) Jordan Tigani (our own!) ...and more. Tickets are $499 for both days: