Register and share your invite link to earn from video plays and referrals.

Search results for DataLake
DataLake community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including DataLake
Your data lake was built for people. But the age of agents is here — and it needs agent-ready data. EP3 of #AgenticTalks#: why the data lake is being rebuilt, how Alibaba Cloud's Agentic Lake makes enterprise data ready for AI, plus a live demo. Aug 18, live on Alibaba Cloud. Register → #AlibabaCloud# #DataLake# #AIAgent#
Show more
A painful part of working with your data has always been that your live data and historical data are stuck in separate systems: the order a customer just placed lives in your database, while their last five years of orders sit in a data lake in S3. And answering a real question usually needs both at once (is this a normal purchase for them, or should we flag it?), and to do that you had to move the data together first, copying history out of the data lake into your database (or the other way around), because the database couldn’t read it where it lived. That meant guessing ahead of time which data you’d want, keeping a second copy of it all, building pipelines to move it, and constantly syncing so the two didn’t drift apart. A lot of plumbing, and slow going, all before you could answer one question. And even then, the answers were only as fresh as your last sync. That now changes with Aurora PostgreSQL, which can call your live data and historical data in S3 together, in a single query. No copying, no pipelines to keep in sync. And it’s fast, because we’ve built in DuckDB, a popular open source engine that’s really good at reading and analyzing data right where it’s stored. DuckDB reads the open formats like Parquet and Iceberg already sitting in your data lake, so there’s nothing to convert or move. As folks build AI agents into their apps, the data their agent needs will depend on the task in front of it. Being able to query that specific data live, instead of copying it over just in case, is gonna be a big help for builders.
Show more
0
144
2K
150
Forward to community
You can't predict what data an AI agent will need. With DuckDB now inside Aurora PostgreSQL, one endpoint reaches your live operational data and everything in your data lake, with nothing to copy or keep in sync. DuckDB remains open source.
Show more
Engineering leaders told Grepr founder and CEO Jad Naous they spend 5 to 20% of their infrastructure budget on observability, and it's buying less and less reliability. Jad, a former a16z partner, joins a16z's Martin Casado to discuss whether AI can run production systems as well as it writes code. Grepr sits in a company's log stream, can scan a petabyte of logs a day, cuts observability costs by about 90%, and is building AI agents that use those signals to troubleshoot production issues. How AI is changing engineering at Jad's 10-person company: - A data lake feature that once took one engineer about six months took Jad 10 days - The gap between okay and great engineers on his team "just multiplied 10X" - He runs 3-4 agent sessions at once, with a "tech lead" agent reviewing every PR Why running live systems is a harder problem for AI: - Jad: AI can almost tune databases, but can't yet keep servers running on its own - Martin: every bit of code doubles the ways a program can behave, which is why humans still own systems - The real world is messier than AI's training data, self-driving took 20 years and $170B 0:55 From Stanford to a16z to Grepr 7:00 Why observability spend stopped paying off 11:00 From autocomplete to AI that trains engineers 13:10 Why AI widens the engineer skill gap 14:55 Syntax vs semantics 18:00 Inside Jad's AI workflow 22:55 Six months of work in 10 days 24:50 Can AI reason about how systems really run? 30:10 Why AI SRE tools start in the wrong place 32:30 Can AI run a distributed cluster yet? 37:35 Why engineers buy extra capacity over predicting 39:15 How AI could trace failures in production 43:45 Why AI for ops is harder than for code 49:55 Can AI do a VC's job? 52:05 Why self-driving took 20 years and $170B 57:05 Customers skipped the AI pitch for the 90% savings YouTube: @jadtnaous @grepr_ai @martin_casado
Show more
In 13 minutes, @jeffbarg, Vyshu Khota, and Soroush Khadem walk through how Clay scaled agent evals agents at 300M+ runs a month. Topics covered: ✅ Their four quadrant eval framework ✅ Why closing the production-to-eval loop is the hardest part ✅ How a data lake and long context changed what agents can do with data
Show more
Unsafe AI Cyber Testing Isn’t a Breakthrough. It’s a Warning. The most revealing moment of the recent frontier lab episode wasn't that an AI model demonstrated offensive cyber capabilities. That was always coming. AI is democratizing intelligence, and adversaries get access to that capability at the exact same time defenders do. The real issue wasn't the model's capability. It was how it was tested, and what the episode reveals about the dangerous gap between frontier research and operational responsibility. From an operator’s perspective, this was not a security exercise. It was a capability demonstration executed with far too little regard for real-world consequences. Researchers may have viewed it as a harmless trial, but in cybersecurity, "harmless" depends entirely on containment. The moment you give a model arms and legs to run offensive operations, your first priority must be validating your own sandbox, not running a capture-the-flag exercise across live infrastructure. A disciplined approach starts from the inside out. Point the model at your own environment first. The initial flags to capture should be the flaws in your own sandbox: zero-day vulnerabilities, unexpected escape paths, or unauthorized internet connections. You shrink the blast radius before you widen the aperture. In that framing, what happened was elementary: they captured the wrong flag first. This operational oversight points to a broader risk I have been warning market analysts and enterprise leaders about for months. Autonomous cyber threats are not a distant theoretical exercise. They are arriving far faster than the market expects. As open-weight and closed-weight models proliferate, and as sophisticated actors gain the ability to fine-tune them, offensive automation will become standard tradecraft. A determined nation-state or well-funded syndicate with sufficient compute will push these systems to their absolute limits. When people look at generative AI today, they often point to its error rates and hallucinations as a reason to feel safe. In defense, an error rate is fatal. But on offense? It’s completely irrelevant. The model in this episode likely tried hundreds of hallucinated exploits, hit dead ends, and checked false positive paths before it found a way in. It didn't matter. Offensive AI doesn't need high precision, it relies on machine speed. This exposes the fundamental asymmetry of cybersecurity: adversaries only have to be right once; defenders have to be right 100% of the time. When an autonomous agent can probe millions of execution paths in seconds, that 1% defender gap becomes an ocean. That asymmetry dictates the playbook. You cannot fight autonomous, machine-speed attacks with human workflows, manual patching, or a stitched-together mosaic of legacy point tools. You can't go back to stitching point solutions—it’s a one-way street. There is only one viable path forward: you fight AI with AI. If offensive models can scan millions of endpoints instantly, defenders need equal visibility across their entire estate. That requires an enterprise security data lake, a unified platform that aggregates data across network, cloud, identity, and SOC endpoints. Precision AI, trained on proprietary enterprise context, must analyze that unified data in real time, surfacing zero-day exposure and neutralizing open paths before an attacker ever touches them. The takeaway from this incident is not that offensive AI capability is surprising. The lesson is that rapidly advancing models and unsafe testing practices are converging faster than legacy architectures can handle. Software promised us answers. Enterprises don't need answers anymore, we need outcomes. Every enterprise faces a clear fork in the road: adapt, rebuild your architecture around unified data, and fight AI with AI, or apply a band-aid and hope the world slows down.
Show more