Marc Klingen
@marcklingen
co-founder @langfuse, YC W23, part of @clickhousedb
1.4K Following 4.3K Followers
Got myself a FT subscription and I really like it
So many Jev-shaped problems in agent observability
Jealous I'm missing this banger of a talk by @rawert live
Just went with @rawert through his Econ History AI talk that is coming up at @aiDotEngineer Paris next week, you don’t want to miss it!
Very happy that I get to work with such a good cofounder who has nuanced takes partially grounded in his history hobby/academics
Show more
You can also access jev via @OpenRouter now on @langfuse to power your evals
We're in Langfuse!
Evals plus more natural language are very strong opus arguments
Based on our experiments, Jev bears big potential for evals
✔️ It is up to 400x cheaper than frontier models, making full scoring of production traces feasible, no more sampling needed, even at huge scale
✔️ It allows for concurrent questions on the same state, where LLMs previously struggled with interaction effects and common wisdom was to not mix evals in the same requests. This makes Jev even cheaper.
✔️ The structure of Jev's output forces thinking in distinct categories during setup, improving the engagement with defining the exact evaluator categories
Play around with this demo here:
Show more
That evals independently evaluate without needing to split them into single model calls is what’s cool here
This multiplies the llm-vs-jev cost savings
@doneyli @typesafeai with Jev
it is less
eval -> eval -> eval
and more
-> eval
-> eval
-> eval
6 Luna going for competing with Jev on price
evaluate ALL your traces with Jev
Very excited what this new class of generalizable/zero-shot classifiers means for agent o11y/evals
Many more things cooking
Jev-powered evals are now available in Langfuse
👉️ Pay 40-400x less than with frontier models
👉️ Run Jev (by @typesafeai) on all production traces without sampling
👉️ Check multiple criteria at minimal extra cost
Show more
Every application should have this (slack 👀)
New: Linear Inbox is now available via MCP.
Enable your agents to read and manage your notifications.
Very impressed by @traderepublic’s dedication to UX
I just opened a new brokerage account for my holding entity in 1 minute
Took me 3 hours with Interactive Brokers
Trade Republic mischt den Markt für Firmenkunden auf!
Kunden, die alleiniger Gesellschafter (leider darauf beschränkt) einer UG oder GmbH sind, können jetzt ein Trade Republic Private Holding Account eröffnen:
Show more
Learned to appreciate the database classes I took at TUM only after graduating
How you know you're about to read a great paper:
TUM
TUM
TUM
TUM
TUM
Just went with @rawert through his Econ History AI talk that is coming up at @aiDotEngineer Paris next week, you don’t want to miss it!
Very happy that I get to work with such a good cofounder who has nuanced takes partially grounded in his history hobby/academics
Show more
i've been surprised at the response to Jev, but it makes sense in retrospect. sure it's just a classifier but it's a zero shot classifier with frontier-ish intelligence. i'm surprised someone hadn't built it before. i wonder what other old ML ideas are also worth rescuing
Show more
Rule number 1: don't add a human blocker to growth
Users / prospective customers can get a great Langfuse demo without ever talking to us
We're here if you have questions unique to your situation (see the toggle on
Show more
@marcklingen sat down to demo all langfuse features in a single video.
Agent observability, monitoring, evals, prompt management, and what is special about Langfuse.
Great place to start if you are new, or catch up on latest changes.
Show more
Guided Diffs in @linear are so impressive
Removes need for stacked diffs even for huge changes
Raycast dictation is very nice
