Beni
@beniduboss
Co-founder stealth neolab | Building Novel Efficiency Infrastructure for LLM | Former Quant | Based in beautiful 🇨🇭
1.5K Following 32.4K Followers
prompt: “is there something wrong?”
AI gf: “I am fine”
(reviewing previous conversation)
(reviewing last 6 months)
(reviewing last 5 years)
(reviewing July 7th 2022)
thought for 58s
“it’s just funny how you…"
Show more
Trump admin is suing Utah back btw lmao
BREAKING: Utah has filed felony and misdemeanor criminal charges against prediction-market platform Kalshi and two of its top executives, Tarek Mansour and Luana Lopes Lara.
Didn’t see anyone mention this so guess I will
My team and I looked into H-Nets pretty seriously when we first started exploring LLM efficiency
It’s genuinely super interesting, and what they’ve done is very impressive, but I’m still skeptical of the inference economics at scale
Dynamic chunking means you deal with unpredictable compute per request, which hence means harder batching and more complicated GPU scheduling/memory management
But there are ways to handle this by using separate queues and grouping requests by expected workloads + custom kernels, but i don’t think it’s sufficient to beat the competitions
A hybrid architecture might be interesting though!
Because you can reduce FLOPs on paper and still end up with worse throughput, higher latency and more expensive inference in prod
At the end of the day what really matters is how many requests you can serve per GPU at the same quality and latency
But again
That really doesn’t take anything from what @_albertgu and his team have achieved, we gave up on H-nets due to the above and a few other reasons but regardless, it still has very significant use cases even if it struggles to scale
Gg
Show more
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Show more
Didn’t see anyone mention this so guess I will
My team and I looked into H-Nets pretty seriously when we first started exploring LLM efficiency
It’s genuinely super interesting, and what they’ve done is very impressive, but I’m still skeptical of the inference economics at scale
Dynamic chunking means you deal with unpredictable compute per request, which hence means harder batching and more complicated GPU scheduling/memory management
But there are ways to handle this by using separate queues and grouping requests by expected workloads + custom kernels, but i don’t think it’s sufficient to beat the competitions
A hybrid architecture might be interesting though!
Because you can reduce FLOPs on paper and still end up with worse throughput, higher latency and more expensive inference in prod
At the end of the day what really matters is how many requests you can serve per GPU at the same quality and latency
But again
That really doesn’t take anything from what @_albertgu and his team have achieved, we gave up on H-nets due to the above and a few other reasons but regardless, it still has very significant use cases even if it struggles to scale
Gg
Show more
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Show more
Hi @bubbleboi
Shame we live so far apart I have a game I’d like to play with you
I’ve got a really good friend, like top tier dude I promise you and coming from me it means something, who’s going through a tough time through no fault of his own
He’s a great full stack dev, honestly I’d hire him if I had any work to throw at him
If you have any work you need help with please contact me and I will put you in relation
You can hold me accountable if anything happens and I will reimburse you
That’s how much I trust this guy
Show more
I’ve got a really good friend, like top tier dude I promise you and coming from me it means something, who’s going through a tough time through no fault of his own
He’s a great full stack dev, honestly I’d hire him if I had any work to throw at him
If you have any work you need help with please contact me and I will put you in relation
You can hold me accountable if anything happens and I will reimburse you
That’s how much I trust this guy
Show more
prompt: “is there something wrong?”
AI gf: “I am fine”
(reviewing previous conversation)
(reviewing last 6 months)
(reviewing last 5 years)
(reviewing July 7th 2022)
thought for 58s
“it’s just funny how you…"
Show more
there’s a world of difference being top 1% and being #1#
This is just obvious especially for small teams I am not sure why people are acting shocked
Revolut CEO says you can't out-compete people who work 9 to 9, six days a week.
@chameleon_jeff said the same thing in 2023. He was building his trading system 5am-10pm, 7 days a week, for months. Then he built Hyperliquid.
I was just in Singapore and around Asia, and I can totally back this up. It's not even about the hours, it's the attitude and motivation. People there are hungry. Coming back to Europe, the contrast is huge.
When the people who actually built these companies all say it, it's hard to argue with.
Show more
It’s quite clear that META nerfed Muse spark open to take the lead on US open weight model rankings and nothing more
It’s a shame, hope the next one is actually a proper model
Why is Muse spark open so much worse than the closed version. Unbelievably disappointing.
@Web3junkiee >web3 in profil
>VC "strategist-researcher"
ofc you wouldn't understand that it's entirely satirical you dense motherfucker
@Web3junkiee >web3 in profil
>VC "strategist-researcher"
ofc you wouldn't understand that it's entirely satirical you dense motherfucker
todays snapshot
Some product feedback for @elonmusk and @poteto. I love GrokBot and its integration with X but every @bot having its own thread in X chat by default is making the chat unusable. Fully buries messages from humans.
Please add a separate tab or something similar for bot threads.
Show more
One must imagine Ender's Game was really just a VC sponsored recruiting challenge to dupe naive children into an extractive term sheet
Any startups out there selling really good RL envs in kernels/compilers/chips (my personal domain of interest)?
Or any cool RL envs in general?
DM me.
> be me
> never mention what you’re actually building
> get into a feud with some idiot on Twitter
> prove him wrong
> somehow end up in the WSJ
> grow Twitter by 50%
> get DMs from your role models and genuine Forbes list billionaires
> get LinkedIn messages asking if you’re hiring from PhDs at Jane Street and Anthropic
>
> life is good sometimes
Show more
@nexora_28 Many do things right
@A_Leutenegger runs his business most ethically
@follis_ is an influencer, runs a paid group and yet is one of the most genuine and honest guys in this space I could give him my seed phrase without thinking twice about it
Show more
