Register and share your invite link to earn from video plays and referrals.

Search results for FineTuning
FineTuning community
One keyword maps to one global community path.
Create community
People
Not Found
Tweets including FineTuning
hey fam, i’m currently working on finetuning the dex, which should go live tonight or tomorrow morning. the biggest liquidity layer for robinhood will go live. thanks @blockaid_ for quickly resolving our wallet issues, as a lot of users were scared due to the warnings. the dex will start with 100k $prism/day emissions for 24 hours for testing, and then it will be increased to a much larger amount. it will mostly be dynamic and will change according to price, pool weightage, and many other factors. we are adding a few new pools too with v1, and you can farm lp to earn emissions and rewards. i’m working on an article to explain how this exactly works, and it’ll drop soon. we are also working on launching both rwa perps and the social trading layer together right after the launch of v4. here’s a personal note i wanted to add: robinhood is going through a tough battle, and it’s necessary for all the successful chains, and it will sustain this dump. pls keep on supporting projects and builders who want to build a long term defi enterprise and build the infrastructure. the dump will be temporary, but tech will be permanent. tech is what brings more retail and institutions to the chain, and that’s how ecosystems win over time. thank you for all the support all these days for $prism, but i request you guys to stay with us during this crazy robinhood dump, as i strongly believe it’s gonna last for a week and it will be back stronger. whatever, we keep shipping let’s build fam.
Show more
One open question for anyone finetuning models is - how "post-trainable" are different open source models? We can theorize that models that reside in shallower loss curves are more amenable to post-training - which makes sense, given that it requires less gradient updates to modify behavior. Then, this paper proposes a pre-training algorithm that makes a model more post-trainable. The idea is straightforward once you wrap you head around it: 1. Find the worst policy locally around the policy pi (found by taking the inverse gradient w.r.t pi), call that pi_prime 2. Take the gradient at pi_prime (call that grad[pi_prime] ) 3. Apply grad[pi_prime] to pi. The intuition is that we're actually moving in the direction that benefits the worst model around you, meaning a model maintains post-trainability because we maintain a shallow loss landscape. Another way to imagine this, is it effectively avoids steep pot-holes during pretraining that would lock your model in a distribution that it can't post-train its way out of. Really great work from @IshaanWatts18, @CatherineL11638, @goyalsachin007, @jacspringer, @AdtRaghunathan. It's our favorite paper of the month at Trajectory!
Show more
Debate Training Reduces Reward Hacking in RLAIF New paper from Google DeepMind: "We demonstrate that RL finetuning an LLM using debate, a two-player adversarial game between a generator and a critic adjudicated by a weaker LLM judge, reduces reward hacking compared to a reinforcement learning from AI feedback (RLAIF) baseline." paper link:
Show more
hey fam, (consider this tweet an ama and ask me anything about prism protocol, prism L3 or anything around the product) im currently working on finetuning V5 for launch but here’s a few things i wanted to share: the whole purpose of v5 is to make the site very smooth and less confusing for users. but that’s not it, we are also working on adding better verification of each asset, multiple liquidity pools aggregated for similar assets under one router, every feature and tool will be upgraded with better efficiency and also better rewards. this version along with the trading competition will act as our first onboarding event and allow us to move on auto pilot for a while. when i say verification, it’s not a simple verification. It will be multiple layers of asset verification, proof of reserves, better osint and self verification by issuers too. enterprise grade verification for each asset in our protocol. also perps will go live, futures and options will go live, dex v2 will go live, higher emissions and rebates will go live, and efficient trading flow too. not everything at once of course, one by one. and a better developer environment will be built for prism test net and more dapps will go live. but when i keep saying more, we are not focusing on just increasing numbers here lol anything that’s not up to industry standard or not efficient will be instantly delisted from the protocol. more dex pools, more transparency layers, better docs and guides etc. this version should mark the beginning of prism as a mainstream platform for RWAs just like how every other protocol takes over and dominates one particular space. whatever, we keep shipping. let’s build.
Show more
hey fam, good morning our first dapp went live yesterday on prism testnet and we already have more than 100 users spending gas on the testnet. also seems like a lot of people now have a better understanding of how prism L3 works compared to last time haha. today we will work on: > finetuning V5 for launch and restructuring the protocol features to make it easy, smooth, less confusing and not exhausting > add and polish all the dex pools with the farm V2 contract for improved emissions and rewards and to build a much better liquidity layer. > work on a few important partnerships add 1000 more cool assets covering $10B in market value > improve OSINT, proof of reserves and verification. > work on finetuning RWA perps for launch > work on installable web apps a lot of people reached out to me and asked in the community how verification is done. you can just go to any asset page or OSINT tool to find out everything about an asset, from its verification to exact custodial information. we are also working on building a network where we can get consistent weekly proof of reserves and expose any asset that fails verification. we also need to fix a few of the bugs and UI issues within the protocol (specifically mobile UI). whatever, we keep shipping. lets build for 16 hours straight today, together. btw the total $PRISM supply staked also went up to $1.2M😃 letsgoooooooo
Show more
hey fam, good morning. weekends back. lets spend this weekend resolving issues within the protocol, fixing bugs etc. today, we will work on: > finetune and polish V5 for launch > work on installable web apps > fix the mobile UI (its been a while since we last upgraded our mobile UI haha) > work on RWA perps and F&O > launch and test more dapps on prism L3 > add more assets and aggregate more liquidity. > permissioned dexs and V4 hooks > finetuning the asset marketplace > issuer vaults and more synthetics and many more, i just listed a few. everyones slowly talking about mass tokenization and with SEC exempt about to go live next month, we will see thousands of issuers issuing thousands of tokens. but no ones really thinking about the accessibility and scalability of those tokens and they will all come to prism. lets discuss more and build together for 16 hours straight today as we did last weekends. letsgoooooo
Show more
All that agent trace data sitting in your production logs — what if it could train its own specialized model? LangSmith just made that a built-in workflow. Title: Introducing LangSmith Fine-Tuning URL: ❓ What is LangSmith Fine-Tuning? It's the `smithtune` CLI, which turns production agent traces into a fine-tuned model without building custom infrastructure. It covers dataset creation, training, evaluation, and deployment end to end. ❓ How does it build the training data? It pulls trajectories — ordered sequences of messages and tool calls — from LangSmith projects, then uses a multi-agent review process with custom rubrics to keep only high-quality examples. Crucially, it preserves the exact context at each turn, including which tools were available. ❓ How do training and deployment work? Supervised fine-tuning via LoRA runs through managed platforms like Fireworks and Baseten, so there's no GPU provisioning to manage. After a replay evaluation against the base model, `smithtune deploy` ships the fine-tuned model straight to production. ❓ What results did they see? On an issue-detection task, fine-tuning lifted Kimi K3's score from 90.0 to 96.0. On code review, it matched or beat base-model quality while cutting model calls by 29.8% and tool requests by 29.4%. #LangSmith# #FineTuning#
Show more
there’s a certain rightness to the idea of a hippocampus for the models. a kind of chain-of-thought cache, built around the intuition that an intelligence ought to be changed by the work it does, even while its weights stay frozen. i keep thinking about the dot product between problems. metaphorically. two questions can share little vocabulary and still contain the same obstruction. different answers, but the same useful decomposition, the same assumption worth checking first. where the structure aligns, some fraction of the discovery cost ought to be recoverable. we wring beautiful deductions out of these intelligence engines and leave them in the sediment of a transcript. the answer survives. the expensive little maneuver that made it possible often remains unextracted. an engineer comes away from a difficult failure with an acquired suspicion. something gets checked earlier next time. expertise lives partly in this altered order of operations. i’d like the models to have somewhere for that alteration to persist. this is above prompt caching. something closer to memoizing how a problem became tractable. a decomposition, a diagnostic procedure, a failed approach with the reason it failed still attached. amortize the discovery, even when the answer must be recomputed. a transcript is a laboratory notebook, not a protocol. extracting the protocol requires deciding what was necessary, what was incidental, and what can be reproduced elsewhere. resemblance doesn’t authorize reuse. the reusable object should be somewhat lemma-like, carrying its assumptions wherever it travels. usually we won’t have a proof. we can still preserve tests, counterexamples, and the distinction between what worked once and what has been independently verified. a successful answer doesn’t certify every step that accompanied it. i used to think an AI hippocampus would mostly remember facts. now i’m interested in experience becoming procedure without first becoming a weight update. episodes, procedures, strategies, with a return path to the evidence whenever an abstraction becomes suspect. acquired competence outside the parameters. this is why i’m increasingly blackpilled on finetuning as the thing to build everything around. my bet is that successive general models absorb enough of today’s narrow specialization that i’d rather build the apparatus they inherit. tools, procedures, reasoning memory that survive a change of model. there’s a loose von neumann instinct here: an intelligence engine drawing on memory that holds both information and instructions. treat the model as an interchangeable cpu, with a hierarchy of reasoning caches backed by durable memory. useful procedures close at hand, the episodes behind them still addressable. let the surrounding architecture carry the burden of remembering, rather than requiring the engine to internalize every new experience. the architectural attraction is giving the engine and its accumulated experience separate lifecycles. some procedures will need rechecking. others will turn out to be workarounds for limitations the new model no longer has. but an upgrade shouldn’t require cold-starting the apprenticeship. does the first useful version look like reasoning traces in postgres, with retrieval, synthesis, and verification on top? the database isn’t the part i’m uncertain about. it’s how much of the work we can turn into a reusable method, and how cheaply we can establish that it applies. surely even a very marginal savings of a few percent tokens would gradually compound over time? isn't this the shape of continual learning?
Show more
Seeing a hype wave around GEN-1.5, and rightfully so. Lots of respect to Pete & Andy for executing so well. The secret is in the naturally repetitive motions in human-collected data. There're 2 main sources for such repetitions: (1) Symmetric patterns. Sorting, tidying, and assembling almost never finish in one motion. Open any assembly manual from IKEA, and you find most objects symmetrical. You drive one bolt, then its twin, then the next pair. Every {bolt A, bolt B} pair is a natural continuation in context, and the second instance is a free training signal that imitates the first ("prompt"). (2) Recovery. Humans drop things all the time, but we pick them up so fast, we don’t even notice. That reflex to fix is half of our physical competence. The key insight is to keep the failed first half instead of trimming it away. If the model consumes the full arc, fumble, catch, continue, then recovery shows up organically at test time. It's funny that in-context improvement results from *NOT* over-sanitizing your data. The other critical ingredient is UMI. I've been saying for a while that teleop will not last, and GEN-1.5 is driving the final nail in the coffin. UMI is essentially a human wearing the robot gripper to collect data directly (human → data). Teleop inserts a layer of separation: human → VR/skeletal device → robot → data, which bleeds out all the human "physical intuition". The subtle sleight of hand we perform constantly with objects, the micro-adjustments, the feel of a part snapping into place, is nearly impossible to capture when you can't feel the environment directly. Once you have enough data, many behaviors can actually be zero-shot. For example, you don't even need finetuning to pick up a novel object. The model "just knows" what to do given a similar scene in the training distribution. Whether in-context learning truly works or not also depends on how far away the test is from training. Currently, the demos are still a bit too simple to conclude. I'm cautiously optimistic. Still, it's a great day in robotics.
Show more
0
46
1.1K
132
Forward to community
Judging billions of trace tokens a day with a frontier LLM was financially untenable 💸 Here's how fine-tuning a small open model matched the accuracy at 10-100x lower cost. Title: Building a 100x Cheaper Trace Judge with Fireworks URL: 💸 Overview LangChain Labs partnered with Fireworks to build a "Perceived Error" detector for agent traces. It detects cases where users felt the assistant made a mistake or needed correction — using a small open model. ❓ Challenges Solved LangSmith processes billions of tokens daily across production traces. ・Evaluating them with frontier LLMs is prohibitively expensive at scale ・The question: how to cost-effectively mine important signals from every trace while maintaining frontier performance? 💡 Methodology & Proposed Approach ・Supervised fine-tuning (SFT) with LoRA on the open-source Qwen-3.5-35B, on Fireworks infrastructure ・Training data from two production datasets: chat-langchain (technical Q&A, 707 examples) and Fleet (no-code agent, 727 examples) ・It learns "Perceived Error," doing the judging without relying on huge frontier models 📊 Experimental Results ・Accuracy: the fine-tuned Qwen matched or exceeded frontier models (96.1% on chat-langchain, 90.8% cross-domain on Fleet) ・Cost: 10-100x cheaper than frontier models depending on trace volume ・Transferability: the model trained on chat-langchain beat all frontier models on Fleet without retraining #LLMEvaluation# #FineTuning#
Show more