Overfishing ist one of the largest threats to our ocean. Our goal is to create innovative tools to offer solutions to this problem. Really nice panel to promote #Sustainablefishing# #FishingTraceability# #BlueEconomy#

When too many fish are caught at once, the breeding population becomes too depleted to recover. Less fish in the ocean means less food and income for coastal communities. See how being #GenerationRestoration# can help prevent overfishing & protect biodiversity:

🧩 Your Agent Model May Be Overfitting the Harness, Not Learning the Task DeepSeek V4 Pro has exposed a growing Agent problem: the same weights can approach their ceiling under DSH’s minimal preset, then degrade under standard or third-party frameworks. Zhihu contributor 曾天真 compares reports from Kimi K3, Qwen, Kwai, and DeepSeek, then connects them to his team’s production experience. The core lesson: a model can understand the task while remaining unable to execute it outside its training interface. 1️⃣ What is harness overfitting? An Agent harness defines system prompts, tool schemas, history layout, result truncation, planning hooks, reflection timing, and stopping rules. Kwai’s KAT-Coder-V2.5 report divides overfitting into three types: 🔹 Format overfitting: changing the tool-call protocol causes parsing failures. 🔹 Context-structure overfitting: rearranging history, truncating results, or enabling compaction changes behavior. 🔹 Control-flow overfitting: planning and stopping depend on scaffolding provided by the training harness. The last type is especially dangerous. If a model stops planning because the new runtime has no todo tool, it has learned a protocol rather than a transferable capability. 2️⃣ Kimi K3: test with an unfamiliar harness The Kimi K3 report treats diverse, verifiable environments as a prerequisite for Agent RL. Its results explicitly name the harness used. Kimi also reserves MIRA as an out-of-distribution harness and evaluates every model under the same environment. K3 even reports the same benchmark under Kimi Code and Claude Code, with only a 0.8-point difference. This is stronger evidence of cross-harness stability than a general robustness claim. But K3 still has protocol coupling. It was trained with preserved thinking history. If a harness does not return the complete reasoning history across turns, generation can become unstable. Kimi therefore offers useful methodology, not immunity: task-level environment diversity cannot remove a hard dependency at the message-protocol level. 3️⃣ Qwen provides the cleanest controlled evidence The Qwen3-Coder-Next report states the problem directly: training with one tool-chat template often makes models memorize a particular output structure. Qwen trains across natural-language descriptions, JSON, Python-style calls, XML schemas, and TypeScript interfaces. Its strongest evidence is a controlled ablation. With data volume and training recipe fixed, increasing the number of tool templates improved SWE-bench Verified. Interface diversity may therefore improve the main benchmark, not merely reduce deployment failures. Qwen also evaluates models across five real CLI and IDE scaffolds. During RL, malformed tool calls receive token-level penalties. Qwen3.8 makes reasoning depth and thinking-history preservation configurable. Kimi treats preserved thinking as a requirement; Qwen exposes it as an option. 4️⃣ Harness Scaling must cover the right dimensions Kwai describes its solution as Harness Scaling, or domain randomization applied to Agent rollouts. The key is not the number of harnesses. It is whether they vary along dimensions that matter: 🔹 Tool protocols: structured function calls, code blocks, or tag-based formats. 🔹 Context management: full history, sliding windows, summaries, compaction, and different truncation policies. 🔹 Control flow: minimal ReAct loops versus explicit planning and self-reflection. This broader design matters because tool-format diversity alone cannot address context and control-flow dependence. Kwai also finds that a model may perform better under a simpler harness. More tools can increase unnecessary exploration and weaken stopping behavior. More scaffolding does not always produce a stronger Agent. 5️⃣ DeepSeek’s transparency made its coupling measurable The DeepSeek-V4 report publishes its XML tool-call schema and RL system prompt. In the open-source DeepSeek Harness, minimal keeps only Bash and str_replace_editor, disables context compression, and reproduces the training interface. A snapshot test is explicitly named: “sends the exact RL prompt and schemas” So minimal is not simply a lighter standard preset. It is a reconstruction of the interface used during RL. DeepSeek’s post-training pipeline also raises a broader concern. Domain specialists are trained with specialized prompts and rewards, then merged through On-Policy Distillation. Interface habits learned by those specialists may be distilled alongside genuine capabilities. DeepSeek also preserves complete reasoning history during tool use. Its report warns that frameworks simulating tools through user messages may not activate the intended context path. As with K3, reasoning-history structure becomes an implicit contract between the model and harness. 6️⃣ Production failures reveal what benchmarks miss The author’s team initially used one internal runtime for RL because it was stable, observable, and easy to connect to rewards. Deployment exposed the hidden coupling: · Unfamiliar tool names pushed the model toward shell workarounds. · Truncated tool results caused it to abandon partially correct work. · Without a planning tool, explicit planning disappeared. · Adding more MCP tools increased exploration and weakened stopping. The model still understood the task. It had learned to solve it inside one runtime. The team replaced that runtime with a randomized family of environments. They varied tool names, parameter styles, tool count, result truncation, and context policies, while keeping one canonical configuration for regression testing. This required more environment engineering and slowed debugging and convergence. But the benefit appeared in the worst integration. Average performance barely changed, while variance narrowed and complaints decreased. Teams should therefore track the worst integration score or the performance range across harnesses, not only the mean. 7️⃣ Distillation can carry interface pollution Teacher trajectories contain tool preferences, fixed call sequences, confirmation phrases, and output conventions from the teacher’s harness. Students may learn these artifacts as mandatory behavior, then request nonexistent tools or repeat unsupported boilerplate in production. Three fixes worked best: 🔹 Label task-related and interface-related trajectory segments, then rewrite or mask the latter. 🔹 Generate the same task under multiple harnesses and mix the resulting trajectories. 🔹 Add explicit examples for missing tools, failed calls, truncated results, and incompatible schemas. Finally, the harness must be treated as a versioned dependency. Its version belongs in experiment metadata. Prompt, schema, and truncation changes need review. Models and harnesses should ship with a compatibility matrix. The harness is no longer just infrastructure around the model. It is part of the training distribution and part of the model’s behavior. 🔗 Full analysis: #AIAgents# #AgentHarness# #DeepSeek# #KimiK3# #Qwen# #ReinforcementLearning# #LLM#

What if letting an agent evolve its own workflow just meant it got better at gaming the test? This paper tackles exactly that overfitting problem. Title: RRSI: Regularized Recursive Self-Improvement of Agent Harnesses URL: ❓ What's a harness? It's everything wrapped around the frozen LLM: prompts, control flow, tooling, memory, context management. The same model can perform very differently depending on this design. ❓ Why does self-improving it overfit? Three failure modes show up: fitting too tightly to the evolve-set benchmark, chasing noise, and letting complexity pile up unchecked. 💡 How does RRSI fix it? It applies classic ML regularization (L0, L1, L2) to harness evolution: an annealed edit budget limits changes per round, benchmark-specific proposals get screened out at selection, and unproductive components get pruned. 💡 What's the payoff? Across 8 benchmarks, RRSI beats baselines by up to 22.9% on unseen tasks, while using 30% fewer tokens. It feels like a genuinely grounded step toward agents that can safely improve their own workflow in production. #AIAgents# #SelfImprovement#

Today I am announcing META-Bench, the first pure intelligence benchmark for AI. It leverages the hit auto-battler strategy game, TFT. I SWEAR I AM NOT TROLLING let me explain. The industry suffers from labs overfitting and giving us models that score high despite being fundamentally low IQ. Over the years there have been many attempts at benchmarking AI with competitive gaming. I am going to explain the failure points, and why META-Bench is truly the first of its kind. Chess. When picking a game to benchmark with, chess is the obvious first choice. It has clear rules, large player base, and a well defined elo system. The issue with static rule games though is that the best strategies can be figured out ahead of time and baked into the model during the training process. Too easily hacked. Memorizing more strategies is not a proof of intelligence. Dota2/ League. We’ve all heard of OpenAI Five. The issue with benchmarking on a MOBA is that reaction speed is a meaningless metric. We do not need our highly intelligent AI to be able to respond at the speed of top human pro players. And truth be told, we are years away from a LLM that is able to play MOBAs at the highest levels off of vision alone, even though the problem is seemingly solved years ago. What we need is a game that: - Has defined rules but cannot be results hacked during the training process - Large ecosystem of human players - Clear cut results and an elo system - Results that is not reaction time dependent There is only ONE game in the world that meet all the requirements needed for this benchmark. Teamfight Tactics. For those unfamiliar, TFT is a strategy based auto-battler created by Riot Games with ~100 million monthly active players worldwide. It is a highly competitive multiplayer turn based game. It’s as if Chess and League of Legends had a baby that’s born to be an AI benchmark: - There is a new set released every 3 months. - Time limitations in the 10-40 second range rather than the milliseconds required for MOBAs - Skill based enough for esports yet uncertain enough to require reasoning over hard scripts “Can’t labs just train models to be good at TFT?” Nope and the reason why it’s unhackable comes down to how the benchmark itself is set up. Due to the fact that the entire game is changed every 3 months and patched every 2 weeks, any data on a previous TFT set is effectively useless when it comes to raw pattern recognition. Strategy wise, there are core concepts that carries over from set to set. That’s why we have the same players hitting the highest elo every season even though each set is so different. Any efforts at overfitting here can be fully negated if the benchmark harness used for all models has every core strategy built in. You are never going to beat a carefully curated harness layer with strategy training at the model layer. By presenting the models in the harness with the same core strategic concepts, the only difference in outputs will be its ability to reason across the different scenarios of each game. The luck elements of TFT already ensures that no 2 games will be the same in the reasoning required. Run the models against each other enough times and you will have a clear winner. Aka, the world’s first true IQ test for AI. I really, really want to know which AI model would win this. So I am going to build this. Not too sure how I’m going to fund it yet so if you would like to invest HMU. I’m also looking to put together a small team of individuals who are both high elo in TFT and highly experienced with agentic AI. And if you are even remotely curious on the results, like and help share this post 🫡

🎮 A game-playing AI that can walk around and drive a car in a commercial game it has never trained on - and it learned to play without ever touching a real game, only synthetic environments. Title: Player-1: Learning to Play from Synthetic Experience URL: Here are 3 highlights. 🕹️ A game-specific action space Relative pointer motion handles camera control, "sustained inputs" let a key stay held down, and "composite actions" (like holding W while tapping jump) let the model express the kind of coordinated controls games actually need. 🏗️ RL training across 16 synthetic environments 16 verifiable 2D/3D environments cover aiming, combat, driving, navigation, and UI interaction, with appearance, camera, and control sensitivity all randomized together to avoid overfitting to any single game. 🚀 Generalizing to games it never trained on On Minecraft (MineDojo-300, outside the RL training set), success jumps from 9.21% to 38.33%; on ViZDoom, scaled return improves 83.7%. In the unseen commercial game Watch Dogs 2, it walks, approaches a vehicle, gets in, and drives through the city. A nice finding on top: scaling synthetic environments from 2 to 16 steadily improves generalization to unseen games, which is a practical takeaway for building future game agents. #GameAI# #ReinforcementLearning#

🎮 A game-playing AI that can walk around and drive a car in a commercial game it has never trained on - and it learned to play without ever touching a real game, only synthetic environments. Title: Player-1: Learning to Play from Synthetic Experience URL: Here are 3 highlights. 🕹️ A game-specific action space Relative pointer motion handles camera control, "sustained inputs" let a key stay held down, and "composite actions" (like holding W while tapping jump) let the model express the kind of coordinated controls games actually need. 🏗️ RL training across 16 synthetic environments 16 verifiable 2D/3D environments cover aiming, combat, driving, navigation, and UI interaction, with appearance, camera, and control sensitivity all randomized together to avoid overfitting to any single game. 🚀 Generalizing to games it never trained on On Minecraft (MineDojo-300, outside the RL training set), success jumps from 9.21% to 38.33%; on ViZDoom, scaled return improves 83.7%. In the unseen commercial game Watch Dogs 2, it walks, approaches a vehicle, gets in, and drives through the city. A nice finding on top: scaling synthetic environments from 2 to 16 steadily improves generalization to unseen games, which is a practical takeaway for building future game agents. #GameAI# #ReinforcementLearning#

What if you could build an epidemic outbreak prediction model in minutes rather than weeks? Title: Planetary prediction engine: Automating global models via Earth AI URL: Google's Planetary Prediction Engine (PPE) automatically builds geospatial prediction models from natural language queries. For problems like public health, food security, and infectious disease outbreaks, it compresses weeks of manual data curation and feature engineering into minutes — no specialized engineering team required. Key Points 🌍 A three-stage automated pipeline — from query to model Stage 1 translates natural language into geographic constraints and retrieves covariates from Data Commons and Google Earth Engine. Stage 2 fuses structured covariates with embeddings from Population Dynamics Foundation Models and AlphaEarth. Stage 3 searches across regularized linear models, gradient-boosted trees, and MLPs. Each stage embeds a "Feature Gate" (leakage prevention) and an "Overfitting Guard Protocol" (self-correcting risk assessment). 🔬 Multimodal fusion outperforms either modality alone — across every experiment Combining structured statistical covariates with latent foundation model embeddings captures information neither source provides individually. This synergy was confirmed across all experiments, establishing a new architectural pattern for geospatial prediction. 📊 Validated across three high-stakes domains ・US CDC health indicators (21 types): R² 76.8% vs 60.0% baseline ・Nigeria food security downscaling: R² 66.1% vs 31.5% baseline (2× improvement) ・DRC Ebola outbreak prediction: Recall@10 83.3% (+10.3 percentage points vs Bayesian baseline) The real impact is democratization: humanitarian organizations and researchers without dedicated engineering teams can now get evidence-based predictions within minutes when it matters most. #EarthAI# #PublicHealth#

Harness optimization is getting real receipts. AutoSaddler is offline harness learning from agent failure traces. Not another prompt tweak loop. It patches prompts, tools, and middleware as code. Then it keeps updates that survive a held out set. On the test sets (Pass@1): GAIA2: 53.0 → 62.0 (+9.0) SWE-Bench Pro: 37.3 → 46.9 (+9.6) Terminal-Bench 2.0: 40.0 → 50.0 (+10.0) That TB2 number also clears the expert tuned Terminus KIRA at 47.5. Kill generalization aware selection and GAIA2 falls to 50.6, under the default agent. Deep diagnosis and structured patches help. Dev set filtering is what stops the harness from overfitting the mini batch. On GAIA2, Figure 1b, about 147 leveraged traces to the best dev score vs about 1,400 for Meta-Harness.