Morgan Stanley's quant research group is about 30 PhD AI researchers, and they've been handing their model-building cycles to an agent harness that writes its own code, submits its own GPU jobs, and grades its own results. @brendanh0gan, a Machine Learning Scientist there, walks through how it works in "Morgan Stanley's ALPHALAB: Multi-Agent Research Across Optimization Domains", on @aiDotEngineer's YouTube. If you're building long-horizon agents that have to produce verifiable work, this one shows a real system, its results, and the failures that forced a redesign. - Three phases, not one loop. AlphaLab takes a data path and a natural-language goal, then runs research, builds its own evaluation, and moves into mass experimentation. - No off-the-shelf agent framework. They wrote the harness themselves, with functional tool calling so it stays provider agnostic across Anthropic, OpenAI, and open source models. - Three tools carry most of the weight. Full shell access, web search for reading arXiv and technical blogs, and a Slurm abstraction so the agent asks for four H100s instead of orchestrating hardware. - The eval gets adversarial review. A builder agent writes the evaluation, then a conceptual critic looks for forward leakage and a programmatic critic writes unit and integration tests. The loop doesn't close until all three are satisfied. - Experimentation as a Kanban board. A strategist agent files experiment cards, workers implement and submit them, and each post-mortem feeds back so the strategist can drop transformers and chase tree methods instead. You can cancel cards, add your own, or chat with the strategist. - Measured results. Top 12% on an NVIDIA-hosted Kaggle competition fine-tuning Nemotron, with only 10 iterations. Internally, several improved models are working through risk toward production. - 2.0 makes the eval strict. Data and description in, containerized models out, public leaderboard score as feedback, private held-out validation for the user. That format turns evals into environments, which turns them into a reinforcement learning signal. - They're AlphaLab-ing AlphaLab. With 10 to 20 careful environments built, an LLM reads the traces and optimizes the harness itself, instead of humans arguing over whether there should be two strategists. - Environments are where the expertise lives. Proprietary data plus qualitative rubrics graded against what a good researcher's thought process looks like. Brendan's argument is that general auto research becomes a commodity, and the enterprise value sits in the environments. I'm working through the published talks from AI Engineer World's Fair sharing summaries and takeaways. Follow for more!