Trying to determine how much it could cost me to benchmark Muse Wonder...every other model I have benchmarked has been able to do it no problem, in a sub. Muse Code's $50/mo tier and can't get through even all the tasks in my benchmark suite at low effort. Looks like I would need a budget between $310-$770 to benchmark Muse Wonder vs. $0 for models from companies like Anthropic and OpenAI since I can run those in my subs. So...trying to think of alternate paths. I've been told that potentially Muse Code just might not be a super optimal harness, and Pi might be a better harness to use it with. Might try this before calling it. Such a bummer because I really like the team at Meta, but I have no horse in this race and have to be independent and just let the data guide me. And right now, I can't possibly understand why I would code with a model that is so much more expensive than the latest models from Anthropic and OpenAI. But, not giving up yet, just can't put $500+ into benchmarking this if I already know I wouldn't recommend it for coding with the early cost-per-task data I'm seeing. Next I'm going to compare a few tasks at low effort in Pi to see if the cost-per-task is lower than in Pi...hoping it's just the harness and not the model 🤞