simple example but i think this is a balanced take about where different agents shine. (spoiler: dots shine on intelligence for hard tasks)
i just gave the exact same task to Muse, Dots, and Instinct (didn't do grok bot because my quota's already tight)
i asked them to research good product names for something i want to ship, and i gave a bunch of criteria for what i look for
1. muse was very fast, but outcome was questionable
muse was the only agent that went straight into the work without asking me any questions - not sure i like this
but boy was it fast. i did 3 rounds of iterations with muse while dots and instinct were both still working on the first iteration
BUT... guess what was the top suggestion from muse?
it was "endframe" 🤦 i was like.. dude, did you not watch silicon valley?
in follow up iterations, i also noticed muse was not quite good at following my intent in a way i like. for example, i said in the very beginning that every name needs to check for domain availability, but later i found out that some names weren't checked
overall, it seems to optimize for quick iterations rather than thoroughness
2. instinct was very slow, but quite autonomous
instinct first asked me some questions to anchor its work. i liked that because it helps steer its work to be more aligned with what i want
then it went dark for 20 minutes to do the work. it did a pretty thorough research and came back with a file i could open and read. the file had a table with all the candidates and their readiness
overall i thought that was pretty good, except for the names themselves.. they aren't that good. this feels like a general problem with LLMs - they seem to be really bad at coming up with names humans would really find appealing
3. dots was also slow, but it's more "asynchronous" and more intelligent
like indistinct, dots also started with asking me some questions, but it didn't let the questions block its work!
it had already started work in the background, and my questions were sent to it asynchronously as steers. i think this shows openai's slightly more experienced with harness engineering
it also didn't go dark for the full duration of its work. it kept giving me updates about its progress, which was useful because i spotted some names i liked and didn't like, and gave it feedback early for what to lean into
also gotta say, having astra in dots is clearly an advantage. the model is just smarter. it kept all my inputs in its mind, including the original goal i stated plus all the follow up steers. it "gets me"
dots was also the only solution that actually gave me a few names i'd consider
just a one-off experience here - take it with a grain of salt. i'll share more examples like this when i give them tasks!
Show more