Chau Tran(@mr_cheu):2/n Model routing is context-dependent Model routing doesn't work at the LLM API level. The same request can have different difficulty level (hence different optimal model) depending on the context around it. Take the task "What is our biggest competitive threat nowadays" as an example. For some organizations, doing a simple search "competitive threat" will bring up an up-to-date canonical report from a competitive analysis team. For others, the agent actually needs to comb through Gong calls, external Slack channels, emails, Salesforce to get to the answer.

2026.08.26 17:07

2/n Model routing is context-dependent Model routing doesn't work at the LLM API level. The same request can have different difficulty level (hence different optimal model) depending on the context around it. Take the task "What is our biggest competitive threat nowadays" as an example. For some organizations, doing a simple search "competitive threat" will bring up an up-to-date canonical report from a competitive analysis team. For others, the agent actually needs to comb through Gong calls, external Slack channels, emails, Salesforce to get to the answer.

显示更多