If you're automating any part of your software development lifecycle (i.e. loops or software factories), you should be on the lookout for these flags: 🚩 Single model or single harness solutions: this is a form of lock-in that will give the model provider leverage over prices, and will make it harder to tune the token costs for your SDLC overtime. Data ownership: make sure that whatever vendor you use doesn’t capture or train on your data. All the data flowing through the SDLC should be owned and housed by your company. Compute inflexibility: you probably want a variety of different hosting solutions, including self-hosting. Forced token reselling: vendors should allow you to bring your own inference endpoints (e.g. Bedrock, Vertex and Azure) and generally not be in the business of selling or re-selling tokens alone. Folks in the token selling business are going to have a conflict of interest when it comes to optimizing your costs, so it’s better to pick a vendor that is agnostic to whether you buy inference through them.