가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

FleetingBits
@fleetingbits
sf thinkcat
가입 September 2023
381 팔로잉 중    10.9K 팬
some thoughts on jev 1) i tend to think that twitter hype is not relevant for ai product releases; especially when someone claims to be doing something novel with the architecture 2) but, i think that jev is interesting in principle; it is a low cost, fast classification service, and it claims to be about equivalent to gpt-5.6-terra in performance 3) something interesting here is that this is what most ai services looked like in 2022; cohere sold a classification api, an embedding api, a reranker 4) the problem was that, in addition to not being as effective as gpt-3 and then gpt-4, these services were much harder to use as a developer 5) you had to decide which apis to use, then write the code to glue them together; so, you got less performance for more work 6) but now, we can imagine coding agents making it much easier to select and integrate a cheaper service with specialized characteristics 7) evals are still a bottleneck, though; you need some way to automatically compare performance between services using their apis 8) and, enterprise companies often have a hard time building evals that reflect their actual use cases; it's not a core expertise 9) but, the better coding agents get at integrating and switching between services, the more valuable those evals become; 10) this means companies have more reason to figure out the evals problem; 11) maybe that means getting rights to use customer data for testing, or getting better at purchasing synthetic test data, or something else 12) so, i think there is an interesting world where coding agents create more market for specialized api services, even specialized model services 13) which can compete either on cost or on capabilities; with the winner determined by cost basis or specialized data, etc... 14) note, it might be unclear but if specialized models services becomes an important direction, i expect the frontier labs to have an edge on cost basis 15) and, i think that, at least for now, these specialized model services probably do not provide enough of a cost difference to be relevant
더 보기