Antaripa Saha
@doesdatmaksense
consulting companies in evals, search, retrieval
466 Following 15.2K Followers
100% of the data is synthetic
people are often skeptical of synthetic data. it just proves how carefully constructed synthetic data can be incredibly powerful
got access to Jev!!
heard a lot of good things about it, especially as a classifier.
really curious to try it for evals. lots of llm-as-a-judge for binary decisions are going to get replaces. give Jev some state + a bunch of questions and get back probabilities you can actually threshold on.
also feels pretty useful inside agent harnesses as a control plane for making all those fuzzy decisions like which model or tool to route to, whether to continue or retry, whether an output looks good enough, whether something needs human review, etc. basically letting the llm handle generation, while Jev handles a lot of the judgment calls around it.
Show more
