An idea to replicate jev: any LLM can give the probably for a given combination of context + question + option
Just make sure the answer is binary (yes/no or true/false), then we can simply get the softmax of their logprobs
Maybe someone can take this idea and try it, idk. I'm too sleepy now.
Show more
