Love the fact that Jev is bringing focus to structured / open-ended classification problems with encoder-style models. There is so much to be unlocked here... it brings a new dimension to LLM workflows with a lot of interesting opportunities. Interestingly enough, this was one of my earliest research interests. See the paper below where I trained a multimodal BERT model to solve >1000 multi-class classification problems with a single model. These types of models are really capable and can do so much if you train / scale them properly! However, it's worth mentioning that Jev goes well beyond basic encoder classification. Some examples of novel components are: - Generalizing to any classification space in a scalable manner with good inference efficiency (probably some improved version of CLIP-style contrastive approach). - In-depth, data-centric research to do this properly and maximize model performance with training scale. - Figuring out how to structure the model in a way that yields really impressive inference efficiency. These are all changes that create something completely different than just encoder + classification. Even small details make a massive difference in adoptability and utility for these types of models. Jev (or similar models) provide an orthogonal tool to standard autoregressive LLMs that is really helpful and can be integrated into LLM workflows with minimal added cost. There are so many directions we can go with these models IMO. Need to do a deeper dive and will follow-up with a full writeup soon.