@liquidai AI just released LFM2.5 encoders small bidirectional models with 8k context.
I've added a training recipe for fine-tuning them on
@huggingface Jobs (works for any encoder), and used it to train a classifier that suggests task categories for Hub datasets from their READMEs.
One command, 20 minutes on an A100, about $0.80.