One challenge facing legal AI is the context window. This is fairly straightforward: as data rooms grow, search becomes more complex. And it doesn't just apply to law: anyone working with long-running tasks has to grapple with context windows.
Currently, methods to optimize context windows in the industry range from summarization to KV cache compaction to true continual learning in weights.
Show more
will open-source models become the majority of inference?
@thdxr weighs in: YES
