In-context continual learning requires models to accumulate experience and reuse it later in the same sequence. But an RNN compresses an ever-growing history into a fixed-size state, where each token gets a single write into memory.
We study dynamic compression: letting the model revisit the past and reorganize its state as it discovers what needs to be reused.
Show more
