Key-Value Cache (KV cache)

The Key-Value Cache (KV cache) stores information from tokens the model has already processed so it can reuse that work.