Hidden state

A hidden state is the model’s temporary numerical representation of a token at a processing stage.