How Does a Machine Think?

There is a peculiar moment that happens to almost everyone who spends enough time with a good AI model.

You begin perfectly aware that you are using software.

You type into a box. A computer performs calculations. Words appear.

Then, perhaps twenty minutes later, you ask something complicated. The machine notices a detail you mentioned ten minutes ago, connects it to something else, catches a contradiction in your reasoning, and produces an answer that is not merely grammatical but apparently relevant.

And at some point the obvious question arrives:

What, exactly, happened inside the machine between my question and that answer?

Not philosophically.

Mechanically.

What did the computer actually do?

Let us follow one question all the way through the machine.

Suppose you type:

My dog stole a whole chicken from the kitchen table. Should I call the vet?

For you, the sentence arrives already wrapped in meaning.

There is a dog.

There was a chicken.

The chicken has apparently changed ownership without authorization.

There may be bones involved.

You are worried.

The dog, one suspects, has no regrets.

A Large Language Model receives none of this as an image or experience. It does not picture a Labrador standing beside an empty plate. It has never smelled roasted chicken. It has never driven an animal to an emergency veterinarian at eleven o'clock at night.

It receives symbols.

And the first thing it must do is turn those symbols into numbers.

Step One: Tokenization — breaking language into machine-sized pieces

The model cannot directly process your sentence as ordinary English.

Before the neural network sees it, a piece of software called a tokenizer breaks the sentence into smaller units called tokens.

A token may be an entire word.

It may be part of a word.

It may be punctuation.

It may be a number, a space, or part of an unfamiliar name.

So:

My dog stole a whole chicken.

might become something conceptually like:

My | dog | stole | a | whole | chicken | .

That example is simplified; different model families use different tokenizers.

Each token is then assigned a numerical identifier called a token ID.

The word dog does not travel through the model as the letters D-O-G.

It might enter as something like:

Token ID 8,214

The actual number is not meaningful to us. It is simply an address in the model's vocabulary.

At this point the machine has converted your sentence into a sequence of integers.

That sounds like progress only in the very generous sense of the word.

Because the number 8,214 does not explain what a dog is.

For that we need another transformation.

From Token IDs to Embeddings

The model converts each token ID into an embedding.

An embedding is a list of numbers representing the token in a mathematical space.

Another useful word here is vector.

A vector, in this context, is simply an ordered collection of numbers.

Instead of representing dog as one meaningless integer, the model may represent it with hundreds or thousands of numerical values.

Very roughly:

dog → [0.17, −0.42, 0.08, 1.31, …]

There may be thousands of numbers in that vector.

Do not try to assign human meanings to individual positions.

The first number does not mean “has fur.”

The second does not mean “occasionally destroys furniture.”

Meaning emerges from the entire pattern.

You can imagine embeddings as coordinates on an enormous map.

But it is not a normal map with north, south, east, and west.

It may have several thousand mathematical dimensions.

That makes it terrible for road trips but very useful for machine learning.

On this strange mathematical map, related concepts tend to develop related representations.

Dog may be closer to puppy than to mortgage.

Veterinarian may have relationships with animal, medicine, clinic, and treatment.

Chicken may connect with food, bird, bones, cooking, and thousands of other contexts.

Nobody typed all those relationships into the model.

They emerged because training repeatedly adjusted the model's weights until useful relationships appeared.

But we are not finished.

There is another problem.

The word bank can mean a financial institution.

Or the side of a river.

The embedding for the word alone cannot tell us which one you mean.

Meaning depends on context.

And context is where the Transformer begins earning its salary.

The Transformer Block: where the real processing begins

Modern Large Language Models are usually built from many repeated processing stages called Transformer blocks.

You can think of each block as one floor of a very tall building.

Your token representations enter the first floor.

They are transformed.

They move upward.

The next floor receives the changed representations and transforms them again.

A modern model may contain dozens or even more than a hundred Transformer layers.

By the time the information reaches the top, the representation of the sentence is vastly richer than when it entered.

Inside each Transformer block are several important mechanisms.

The famous one is called:

self-attention.

And this is where we need three of the strangest names in modern AI:

Query, Key, and Value.

Usually abbreviated:

Q, K, and V.

Query, Key, Value — terrible names, useful idea

The terminology comes partly from information-retrieval systems, and unfortunately nobody stopped to ask whether ordinary people would ever have to learn it.

But the idea is manageable.

Imagine a cocktail party.

Everyone in the room is wearing a name tag containing information about themselves.

One person is currently looking for:

“Someone who knows about veterinary medicine.”

That is like a Query.

Each other person's name tag contains information describing what they represent.

That is somewhat like a Key.

When the query strongly matches a particular key, we pay more attention to the information carried by that person.

That information is the Value.

In a Transformer, every token creates its own Query, Key, and Value vectors.

Not metaphorically—actual numerical vectors are calculated using learned weight matrices.

The word dog creates Q, K, and V representations.

So does chicken.

So does vet.

So does every other token.

The model then compares Queries against Keys to calculate:

Which other tokens should this token pay attention to, and how much?

Attention scores: deciding what matters

Let us return to our unfortunate chicken.

The model reaches the phrase:

Should I call the vet?

The representation associated with vet may develop strong relationships with earlier tokens such as:

dog
chicken
stole
whole

and weaker relationships with less important pieces such as:

the

This relationship is represented mathematically as an attention score.

Technically, the model calculates something called a dot product between Query and Key vectors.

A dot product is simply a mathematical way of measuring how strongly two vectors align.

You do not need the detailed arithmetic.

Conceptually:

Strong match → larger score
Weak match → smaller score

The scores then pass through a mathematical operation called softmax.

Softmax turns a collection of raw scores into normalized weights.

In plain English:

It converts “who seems important?” into “how much attention should I actually give each one?”

Perhaps, purely as an illustration:

dog → 35% attention
chicken → 28%
vet → 21%
table → 6%
the → almost nothing

Real attention is much more complicated, and multiple attention heads operate simultaneously, but the intuition is useful.

The model then combines the corresponding Value vectors according to those attention weights.

This produces a new representation containing information gathered from relevant parts of the context.

The token no longer represents merely itself.

It now carries information about its relationships with other tokens.

The equation engineers use

An engineer will often write self-attention like this:

Attention(Q, K, V) = softmax(QKᵀ / √d) V

At first glance this looks like somebody dropped a calculator down a staircase.

But now we can translate it.

Q = what this token is looking for.

K = what the other tokens advertise about themselves.

QKᵀ = compare what is being sought with what is available.

√d = a scaling adjustment that keeps the numbers mathematically well behaved.

softmax = convert the comparison scores into attention weights.

V = collect the actual information from the tokens according to those weights.

That intimidating equation is essentially saying:

Look around, decide what matters, and combine the relevant information.

Suddenly it is not quite so terrifying.

Multi-Head Attention: looking at the same sentence several ways

A Transformer usually does not perform only one attention calculation.

It uses Multi-Head Attention.

An attention head is one independent attention mechanism.

Why have several?

Because the same sentence contains several kinds of relationships at the same time.

Take:

The dog that stole the chicken is now sleeping peacefully because it has no conscience.

One attention head might become useful for grammatical relationships.

Another may focus on which noun it refers to.

Another might track the main subject.

Another may capture longer-range semantic relationships.

We should be careful not to imagine that every head has a neat human-readable job such as:

Head Number 17 = Chicken Theft Department.

The specialization is not necessarily that clean.

But multiple heads allow the network to examine the same context through different learned relationship patterns simultaneously.

That is why the model is not merely reading words from left to right.

It is constructing a web of relationships among them.

The Hidden State: what the model currently “knows” about the token

After attention, each token now has a richer internal representation.

This representation is commonly called a hidden state.

The word hidden simply means that these numbers exist inside the neural network rather than being directly visible to the user.

This is an important idea.

The model begins with:

“dog”

as a token.

Several layers later, its hidden state may implicitly represent something more like:

the dog belonging to the user, which has just consumed a whole chicken and is the subject of a possible medical concern

There is no English sentence inside the model saying that.

The hidden state is still a vector of numbers.

But mathematically it now carries much more contextual information.

And then it goes into the next Transformer block.

Attention again.

Transformation again.

Another hidden state.

Again.

Again.

Layer after layer.

The Feed-Forward Network: thinking about what attention found

Attention gets most of the publicity, but Transformer blocks contain another major component called the Feed-Forward Network, often shortened to FFN.

You can think of attention as gathering relevant information.

The Feed-Forward Network then performs additional transformations on that information.

If attention is the newsroom saying:

“These facts are related,”

the Feed-Forward Network is closer to:

“All right. What should we make of them?”

Technically, this involves large matrix multiplications.

A matrix is essentially a rectangular table of numbers.

Modern neural networks spend an extraordinary amount of time multiplying enormous matrices together.

This may sound mundane.

It is also the reason GPUs are so useful.

Graphics Processing Units were designed to perform many mathematical operations simultaneously.

Rendering millions of pixels and processing neural-network matrices turn out to have surprisingly compatible hardware requirements.

Humanity spent decades building faster graphics cards so computer games could have more realistic explosions.

Then researchers discovered that the same hardware was remarkably good at artificial intelligence.

Technology rarely takes the route originally advertised in the brochure.

Activation Functions: introducing nonlinearity

Inside the Feed-Forward Network we encounter another term:

activation function.

Without activation functions, stacking many neural-network layers would be far less powerful.

An activation function introduces nonlinearity.

That phrase deserves translation.

If the entire network only performed simple linear combinations, many layers would mathematically collapse into something equivalent to one large linear transformation.

Nonlinear activation functions let the network represent far more complicated relationships.

Modern Transformers often use functions such as GELU, SiLU, or related variants.

You absolutely do not need to memorize those names to use an LLM.

The important idea is:

Activation functions allow the network to bend, gate, and reshape information instead of merely scaling and adding it.

Think of the difference between a ruler and a box of Lego.

Linear operations give you excellent rulers.

Nonlinearity lets you start building things.

Residual Connections: don't throw away what you already knew

Transformer blocks also use something called a residual connection.

This is a wonderfully sensible idea hidden behind another technical term.

Instead of completely replacing the previous representation with a new transformation, the model adds much of the original information back in.

Conceptually:

new representation = old representation + new information

Why?

Imagine editing a document.

You would not normally rewrite the entire document from memory every time you correct one paragraph.

You preserve what already worked and add the improvement.

Residual connections help information flow through very deep neural networks without being repeatedly destroyed or distorted.

They were one of the important innovations that made very deep networks practical.

Layer Normalization: keeping the mathematics under control

There is also Layer Normalization.

As values travel through dozens of layers, their numerical scales can become difficult to manage.

Layer normalization rescales the internal values into a more stable range.

Think of an audio engineer receiving tracks recorded at wildly different volumes.

One microphone whispers.

Another sounds like it was placed inside a jet engine.

Before mixing the music, the engineer needs reasonable levels.

Layer normalization performs an analogous mathematical housekeeping job.

It does not provide intelligence.

But without good housekeeping, the intelligent part becomes much harder to train.

Even artificial intelligence apparently needs somebody to tidy the room.

And this happens layer after layer

Now let us pause.

Your original sentence:

My dog stole a whole chicken from the kitchen table. Should I call the vet?

has gone through:

Tokenization

→ converted to token IDs

→ converted to embeddings

→ transformed into vectors

→ passed into a Transformer block

→ processed through self-attention

→ Queries compared with Keys

→ attention weights calculated with softmax

→ Values combined

→ transformed by a Feed-Forward Network

→ passed through activation functions

→ preserved with residual connections

→ stabilized through Layer Normalization

→ turned into richer hidden states

And then the whole process repeats through another Transformer layer.

And another.

And another.

This is much closer to what is physically happening when we casually say:

“The model is thinking.”

But there is one more thing that is easy to miss when we write this process out slowly on a page.

The computer is doing all of this at extraordinary speed.

Somewhere, a data center just woke up

If you are using a cloud AI service, your question may travel across the Internet to a data center filled with racks of computers.

Inside those racks are powerful Graphics Processing Units (GPUs) designed to perform huge numbers of mathematical operations in parallel.

The model's learned weights are already loaded into the memory of those machines.

They are not being created when you ask your question.

They were created earlier, during training.

You can think of those weights as the permanent mathematical structure the model acquired during its education.

Your question does not get written into them.

Instead, your question travels through them.

That distinction is important.

When you type:

My dog stole a chicken. Should I call the vet?

the model does not stop and retrain itself.

Its billions of learned weights are already sitting there, waiting.

Your input enters.

It becomes tokens.

Tokens become embeddings.

Embeddings become hidden states.

Those hidden states move through layer after layer of learned weight matrices.

Attention compares relationships.

Feed-Forward Networks transform them.

Residual connections preserve useful information.

Normalization keeps the whole numerical operation stable.

The incoming text is gradually transformed into richer and richer internal representations.

You might loosely say that the original sentence is being compressed into mathematical meaning, although “compression” here does not mean that the information is being stored back into the weights.

The weights remain largely fixed.

What changes is the temporary internal state produced by passing your question through those weights.

And all of this can involve billions upon billions of multiplication and addition operations.

Yet on modern hardware, the first part of the answer may begin appearing in a fraction of a second or within a few seconds, depending on the model, hardware, system load, and size of your prompt.

Think about that for a moment.

You press Enter.

Your sentence leaves your computer.

Somewhere, perhaps hundreds or thousands of miles away, GPUs begin multiplying enormous matrices.

The sentence is broken apart, turned into vectors, compared across attention heads, transformed through dozens of layers, reduced to probabilities for possible next tokens—

and then an answer starts coming back.

To you, it looks like:

“If your dog ate a whole chicken, the important question is whether it contained cooked bones…”

It feels like a conversation.

Inside the data center, an enormous mathematical factory has just gone to work.

And if you are running the model locally, the same general process is happening on the GPU sitting inside your own computer.

No distant data center.

No mysterious cloud.

Just your graphics card quietly performing an absurd quantity of mathematics while its cooling fans reconsider their life choices.

But eventually the model must say something

After all of those layers, the model has built a rich hidden representation of the current context.

Now it must convert that internal state back into language.

The final hidden state is transformed into a large list of numbers called logits.

A logit is basically a raw score for a possible next token.

Suppose the response is about to begin.

The model might produce something conceptually like:

If → 8.4
Because → 7.9
Your → 6.7
Chicken → 2.1
Volkswagen → −3.8

Again, these are invented numbers for illustration.

The model has not yet chosen the word.

It has produced scores.

Those logits are then converted into probabilities, typically using softmax again.

Now perhaps:

If → 31%
Because → 22%
Your → 14%
Since → 11%
hundreds of other tokens → the remaining probability

The system must decide which token to generate.

Temperature and Sampling: why the model does not always say the same thing

If the model always selected the single highest-probability token, responses could become repetitive and rigid.

Instead, generation systems often use sampling.

Sampling means choosing among probable tokens according to their probability distribution.

A setting called temperature changes how adventurous that choice becomes.

At low temperature, the model strongly favors the most probable tokens.

It behaves more conservatively.

At higher temperature, lower-probability alternatives become more competitive.

The writing may become more varied or creative.

It may also become more eccentric.

Think of temperature as the difference between hiring:

a careful accountant

and

the same accountant after three espressos and an improv class.

Both may be useful.

Not necessarily for the same assignment.

Other generation controls include top-k sampling and top-p sampling, which restrict which candidate tokens are allowed into the selection pool.

Again, the details are configurable, but the central point is simple:

The neural network produces possibilities.

The decoding strategy chooses among them.

Autoregressive Generation: one token changes the next

Now the model selects its first output token.

Perhaps:

If

That token is added to the context.

The model runs again.

Now it is processing:

My dog stole a whole chicken from the kitchen table. Should I call the vet?
If

It predicts the next token.

Perhaps:

your

Then:

dog

Then:

ate

And so forth.

This is called autoregressive generation.

Auto means the system feeds its own generated output back into the next prediction.

Regressive, in this context, means the next value depends on previous values in the sequence.

So the answer is not written all at once.

It grows.

Token by token.

Every generated token slightly changes the context from which the next token will be predicted.

That is why the jazz analogy works so well.

A musician plays a note.

That note changes what makes musical sense next.

The next note changes the possibilities again.

A language model does this with tokens.

So is there a point where the model “has the answer”?

This is an interesting question.

Usually, not in the way a human imagines it.

There is no clear instant when the model internally writes:

COMPLETE ANSWER FOUND. NOW PRINT.

Instead, there is a continuously evolving internal state.

Each layer refines representations.

Each generated token modifies the context.

The answer emerges sequentially.

This helps explain some strange behaviors.

A model may begin an answer confidently and later realize—through its own newly generated context—that another direction makes more sense.

It may correct itself.

Or contradict itself.

Or occasionally drive directly into a conceptual ditch while maintaining excellent grammar.

Generation is a process, not retrieval of a finished object.

Where does the Key-Value Cache fit into all of this?

Now we can finally understand the Key-Value Cache (KV cache) properly.

Remember Query, Key, and Value?

Every Transformer layer calculates Keys and Values for the tokens it processes.

When generating the next token, most of the earlier Keys and Values do not need to be calculated again.

They have already been computed.

So the inference software stores them in the KV cache.

The KV cache is therefore not some mysterious extra feature glued onto the model.

It is directly connected to the attention mechanism we just described.

The cache says:

“We already calculated the Keys and Values for the previous tokens. Keep them nearby so we don't redo the same work.”

As the conversation grows, there are more previous tokens.

More tokens mean more cached Keys and Values.

So the KV cache grows.

And because it occupies VRAM, longer context requires more GPU memory.

Now the relationship becomes logical rather than something you simply have to memorize:

More context → more stored K/V information → larger KV cache → more VRAM used.

That is precisely the kind of relationship Green Zone is trying to make visible.

So, finally: How does the machine think?

Let us return to the dog and the stolen chicken one final time.

You typed a sentence.

The computer did not understand that sentence in the effortless, immediate way you did.

It had to build that understanding.

First it broke your language into tokens.

Then it turned those tokens into vectors.

Those vectors entered a network containing billions of learned weights.

Self-attention allowed different parts of the sentence to interact.

Query, Key, and Value vectors helped the model decide which relationships mattered.

Multi-Head Attention allowed several different relationship patterns to be examined at the same time.

Feed-Forward Networks transformed the resulting information.

Activation functions allowed complicated nonlinear relationships.

Residual connections carried useful earlier information forward.

Layer Normalization kept the numerical system stable.

And each Transformer layer produced richer hidden states than the layer before it.

By the time the information reached the final layers, the internal mathematical representation no longer corresponded merely to isolated words such as:

dog
chicken
vet

The network had constructed something much closer to a representation of the situation:

A person is worried because a dog has eaten something that may contain dangerous bones, and the person wants to know whether medical attention may be necessary.

There is no English sentence inside the model saying that.

There is no tiny doctor examining the dog.

There is a complicated state made of numbers.

But those numbers now encode relationships rich enough that the model can generate an appropriate continuation.

The final hidden state is turned into logits.

The logits become probabilities.

A decoding algorithm chooses a token.

That token becomes part of the next input.

Another token is generated.

Then another.

While this happens, the previously calculated Keys and Values remain available in the KV cache, allowing the model to continue the conversation without repeating every calculation from scratch.

And all of this—tokenization, embeddings, attention, matrix multiplication, hidden-state transformation, probability calculation, decoding, caching—can happen fast enough that the answer begins appearing almost immediately.

There is no single operation inside the machine called thought.

There is no special “intelligence equation.”

Instead, something more interesting happens.

Thousands of relatively simple mathematical operations are repeated across billions of learned parameters, over and over, layer after layer, until a complex representation emerges.

Think of a piano.

A single hammer is not music.

A string is not music.

A wooden key is not music.

The pedal is certainly not music.

You can disassemble the entire instrument and examine every piece without finding Beethoven hiding underneath the soundboard.

Music appears when the parts are arranged correctly and set into motion.

A neural network is similar in one important way.

A single weight is not intelligent.

A matrix multiplication is not intelligent.

A token is not intelligent.

Even an attention head, by itself, is not intelligent.

The remarkable behavior appears from the interaction of the whole system.

And perhaps that is the strangest part of modern artificial intelligence.

We did not discover a mathematical symbol meaning:

intelligence = 42.

We learned how to build enormous systems in which simple mathematical operations, carefully structured and trained on vast amounts of data, can produce behavior that looks astonishingly sophisticated.

You type:

Should I call the vet?

Somewhere, billions of numbers go to work.

And a moment later, the machine answers.

Which leaves us with the obvious next question:

Where did those billions of useful weights come from?

Because when the model was first created, those weights knew nothing about dogs.

Or chickens.

Or veterinarians.

Or Python.

Or France.

Or Shakespeare.

They had to learn.

And that is where the next story begins.