64 GB of RAM looking for work. Preferably something intelligent.

Read a few guides to running AI at home, and before long you get the feeling that the first step is to buy a graphics card. Then it turns out you should buy a different one. Then you discover it will need a new power supply. At this point, you feel like closing the browser, because you were only planning to try chatting with a model, not rebuild your computer and revise the family budget.

And all this time, a perfectly decent machine is sitting under your desk.

Maybe it's a few years old. Maybe it's an old workstation that was once bought for serious work. Some people even have a server left over: two processors, plenty of memory, an imposing metal case. It used to serve an office. These days, it mostly serves its own fans.

It seems a shame. The computer works, and it has 32, 64, or even 128 gigabytes of memory. Is all of that really useless for a language model without an expensive graphics card?

No, it isn't. And this is where things get rather interesting.

Language models can run on a processor using ordinary system memory. Programs such as llama.cpp support CPU computation. Your chosen model needs to be supported by the particular version of the program, but the basic capability has long since moved beyond impressive demos. llama.cpp documentation

We sometimes say “run a model in RAM,” which is a convenient shorthand. But memory doesn't do any computing by itself. It holds the model's weights and working data, while the processor performs the operations. Think of a pantry and a cook: a spacious pantry is useful, but dinner won't appear just because it's there.

Incidentally, you don't necessarily need a permanent internet connection to chat with a model this way. Once you've downloaded the files, it can run locally, provided the application itself doesn't use the cloud or external tools. The internet has already helped deliver a few gigabytes and can now take a well-earned rest.

This all sounds encouraging, but it's better to look at what happened when people actually tried it.

We found several published examples. We haven't repeated these measurements ourselves, so what follows is the authors' own results, using their hardware, software, and settings.

The second example is the one that particularly appeals to me. A processor from 2013, DDR3, an old workstation — all the things that are easy to dismiss in a conversation about modern AI. And someone simply went ahead and tried it.

The third example is, of course, in a different weight class altogether. The RAM alone takes up more space in my imagination than my entire original idea for this article. But you certainly don't need a server like that to get started. You don't have to rent the Atlantic Ocean to learn to swim.

One more thing: don't read these rows as a processor competition. The models are different, the programs are different, and so are the measurement conditions. Declaring the old Xeon the winner over the Ryzen just because one row has a bigger number would be quite a leap.

What we can see, though, is that the lack of a modern graphics card doesn't mean the experiment is off. Even DDR3, it turns out, doesn't turn into a pumpkin on the day DDR5 arrives.

Now let's return to a more familiar situation. Suppose you have a computer with 64 gigabytes of RAM. You open a model catalog, find a file of around 60 gigabytes, and think: there we go, almost exactly the right size. There'll even be a little room left.

This is where I'd slow down a little.

The operating system isn't moving out, after all. The browser is staying too — along with all those tabs you will definitely read, just not today. And the program running the model will need some room to work.

Besides the weights, memory also holds working buffers and data for the conversation being processed. In many models, a substantial part of that data is the KV cache: stored intermediate results that let the model avoid repeating some of the computation for text it has already read. Its size depends on the model's architecture, the context length, and the settings. KV cache explained

So the file size on the download page is a useful number, but it isn't the final answer to “how much memory will I need?”

In that same HP Z420 experiment, the author gives a telling comparison: the model file was approximately 29.3 GB, while the process's memory usage, measured as RSS, was around 34.8 GB. And that still isn't the whole computer's memory use. We shouldn't turn the difference between those numbers into a universal allowance, but the principle is clear enough. The author's measurements

Naturally, we'd like a simple rule: “leave this many gigabytes free, and everything will be fine.” That would be handy. But a short chat with a small model and a large document handled by a completely different model will need different amounts of spare memory.

We can at least start with a straightforward plan. Let's take an illustrative budget of 64 GiB, using the units in our calculator, and divide it up like this:

This is an example, not a prescription for every computer. Eight gigabytes for working data could be a generous allowance in one case and too little in another. It's better to check your own system's usage and estimate the cache and other requirements for your chosen model and task.

Notice the last row. There's free space left, and that's perfectly fine. Nobody awards a medal for filling every last byte of RAM.

If memory starts running short, the system may move some data to disk. Sometimes this lets the work continue, but constant swapping to disk can turn a conversation into a very slow exchange of letters. The process may also stop because it has run out of memory. An SSD is useful for loading a model, but you shouldn't count on it as an equivalent substitute for RAM.

All right, that covers capacity. Suppose the model fits with room to spare. Why might it still respond slowly?

This is where the processor returns to the conversation, after all our attention has gone mostly to counting gigabytes.

The CPU has to perform many operations and fetch the data for them from memory. Its architecture, supported instructions, clock speeds, cache, core count, and how well the software uses all of these things matter.

It's particularly easy to jump to conclusions about cores. Eight is good, sixteen is better, so thirty-two must be twice as fast again?

Sometimes additional cores really do help. But if computation is held back by data transfer from memory, some threads will be left waiting. Too many threads can even make the result worse — something the llama.cpp developers specifically point out. The developers' recommendations

Thirty-two cooks around a single cutting board is an impressive workforce. It just doesn't necessarily mean lunch will arrive any sooner.

Memory has a similar story. We're used to asking how much is installed, but a model also cares about how quickly the processor can get data from it.

So DDR3, DDR4, or DDR5 is only part of the description. The actual memory speed and number of channels in use also matter. And four sticks don't necessarily mean four channels: that depends on the platform and how the modules are installed. Bandwidth is related to the transfer rate, interface width, and number of channels. Intel's explanation

That's why an old server platform can sometimes be more interesting than its release year might suggest. It may have advantages in how its memory is organized. It may also have drawbacks that only become apparent when you run a particular program.

With two processors, you'll need to get to know the arrangement a little better too. Each usually has its own local memory, and accessing the other processor's memory comes at a higher cost. This is called NUMA. How useful the second CPU turns out to be depends on how the program distributes its computation and data. Intel documentation

Our kitchen now has a second cook with a pantry of their own. Good news. Though sometimes they have to cross the yard to fetch ingredients.

You don't have to understand every detail before your first run. But it's useful to keep them in mind when the results suddenly fail to match the tidy arithmetic of “twice as many processors means twice as fast.”

There's another thing that's easy to mistake for a malfunction.

You ask a question, and the program says nothing. The fans are running. Nothing on the screen yet.

The model may still be reading your prompt. Before generating a response, it needs to process the input text, and a short question takes a different amount of work from an attached fifty-page contract.

Go back to the first row of our table for a moment. Generation ran at 7.2 tokens per second, but almost a hundred seconds passed before the first token appeared. Those numbers don't contradict each other: they describe different stages.

That's why comparisons should consider both the time before the response begins and the speed of subsequent generation. In llama-bench, input processing and generation are measured separately. How the measurements work

A long response will naturally take time too. If the model supports a reasoning mode, it may generate extra tokens before its final answer. Sometimes that's useful. Sometimes you only want the delivery date from an email, and the model has already convened a philosophical symposium on the nature of time.

So where should you start if you don't want to spend your entire first evening waiting?

I'd choose a small instruct model in a supported quantized format and give it a short, familiar task. Options to explore include Qwen3.5-4B and Phi-4-mini with 3.8 billion parameters. Then you can try the 7–8B class and see whether a larger model offers an advantage in your particular work. All of these options are available in model runner catalogs. Qwen3.5-4B, Phi-4-mini, Qwen3-8B

For a first attempt, you could consider Q4_K_M if a file in that format is supported, then compare it with a different precision. Quantization reduces the size of the weights, but it can affect responses. How noticeable and important that is depends on the model and the task. Quantization documentation

And there's no need to feel awkward about running a small model on a computer with lots of memory. The computer won't take offense. If it does the job well, you can happily leave those spare gigabytes spare.

MoE models are also interesting for owners of machines with plenty of memory. They use a subset of experts to process each token, which can reduce the computational load compared with a dense model of a similar total size. But the full set of weights still needs to be stored.

For example, Qwen3.6-35B-A3B is listed as having around 35 billion total and 3 billion active parameters. In memory terms, it isn't a small 3B model, and in terms of computational load, you can't automatically equate it with a conventional dense 35B model. Model card

The experts take turns working, but nobody has moved out of the apartment.

Perhaps the most useful thing you can do at the beginning is come up with a few tasks whose results you can judge. Ask it to summarize a short document. Extract names, dates, and amounts from an email. Answer a question you already know the correct answer to.

Start with a small context, one request at a time, and a limited response length. See how long you had to wait, how much memory was left, and, most importantly, whether the result helped you.

Then change one thing at a time: the model, weight precision, context length, or thread count. That way, you gradually learn what actually affects your system. Change everything at once, and the experiment starts to resemble a family furniture rearrangement: things are different, but nobody remembers why you started.

Running on a CPU has real limitations. Large documents and long stretches of reasoning require patience. Serving many users at once is a different workload from one person at a home computer. An old server can use a noticeable amount of electricity and announce the fact with the sound of an aircraft taking off.

But for unhurried work with text, extracting data, small automations, and getting to know local AI, the machine you already own deserves at least one attempt.

That's the point of our calculator: to help estimate memory needs and choose some options to start with. It doesn't issue a speed certificate, and it doesn't know how many seconds of waiting it takes before you start drumming your fingers on the desk.

After a few evenings, you may settle on a small model that does a particular job well. You may find a suitable MoE. Or you may conclude that your tasks do call for a graphics card after all. That happens too, and it doesn't make the experiment a failure.

At least you'll now know exactly what improvement you're planning to pay for.

And if you find a suitable model, the old computer will have a new job. I can picture it: a few years ago, it was written off, taken home, and put under a desk — and now it's helping sort through documents.

For a server, that's not a bad career move.

Until then, its job description had just one line: “Make noise.”