{/* banner prompt: A playful, polished 3D clay-render in matte sage green, cream and terracotta: a stylized NVIDIA RTX 3060 graphics card with a softly glowing rounded brain resting on top of it, a few small thought-bubble and gear props floating beside it to suggest reasoning, soft studio shadows, simple background, 16:9. */}

The first three posts in this series were about *making* things — the same RTX 3060
learning to generate [images](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/),
then [video](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/),
then [video with sound](/blog/2026/08/04-now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/).
This one is different. This time the card isn't drawing or filming anything — it's
*thinking*. I put a **27-billion-parameter language model on the same 12GB card**,
and it works.

The headline: **Qwen3.8-27B**, a reasoning model released only a few days ago, runs
locally on a consumer RTX 3060 that shipped back in **2021**. It answers questions, writes code,
reasons step by step — and it can even look at a picture and tell you what's in it.
The kind of model people assume needs a data centre, or at least a $2,000 graphics
card, runs on hardware a small business already owns.

## The machine (still the same one)

Same box as every post in this series: an RTX 3060 with **12GB of VRAM**, 64GB of
system RAM, Debian. Nothing upgraded. The card is the constant.

## Why this is surprising

A 27B model is *big*. The rule of thumb people quote is that you need a 24GB card
(an RTX 4090 or better) to run one at all — and its full-precision form is over
50GB, firmly data-centre territory. So "27B on 12GB" sounds like it shouldn't be
possible.

It is, because of two things working together: **quantization** and **offloading.**

## Fitting an 18GB model into 12GB

Quantization compresses the model's numbers to 4 bits each, shrinking it from 50GB+
down to about **18GB** with very little quality lost. That still doesn't fit in
12GB — but it doesn't have to. The card holds a bit more than half the model
(**about 57%**), and the rest lives in the 64GB of system RAM, streamed onto the
GPU as it's needed. The GPU does the heavy math; the spare layers wait in RAM.

The remarkable part is how *easy* it was. The video-with-sound model in the last
post took patches and careful memory tuning. This time it was genuinely two
commands — download, and run. The tool (Ollama) works out the GPU/RAM split
automatically.

## The honest numbers

<Table data={
    {
        headers: ['', 'On the RTX 3060'],
        rows: [
            ['Model', 'Qwen3.8-27B (released this month)'],
            ['Size on disk', '~18GB (4-bit)'],
            ['Runs on GPU', '~57% on the card, ~43% streamed from RAM'],
            ['VRAM used', '11 of 12 GB'],
            ['Speed', '~4.7 words-per-second (reads about as fast as you do)'],
            ['Can it see images?', 'Yes — it is multimodal'],
            ['Best for', 'Questions, coding help, explanations, one-image analysis'],
        ]
    }
} />

Four-ish words a second isn't fast — you watch the answer type itself out, roughly
at reading speed. This is not the tool for churning through a thousand requests an
hour. It's the tool for asking one good question and getting a thoughtful answer,
privately, on your own machine, for the cost of the electricity.

## It reasons — out loud

Qwen3.8 is a *reasoning* model: before it answers, it thinks. Ask it a coding
question and you can watch it work the problem — considering approaches, catching
its own mistakes, checking its logic — then give a clean final answer. Ask it to
find the longest palindrome in a string, and it doesn't just spit out code; it
reasons about edge cases and complexity first. That visible thinking is a big part
of why these newer models are so much more useful than the ones from a year ago.

## And it sees

Here's the part that closes the loop on this whole series. The model is multimodal,
so I showed it one of the AI-generated animal photos from our image experiments and
asked: *is this a real photograph, or AI-generated?*

It nailed it. It described the scene correctly — a brown bear in a rushing river,
autumn forest blurred behind it — and then, unprompted, listed exactly *why* it
looked artificial: fur that was "suspiciously smooth and painted," the
"characteristic milky AI bokeh" in the background, "melted" rocks, a glassy eye
lacking real reflections. A local model, on a 12GB card, giving a genuinely sharp
critique of a locally-generated image. The tools in this series can now check each
other's work.

## Try it yourself

If you have an RTX 3060 (or any 12GB card) and [Ollama](https://ollama.com)
installed, this is the whole thing:

```bash
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Q4_K_M
```

The first run downloads the model; after that it's instant to start. Ask it
anything. To feed it an image, just put a file path in your message.

## Where this leaves a small business

Four posts, one unchanged computer. The same RTX 3060 went from a single 512px
still, to short video, to video with a soundtrack — and now to a reasoning model
that can answer questions, help write code, and read an image, all without a single
byte leaving the building.

That last point matters more than the speed. Running the model *locally* means your
questions, your customer data, your documents never touch someone else's server.
No per-message cloud bill, no data leaving your office, no dependency on a
subscription that can change its terms next quarter. For a lot of small businesses,
"good enough, private, and paid-for-once" beats "fastest" every time.

It isn't a data centre. It doesn't need to be.

*It's a five-year-old graphics card. What it can do keeps surprising me.*


0a2a03c8bcbf34fd700c45afe3e7e1d41f71654c0dd66b371de37159ffb3f7b5

Running Qwen3.8-27B, a brand-new 27B multimodal reasoning model, locally on a consumer RTX 3060 with only 12GB of VRAM: how an 18GB model fits, honest tokens-per-second, what it can and can't do, and the one command to try it yourself with Ollama.

The fourth in our RTX 3060 series. Qwen3.8-27B — a 27-billion-parameter reasoning model released this month — runs on the same 12GB card. Not in the cloud, not on a $2,000 GPU. One command, and it thinks, codes, and even sees.

Tag: Rtx 3060

Tag: Rtx 3060

Now It Thinks: A 27B Language Model on the Same RTX 3060