Local LLM + Agent + Image Gen on Your PC

A plan for Christina, written by Rosey · October 20, 2026. One self-contained file — no internet required to read it, internet required to do anything.

0. What you're building

Three things, in one afternoon-ish:

Ollama — runs open LLMs on your RTX 5070. The brain.
Hermes — an agent that drives that brain: edits files, runs commands, browses the web. Including your Obsidian vault.
ComfyUI — local image generation, in a node-based UI that the agent (or you) can drive.

Everything below is specific to your machine: a Windows gaming rig with a GeForce RTX 5070 (12 GB of video memory). That's a genuinely good machine for this — it will comfortably run 8–14 B models at full quality straight on the GPU. The rest of this is just setup.

One number that matters more than the GPU: your system RAM. The 12 GB is the GPU's video memory — that part is great. But big models that don't fit in 12 GB spill into regular system RAM, and 16 GB of system RAM isn't enough headroom for a 30 B model plus Windows plus a browser. If your rig has 32 GB or more, everything in this plan works as written. If it has 16 GB, skip the 30 B model in section 2.3 for now — you'll live on the 8–14 B class, which is a fine agent experience. (Upgrading to 32 GB is a cheap fix and the single best upgrade you can make for this project.)

1. Your hardware, briefly

RTX 5070, 12 GB VRAM. Blackwell card, fully supported by Ollama and ComfyUI both. The one real requirement: current NVIDIA drivers. Install at least the 550-series driver; for a 5070 just grab the latest Game Ready driver from nvidia.com and you're done.
12 GB VRAM is a shared resource. It hosts the LLM and image generation, and it can only host one model comfortably at a time. Ollama unloads idle models after 5 minutes, so in practice you won't fight with this much: run your agent, let it idle, then fire ComfyUI. (Section 5.5 covers the "I want both at once" case for images.)
Windows. Good — every tool in this plan ships first-class Windows support, and two of them have one-line install commands.

2. Part 1 — Ollama (the LLM server)

2.1 Install

Open PowerShell and run:

irm https://ollama.com/install.ps1 | iex

That's the whole install — it pulls the app (with a little system-tray icon) and starts serving. Verify it works:

ollama --version curl http://localhost:11434/api/tags # should return {"models":[]}

If curl isn't on Windows yet, the web UI check works too — Ollama serves a small interface at http://localhost:11434.

2.2 Pull a model

ollama pull qwen3:14b

Then confirm the GPU is actually doing the work:

ollama ps     # you want "100% GPU" next to the model

If it says CPU, your drivers are the problem (see section 1), not Ollama.

2.3 Which model fits a 12 GB card

This is the part people get stuck on, so here's the honest picture. The rule: a 4-bit model needs roughly its size in GB of VRAM plus overhead for the context window. Tool-calling support is the dealbreaker for agent use — an agent is a model that can call tools.

Model	Fits 12 GB?	Tool calls	Vibe
`qwen3:14b`	Yes — fully on GPU, comfortably	Yes, reliable	The sweet spot. Fast, smart enough, no spilling.
`qwen3:8b`	Yes, easily	Yes	Faster, dimmer. Great for quick Q&A, fine for simple agent work.
`qwen3:30b-a3b`	Partially — ~19 GB at 4-bit, spills to system RAM	Yes, well supported	Smartest on the list. It's a Mixture-of-Experts model, so only a sliver (~3B) activates per token — it runs at usable speed even when it can't fully fit the GPU. This is the pick if you have 32 GB+ system RAM.
`gemma4:31b`	Partially — ~20 GB on disk	Yes (Hermes' docs call it the best local option)	Dense 31B — when it spills to RAM, it slows down hard, because every token touches the whole model. Only if you like pain.
`gemma2:9b` / `llama3.2:3b`	Yes, easily	No / No	Chat-only. Fine as a fast fallback model in Hermes, not as the agent brain.

My recommendation: start with qwen3:14b — it runs fully on your card at high quality. When you're bored, pull qwen3:30b-a3b and compare. And keep in mind (Hermes' own docs say it): tool-calling support in local models is expanding fast — check Ollama's model library when you start; something released after this doc was written may be smarter.

2.4 Two Ollama settings you'll actually need for an agent

Context window. Ollama's default context is 2,048 tokens. That's about four paragraphs — not enough for an agent, which carries its system prompt + tool definitions + your whole conversation on every call. Hermes wants at least 64,000. You make a bigger-context clone of the model with a Modelfile:

# 1. make a file. C:\ollama-tmp\Modelfile will do; Notepad works fine
#    its entire contents:
FROM qwen3:14b
PARAMETER num_ctx 64000

# 2. build it
ollama create qwen3-64k -f C:\ollama-tmp\Modelfile

# 3. that new model name (qwen3-64k) is now the one you give Hermes

Keep-alive. Ollama unloads models after 5 minutes of inactivity (saves VRAM — good in general, but re-loading a model takes minutes). If the reloads annoy you, set the OLLAMA_KEEP_ALIVE environment variable to 24h (System → Advanced system settings → Environment variables), or pass "keep_alive": "24h" in API calls.

That's Ollama. It's now a local LLM server at http://localhost:11434 that speaks the OpenAI-compatible API. Everything else plugs into that.

3. Part 2 — Hermes (the agent)

Hermes is Nous Research's agent: it runs a model (your Ollama), then does real work — edits files, runs terminal commands, browses the web, and can use skills (packaged capabilities). Your use case — an agent that browses and works on your Obsidian repository — is exactly what it's for.

3.1 Install

Easiest path — the desktop package:

Download the Windows package from hermes-agent.nousresearch.com and open the .appinstaller file.
Done. It installs via Windows App Installer.

Prefer terminal-only? Same website's CLI installer, in PowerShell:

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

(Its settings and data live in %LOCALAPPDATA%\hermes\ on Windows.)

3.2 Point Hermes at Ollama

hermes setup

The wizard asks where the model lives. Choose Custom Endpoint and enter:

Base URL:  http://localhost:11434/v1
API key:   (leave empty, or type: no-key)
Model:     qwen3-64k          # the one from 2.4 — the 64K-context clone

Or edit %LOCALAPPDATA%\hermes\config.yaml by hand:

providers:
  ollama:
    base_url: "http://localhost:11434/v1"
model:
  default: "qwen3-64k"
  provider: "custom"

If Hermes ever complains about endpoints, hermes doctor tells you exactly what's missing, and hermes model re-runs just the provider/model picker.

3.3 The speed reality (read this before you panic)

The first reply of every fresh session can take a long time. That's normal, not a crash. On every call Hermes sends a fixed payload before your message — its system prompt plus the schemas for all its tools — and on a local model it has to process all of that before generating ("prefill"). On a 12 GB card with a partially-offloaded model, the first turn can sit silent for a couple of minutes, then stream at normal pace. Subsequent turns are much faster because the context is already built. If it's unbearably slow on turn one: disable toolsets you don't use (hermes tools) and prune installed skills (hermes skills) — every enabled tool makes the prefill bigger. And for extra insurance, add HERMES_API_TIMEOUT=1800 to %LOCALAPPDATA%\hermes\.env.

3.4 Point it at your Obsidian vault — the fun part

Here's the good news about Obsidian: an Obsidian vault is just a folder of Markdown files. The graph view, backlinks, and the app magic are all on top of plain text; none of it is locked away. So an agent with file tools can already browse, search, read, and edit your whole vault out of the box — no Obsidian plugin required.

Set it up:

Open your vault folder in File Explorer, then cmd in the menu bar (or cd to it in PowerShell). This makes the vault folder the agent's home.
Run hermes from there.

Talk to it:

"List my notes on machine learning, and summarize what I already know about neural nets"
"Find every note that mentions 'Q3 planning' and tell me what's in them"
"Merge my three scattered draft notes about the trip into one clean note"

From there you can go two ways, pick either or both:

A skill. Hermes has a hermes skills command for exactly the thing you asked for — packaged capabilities you add to the agent. There are community Obsidian-vault skills; even a simple one of your own is just a Markdown file with instructions ("when the user asks about notes, search the vault, follow [[wikilinks]] to related notes, keep their formatting conventions"). Writing it yourself is a good first experiment.
An MCP server. If you'd rather have the full Obsidian app-level API (search that understands backlinks, canvas, etc.) instead of plain file access, the MCP server ecosystem has Obsidian servers (e.g. mcp-obsidian) that Hermes can attach. More moving parts, more power — skip it until the plain-text path leaves you wanting.

3.5 Optional extras

Local Models (zero-config path). Hermes' desktop app can also run and manage its own llama.cpp server under the hood — Settings → Providers → Local Models → Install runtime → pick a model → Use. It picks the right model build for your card, sizes the context so it fits, and never sends anything off the machine. If you want to skip Ollama entirely, this works; we're using Ollama because it leaves you with a general-purpose local LLM server that other tools (including ComfyUI-adjacent assistants) can reuse.
Cloud fallback. Local models do struggle with gnarly multi-step tasks. If you have an OpenRouter (or similar) key, Hermes can fall back to a cloud model only when local fails — about 90% free local, expensive help on the rest.
Messaging bot. Hermes can expose itself as a Telegram/Discord bot run entirely by your PC, via hermes gateway. Your PC has to stay on for it to work.

Sources: hermes-agent.nousresearch.com — Installation · Local Models · Run Hermes Locally with Ollama.

4. Part 3 — ComfyUI (image generation)

4.1 Install the portable build

Grab 7-Zip (you'll need it), and the NVIDIA build of ComfyUI Portable (the standard CUDA 13 package — for modern RTX cards).
Extract the .7z somewhere big and fast — an SSD drive, not the C: root if your C: drive is getting full. Models are chunky.
Double-click run_nvidia_gpu.bat. Wait for a console line that says To see the GUI go to: http://127.0.0.1:8188, then open that in your browser.
Don't close that black console window. It is ComfyUI. Close it = ComfyUI stops. (If you want to use it from other devices on your network, add --listen to the command line in that .bat file with Notepad.)

A less-hacky alternative: Comfy Desktop — a proper Windows app with its own window, an update channel, and multiple-instance support. The portable build is the classic choice and tracks bleeding-edge commits; pick whichever you like — everything after this works the same.

4.2 Your first image

The default text-to-image workflow loads automatically: a prompt box, a model loader, a sampler, a save node, wired together.
You'll get a warning about a missing model. Click it: the missing-model panel gives you a direct download link.
Save the model into ComfyUI_windows_portable\ComfyUI\models\checkpoints\.
Press R to refresh the model list, pick it in the Load Checkpoint node, write a prompt, hit Ctrl+Enter. The image appears in the Save Image node — right-click to save it.

4.3 Model notes for a 12 GB card

SD 1.5 / SDXL — tiny, run at top quality, lots of headroom. The classic starting point.
Flux — the current workhorse for prompt-following photorealism. Loads it as fp8 (~11–12 GB): it fits your card, but snugly, and will slow big-res images down.
Fast/single-step models (the Z-Image-Turbo line, LTX-style) — newer "one step" models that generate in a few seconds instead of a minute. Great for the iterate-fast loop; check the Comfy blog/HF for the current names, they move quickly.

VRAM contention, the one gotcha. If Hermes is mid-session with its 14B/30B model in the 12 GB and you launch a Flux image, somebody spills to system RAM — ComfyUI will still work, just slower (or, worst case, an OOM). The lazy fix: quit the agent session, or let Ollama's 5-minute timeout unload the LLM first. The 32 GB-RAM system handles the spill gracefully, so this rarely bites; it mostly means "don't run a huge image batch and a long agent conversation at the exact same time."

4.4 Handy stuff to know

Upgrading: the update\ folder has bat files — update_comfyui_stable.bat for the version track.
Missing-model links in generated images: every image ComfyUI makes has its workflow embedded as metadata — drag a ComfyUI-made image back into the window and the whole node graph reappears. Community workflow sharing rides on this; it's a feature.
Outputs land in ComfyUI\output\.
The agent connection: ComfyUI exposes an HTTP API at http://127.0.0.1:8188 (prompt queue, upload, results). Later on, you can have Hermes POST image jobs to it — but don't start there. Learn the nodes by hand first; the API is just the same workflow you built by hand, in JSON.

Sources: docs.comfy.org — ComfyUI Portable for Windows · Getting Started with AI Image Generation · Comfy Desktop for Windows.

5. The map of alternatives

You chose a sensible default (Ollama + Hermes). But the landscape is crowded and it moves fast — here's what else exists, so you can find your way back if the winds change.

5.1 LLM servers (the Ollama layer)

Tool	What it is	When you'd pick it
Ollama	The friendly default. One-line install, tray icon, OpenAI-compatible API.	You — this doc. Best first choice for a Windows gaming rig.
LM Studio	GUI app: browse + download models from a nice model library, chat from the app, and expose an OpenAI-compatible server from a side panel.	If you'd rather click than type. It can replace Ollama entirely (same `base_url` trick: `http://localhost:1234/v1`). Good for A/B testing models side by side.
llama.cpp `(llama-server)`	The engine under everything — the fastest local inference, no magic. Hermes can actually manage it for you (that "Local Models" feature).	Tinkerer mode. More knobs, more control, more typing.
vLLM	High-throughput production serving. Batched inference, fast, RAM-hungry.	Not for a gaming rig. Listed for completeness — it's what companies run.

5.2 Agent harnesses (the Hermes layer)

Harness	Shape	Character
Hermes (your pick)	Terminal + desktop app, skills, browser/computer-use tools, messaging gateways	The full local citizen. Terminal-first, very capable, works great with local models.
Cline / Roo Code	VS Code extensions	Agent inside your code editor. Best if your Obsidian vault or projects live in editor land (VS Code has a Markdown-friendly extension for vaults).
Aider	Terminal, chat-pair style	The veteran: a clean conversation that edits files and manages git commits for you. Less "autonomous agent", more "smart coding partner".
OpenHands	Web UI + terminal; can run in Docker	Full "software agent" — own browser, terminal, file ops; more infrastructure, more spectacle.
Goose	Terminal, extensible "extensions" model	Close cousin to the skills idea — good local-model support, from Block.
Cloud agents (Claude Code, Codex, etc.)	Cloud API	The honest footnote: for hard long-horizon tasks, frontier cloud models are still noticeably better than the best local one. The good setup is local for 90% of daily work (free, private, instant-feel) with a cloud key as fallback — Hermes does this natively.

One thread runs through all of it: MCP. Model Context Protocol is the de-facto standard for hooking external things (Obsidian, browsers, APIs, dbs) into agents, and most of the harnesses above support it — so if you outgrow Hermes, the Obsidian wiring you build can largely travel with you.

6. Troubleshooting cheatsheet

It's doing	It means
First agent reply takes minutes	Normal prefill (see 3.3). Trim tools/skills; set `HERMES_API_TIMEOUT=1800`.
`ollama ps` shows 0% GPU	Old NVIDIA drivers. Update to the latest — RTX 50xx is new and Ollama needs a current driver to see it. Then `ollama serve` restarts clean from the tray icon.
Hermes prints raw JSON like `{"name": "web_search"…}` instead of acting	The model isn't tool-calling, or the server didn't parse it. Switch to a tool-capable model (14b/30b-a3b); with Ollama you'd need tool-capable chat formats (that's one reason Ollama over a raw llama-server — it handles the format for you).
"Context window too small" / weird mid-answer cut-offs	You pointed Hermes at the 2K-context model instead of the `-64k` clone from 2.4.
Hermes: "connection refused"	Ollama isn't running. Click its tray icon / run `ollama serve`.
Hermes: "provider 'ollama' has no endpoint configured"	Add a `providers: ollama: base_url: http://localhost:11434/v1` block to its config (3.2).
ComfyUI: missing model warning, forever	You saved the file to the wrong folder. Checkpoints go in `ComfyUI\models\checkpoints\`, then press `R`.
ComfyUI OOM / super slow	The LLM is loaded in the same 12 GB (4.3). Load a smaller image model, or lower the image resolution, or close the agent.
Anything else	`hermes doctor` tells you exactly what's wrong with Hermes. Check the Ollama tray icon console and the ComfyUI console window — the answers are in those three places.

7. Links

Ollama — Windows download · docs · GPU support · model library
Hermes Agent (Nous Research) — docs: install · local models · Hermes + Ollama guide · providers
ComfyUI — portable for Windows · first image · Comfy Desktop
Models: Hugging Face, Ollama library
MCP ecosystem: modelcontextprotocol.io · mcpservers.org (Obsidian servers live here)
Obsidian: obsidian.md (remember: it's a vault of text files)

The whole afternoon, in order: update the NVIDIA driver → Ollama (irm … | iex) → ollama pull qwen3:14b → make the 64K Modelfile → Hermes (.appinstaller) → hermes setup with the custom endpoint → cd into the vault and talk to it → 7-Zip + ComfyUI portable → run_nvidia_gpu.bat → first checkpoint → first image. Everything after the first day is just fiddling with prompts and models.