Build a Tutorial Agent Stack: Record, Screenshot, Write — and Host Your Own LLM

Evidence note: Deploy Desk synthesis as of 2026-08-30. This guide starts from a real product question—whether the market already sells an agent that takes a task, records its own screen, captures key steps, and writes an article—then expands that ask into a deployable three-layer stack: orchestration agent, local inference, and optional fine-tuning. It is not a vendor ranking, not a promise that any single SaaS ships all three layers today, and not investment advice. Hardware numbers are planning ranges; verify against your GPU SKU and driver stack.

Quick answer

Yes, pieces of that product exist—but almost nobody ships the full fantasy out of the box. Most “AI screen recorders” still need a human to click; a smaller set of computer-use agents can operate a browser for you; almost none also host and fine-tune your own model on the same box. The workable design is Scheme B: treat documentation as an agent loop (brief → act → record → screenshot → write), put planning/vision/writing behind a local OpenAI-compatible LLM API, and only then—if you have GPU memory and a real corpus—add LoRA fine-tuning. Do not start by “training a large model.” Start by shipping one reproducible walkthrough from one allowlisted task.

Key takeaways

• Split the market into Mode A (you record, AI writes) and Mode B (agent records itself and writes). Your original ask was Mode B.
• Mode B commercial options are thin; Vidocu-style AI recorders and DIY computer-use + tutorial skills are the closest fits.
• Expanding Mode B to “deploy LLM + train on my server” means a three-layer stack, not one product page.
• Deploy in phases: agent loop + Ollama first; vLLM production second; LoRA third. Skip from-scratch pretraining on a single machine.
• Pair this with existing Deploy/Agents reading: tool use vs computer use, minimal tool agent, local LLM checklist, vLLM production.

Who this guide is for

• Founders and ops leads who want “give a task → get a how-to article with screenshots” without hiring a full content studio.
• Platform engineers who already run Linux GPU boxes and want an agent that calls a local model instead of shipping every frame to a cloud API.
• Teams publishing SOPs, Deploy docs, or SEO how-tos who need a reproducible pipeline—not a one-off demo reel.

Who should skip

• If you only need chat Q&A over PDFs, build RAG first.
• If you expect unsupervised overnight agents that buy, email, and admin every SaaS login—this stack deliberately refuses that scope.
• If “train a large model” means foundation pretraining from scratch, you need a cluster and a research budget, not this article.

The originating question (and why it matters)

The product ask usually arrives in one sentence:

Are there AI agents on the market that I can give a task to—then they record the screen themselves, screenshot important steps, and write an article?

That sentence hides three separate engineering jobs:

1. Autonomy: plan and execute UI actions (browser or desktop).
2. Capture: decide what is an “important step,” record video, and store stills.
3. Authoring: turn action logs + frames into readable prose (often SEO-shaped).

Most products only own job 2–3 after a human owns job 1. A smaller set owns job 1–2 for browsers and leaves job 3 as a thin draft. Almost none also own “host my LLM” and “fine-tune on my SOP corpus” on the same machine. Collapsing all of that into one SaaS SKU is how demos look magic and deployments fail on login walls.

If you later add: “Not only the browser—I also want to deploy a large model on the server and train on the server,” you have left the category of screen-recorder SaaS and entered Scheme B: a self-hosted agent platform. The rest of this guide is that platform.

Market map: what already exists

For procurement language and failure modes of GUI agents, read ships vs demos and browser agents for research desks. Do not buy a highlight reel; ask for allowlisted domains, login strategy, and screenshot selection rules.

Scheme B: three layers, one server story

Layer 1 — Tutorial agent (orchestration)

Responsibility: accept a brief, plan steps, drive the UI, record, screenshot, write, store artifacts.

• Web flows: Playwright / Chromium (headful when recording quality matters).
• Desktop / CLI flows: terminal capture (asciinema/script) plus OS screenshots—harder, allowlist narrowly.
• API surface: FastAPI (or similar) with job IDs, status, download links.
• Queue: Redis + worker so one hung browser does not freeze the API.
• Artifacts: runs/{run_id}/video.mp4, steps/*.png, guide.md, meta.json.

Build the loop like a minimal tool agent: one run_id, budgets, allowlisted tools, structured logs—see minimal tool agent with observability and traces and tool-call logs.

Layer 2 — Local LLM inference

Responsibility: planning, “is this frame a key step?”, and article drafting without shipping every pixel to a third party by default.

• Fast path: Ollama with a quantized 7B–14B chat model for planning and prose.
• Production path: vLLM (or TGI / TensorRT-LLM) behind an OpenAI-compatible /v1/chat/completions endpoint—see vLLM production.
• Vision path: a VLM (or cloud VLM as fallback) to rank frames and write captions.

Expose one base URL to the agent. The agent should not hardcode vendor SDKs. That keeps you free to swap Ollama → vLLM → cloud without rewriting the capture loop. Hardware checklist: run an open-weight LLM locally.

Layer 3 — Training / fine-tuning (optional)

Responsibility: adapt writing style and domain vocabulary to your SOPs—not invent a new foundation model.

• Realistic: LoRA / QLoRA SFT with LLaMA-Factory, Unsloth, or Axolotl.
• Data: your approved tutorials, Deploy docs, Research how-tos, redacted run logs.
• Unrealistic on one box: full pretraining. If someone sells you that on a single consumer GPU, walk away.

For cost framing when you compare API vs self-host, use the narrative in self-host vs API TCO and the live planner at /tools/ai-inference-tco.

Hardware reality check

Disk matters: screen recordings grow fast. Plan 50–200 GB for artifacts if you keep raw video. Network matters if you must pull Hugging Face weights from a restricted region—use a mirror or ship weights offline.

Deploy in three phases

Phase 1 — Ship the loop (1–2 days on a ready Linux box)

Goal: one allowlisted brief → one Markdown guide with step screenshots a human can edit.

1. Provision Ubuntu 22.04/24.04, Docker, and (if GPU) NVIDIA drivers + Container Toolkit.
2. Run Chromium via Playwright in a worker container; mount an artifacts/ volume.
3. Stand up Ollama; pull one chat model (7B–14B quantized is enough for planning + draft prose).
4. Implement a job API: POST /runs with brief + start URL + allowlist; GET /runs/{id} for status.
5. On each meaningful action (navigate, click, type, assert), save a PNG and append a step record.
6. After the run stops (success, max steps, or timeout), call the local LLM with step list + selected frames captions to draft guide.md.
7. Human review gate before anything publishes to a CMS.

Success criterion: three consecutive runs on the same allowlisted flow produce guides that a junior editor can ship with under 30 minutes of edits.

Phase 2 — Productionize inference and ops

Goal: concurrent jobs, auth, and logs you can debug at 2 a.m.

1. Move chat serving to vLLM if Ollama cannot hold your concurrency or latency SLO.
2. Put Nginx (or Caddy) in front with TLS and a shared secret / OAuth for the job UI.
3. Add Redis queue, retries, dead-letter, and per-run budgets (max_steps, wall clock, max video minutes).
4. Optional: dedicate a VLM endpoint for frame ranking so chat models stop guessing which screenshot is “important.”
5. Redact secrets from logs and from screenshots that may contain cookies or PII—see prompt injection and agent risks.

Success criterion: you can answer “which tool ran with which arguments on run_id X?” without SSHing into a random container.

Phase 3 — Fine-tune writing (only with GPU + data)

Goal: house style and domain vocabulary, not a new base model.

1. Export 50–500 high-quality tutorials as instruction pairs: (brief + step outline → final article).
2. Run LoRA/QLoRA via LLaMA-Factory or Unsloth; keep a held-out eval set.
3. Eval on: factual alignment to screenshots, step order, forbidden claims, and length discipline.
4. Deploy the adapter behind the same OpenAI-compatible URL; keep the base model pinned.

Success criterion: editors prefer fine-tuned drafts on a blind A/B for your own domain—not a public leaderboard score.

Reference compose shape (conceptual)

You do not need twenty microservices. A pragmatic Phase 1–2 compose looks like:

services:
  api:          # FastAPI job submission
  worker:       # Playwright + ffmpeg + artifact writer
  redis:        # queue + locks
  ollama|vllm:  # OpenAI-compatible LLM
  nginx:        # TLS + auth (Phase 2)
volumes:
  artifacts:
  model-cache:

Keep the worker’s browser profile disposable. Persist only artifacts and job metadata. Never bake production cookies into the image.

What “important step” selection should mean

Do not ask the model to invent importance from vibes alone. Prefer deterministic hooks, then VLM polish:

1. Screenshot on every navigation commit and every successful click that changes URL or opens a dialog.
2. Collapse near-duplicate frames with pixel or perceptual hash thresholds.
3. Ask a VLM only to caption and rank the surviving set for the article figure list.
4. Always keep the raw timeline so editors can rescue a missed frame.

This is the same discipline as chunking in RAG: over-capture, then filter with measurable rules—see chunking and embedding choices for the analogy, and use /tools/rag-chunk-planner when your corpus of SOPs grows into a retrieval desk.

Failure modes you should expect

For a broader agent failure taxonomy, read agent failure modes and observability.

Security and policy baseline

• Allowlist start URLs and hosts; deny arbitrary navigation in Phase 1.
• Separate credentials per environment; never reuse personal admin sessions in the worker.
• Treat page content as untrusted input (prompt injection).
• Keep the training corpus rights-cleared; do not fine-tune on scraped paywalled manuals you cannot redistribute.
• Publish only after human review if the article will go to a public Research/Deploy channel.

Practical builder safety: practical AI safety for builders.

How this maps to a RomeWay-style content desk

If your CMS already ingests HTML/Markdown articles (as this site does under content/articles/), Scheme B’s happy path is:

1. Brief: “Document the RAG Chunk Planner calculator end-to-end.”
2. Agent runs against your staging URL, records, screenshots, drafts Markdown.
3. Editor fixes facts and internal links.
4. Import / publish via your existing SEO import path.

That is still Mode B autonomy for capture—but Mode A quality for publication. That hybrid is the honest product. Pure unsupervised publish is how you get confident wrong screenshots.

Checklist: what to send an engineer before they SSH

1. Server IP + SSH access method.
2. GPU model, VRAM, driver status (or “CPU only”).
3. CPU / RAM / disk free.
4. OS version; Docker present? NVIDIA toolkit present?
5. Outbound access to model registries (or offline weight plan).
6. Domain + TLS plan for the job UI (optional for Phase 1).
7. LLM policy: all-local / local+cloud VLM fallback / API-only until GPU arrives.
8. Fine-tune goal (writing style? domain terms?) and approximate corpus size.

Without GPU facts, refuse Phase 3 promises. Without an allowlisted first task, refuse Phase 1 demos that “browse the whole internet.”

FAQ

Is there already a SaaS that does everything in Scheme B?

Not as a mature, general product. Closest Mode B capture products exist in beta; self-host inference and LoRA remain your stack. Expect to assemble.

Can I skip local LLM and only use cloud APIs?

Yes for Phase 1. Many teams should. Local LLM becomes urgent when privacy, cost at volume, or offline constraints matter—or when you want VLM frame ranking without per-frame cloud bills.

Should the agent control the full desktop?

Default to browser-only. Desktop computer use fails more often and expands the blast radius. Expand only after browser SOPs are boringly reliable.

Where do coding agents fit?

Coding agents (Cursor, Devin-class tools) optimize repositories, not always tutorial capture. Use them to build Scheme B; do not confuse them with the recorder itself—see coding agent tools compared.

How do I know the article math/docs are correct?

Separate deterministic calculators from narrative agents. Keep formulas in pure functions with tests (as this site does for free planners under /tools). The agent may demonstrate the UI; it must not invent the formula.

Related reading

• From chatbot to agent: a practical map
• Tool use and computer use: ships vs demos
• Build a minimal tool agent with observability
• Run an open-weight LLM locally: checklist
• vLLM, TensorRT-LLM, TGI in production
• Self-host vs API: TCO narrative
• Open-source short drama stack (another multi-service GPU compose pattern)

Bottom line

Start from the human question—give a task, record, screenshot, write—then refuse the marketing collapse into one SKU. Ship Scheme B as three layers: a tutorial agent with budgets and artifacts, a local OpenAI-compatible LLM for plan/vision/write, and optional LoRA when you have GPU and a rights-cleared corpus. Phase 1 is a walkthrough you can edit. Phase 2 is an operable service. Phase 3 is style adaptation. Everything else is a demo.