gputaskman Guide: How the Open-Source GPU Task Manager Shows VRAM, Engine Load and LLM Memory per Process

What gputaskman is and why it exists

The README opens with an example: Task Manager says "15 GB used," and gputaskman tries to answer "by what?" It then asks whether that memory is model weights, KV cache, the browser or the desktop compositor, and why so much GPU memory sits in system RAM.

The scope is local GPU work. Decision models, vision, speech, LLM inference, training and fine-tuning, graphics and video are in scope, while CPU-only, NPU-only and cloud workloads are not. The project also states what it does not do: it does not dump VRAM and cannot list a game's textures. It labels itself "early, experimental software": device and process counters are measurements, while process classifications and breakdowns derived from GGUF model files are estimates.

The repository was created on November 3, 2026, and version 0.1.0 for Windows shipped the same day. Linux and WSL followed in 0.2.0, and later releases added AMD, macOS, diff, Intel on Linux, record and a Huawei Ascend backend. Version 0.4.1 added Intel activity from Windows counters, NVIDIA TCC-mode GPUs and Foundry Local models on Snapdragon.

How gputaskman works

On Windows, GPU drivers follow the Windows Display Driver Model (WDDM), which Microsoft describes as "the graphics display driver architecture for Windows." The graphics kernel publishes per-process GPU memory and engine counters, which programs read through PDH, a high-level API for collecting performance counter data, while adapters are enumerated through DXGI. On NVIDIA hardware, the NVIDIA Management Library (NVML) is the library behind nvidia-smi.

gputaskman's Windows backend reads what its source code calls "the same data Task Manager uses": the GPU Process Memory, GPU Engine and GPU Adapter Memory counters. It then combines that with other evidence, "rather than reading arbitrary GPU memory," as the README puts it:

Why not just use nvidia-smi?

On GeForce cards under WDDM, Windows manages GPU memory and NVML may report per-process memory as N/A. The Windows counters can still attribute memory and activity across Direct3D, Vulkan, OpenGL and CUDA, so gputaskman uses them for processes and keeps NVML for device readings. Inside WSL, nvidia-smi lists no processes at all.

Measured, inferred and estimated numbers

The contributor guide, AGENTS.md, sets one rule: every number is measured, inferred, estimated or reported by a runtime, and the UI and JSON must say which. Only measured bytes are added to totals, and an unavailable counter appears as N/A in the interface and null in JSON, never as zero. The guide also states that the tool never reads another process's memory, launches runtimes or sends inference requests.

Supported GPUs and operating systems

Windows needs a current WDDM driver, and the troubleshooting table points to WDDM 2.0 or later in dxdiag. On Intel under Linux, device load and video load need root, as does power on an integrated GPU. The README names the GPU, driver and workload of each live check and marks behaviour covered only by mocks or unit tests as not validated.

How to install gputaskman

From PyPI with uv or pipx

The gputaskman package on PyPI works on Windows, Linux, WSL and macOS with Python 3.11 or newer.

1. Install uv or pipx if needed.
2. Run uv tool install gputaskman or pipx install gputaskman. Both keep the app in an isolated environment.
3. Start it with gputaskman or the shorter gtm.

To upgrade, run uv tool upgrade gputaskman or pipx upgrade gputaskman.

Portable Windows build

1. From the Releases page, download gputaskman-VERSION-windows-x64.zip for Intel or AMD PCs, or gputaskman-VERSION-windows-arm64.zip for Windows on ARM.
2. Extract the entire ZIP, keeping the _internal folder next to gputaskman.exe.
3. Open Windows Terminal or PowerShell in that folder and run .\gputaskman.exe.

No Python, uv, CUDA Toolkit, model download or installation is needed. The builds are unsigned, and each ZIP has a .zip.sha256 companion file for integrity checks.

Portable Linux build

The Linux tarballs for x64 and ARM64 need glibc 2.35 or newer (Ubuntu 22.04, Debian 12 or newer) and a supported driver:

tar -xzf gputaskman-VERSION-linux-x64.tar.gz
cd gputaskman-VERSION-linux-x64
./gputaskman

Clone and run, NixOS and macOS

From a clone, uv run --locked gputaskman fetches the pinned Python version if needed, creates .venv and installs the locked dependencies. The first run needs internet access, and no CUDA Toolkit, Visual Studio, administrator session, API key or model download is required.

With flakes enabled, nix run github:shanselman/gputaskman runs the published flake for x86_64-linux and aarch64-linux. It was contributed and checked on x86_64 NixOS 26.05; CI does not build it and ARM64 is not validated.

macOS has no portable download yet, so the PyPI route or a clone is used. Other users' processes, such as WindowServer, show GPU% without memory unless gputaskman runs with sudo.

How to use gputaskman: commands and options

Running gputaskman with no arguments opens the live TUI. The options, as defined in the CLI source, are:

Keyboard shortcuts

The device view

Each GPU gets a header with a VRAM bar split by process category, GPU activity, memory activity, PCIe traffic, temperature, power and clocks. Eight history charts are shown by default: GPU, SM clock (GFX clock on AMD), VRAM, memory activity, temperature, power, encoder and decoder. The Charts dropdown or c focuses on one metric. Mem activity is NVML's memory-activity percentage, not bandwidth in GB/s.

The process table and inspector

The columns are PID, Process, Kind, Runtime, VRAM, Shared, Paged, GPU%, 3D, Copy and Video, plus RAM and CPU% unless --no-system is set, and "What's inside". Kinds include LLM, decision, embedding, speech, diffusion, AI, compute, 3D/game, video, browser, app, compositor, kernel and system. GPU% is the busiest engine, as in Task Manager.

Selecting a process opens the inspector: the model it holds, how it was identified, architecture, quantization, layer count, weights, an estimated KV-cache cost and buffer sizes from compatible load logs. Model discovery is throttled, so a newly loaded model can take about 15 seconds to appear.

Slim monitoring, system panel and multiple GPUs

The TUI starts in full diagnostics. Pressing m switches to slim monitoring, which hides driver and CUDA versions, PCI address, PCIe generation, power and clock limits, while readings, charts and processes stay. The toggle lasts for the session only.

A System panel and RAM and CPU% columns show what each process costs the machine, as the operating system reports it. RAM is the working set, so it will not match Task Manager's default memory column, and CPU% is a share of the whole machine. These figures are never added to GPU totals.

On multi-GPU systems, each GPU gets its own header and charts, and the table adds a GPU column. On Windows, the TUI re-checks adapter topology every five seconds and reconnects an adapter that is disabled and re-enabled.

Snapshots, diffs and recordings

--once and --json print one text or JSON snapshot from the same collector as the TUI. Unavailable counters are null, and fields such as counter_status, vram_inferred_by and started_at record where values came from.

Comparing two snapshots

To see which memory bucket grew after a model load or context change, the README gives these steps:

1. Save a snapshot: gputaskman --json > before.json.
2. Load the model, or restart the server with a different -c value, then send it one request.
3. Save a second snapshot: gputaskman --json > after.json.
4. Run gputaskman diff before.json after.json, adding --json for machine-readable output.

Measured counters are compared separately from runtime-reported segments and GGUF estimates, which are never added to measured totals. Processes are paired by PID, then by name and listening ports, because a model reload often restarts a server under a new PID. Reused PIDs are detected from start times, and diff refuses snapshots from different adapters that share a GPU name.

In a README example on an RTX 4080 SUPER, changing a gemma-3-270m server from -c 4096 to -c 32768 estimated +504 MB of KV cache, but measured dedicated memory grew by only +83 MB. Gemma 3 uses sliding-window attention, so the estimate is an upper bound.

Recording a long session

gputaskman record appends one JSON snapshot per interval to a JSON Lines file:

gputaskman record --gpu 0 --interval 5 --duration 300 -o session.jsonl

-o is required, --gpu defaults to 0, --interval to one second, and without --duration recording runs until Ctrl+C. Each line is flushed as it is written, so stopping the process keeps every sample so far. The README gives about 27 KB per 52-process snapshot, roughly 100 MB an hour at one-second intervals. Any two lines can be passed to diff. Replaying or summarising a whole recording is not built yet.

Using gputaskman with local LLMs

The README documents model support per runtime:

Runtimes on iThub that gputaskman reads include Ollama and llama.cpp. It also names stable-diffusion.cpp files and Hugging Face cache paths.

How models are identified

Two of api, mapped and cmdline, including api or mapped, mark a model as confirmed. The model kind comes from GGUF header keys, which separate an LLM from an embedding, speech or decision model.

Exact buffer sizes versus estimates

Without logs, weights and KV cache are estimated from GGUF metadata and the context size, assuming llama.cpp-style placement. For exact sizes, the README shows a llama-server started with logging, as an example workload rather than a prerequisite:

llama-server.exe -m .\model.gguf -ngl 99 -c 8192 --metrics --log-file server.log -lv 4

The README's spot checks (llama.cpp b12061, CUDA 13.4, RTX 4080 SUPER), which it says validate these examples only:

Local server probes

To read context size, busy slots and throughput, the collector sends GET requests to ports that local processes already listen on, for paths such as /props, /slots, /metrics, /v1/models and /api/ps. It does not launch ollama, lms or an inference request. If llama-server was started with --api-key, that key is sent only to the same server's port and is never stored, printed or written to JSON.

Changes merged to the main branch after v0.4.1 also read request counts and KV-cache pool usage from the /metrics endpoints of llama-server and vLLM, labelled runtime-reported, not GPU memory, and fixture-tested only.

Reading the compositor, Shared and Paged numbers

The README has a section headed "read these carefully" about three readings:

• The compositor. Windows' per-process Dedicated Usage counter includes shared resources, so one surface can be counted against an application, csrss.exe and dwm.exe. Microsoft's GPU accounting article notes that these counters include cross-process shared memory. gputaskman prefers Local Usage and Non Local Usage. In one README sample, DWM's older counter read about 15.3 GiB while Local Usage was about 1.4 GiB.
• Shared. GPU-accessible system RAM, which the README calls spill. It can include buffers placed there on purpose, so it is not proof of overflow.
• Paged. An estimate: committed memory minus local and non-local usage. It is not an eviction trace or a pagefile measurement.

With NVIDIA on native Linux, NVML has no equivalent counters, so Shared and Paged stay blank rather than showing zero. On AMD, GPU-accessible system RAM (GTT) fills the Shared column.

WSL and containers

Under WSL, Windows charges all Linux GPU memory to the VM worker process, vmwp.exe. With WSL interop enabled, gputaskman starts one powershell.exe that streams the Windows GPU counters. The WSL VM row is then the exact total, Windows apps appear as W:<pid> rows, and Linux processes get sizes by matching the VM's GPU contexts: * marks a match and ? a heuristic guess. --no-host turns this off.

In Docker, Kubernetes or rented GPU containers, NVML reports host PIDs. gputaskman pairs them with container processes that have libcuda.so mapped, using the same markers, and shows the rest as H:<pid>.

Privacy

According to the README, the monitor reads local process metadata, model headers, load logs and local server responses, and does not dump model weights or send telemetry. It does not read model names out of another process's memory, skips API-key and key-file arguments when scanning model paths, and on macOS skips other apps' sandbox containers. The README warns that inspector output, JSON and recordings can contain paths, command-line flags, model names and local ports, and asks users to review them before posting publicly.

Limitations

Troubleshooting

The README's troubleshooting table covers these cases:

• Empty or incomplete GPU readings. Check the driver and WDDM 2.0 or later support in dxdiag, and use a normal local desktop session.
• WSL host counters are off. Enable [interop] in /etc/wsl.conf, then run wsl --shutdown.
• WSL shows starting. The first Windows counter sample takes about 2.5 seconds.
• Linux rows show ?. The processes were running before gputaskman started; they become * once there is evidence.
• A Linux GPU process is missing. Processes owned by other users need sudo.
• The screen looks cramped. Widen the terminal or use --once or --json.

How gputaskman compares with other tools in the directory

The table below describes scope only, using each tool's iThub directory summary.

Argus and Mole report on the whole system, while gputaskman reports GPU memory and engine load per process, together with the runtimes and models behind them. Its README contrasts it with Task Manager's GPU memory total and with nvidia-smi, which may show N/A for per-process memory under WDDM and lists no processes inside WSL.

Summary

gputaskman is an MIT-licensed Python terminal tool that shows per-process GPU memory and engine load from Windows WDDM counters, NVML, Linux kernel interfaces and Apple Silicon statistics. It adds runtime detection, model identification with evidence labels, LLM weights and KV-cache estimates, and JSON snapshots, diffs and recordings. It installs from PyPI, as portable builds or through Nix, and the gputaskman listing keeps a short profile alongside related tools.