Skip to content

Recommender Results Model Deep Dive

Home Recommender Results Model Deep Dive

Loading GPU benchmark data…

Home / GPUs / GPU

← All GPU benchmarks

GPU Benchmark

Tokens/sec by Model

Median tokens per second across all benchmark runs on this GPU. CPU offload means the model exceeded VRAM and ran partly on system RAM.

Loading…

Legend: CPU offload — model larger than VRAM, runs on system RAM (2–5× slower) | Partial — some layers spill to system RAM, minor speed penalty

Speed vs Other GPUs

Model:

Buying Options

Methodology

Benchmarks run on Vast.ai consumer GPU instances using Ollama with fixed 2048-token context and fixed-length generation. Results are the median of available runs. Platform-rejected results (EPYC multi-GPU server boards) are excluded.

As an Amazon Associate, I earn from qualifying purchases. This does not affect the price you pay.

All GPUs Recommender Vast.ai Ollama