From e0ad62c5d176d021b2bd782dfe4d9595fa5ebfe1 Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Thu, 22 Oct 2026 13:12:31 +0200 Subject: Default to Gyro-S, record B580 test results Gyro-M does not fit a 12 GB B580 with 31 GiB RAM on the xe stream-experts path: the engine refused with 5,309 experts neither in VRAM nor in the pinned RAM mirror. Gyro-S fits and answers at 22-29 tok/s, but hit UR_RESULT_ERROR_DEVICE_LOST in the verify step on 2 of about 6 requests. AGENTS.md records the upstream fake-RAM bug that adds --ple-io ram to Gyro configs, the measurements, the suspected spin-bound cause and the untried fixes. Logs are now ignored. Co-Authored-By: Claude Opus 5.5 --- .gitignore | 1 + AGENTS.md | 32 ++++++++++++++++++++++++++++++++ README.md | 8 ++++---- strata-docker | 2 +- 4 files changed, 38 insertions(+), 5 deletions(-) diff --git a/.gitignore b/.gitignore index 3a3cf48..b96dcd3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ /Strata/ /Strata-data/ +*.log diff --git a/AGENTS.md b/AGENTS.md index caddcd5..5d97a07 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,9 +31,41 @@ reimplement anything upstream's scripts already do. 12 GB card, so close other programs before starting it. - Gyro on Arc was tested upstream only on an A750 (`i915`). The B580 is untested natively; one WSL2 B580 report saw device loss. +- Measured here (v0.2.2, B580, 31 GiB RAM): on `xe` the engine runs with + `--stream-experts` and `STRATA_VERIFY_NO_HOST`, so every expert must sit in + the VRAM cache or a pinned RAM mirror (capped at MemAvailable minus 4 GiB). + Gyro-M does not fit: 4.6 GiB VRAM cache + 21.6 GiB mirror left 5,309 + experts out and the engine REFUSED. setup's "35 GB of VRAM + RAM: fits" + check is wrong for this path. Gyro-S fits: 3,062 experts in VRAM, 21,514 + mirrored (21 GiB), about 6 GiB of RAM left, decode 22-29 tok/s. +- Gyro-S is not stable here: 2 of about 6 requests hit + `UR_RESULT_ERROR_DEVICE_LOST` in the verify (speculative decoding) step, + the engine segfaulted (139) and the chat showed "verify error". The server + restarts the engine and the card recovered each time. +- Likely cause, not verified: device spins on host flags are capped at + 20,000 reads on B-series cards (`sycl/include/strata/sycl_doorbell.hpp`), + tuned on a B70 that holds every expert in VRAM. Here 21 GiB of experts + come from the RAM mirror over PCIe, so a wait can outrun the cap. Upstream + documents the same 139 crash pattern for a cold first request + (`docs/INTEL.md`, `STRATA_SYCL_SPIN_MAX`). +- Untried next steps: (1) `"env": {"STRATA_SPIN_MAX": "2000000"}` in the + config, read at engine start. Risk: on `xe` an overlong spin can wedge the + GPU until reboot, and the B580 drives the desktop. (2) Drop `--spec`, + `--spec-min-p` and `--mtp` from the config to avoid the draft verify, at + lower decode speed. +- Status 2026-10-22: testing stopped. Gyro-M deleted. Gyro-S (55 GiB) and + the shared draft layer remain in `Strata-data/`. - On `xe`, asking for more VRAM than the card has does not fail: buffers spill into RAM and the host can livelock. Never raise `--context` or lower `--vram-reserve-mib` past what setup's fit rule picked. +- Upstream bug (v0.2.2): `setup_intel.py` tells `setup.py` the host has + 1024 GB of RAM (`S.ram_gb = lambda: fake_ram`), so on the `xe` path + `setup.py` adds `--ple-io ram` to the Gyro config. That loads the 54 GB + n-gram table into RAM and would OOM this host. The Intel code itself says + Gyro's table must stay on the SSD (it only guards the `i915` branch). After + every `setup` run, check `Strata/strata-gyro-gyro-m.json` and drop + `--ple-io ram` from `args`. Start the model with + `Strata/run-gyro-gyro-m.sh`, which does not rewrite the config. - Pictures are off by default on Battlemage (they reset a B70). Do not pass `--vision yes`. - `docker build --pull` matters: a stale base can carry an ocloc that crashes diff --git a/README.md b/README.md index a86e491..2ab4463 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Run [Strata](https://github.com/Niko1221/Strata) on an Intel Arc card under Slackware, without installing oneAPI on the host. It wraps upstream's documented Docker route: the SYCL engine is built and run inside the `strata-sycl-dev` image, the setup and server run on the host. Defaults to an -Arc B580 and the Gyro-M model. +Arc B580 and the Gyro-S model. Upstream's Intel engine is experimental. Expect rough edges. @@ -14,8 +14,8 @@ Upstream's Intel engine is experimental. Expect rough edges. usable by your user - Docker, your user in the `docker` group - Python 3.10+ with venv, git -- Disk: about 13 GB for the image, about 92 GB for Gyro-M -- Memory: Gyro-M needs about 35 GB of RAM and VRAM together +- Disk: about 13 GB for the image, about 58 GB for Gyro-S +- Memory: Gyro-S needs about 21 GiB of free RAM for the expert mirror ## Usage @@ -23,7 +23,7 @@ Upstream's Intel engine is experimental. Expect rough edges. ./strata-docker fetch # clone Strata, check out the pinned tag (v0.2.2) ./strata-docker image # build the strata-sycl-dev image (~6 min, 13 GB) ./strata-docker build # build the engine for bmg-g21 (~3 min) -./strata-docker setup # download Gyro-M and start serving +./strata-docker setup # download Gyro-S and start serving ./strata-docker setup --check # what fits, changes nothing ``` diff --git a/strata-docker b/strata-docker index 3fc5cb7..7e19cc5 100755 --- a/strata-docker +++ b/strata-docker @@ -44,7 +44,7 @@ build) setup) shift cd "$src" - exec ./setup.sh --backend sycl --family gyro --model Gyro-M \ + exec ./setup.sh --backend sycl --family gyro --model Gyro-S \ --data-dir "$here/Strata-data" "$@" ;; *) -- cgit v1.2.3