aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-22 13:12:31 +0200
committerDanilo M. <danix@danix.xyz>2026-10-22 13:12:31 +0200
commite0ad62c5d176d021b2bd782dfe4d9595fa5ebfe1 (patch)
tree72cb5b088e8d5eaa98773f6dd36580fd5897b7f1
parent4738197c3f005ce2f563b7627ab345fb33956ee3 (diff)
downloadstrata-docker-master.tar.gz
strata-docker-master.zip
Default to Gyro-S, record B580 test resultsHEADmaster
Gyro-M does not fit a 12 GB B580 with 31 GiB RAM on the xe stream-experts path: the engine refused with 5,309 experts neither in VRAM nor in the pinned RAM mirror. Gyro-S fits and answers at 22-29 tok/s, but hit UR_RESULT_ERROR_DEVICE_LOST in the verify step on 2 of about 6 requests. AGENTS.md records the upstream fake-RAM bug that adds --ple-io ram to Gyro configs, the measurements, the suspected spin-bound cause and the untried fixes. Logs are now ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-rw-r--r--.gitignore1
-rw-r--r--AGENTS.md32
-rw-r--r--README.md8
-rwxr-xr-xstrata-docker2
4 files changed, 38 insertions, 5 deletions
diff --git a/.gitignore b/.gitignore
index 3a3cf48..b96dcd3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,2 +1,3 @@
/Strata/
/Strata-data/
+*.log
diff --git a/AGENTS.md b/AGENTS.md
index caddcd5..5d97a07 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -31,9 +31,41 @@ reimplement anything upstream's scripts already do.
12 GB card, so close other programs before starting it.
- Gyro on Arc was tested upstream only on an A750 (`i915`). The B580 is
untested natively; one WSL2 B580 report saw device loss.
+- Measured here (v0.2.2, B580, 31 GiB RAM): on `xe` the engine runs with
+ `--stream-experts` and `STRATA_VERIFY_NO_HOST`, so every expert must sit in
+ the VRAM cache or a pinned RAM mirror (capped at MemAvailable minus 4 GiB).
+ Gyro-M does not fit: 4.6 GiB VRAM cache + 21.6 GiB mirror left 5,309
+ experts out and the engine REFUSED. setup's "35 GB of VRAM + RAM: fits"
+ check is wrong for this path. Gyro-S fits: 3,062 experts in VRAM, 21,514
+ mirrored (21 GiB), about 6 GiB of RAM left, decode 22-29 tok/s.
+- Gyro-S is not stable here: 2 of about 6 requests hit
+ `UR_RESULT_ERROR_DEVICE_LOST` in the verify (speculative decoding) step,
+ the engine segfaulted (139) and the chat showed "verify error". The server
+ restarts the engine and the card recovered each time.
+- Likely cause, not verified: device spins on host flags are capped at
+ 20,000 reads on B-series cards (`sycl/include/strata/sycl_doorbell.hpp`),
+ tuned on a B70 that holds every expert in VRAM. Here 21 GiB of experts
+ come from the RAM mirror over PCIe, so a wait can outrun the cap. Upstream
+ documents the same 139 crash pattern for a cold first request
+ (`docs/INTEL.md`, `STRATA_SYCL_SPIN_MAX`).
+- Untried next steps: (1) `"env": {"STRATA_SPIN_MAX": "2000000"}` in the
+ config, read at engine start. Risk: on `xe` an overlong spin can wedge the
+ GPU until reboot, and the B580 drives the desktop. (2) Drop `--spec`,
+ `--spec-min-p` and `--mtp` from the config to avoid the draft verify, at
+ lower decode speed.
+- Status 2026-10-22: testing stopped. Gyro-M deleted. Gyro-S (55 GiB) and
+ the shared draft layer remain in `Strata-data/`.
- On `xe`, asking for more VRAM than the card has does not fail: buffers
spill into RAM and the host can livelock. Never raise `--context` or lower
`--vram-reserve-mib` past what setup's fit rule picked.
+- Upstream bug (v0.2.2): `setup_intel.py` tells `setup.py` the host has
+ 1024 GB of RAM (`S.ram_gb = lambda: fake_ram`), so on the `xe` path
+ `setup.py` adds `--ple-io ram` to the Gyro config. That loads the 54 GB
+ n-gram table into RAM and would OOM this host. The Intel code itself says
+ Gyro's table must stay on the SSD (it only guards the `i915` branch). After
+ every `setup` run, check `Strata/strata-gyro-gyro-m.json` and drop
+ `--ple-io ram` from `args`. Start the model with
+ `Strata/run-gyro-gyro-m.sh`, which does not rewrite the config.
- Pictures are off by default on Battlemage (they reset a B70). Do not pass
`--vision yes`.
- `docker build --pull` matters: a stale base can carry an ocloc that crashes
diff --git a/README.md b/README.md
index a86e491..2ab4463 100644
--- a/README.md
+++ b/README.md
@@ -4,7 +4,7 @@ Run [Strata](https://github.com/Niko1221/Strata) on an Intel Arc card under
Slackware, without installing oneAPI on the host. It wraps upstream's
documented Docker route: the SYCL engine is built and run inside the
`strata-sycl-dev` image, the setup and server run on the host. Defaults to an
-Arc B580 and the Gyro-M model.
+Arc B580 and the Gyro-S model.
Upstream's Intel engine is experimental. Expect rough edges.
@@ -14,8 +14,8 @@ Upstream's Intel engine is experimental. Expect rough edges.
usable by your user
- Docker, your user in the `docker` group
- Python 3.10+ with venv, git
-- Disk: about 13 GB for the image, about 92 GB for Gyro-M
-- Memory: Gyro-M needs about 35 GB of RAM and VRAM together
+- Disk: about 13 GB for the image, about 58 GB for Gyro-S
+- Memory: Gyro-S needs about 21 GiB of free RAM for the expert mirror
## Usage
@@ -23,7 +23,7 @@ Upstream's Intel engine is experimental. Expect rough edges.
./strata-docker fetch # clone Strata, check out the pinned tag (v0.2.2)
./strata-docker image # build the strata-sycl-dev image (~6 min, 13 GB)
./strata-docker build # build the engine for bmg-g21 (~3 min)
-./strata-docker setup # download Gyro-M and start serving
+./strata-docker setup # download Gyro-S and start serving
./strata-docker setup --check # what fits, changes nothing
```
diff --git a/strata-docker b/strata-docker
index 3fc5cb7..7e19cc5 100755
--- a/strata-docker
+++ b/strata-docker
@@ -44,7 +44,7 @@ build)
setup)
shift
cd "$src"
- exec ./setup.sh --backend sycl --family gyro --model Gyro-M \
+ exec ./setup.sh --backend sycl --family gyro --model Gyro-S \
--data-dir "$here/Strata-data" "$@"
;;
*)